Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ Each command directory has its own `CLAUDE.md` (domain knowledge) and `README.md
| `cmd/objectstorage/` | CLAUDE.md, README.md | S3-style object storage: configure, mb/rb, cp/mv/sync/ls/rm, uploads, presign |
| `cmd/network/` | CLAUDE.md, README.md | Private networks (beta): list, create, delete; subnet list, create, delete |
| `cmd/serverless/` | CLAUDE.md, README.md | Serverless containers (`verda container`: create, update, list, describe, scale, replicas, pause/resume/restart, purge-queue, delete, env, secret, file-secret, registry-creds) and batch jobs (`verda batchjob`: create, update, list, describe, delete, pause/resume, purge-queue) |
| `cmd/mk8s/` | CLAUDE.md, README.md | Managed Kubernetes (experimental): locations; cluster list, describe, create, upgrade, delete; node-pool list, describe, create, scale, delete; kubeconfig list, describe, create, save, delete |
| `cmd/doctor/` | — | Environment diagnostics |
| `cmd/availability/` | — | Instance availability by location |
| `cmd/cost/` | — | Balance, running costs, estimates |
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ require (
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
github.com/verda-cloud/verdacloud-sdk-go v1.9.0
github.com/verda-cloud/verdacloud-sdk-go v1.9.1
go.yaml.in/yaml/v3 v3.0.5
)

Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -186,8 +186,8 @@ github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/vbatts/tar-split v0.12.2 h1:w/Y6tjxpeiFMR47yzZPlPj/FcPLpXbTUi/9H7d3CPa4=
github.com/vbatts/tar-split v0.12.2/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
github.com/verda-cloud/verdacloud-sdk-go v1.9.0 h1:uKFMwgpHRheKurx0ANhxAsUe21znJn68ndN/MkpH/SQ=
github.com/verda-cloud/verdacloud-sdk-go v1.9.0/go.mod h1:va7s1wnZpFCqY/WVFxuUb66FpJzXgqHBYEWtEJTWx8o=
github.com/verda-cloud/verdacloud-sdk-go v1.9.1 h1:kttXszwkad1mVvpwC9UhkBms2uLzTO52bRI4/sxCT38=
github.com/verda-cloud/verdacloud-sdk-go v1.9.1/go.mod h1:va7s1wnZpFCqY/WVFxuUb66FpJzXgqHBYEWtEJTWx8o=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
Expand Down
7 changes: 7 additions & 0 deletions internal/verda-cli/cmd/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import (
"github.com/verda-cloud/verda-cli/internal/verda-cli/cmd/instancetypes"
"github.com/verda-cloud/verda-cli/internal/verda-cli/cmd/locations"
mcpcmd "github.com/verda-cloud/verda-cli/internal/verda-cli/cmd/mcp"
"github.com/verda-cloud/verda-cli/internal/verda-cli/cmd/mk8s"
"github.com/verda-cloud/verda-cli/internal/verda-cli/cmd/network"
"github.com/verda-cloud/verda-cli/internal/verda-cli/cmd/objectstorage"
"github.com/verda-cloud/verda-cli/internal/verda-cli/cmd/registry"
Expand Down Expand Up @@ -167,6 +168,12 @@ func NewRootCommand(ioStreams cmdutil.IOStreams) (*cobra.Command, *clioptions.Op
serverless.NewCmdBatchjob(f, ioStreams),
},
},
cmdutil.CommandGroup{
Message: "Kubernetes Commands:",
Commands: []*cobra.Command{
mk8s.NewCmdMK8s(f, ioStreams),
},
},
cmdutil.CommandGroup{
Message: "Resource Commands:",
Commands: resourceCmds,
Expand Down
66 changes: 66 additions & 0 deletions internal/verda-cli/cmd/mk8s/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# MK8s Command Knowledge

> Go house style lives in the root `CLAUDE.md`. This file carries managed-Kubernetes idioms only. User-facing behavior is in `README.md` and in `verda mk8s <command> --help`.

## Quick Reference

- Parent: `verda mk8s` (aliases `k8s`, `kubernetes`), tagged `(experimental)` via `cmdutil.TagAnnotation`, in its own "Kubernetes Commands:" group in `cmd/cmd.go`. No env-var gate: the server's per-project flag controls access.
- Verbs: `locations`; `cluster` list/describe/create/upgrade/delete; `node-pool` (`pool`, `np`) list/describe/create/scale/delete; `kubeconfig` list/describe/create/save/delete. One file per verb, plus:
- `resolve.go` -- `clusterTarget`, `resolveCluster`, `apiCall`/`apiRun` (timeout + spinner + `mapErr`), pickers' shared constants.
- `nodepool.go` -- `poolScope` (`--cluster` + `-l`, reused by every pool and kubeconfig command: `missing`/`check` validate, `connect` gets the client and the cluster), `resolvePool`, `poolCmd`.
- `confirm.go` -- `checkConfirmed`, `confirmWarning`, `askConfirm`. `errors.go` -- `mapErr`, `createFailed`.
- `wait.go` -- `waitFor` and the done conditions. `views.go` -- every JSON/YAML shape.
- `kubeconfig_file.go` (file rules), `kubeconfig_save.go` (`--file`/`--force`, recovery command), `kubeconfig_prompts.go` (direct prompts of `kubeconfig create`).
- `wizard_cluster.go`, `wizard_nodepool.go`, `wizard_cache.go` -- the two create wizards; `runWizard`, the step builders (`textStep`, `selectStep`, `cidrStep`) and `validateName` are in `wizard_cluster.go`.
- SDK: `client.ManagedKubernetes` (SDK v1.9.1); the wizards also read `PrivateNetworks`, `InstanceAvailability`, `SSHKeys`. mk8s does not import `cmd/network`.
- Help: every `Long` goes through `longDesc`, which puts `experimentalCaution` first. `TestMK8s_HelpStartsWithCaution` checks only the direct children of `mk8s`, not the leaf commands: do not build a `Long` without `longDesc`. Shared help strings (`yesUsage`, `clusterUsage`, `locationUsage`, `waitTimeUsage`) live in `mk8s.go`; reuse them so one rule reads the same everywhere.

## Domain-Specific Logic

- **Names are unique per project.** Cluster and node pool names are unique in the project (pool names too, not only in their cluster). So `resolveCluster` finds a cluster's location with one `List`; with a name and `-l` it makes no call; with no name it shows a picker (filtered by `-l`). No match is a plain `*verda.APIError` 404 (NOT_FOUND via `ClassifyError`); a name in several locations, which cannot happen today, is a `UsageErrorf` asking for `-l`. Every API path needs the location, so all commands go through `clusterTarget`.
- **Network and subnet: IDs only, never created.** `cluster create` sends `--network`/`--subnet` as given; the API checks them. The wizard lists networks, then that network's subnets in the chosen location (`subnetLoader` filters by `Location.Code`), and stops with a `stopError` that names `verda network create` / `verda network subnet create` when a list is empty.
- **Creates are sent exactly once.** The API can save an object and still fail the answer (create saves, then reads back; a kubeconfig gets a server-chosen ID), so a retry would give a 409 or a hidden second kubeconfig. `createFailed` + `createMayHaveSucceeded` print "may have been created anyway" (stderr for people, `details.warning` in agent mode) for a 5xx, a 408, a timeout or a network error after the request left. `neverSent` excludes a failed token fetch and a dial error. Some of these are matched on SDK error text (`sdkAuthFailed`, `sdkResponseUnread`, `sdkResponseUnparsed`): an SDK upgrade can change them; `TestCreateMayHaveSucceeded` guards them. `createFailed` itself has no test.
- **SDK retries.** `factory.go` installs `ExponentialBackoffRetryMiddleware`, but in SDK v1.9.1 the request middleware chain wraps a no-op handler and the HTTP call happens after it, so nothing is retried (the comment in `factory.go` says so). An SDK upgrade that makes retries real must keep mk8s creates at one attempt. Nothing guards that today: the unit tests build the client with `newFakeClient`, without the factory's middleware, and there is no mk8s contract test through the real binary.
- **One confirmation rule.** Deletes, scale-down and `cluster upgrade`: red warning and confirm on a terminal; elsewhere `--yes` is required, else `CONFIRMATION_REQUIRED`. Creates and scale-up: summary and confirm on a terminal only, never `--yes`. `checkConfirmed` never prompts and runs before `f.VerdaClient()` for deletes and upgrade; scale decides in `nodePoolScaleOptions.confirm`, after it reads the pool, because only then is "down" known. `interactiveTTY` = stdin, stdout and stderr are terminals, not agent mode, table output: `-o json` on a terminal is non-interactive.
- **Deletes cascade in the background and have no `--wait`.** The API deletes pools, worker VMs and kubeconfigs after it answers; a cluster or pool GET is 404 at once while VMs are still being deleted, so a wait until 404 would report "completed" while VMs still bill. The CLI does not check for pools before a cluster delete. Named pool and kubeconfig deletes do not read the object first, so one whose read fails stays deletable (`validUntil` then says "until they expire").
- **A scale to the current count is a no-op.** `node-pool scale` reads the pool first; with `replicas == current` it sends nothing, prints "already has N replicas; nothing to do." and returns `{"status": "unchanged", ...}` (exit 0). With `--wait` and a pool that does not yet meet `poolScaled` (for example after a scale-up timeout: replicas 2, ready 1) it skips the update and the confirmation, waits like a real scale (`poolWaitSettle`: a timeout has no scale-back advice, there is no previous count) and returns the normal `completed`/`accepted` + `wait_error` result. `cluster upgrade` has no such check: it would need an extra read and `kubernetes_version` is the requested version.
- **Timeout advice.** A failed wait prints the advice once, in the error (`poolWaitFailure`); the "Check progress with" line is only for a result without `wait_error`. After a scale-up timeout the advice is to scale back to the previous count (with `--yes`: it is a scale-down, which scripts and agent mode must confirm), after a create it is to delete the pool.
- **Dates.** Every date in mk8s output is UTC and says so (`utcTime`, `timeColumn`).
- **`cluster upgrade` has no `--wait`.** `kubernetes_version` is the requested version, not the running one, so polling it could end before the rollout starts. Add `--wait` only when the API reports the running version.
- **Wait conditions** (`wait.go`): cluster `running` with an endpoint; pool create `active` and `ready_replicas == requested`; scale `active` and `replicas == ready_replicas == requested`; kubeconfig `ready` and YAML present. They compare with the requested count, not the count read, so a stale first read cannot end the wait. A `degraded` pool keeps waiting (the backend keeps retrying and can still bill); the status line shows its error reasons.
- **Wait error handling** (`waitFor`): each read has its own `--timeout`; the whole wait ends after `--wait-timeout` (defaults per command: 15m cluster and pool, 3m kubeconfig; the shared `WaitOptions.AddFlags` 5m is not used). Up to 3 read failures in a row are tolerated (`maxReadFailures`); a 404 is tolerated until the first good read (the read lags the create); other 4xx except 408/429 fail at once (`fatalRead`). A read cut by the overall deadline counts as a timeout. A failed wait still writes the structured result (with `wait_error`) and exits non-zero (`writeResult`).
- **403 `feature_not_enabled`** becomes "Managed Kubernetes is not enabled for this project." (`mapErr`), still AUTH_ERROR. The message carries no instructions on how to get access.
- **Kubeconfig create** asks with direct prompts (`kubeconfig_prompts.go`), not the wizard engine; the others use `pkg/tui/wizard`. `--admin` adds `system:masters`; in scripts at least one of `--admin`/`--group` is required (`NewMissingOneOfFlagsError`; its message names every missing flag, `details` keeps `missing` and `one_of`). `--duration` is left out of the request when unset (API default 90 days).

## Gotchas & Edge Cases

- **Kubeconfig path problems are usage errors.** `kubeconfigTarget` returns an `*inputError` for a taken file, a symlink, a directory, a non-regular file, a missing parent directory and an unwritable directory; the commands convert it with `asUsage` (`UsageErrorf`: VALIDATION_ERROR, exit 2). Failures while creating, writing or renaming stay plain errors.
- **Human errors must be plain errors or `cmdutil.UsageErrorf`.** `main` prints any `*AgentError` as a JSON envelope even without `--agent`. Missing flags (`NewMissingFlagsError`) and `CONFIRMATION_REQUIRED` (`confirmationRequired`: one text for scripts, no terminal, agent mode and `-o json|yaml`) are AgentErrors on purpose; everything else returns `*verda.APIError`, `UsageErrorf` or `fmt.Errorf`, which `ClassifyError` still maps for agents. Only `createFailed` and `kubeconfigRecovery` build an AgentError, and only in agent mode.
- **Validate input before `f.VerdaClient()`** (`checkClusterArg`, `poolScope.check`/`missing`, the commands' own `missing`), so bad input fails the same with or without credentials. A blank `--cluster` or positional is always rejected, even on a terminal.
- **The kubeconfig YAML holds a client private key.**
- `KubeconfigView` has no YAML field; never marshal `verda.ManagedKubernetesKubeconfig` to output or `DebugJSON`. `listKubeconfigViews`/`getKubeconfigView` convert inside the API call so the SDK struct never escapes. The last read of `waitForKubeconfig` holds the YAML: keep it out of every printer.
- `cmdutil` `debugBody` prints `<omitted: kubeconfig endpoint>` for any `/kubeconfigs` path, and `sensitiveJSONFieldRe` lists `kubeconfig` (`redact_test.go`). Keep both if you touch `factory.go`.
- **Kubeconfig file rules** (`kubeconfig_file.go`): mode 0600; a new file is created with `O_EXCL` (the open decides, not the earlier check); `--force` writes a temp file in the same directory, syncs and renames; a symlink, a directory or a non-regular file is always refused; `~/.kube` is created 0700 only when missing (`options.MkdirSecure`); the parent of a custom `--file` must exist. The default name is `verda-<cluster>.<common-name>.yaml`, each part sanitized to `[A-Za-z0-9_-]`: the `.` separator cannot come from either part, so two clusters never share a file (with `-`, cluster `a-b`/cn `c` and cluster `a`/cn `b-c` would collide and `--force` would overwrite another cluster's key). Common names that differ only in replaced characters still share a file. `--file` equal to the default path is treated as the default. Modes are not enforced on Windows; mode tests skip there.
- **Check before `CreateKubeconfig`.** Order: flag validation, then (in scripts) the request (`validRequest`) and `destination()`, which checks the final path and probes that the directory is writable, then the cluster lookup. On a terminal a named `--file` is checked (`checkFree`) before any prompt, and the request and `destination()` after the prompts. A failure costs no certificate.
- **`--force` hint.** When `kubeconfig create --force` replaced an existing file (`kubeconfigDest.replaced`), it prints on stderr that the earlier kubeconfig is still valid and names `kubeconfig list`; nothing in the structured result. `kubeconfig save --force` prints none: it issues no certificate, so the file held the same kubeconfig.
- **`--file -`** writes only the YAML to stdout and is a `UsageErrorf` with `--agent` or `-o json|yaml` (it would share stdout with JSON). The interactive file prompt refuses `-` (it would print the key on the terminal). Recovery commands never use `--file -`.
- **Kubeconfig recovery.** After `CreateKubeconfig` succeeded, a wait timeout, failed reads or a failed write return the kubeconfig ID and the exact `kubeconfig save` command (`kubeconfigSaveCmd`: shell-quoted with `shellQuote`, which treats `:` as unsafe because it splits `KUBECONFIG`; default path and stdout left out; `--force` kept for files) plus "Running create again issues another certificate." In agent mode `kubeconfigRecovery` puts `kubeconfig_id`, `save_command` and `warning` in the details and keeps the recovery text as the message.
- **`node-pool create` wizard.** A cluster given by flag is resolved before the wizard, so a wrong name fails before any prompt. The cluster step sets the location from the picked cluster; its `Resetter` restores the `-l` value (`locationFlag`), so going Back does not keep a stale location. Instance types come from the availability API, which ignores private-network capacity: a listed type can still fail. The API data has no prices, so the picker lists types smallest first (`instanceTypeRank`: CPU-only, then GPU count) and Enter on the first row picks the smallest. The CLI never checks `--instance-type`.
- **Wizard hint bar.** `wizardLayout` has only the progress view: the engine's composite renders the one hint bar (prompt hints plus wizard keys). Adding a `HintBarView` shows a second, contradicting bar.
- **`cluster list` width.** The table drops the ENDPOINT column (after stripping `https://`) when a row is wider than `clusterTableWidth` (120); the endpoint holds the project ID, so it usually does.
- **Cluster wizard defaults.** The CIDR flags have non-empty defaults, so `podCIDRSet`/`serviceCIDRSet` (from `Flags().Changed`) decide whether the step runs; the steps are not `Required`, so their `Default` applies.
- **`cluster describe`** still prints the cluster when `ListNodePools` fails: `NodePools` is a pointer so JSON and YAML both say `null`, with `node_pools_error`.

## Tests

- `fake_test.go` is an in-memory mk8s API on `httptest` plus every shared helper and fixture (`seededClusters`, `poolAPI`, `setHome`, `readFile`, `assertMode`, `assertNoPrivateKey`, ...): pools and kubeconfigs keyed by cluster name, `failures` ("POST /mk8sclusters" → status/body, every request), `disabled` (403 `feature_not_enabled` everywhere), and hooks `onClusterGet`, `onPoolGet`, `onKubeconfigGet` that move objects through states while a command waits. The hooks run under the fake's lock: read `a.requests` directly, never `requestsMatching`/`paths` inside them.
- `TestMain` forces `isInteractiveTerminal` to false, so every test starts off a terminal. `withTerminal`, `fastPoll` (`pollInterval`), `withWizardAnswers` (`newWizardEngine`) and `currentUserName` swap package variables: never call `t.Parallel` in or under a test that uses them.
- Factories: `tableFactory`, `jsonFactory`, `agentFactory`; `runMK8s` runs the command tree. Kubeconfig fixtures (`kcFixtureAPI`, `kcAPI`, `kcPrivateKeyMarker`) are in `kubeconfig_list_test.go` and `kubeconfig_create_test.go`.
- Tests check the API request, the table and JSON output, the `--wait` result and one error per command; the `--yes`/decline refusal paths and the "may have been created" warning are not tested end to end. The security guards are: `TestKubeconfigOutputNeverHasThePrivateKey`, `TestKubeconfigTarget_CreateIsExclusive`, `TestKubeconfigTarget_RejectsSymlink`, `TestNewKubeconfigTarget`, the 0600/0700 checks and `redact_test.go`'s `TestDebugTransport_KubeconfigBodies`.

## Relationships

- `cmdutil` -- `WithSpinner`, `Poll`, `WaitOptions`, `DebugJSON`, `WriteStructured`, `ClassifyError`, `NewMissingFlagsError`, `NewMissingOneOfFlagsError`, `NewConfirmationRequiredError`, `UsageErrorf`, `TextColumn`; `factory.go` (`debugBody`, `sensitiveJSONFieldRe`, the retry middleware).
- `options.MkdirSecure` -- the 0700 `~/.kube`.
- `pkg/tui/wizard` -- the cluster and node pool create flows; `wizardCache` memoizes their lookups.
- `cmd/network` -- users create the network and subnet there; no code dependency.
Loading
Loading