Agent Forge Operator bridges hosted-cluster autoscaling to VM capacity for HyperShift Agent platform clusters on vSphere and OpenStack.
HyperShift and CAPI remain the source of truth. Agent Forge watches the
AgentMachine objects rendered for a HyperShift NodePool; when they report
Ready=False with Reason=NoSuitableAgents, it creates provider-managed VMs so matching
Assisted Installer Agent objects can appear.
OpenStackAgentPool follows the same AgentMachine demand as VsphereAgentPool.
Each OpenStackAgent owns a Neutron port, a bootable persistent Cinder OS volume,
a Cinder discovery ISO volume, and a Nova instance. Glance images are shared through
an ownership-scoped content cache.
OpenStack discovery media stays attached for the worker's entire lifetime. The OS volume has boot index 0; the ISO CD-ROM has boot index 1. Firmware must support falling back to the CD-ROM while the OS disk is blank, then booting RHCOS from the OS disk after installation. Validate this before production use. The attached ISO can contain sensitive bootstrap material; restrict image and volume access. There is no OpenStack ISO ejection option.
- OpenStack setup, authentication, security, and troubleshooting
- OpenStack CRD reference
- Provider architecture and cleanup
- Example OpenStackAgentPool
OpenStack support is available starting with operator v1.3.0. Apply all four
CRDs before upgrading the controller, including when upgrading through Helm.
Operator v1.3.1 adds safe service/operation diagnostics in worker conditions and
logs, plus explicit HTTP opt-in through the credentials Secret. HTTPS remains the
default; see the setup guide for the plaintext transport trade-off.
Operator v1.3.2 uploads the exact ISO snapshot it hashed and automatically recovers
workers stuck before image upload without replacing their OS volumes.
Automatic ejection is disabled by default. Opt in each VsphereAgentPool
explicitly, starting with a canary pool:
spec:
iso:
ejectAfterInstall: trueOmitting the field or setting it to false preserves the existing media behavior.
Enabling it includes existing installed workers in that pool. Disabling it stops
new ejections and does not reattach media; a previously recorded operation still
finishes, including its CD-lock confirmation. ISO caching and pruning remain
independent, so a pool without ejection enabled can still report ISO pruning errors.
For opted-in pools, the operator disconnects and ejects cached media when the matching
Assisted Installer Agent reports Installed=True. This condition covers both
installed and added-to-existing-cluster; assignment (Bound) is not sufficient.
Red Hat documents removing discovery media after installation writes the OS and
reboots.
Waiting for installation completion is the operator's conservative automation policy.
Ejection verifies VM ownership and hardware identity and only touches
content-addressed ISO files in the pool's configured ISO datastore and path prefix.
It also handles existing installed workers. Unrelated media is left attached.
If the guest locks the CD door, the operator answers only the specific vSphere
CD-ROM disconnect lock question for its recorded disconnect operation; it does not enable
global automatic answers. Failures are retried and reported in the ISOEjected
condition (Disabled, WaitingForInstallation, EjectionInProgress, Ejected,
or EjectionFailed) and ISO operation metrics. The last successful Ejected
condition is preserved if the feature is subsequently disabled.
Before disconnecting, status.isoEjection records the VM UUID, ownership, datacenter,
device key, and exact ISO path. It checkpoints the question ID before answering.
After a restart, recovery resumes only that recorded operation, refuses changed
media or a different question, and clears the record only after success. Apply
updated CRDs together with the controller so these recovery fields are preserved.
Normal ISO retention removes the unused file
on a later refresh after every VM sharing it releases it.
Scale-down remains controlled by CAPI: drain, Agent unbind/reclaim, Machine deletion, then VM and Agent cleanup. The operator does not start new ejections for a Machine already being deleted or a released VM. A previously recorded ejection is finished first. Assisted Installer's automatic reclaim path downloads discovery boot artifacts to the node's disk and creates a boot entry before rebooting; this path does not require the original virtual CD. CAPI's deletion hook still waits for reclaim completion (or its failure state). Environments that rely on manually booting discovery media to reuse hosts must reattach an ISO for that workflow. Ejection does not bypass drain or deletion hooks.
User documentation is published on the containeroo website:
Requirements:
- Go 1.26 or newer.
kubectloroc.- Docker or another compatible container tool.
- Access to a Kubernetes or OpenShift cluster for deployment tests.
Common commands:
make test
make lint
make manifests
make install
make deploy IMG=<registry>/agent-forge-operator:<tag>Run locally against the active kubeconfig:
make install
make runThe controller uses govc for vSphere operations. The container image includes
govc; local make run expects govc at /usr/local/bin/govc unless
GOVC_PATH is set.
make test-vcsim exercises ISO caching and VM status/deletion against the vSphere
simulator. make test-e2e requires Docker and uses Kind to verify deployment,
metrics access, and a VsphereAgent status update. It installs minimal external
CRD fixtures for watches; it does not run HyperShift, CAPI, or Assisted Installer.