diff --git a/README.md b/README.md index 0d0f83c..33035e8 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,9 @@ flow browse | [`go-project/`](go-project/) | Full developer lifecycle (build, test, lint, release) — same pattern applies to any ecosystem (Node, Rust, Python…) | | [`git/`](git/) | Git workflow helpers (commit, fetch/rebase, branch cleanup) | | [`api/`](api/) | HTTP automation — GitHub REST API, webhook dispatch | -| [`docker/`](docker/) | Container workflows (build, run, push, clean) | +| [`docker/`](docker/) | Driving the docker CLI as the task — build, run, push, clean | +| [`container/`](container/) | Running executables *inside* an image with exec's `container` block — pinned toolchains, volumes, entrypoints | +| [`python/`](python/) | Python executables — inline `interpreter: python`, `.py` files, params via `os.environ`, mixed shell/Python steps | | [`kubernetes/`](kubernetes/) | kubectl automation (context, apply, pods, logs, shell) + Helm shared-library pattern (reusable installer called by app deployers) | | [`setup/`](setup/) | Project onboarding (prereq checks, tool install, env config) | | [`assets/`](assets/) | Supporting scripts and templates referenced by examples | @@ -40,8 +42,12 @@ flow template generate NAME exec-template.flow.tmpl ## Validation -All `.flow` files in this repo are schema-validated in CI: +Every `.flow` file, the workspace config, and the template are schema-validated +in CI against flow's `main` build: ```sh flow validate ``` + +Examples using recently-added fields are validated against `main` rather than the +latest release, so a feature can be demonstrated here as soon as it lands. diff --git a/assets/scripts/report.py b/assets/scripts/report.py new file mode 100755 index 0000000..d47141c --- /dev/null +++ b/assets/scripts/report.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 +# f:name=report f:verb=generate +# f:description=Summarize a JSON file, or the built-in sample when none is given. +# f:tags=python|fromfile +# f:args=pos:1:DATA_FILE +"""Imported by python/scripts.flow via `imports`. + +The f: comments above are read at sync time, exactly as they are for .sh +scripts - Python uses the same # comment prefix, so the syntax is unchanged. +""" + +import json +import os +import sys + +SAMPLE = {"passed": 48, "failed": 0, "skipped": 2} + + +def main() -> int: + path = os.environ.get("DATA_FILE", "") + if path: + with open(path, encoding="utf-8") as handle: + data = json.load(handle) + else: + print("no DATA_FILE given, using the built-in sample") + data = SAMPLE + + total = sum(data.values()) + print(f"total: {total}") + for key, value in sorted(data.items()): + share = (value / total * 100) if total else 0 + print(f" {key:<8} {value:>4} ({share:.1f}%)") + + # A non-zero exit propagates: flow marks the run failed and shows the + # traceback or message, the same as any other executable. + return 1 if data.get("failed") else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/basics/launch.flow b/basics/launch.flow index 973ec81..ac695be 100644 --- a/basics/launch.flow +++ b/basics/launch.flow @@ -20,7 +20,6 @@ executables: is a built-in variable pointing to the workspace root directory. launch: uri: $FLOW_WORKSPACE_PATH - wait: true - verb: open name: app diff --git a/container/toolchains.flow b/container/toolchains.flow new file mode 100644 index 0000000..6a3f3e4 --- /dev/null +++ b/container/toolchains.flow @@ -0,0 +1,109 @@ +# yaml-language-server: $schema=https://flowexec.io/schemas/flowfile_schema.json +namespace: container +description: | + Running executables inside a container image, using exec's `container` block. + + This is different from the `docker` namespace: those examples drive the docker + CLI as the task itself. Here the container is where the task runs — flow starts + it, mounts the workspace, passes the resolved environment in, and cleans up. + Requires `docker` or `podman` on the PATH. +tags: + - container + - docker +executables: + - verb: build + name: pinned-toolchain + description: | + Run a command in a pinned toolchain image instead of installing it locally. + The workspace root is mounted at /workspace and used as the working + directory, so relative paths behave as they do on the host. + exec: + dir: // + cmd: go version && go build ./... 2>/dev/null || echo "no go module here - the point is the toolchain" + container: + image: golang:1.26-alpine + + - verb: run + name: pinned-node + description: | + The same pattern with a different ecosystem. Nothing about the executable + changes except the image, which is what makes this useful for a repo whose + contributors do not all have the same tools installed. + exec: + cmd: node --version && echo "ran in the image, not on your machine" + container: + image: node:22-alpine + + - verb: run + name: with-env + description: | + Parameters and arguments reach the container as environment variables, the + same as a host run. flow passes them through an env file rather than the + command line, so secrets stay out of `docker inspect` and the process list. + FLOW_IN_CONTAINER is set so a script can tell where it is running. + exec: + params: + - envKey: GREETING + text: hello from the host + args: + - envKey: TARGET + default: container + pos: 1 + required: false + cmd: | + echo "${GREETING}, ${TARGET}" + echo "in container: ${FLOW_IN_CONTAINER}" + container: + image: alpine:3 + + - verb: run + name: with-volumes + description: | + Extra mounts beyond the workspace. A host path may be absolute, `~/`-prefixed, + or `//`-prefixed for workspace-relative. The container side must be absolute. + Append `:ro` to mount read-only. + exec: + cmd: ls /assets && echo "---" && cat /assets/scripts/hello.sh + container: + image: alpine:3 + volumes: + - "//assets:/assets:ro" + + - verb: run + name: custom-workdir + description: | + By default flow mounts the workspace at /workspace and runs there. Override + either side when an image expects a particular layout. + exec: + dir: // + cmd: pwd && ls + container: + image: alpine:3 + mountWorkspace: /src + workdir: /src/basics + + - verb: run + name: custom-entrypoint + description: | + flow overrides the image entrypoint with `sh` by default, so `cmd` behaves + as a shell command on any image. Name a different one when the command needs + it - here bash, for a bashism that would fail under sh. Setting `entrypoint` + to an empty string uses the image's own ENTRYPOINT instead. + exec: + cmd: | + shopt -q login_shell; echo "bash-only builtin ran fine" + echo "shell: ${BASH_VERSION:-not bash}" + container: + image: debian:bookworm-slim + entrypoint: bash + + - verb: test + name: isolated + description: | + A practical use: run the test suite against a pinned runtime so a local + version drift cannot change the result. Combine with `serial` to run + several such steps in order. + serial: + execs: + - ref: build container:pinned-toolchain + - ref: run container:pinned-node diff --git a/docker/containers.flow b/docker/containers.flow index 95ca081..1f2cef3 100644 --- a/docker/containers.flow +++ b/docker/containers.flow @@ -45,11 +45,11 @@ executables: docker tag app:latest "${REGISTRY}/app:${TAG}" docker push "${REGISTRY}/app:${TAG}" echo "Pushed ${REGISTRY}/app:${TAG}" - params: - - envKey: REGISTRY - prompt: Registry URL (e.g. ghcr.io/myorg) - - envKey: TAG - prompt: Image tag (e.g. v1.0.0) + params: + - envKey: REGISTRY + prompt: Registry URL (e.g. ghcr.io/myorg) + - envKey: TAG + prompt: Image tag (e.g. v1.0.0) - verb: clean aliases: [prune] diff --git a/flow.yaml b/flow.yaml index e5f40ca..655fe3f 100644 --- a/flow.yaml +++ b/flow.yaml @@ -6,6 +6,8 @@ tags: - node - git - docker + - container + - python - k8s - api - setup diff --git a/go-project/release.flow b/go-project/release.flow index 2ea5155..fbf8bd2 100644 --- a/go-project/release.flow +++ b/go-project/release.flow @@ -22,9 +22,9 @@ executables: git tag -a "v${VERSION}" -m "Release v${VERSION}" git push origin "v${VERSION}" echo "Tagged and pushed v${VERSION}" - params: - - envKey: VERSION - prompt: "Release version (e.g. 1.2.0)" + params: + - envKey: VERSION + prompt: "Release version (e.g. 1.2.0)" - verb: build name: snapshot diff --git a/python/basics.flow b/python/basics.flow new file mode 100644 index 0000000..ca4addc --- /dev/null +++ b/python/basics.flow @@ -0,0 +1,106 @@ +# yaml-language-server: $schema=https://flowexec.io/schemas/flowfile_schema.json +namespace: python +description: | + Running Python through flow. Set `interpreter: python` to run `cmd` as a + script, or point `file` at a `.py` and the extension implies it. + + Parameters, arguments, and secrets arrive as environment variables, so read + them from `os.environ` exactly as a shell command would read `$VAR`. +tags: + - python + - basics +executables: + - verb: run + name: inline + description: | + `interpreter: python` runs `cmd` as Python instead of a shell command. Use + a block scalar for anything multi-line - indentation is preserved, so + normal Python formatting works. + exec: + interpreter: python + cmd: | + import sys + + print("running", sys.version.split()[0]) + print("interpreter:", sys.executable) + + - verb: run + name: with-params + description: | + Params and args reach Python through the environment, the same as a shell + command. Nothing python-specific is needed to read them. + exec: + interpreter: python + params: + - envKey: GREETING + text: hello + args: + - envKey: NAME + default: world + pos: 1 + required: false + cmd: | + import os + + print(f"{os.environ['GREETING']}, {os.environ['NAME']}!") + + - verb: run + name: script + description: | + A `.py` file needs no `interpreter` at all - the extension implies it. + Setting `interpreter` explicitly overrides whatever the extension says. + exec: + dir: // + file: assets/scripts/report.py + + - verb: run + name: traceback + description: | + flow runs `cmd` from a temporary file rather than `python -c`, so a + traceback reports the real line number. Run this to see it - the error is + on line 4 and says so. + exec: + interpreter: python + cmd: | + import json + + payload = '{"not": "valid", }' + json.loads(payload) + + - verb: run + name: mixed-steps + description: | + Steps inside `serial` and `parallel` take their own `interpreter`, so one + workflow can mix shell and Python without splitting into separate + executables. A step that omits it runs under the shell as before. + serial: + execs: + - name: collect (shell) + cmd: | + echo '{"passed": 3, "failed": 0}' > "${FLOW_TMP_DIRECTORY:-/tmp}/data.json" + - name: summarize (python) + interpreter: python + cmd: | + import json + import os + + path = os.path.join(os.environ.get("FLOW_TMP_DIRECTORY", "/tmp"), "data.json") + with open(path, encoding="utf-8") as handle: + data = json.load(handle) + print("summary:", data) + + - verb: run + name: in-container + description: | + Combine `interpreter: python` with `container` for a pinned interpreter + that does not depend on what is installed locally. The image's own python + is used - a host virtualenv is deliberately not carried in, so install + dependencies in the image or mount them with `volumes`. + exec: + interpreter: python + cmd: | + import sys + + print("python", sys.version.split()[0], "from the image") + container: + image: python:3.13-alpine diff --git a/python/scripts.flow b/python/scripts.flow new file mode 100644 index 0000000..33a1d2c --- /dev/null +++ b/python/scripts.flow @@ -0,0 +1,14 @@ +# yaml-language-server: $schema=https://flowexec.io/schemas/flowfile_schema.json +namespace: py-scripts +description: | + Turning existing `.py` files into named executables with `imports`, rather + than wrapping each one in a flow file by hand. + + Python uses `#` line comments, the same prefix shell scripts use, so the + `f:` metadata syntax is identical - see assets/scripts/report.py. Imported + executables are tagged `generated` and run like any other. +tags: + - python + - imports +imports: + - ../assets/scripts/report.py diff --git a/setup/database.flow b/setup/database.flow index 7aa7f8c..0247445 100644 --- a/setup/database.flow +++ b/setup/database.flow @@ -84,11 +84,11 @@ executables: description: | Drop all tables and re-apply migrations from scratch. Useful during development — destructive in production. - params: - - envKey: DB_PATH - text: .data/app.db serial: failFast: true + params: + - envKey: DB_PATH + text: .data/app.db execs: - cmd: | echo "=== Dropping all tables ===" diff --git a/validate.flow b/validate.flow index 1a4617f..1f2bc0d 100644 --- a/validate.flow +++ b/validate.flow @@ -1,8 +1,29 @@ # yaml-language-server: $schema=https://flowexec.io/schemas/flowfile_schema.json executables: - verb: validate - description: Schema-validate all *.flow files in the workspace. + description: | + Schema-validate every flow file, the workspace config, and the template. + + The file list is built with `find` rather than a `*.flow` glob: the glob + only matches the workspace root, so subdirectories - which is where nearly + every example lives - went unchecked. exec: dir: // cmd: | - flow schema validate *.flow --strict + set -e + + flowfiles=$(find . -name '*.flow' -not -path './.git/*' | sort) + count=$(echo "$flowfiles" | wc -l | tr -d ' ') + echo "validating ${count} flow file(s)..." + flow schema validate $flowfiles --strict + + echo "validating workspace config..." + flow schema validate flow.yaml --strict + + templates=$(find . -name '*.flow.tmpl' -not -path './.git/*' | sort) + if [ -n "$templates" ]; then + echo "validating template(s)..." + flow schema validate $templates --type template --strict + fi + + echo "all files valid"