Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
4cc8dfa
Cockpit: Decide now cards, one click per open question, saved by the …
MendixMau Nov 16, 2026
442f43a
Cockpit: questions as Blocked on you in the sidebar, a dialog per que…
MendixMau Nov 16, 2026
ca7523f
cockpit: plan question for a check that did not run, richer cells, pe…
MendixMau Nov 16, 2026
63506b3
cockpit: step Approve/Reject save through the live server (they staye…
MendixMau Nov 16, 2026
1a6aaa3
cockpit: reload after a saved answer, so the item leaves Blocked on y…
MendixMau Nov 16, 2026
2fecbec
cockpit: a sign-off counts only for the plan it was given on (a chang…
MendixMau Nov 16, 2026
7d1474d
cockpit: sidebar note, page not up to date or stuck: talk to your age…
MendixMau Nov 16, 2026
d6b276f
cockpit: one test row per kind of app part (data, pages per role, but…
MendixMau Nov 16, 2026
0fb1dfd
cockpit: buttons and journeys row stays grey when unmeasured; band co…
MendixMau Nov 16, 2026
2c2f9a5
pipeline: containers on a Rosetta VM on Apple Silicon (bin/dockerhost…
MendixMau Nov 16, 2026
0ccec87
themefiles: an empty USER/EXTRA CODE block is no code (NanoflowCommon…
MendixMau Nov 16, 2026
4eb72df
Dojo to React on every upgrade: dojo_report.py runs in triage (and in…
MendixMau Nov 16, 2026
00649af
cockpit: withdraw dropped plan items, hide the stop bar while running…
MendixMau Nov 16, 2026
e0deafd
proposal: modules kept by the update no longer clear the sign-off
MendixMau Nov 16, 2026
b330b79
cockpit: every plan question card shows what exactly is behind it (ev…
MendixMau Nov 16, 2026
8b700c8
update instead of keep: a module the update refused is routed to an u…
MendixMau Nov 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions upgrade-toolkit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,9 @@ rung whether the app can be judged, what the customer must supply, and the best
associations that other modules own; all of them hold data. List the targets with `idcheck.py app.mpr --xmod` and
update those modules LAST (or not headless). Step idcheck catches it (E1: 27 associations).
- Updating an edited module replaces it with the package: the local edit is gone, and re-creating it gives new GUIDs
(data dropped, while mx check, mxbuild and the start all succeed). See the edited-modules playbook.
(data dropped, while mx check, mxbuild and the start all succeed). See the edited-modules playbook. `bin/editroute.py`
handles the safe cases (diagram-only, UI, constants, unused vendor drops, other modules' associations with their GUIDs).
A real logic edit belongs in `<Module>_Ext`, a module of the app's own: never edit a Marketplace module in place.
- A module update resets the module's constants to package defaults (E1: `MFAmodule.EnabledMFA` True to False, MFA
silently off) and can ADD a switch whose default hides data (E4 and E5b: a new boolean emptied the dashboard
charts, every journey PASSed). constcheck.py lists an added boolean as REVIEW; shotdiff shows the empty page.
Expand Down Expand Up @@ -172,8 +174,10 @@ plain PASS; a failure that is not a call into an excluded system stays FAIL, an

- Proposal not signed: `$W/PROPOSAL.md` section 6 has no `Signed-off: <name> <date>` (pipeline exit 14), or it says `Status: PAUSED` (exit 15), or a plan input changed after the sign-off (the sign-off is cleared). Do not convert, update or repair. An agent never writes the sign-off line. Unattended runs sign nothing: ASSUMED (unattended) in DECISIONS.md, "proposal not signed" in every report.

- RED line in localedits for a module on the plan: ask (steps/decide.md). UNATTENDED default: keep that module at its version, carry
on with the rest, and report "upgrade complete except kept modules: <list>" (never PASS for the kept ones).
- RED line in localedits for a module on the plan: ask (steps/decide.md). The recommendation is the update with the edits carried
over (`bin/editroute.py`), never "keep" without a reason and an end date. UNATTENDED: editroute routes what it can; a module it
HOLDs (real logic edit, Java user code, a removal still used) stays at its version, the rest carries on, and the report says
"upgrade complete except held modules: <list>" (never PASS for the held ones).
- `bin/steps.py check --work $W` names a D row of AGENT-DECISIONS.md that has no step ("MISSING D71: ..."): open the step
(`steps.py propose --new`), write its `S-n` in the row, then go on. Do not start the next change of that kind first.

Expand Down
5 changes: 3 additions & 2 deletions upgrade-toolkit/bin/brain_mine.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

usage: brain_mine.py [--brain DIR] [--table replacements.tsv] [--to VER] [--min N] [--out brain-patterns.md]
[--tsv candidates.tsv]
--brain default $UPG_BRAIN or ~/Mendix/<company>-brain (read only, never written)
--brain default: bin/brainpath.py, a ~/Mendix/*-brain with version tables (read only, never written)
--table default playbooks/replacements.tsv next to this bin/
--to drop rows of component versions whose min Mendix is above VER (not installable on the target)
--min components a successor needs to count as recurring (default 2)
Expand Down Expand Up @@ -107,7 +107,8 @@ def covered_by(names, cov):
def main():
here = Path(__file__).resolve().parent
ap = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
ap.add_argument("--brain", default=os.environ.get("UPG_BRAIN")
import brainpath
ap.add_argument("--brain", default=brainpath.find()[0] or os.environ.get("UPG_BRAIN")
or str(Path.home() / "Mendix/<company>-brain"))
ap.add_argument("--table", default=str(here.parent / "playbooks" / "replacements.tsv"))
ap.add_argument("--to")
Expand Down
50 changes: 50 additions & 0 deletions upgrade-toolkit/bin/brainpath.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/env python3
"""Find the company brain: the folder whose baseline/versions holds the Studio Pro version tables.

Order: $UPG_BRAIN, then UPG_BRAIN= in ~/.config/upgrade-toolkit/local.env, then the first ~/Mendix/*-brain folder with
version tables (a folder ending in -test only when nothing else has tables). Prints the path, or nothing and exit 1.

Cold test WM1: the default was the literal text ~/Mendix/<company>-brain, which never matches a folder, so the
release-note check never ran unless a person had set UPG_BRAIN.
"""
import glob
import os
import sys
from pathlib import Path


def has_tables(d):
return bool(glob.glob(os.path.join(d, "baseline", "versions", "studio-pro-upgrade-*.md")))


def from_local_env():
p = Path.home() / ".config" / "upgrade-toolkit" / "local.env"
try:
for line in p.read_text().splitlines():
line = line.strip()
if line.startswith("UPG_BRAIN="):
return os.path.expanduser(line.split("=", 1)[1].strip().strip("'\""))
except OSError:
pass
return None


def find():
"""(path, how) of the brain, or (None, why not)."""
for how, d in (("UPG_BRAIN", os.environ.get("UPG_BRAIN")), ("local.env", from_local_env())):
if d:
return (d, how) if has_tables(d) else (None, f"{how} points at {d}, which has no baseline/versions tables")
cands = sorted(c for c in glob.glob(str(Path.home() / "Mendix" / "*-brain*")) if os.path.isdir(c) and has_tables(c))
cands.sort(key=lambda c: c.endswith("-test"))
if cands:
return cands[0], "found under ~/Mendix"
return None, "no folder under ~/Mendix holds brain version tables (set UPG_BRAIN, or UPG_BRAIN= in ~/.config/upgrade-toolkit/local.env)"


if __name__ == "__main__":
path, how = find()
if path:
print(path)
sys.exit(0)
print(how, file=sys.stderr)
sys.exit(1)
54 changes: 49 additions & 5 deletions upgrade-toolkit/bin/cloud/upgrade-pipeline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,13 @@ set -u
: "${UPG_TOOLS:=$RUN/tools}"
: "${MXCLI:=${UPG_MXCLI:-}}"
B="$TK/bin"
# Apple Silicon: every container call goes to a VM with Rosetta (bin/dockerhost.sh). Cold test WM1: the run used a qemu VM,
# mxcli crashed on all 10 module updates and each was recorded as "kept".
if DH=$(bash "$B/dockerhost.sh"); then
[ -z "$DH" ] || [ "$DH" = "${DOCKER_HOST:-}" ] || { echo "upgrade-pipeline.sh: containers run on $DH (Rosetta)"; export DOCKER_HOST="$DH"; }
elif [ "${DRY:-0}" != 1 ]; then
exit 2
fi
PORT_OFF=${PORT_OFF:-0}
APP_PORT=$((8080 + PORT_OFF)); ADMIN_PORT=$((8090 + PORT_OFF))
export TK RUN UPG_TOOLS
Expand Down Expand Up @@ -243,6 +250,11 @@ decisions_apply() {
cockpit_regen() {
[ "$DRY" != 1 ] || return 0
python3 "$B/cockpitpage.py" --work "$WORK" --out "$WORK/COCKPIT.html" >> "$WORK/logs/decisions.log" 2>&1 || true
# the page must not contradict the run (cold test WM1: ghost "Not approved yet" rows, a stop bar while running); a problem is
# said in the step log and kept in cockpit-check.txt, never a failed step
if ! python3 "$B/cockpitcheck.py" --work "$WORK" > "$WORK/cockpit-check.txt" 2>&1; then
say "cockpit: the page contradicts the run, see $WORK/cockpit-check.txt"; sed 's/^/ /' "$WORK/cockpit-check.txt"
fi
# rung 2 also refreshes the cockpit of rung 1, so the run keeps one page from start to end
[ -z "${UPG_COCKPIT_OUT:-}" ] || [ ! -f "$WORK/COCKPIT.html" ] || cp "$WORK/COCKPIT.html" "$UPG_COCKPIT_OUT" 2>/dev/null || true
# UPG_COCKPIT_HISTORY=1 keeps every version of the page (a demo video shows the cockpit filling up), next to the one cockpit
Expand Down Expand Up @@ -601,6 +613,7 @@ Decide, in writing: (a) wrap it in a host app (an app with a home page, a theme
note "no module-jumps.txt: release-impact.md holds Studio Pro rows only; module rows NOT generated"
fi
[ "$rc" = 0 ] || note "release_impact.py exit $rc (2 = a module is UNKNOWN to the brain: fill its row by hand), see $WORK/release-impact.out"
dojo_report
# judgment: the plan
if [ "$DRY" != 1 ] && [ ! -f "$WORK/plan.txt" ]; then
nj_add 13 "Module plan missing: write $WORK/plan.txt" "No script picks module target versions (steps/triage.md item 3: newest version whose min Mendix fits the target, from the brain's modules-index.md and \`mxcli marketplace versions <content id>\`).
Expand Down Expand Up @@ -765,8 +778,9 @@ proposal_stop() { # proposal_stop RC: rc 14 / 15 end the run with NEEDS-JUDGMENT
case "$1" in
0) return 0 ;;
14) nj_add 14 "Upgrade proposal awaits sign-off: read $WORK/PROPOSAL.md" "$(head -n 3 "$WORK/proposal.out" 2> /dev/null)
Nothing in the app model has changed. Read PROPOSAL.md, answer the section-3 items in chat (steps/decide.md; the P-n lines go to $WORK/DECISIONS.md), then a person writes one line in section 6:
\`Signed-off: <name> <yyyy-mm-dd>\` to go on, or \`Status: PAUSED <reason>\` to stop on purpose. An agent never writes that line. Then run the command at the top of this file."
Nothing in the app model has changed. Answer the questions under Blocked on you in the live cockpit and approve the plan there
(the approval carries your Git name), or answer the section-3 items in chat (steps/decide.md) and approve on the page.
To stop on purpose, write \`Status: PAUSED <reason>\` in PROPOSAL.md section 6. Then run the command at the top of this file."
nj_file 14 "PROPOSAL.md (head)" "$WORK/PROPOSAL.md" 30 ;;
15) nj_add 15 "Upgrade proposal PAUSED by the owner" "$(head -n 3 "$WORK/proposal.out" 2> /dev/null)
The run stopped on purpose; no step after propose has run. Resume: remove the \`Status: PAUSED\` line from $WORK/PROPOSAL.md section 6, write \`Signed-off: <name> <yyyy-mm-dd>\`, then
Expand All @@ -775,8 +789,20 @@ The run stopped on purpose; no step after propose has run. Resume: remove the \`
esac
nj_flush
}
# Dojo to React (B6-4) on every run, against the last rung: the move is part of each upgrade, not a later project. Cold
# test WM1: the pipeline never ran dojo_report.py, so 144 legacy core widgets in 83 documents showed up nowhere.
dojo_report() {
[ "$DRY" != 1 ] || { say "+ dojo_report.py $WORK/original.mpr --to ${TARGET2:-$TARGET} --out $WORK/dojo-react.md"; return 0; }
[ -f "$WORK/original.mpr" ] || return 0
local rc=0
xo "$WORK/dojo-react.out" python3 "$B/dojo_report.py" "$WORK/original.mpr" --to "${TARGET2:-$TARGET}" --out "$WORK/dojo-react.md" || rc=$?
[ "$rc" = 0 ] || note "dojo_report.py exit $rc, see $WORK/dojo-react.out"
}

step_propose() {
local rc=0
# a run whose triage predates the report
[ -s "$WORK/dojo-react.md" ] || dojo_report
if [ "$DRY" = 1 ]; then
say "+ action_inventory.py $WORK/original.mpr --out $WORK/actions.json; apimock.py inventory $WORK/original.mpr --json $WORK/apimock-inventory.json"
say "+ proposal.py --work $WORK --source $SOURCE --target $TARGET (exit 14 unsigned, 15 PAUSED, 0 signed or unattended)"
Expand Down Expand Up @@ -899,7 +925,8 @@ replay_edits() { # replay_edits [pre]: pre = only modules the post-convert plan
[ -f "$f" ] || continue
mod=$(basename "$f"); mod=${mod%.*}
if [ "${1:-}" = pre ] && grep -qE "^[0-9]+ [^ ]+ $mod\$" "$WORK/plan.txt" 2> /dev/null; then continue; fi
if [ "$DRY" != 1 ] && git_app log --format=%s 2> /dev/null | grep -qxF "replay: $mod"; then note "replay $mod skipped: replayed already"; continue; fi
# one replay per module per rung: the 10.24 rung's replay must not skip the 11 rung's (editroute writes a new replay/ each rung)
if [ "$DRY" != 1 ] && git_app log --format=%s 2> /dev/null | grep -qxF "replay: $mod${UPG_PHASE:+ ($UPG_PHASE)}"; then note "replay $mod skipped: replayed already"; continue; fi
if [ "$DRY" != 1 ] && grep -q "^$mod " "$APP/.stepupdate/kept.txt" 2> /dev/null; then note "replay $mod skipped: the module was kept, its edits are in place"; continue; fi
rc=0
case "$f" in
Expand Down Expand Up @@ -928,7 +955,7 @@ replay_edits() { # replay_edits [pre]: pre = only modules the post-convert plan
else rc=1; fi ;;
esac
[ "$rc" = 0 ] || { nj_file 13 "replay of $mod failed (exit $rc): $f" "$WORK/replay-$mod$([ "${f##*.}" = patch ] && echo -patch).out" 30; nj_flush; }
commit_app "replay: $mod" replay "replay-$mod"
commit_app "replay: $mod${UPG_PHASE:+ ($UPG_PHASE)}" replay "replay-$mod"
note "replayed the local edits of $mod from $f"
done
}
Expand All @@ -950,10 +977,27 @@ assoc_unreplayed() {
done
}

# update: module updates with the post-convert plan; kept modules are recorded (SKILL stop rules), decided at the gate.
# route_kept (cold test WM1): a module the update refused is not kept at its old version by default. editroute.py routes each one
# (diagram-only, UI and constants replayed, unreadable but unedited, files kept as copies, vendor drops nothing uses, associations
# of other modules replayed with their ids) and a second stepupdate pass updates the routed modules; only HOLD rows stay kept.
route_kept() {
if [ "$DRY" = 1 ]; then x python3 "$B/editroute.py" plan --work "$WORK" --app "$APP" --original "$WORK/original.mpr" --plan "$WORK/plan.txt"; return 0; fi
[ -s "$APP/.stepupdate/kept.txt" ] || return 0
local rc=0 le=()
[ ! -s "$WORK/localedits.json" ] || le=(--localedits "$WORK/localedits.json")
xo "$WORK/editroute.out" python3 "$B/editroute.py" plan --work "$WORK" --app "$APP" --original "$WORK/original.mpr" --plan "$WORK/plan.txt" "${le[@]}" || rc=$?
cat "$WORK/editroute.out" >> "$LOG"
[ "$rc" = 0 ] || { note "editroute: no kept module could be routed, they stay kept (see $WORK/editroute.out)"; return 0; }
run_stepupdate "$WORK/route-plan.txt" routed
rc=0; xo "$WORK/editroute-collect.out" python3 "$B/editroute.py" collect --work "$WORK" || rc=$?
note "editroute: $(grep -c '^ROUTE' "$WORK/editroute.out") kept module(s) routed, $(grep -c ' HOLD:' "$WORK/editroute.out") held for a person (edit-routes.json)"
}

# update: module updates with the post-convert plan; kept modules are routed (route_kept), the rest decided at the gate.
step_update() {
[ -f "$WORK/plan.txt" ] || [ "$DRY" = 1 ] || { nj_add 13 "plan.txt missing" "$WORK/plan.txt does not exist"; nj_flush; }
run_stepupdate "$WORK/plan.txt" post-convert
route_kept
replay_edits
assoc_unreplayed
if [ "$DRY" != 1 ]; then
Expand Down
85 changes: 85 additions & 0 deletions upgrade-toolkit/bin/cockpitcheck.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#!/usr/bin/env python3
"""cockpitcheck.py: the cockpit must never contradict itself or the run. Builds the page model exactly as cockpitpage.py does
and checks what a person reads against the run's own files.

Why (cold test WM1, 2026-11-16): the owner saw rows that said "Not approved yet" and "Test gap" with nothing to approve, a red
"waits for your answers" bar after the plan was approved and the run was going, and "2 items to decide" next to "0 for you
now". Each was a separate cockpit bug; this check makes the whole class fail a run instead of confusing a person.

Rules (each line of output names one):
ghost-step an undecided plan-item step (P-n alias) on the page that the current PROPOSAL.md no longer holds
wait-no-ask a row whose plan cell asks for an approval ("Not approved yet", "Needs your OK") while the plan is signed
and Blocked on you does not hold that item
stop-no-ask the red stop bar says the run waits for you, and Blocked on you is empty
stop-and-running the red stop bar and the "Running now" bar at the same time
ask-no-evidence a plan question card that lists no element, file or finding behind it (evidence.py found nothing)
count-mismatch a Blocked on you card that "for you now" does not list while the run is stopped for a person
keep-recommended a module card that recommends keeping the old version without a reason and an end date (an edit is no
reason to stay behind: editroute.py carries edits over; cold test WM1, all seven cards said "keep")

cockpitcheck.py --work W prints CONSISTENT, or one line per problem; exit 0 / 1 (2 usage)
Run by: upgrade-pipeline.sh after each cockpit render (a problem is a note and a cockpit item, never a silent page), and controls.
"""
import argparse, os, re, sys

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import cockpitpage # noqa: E402
import cockpitmatrix # noqa: E402
import steps as stp # noqa: E402

ASK = ("Not approved yet", "Needs your OK")


def check(work):
model = cockpitpage.build_model(work)
M = cockpitmatrix.build(work, model)
out = []
asks = cockpitmatrix.open_questions(model)
ask_names = {re.sub(r"\s*\((module|widget|jar|java)\)$", "", i.get("title", "")) for i in asks}
plan = cockpitpage.rd(os.path.join(work, "PROPOSAL.md")) or ""
in_plan = set(re.findall(r"\[(P-\d+)\]", plan))
recs, _ = stp.load(work)
for r in recs or []:
decided = (r.get("approval") or {}).get("state") in ("approved", "rejected") # a module the update kept after the sign-off
if plan and r.get("alias", "").startswith("P-") and stp.mine(r) and r["alias"] not in in_plan and not decided:
out.append(f"ghost-step {r['id']} ({r['alias']}, {r['title']}): not in the current plan")
signed = bool(model.get("signed"))
for row in M["rows"]:
c = row["cells"].get("plan") or {}
if signed and c.get("l") in ASK and not any(n and n in row["name"] for n in ask_names):
out.append(f"wait-no-ask {row['id']} ({row['name']}): says \"{c['l']}\" but the plan is signed and nothing asks for it")
for i in asks: # a question with nothing under it cannot be decided (cold test WM1: "local edits refuse the update", no list)
if i["kind"] == "plan-choice" and not (i.get("evidence") or {}).get("what"):
out.append(f"ask-no-evidence {i['id']} ({i['title']}): the card asks a person to decide but shows no element, file or finding")
rec = (i.get("rec_text") or "").lower()
if i["kind"] == "plan-choice" and i.get("title", "").endswith("(module)") and rec.startswith("keep") \
and not re.search(r"\buntil\b|\bend date\b|\d{4}-\d{2}-\d{2}", rec):
out.append(f"keep-recommended {i['id']} ({i['title']}): recommends the old version without a reason and an end date")
stopped, running = model.get("stopped"), model.get("running")
if stopped and not asks and str(stopped.get("code")) in ("14", "15"):
out.append("stop-no-ask the red bar says the plan waits for your answers, and Blocked on you is empty")
if stopped and running:
out.append(f"stop-and-running the red stop bar (exit {stopped.get('code')}) shows while the run is running")
if stopped:
mine = [o for o in M["band"]["open"]["items"] if o["owner"] != "agent" and o["group"] != "At deploy"]
miss = [i["title"] for i in asks if i["kind"] != "signoff" and not any(
re.sub(r"\s*\((module|widget|jar|java|check)\)$", "", i["title"]) in o["text"] for o in mine)]
if miss:
out.append(f"count-mismatch the run waits for a person: Blocked on you holds {len(asks)}, \"for you now\" lacks {len(miss)} of them ({', '.join(miss[:5])})")
return out


def main():
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
ap.add_argument("--work", required=True)
a = ap.parse_args()
if not os.path.isdir(a.work):
print(f"cockpitcheck: no work folder {a.work}", file=sys.stderr)
return 2
probs = check(os.path.abspath(a.work))
print("\n".join(probs) if probs else "CONSISTENT")
return 1 if probs else 0


if __name__ == "__main__":
sys.exit(main())
Loading
Loading