Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Contributing

This repo is private. Contributions from outside the team aren't open yet, but the same bar
applies to anyone with access: every change should be backed by a field run or a control, not
a guess.
Contributions are open. Ran the toolkit on your own app and hit a wall, a wrong verdict or a missing step? Send it:
an issue, a learnings draft or a PR are all welcome. One bar for everyone: every change is backed by a field run or a
control, not a guess.

## Three ways in

Expand All @@ -12,6 +12,9 @@ a guess.
made by hand, unknown error codes, scripts and toolkit items into the inbox. Then
`upgrade-toolkit/bin/submit-learnings.sh toolkit` runs the leak checks and opens one PR from a `learnings/` branch.
3. **A direct PR** for a finished change, with the checks below.
4. **An issue** for a bug you can't fix yourself. Every run ends with `FEEDBACK-REPORT.md` in its work folder: where it
stopped, the exit code and the drafts it harvested. Paste its summary into a GitHub issue after you have read it for
names, hosts and paths.

Toolkit or brain: the toolkit takes generic lessons under code names. Anything internal (customer context, notes from
the company brain, `confidentiality: internal`) goes to the brain with `submit-learnings.sh brain`, never here;
Expand Down
4 changes: 4 additions & 0 deletions upgrade-toolkit/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,10 @@ An item is done only when its "done when" check holds on a real app, not when th
| 148 | React-client readiness and safety: `dojo_report.py` only runs when a React switch is already decided, so legacy-widget and Dojo-only-package counts surface late instead of in triage for every 10-to-11 or 11-to-11 rung ([dojo-to-react](playbooks/dojo-to-react.md) 1); a non-ES-module pluggable (DropdownContainer 1.2.0) fails the React export but is invisible to `mx check` ([dojo-to-react](playbooks/dojo-to-react.md) 3.1); the React mxbuild bundler can OOM (exit 137) under ~4 GB free container memory with no warning ([dojo-to-react](playbooks/dojo-to-react.md) 3.5) | triage runs `dojo_report.py` on every 10-to-11/11-to-11 rung regardless of client decision; a readiness check flags a non-ES-module pluggable before build; mx10.sh/mx11.sh warns below ~4 GB free before a React build | M | FSM |
| 149 | `convert_legacy.py` (dump-mpr-driven Dojo-to-React converter, [dojo-to-react](playbooks/dojo-to-react.md) 2) silently drops `SelectFirst` instead of naming every grid that had it in `CONVERT-REPORT.md`; `run_journeys` reports a client-swap regression without first reconfirming it on a fresh database copy, so Dojo-run state leakage (a destructive journey, an added row) can read as a false regression (FSM: 8 flagged, 0 confirmed after rerun); `runapp` reports a stopped app generically ("app went down") instead of the runtime's own stop reason (FSM: a 3 h trial-license limit, not a crash) | `CONVERT-REPORT.md` lists every `SelectFirst` grid; every REGRESSION gets an automatic fresh-data rerun before being reported; a license/known-limit stop is named, not generic | M | FSM |
| 150 | `mxcli widget init` writes `.claude/skills/widgets` into the app folder, which then needs a manual exclude from every app commit | the app's `.gitignore` (or the toolkit's own commit step) excludes it without a manual step | S | FSM |
| 151 | Rung 2 stops at triage (exit 13) for a missing `plan.txt` although rung 1's plan already names most modules: write the rung 2 plan by hand every time (cold CMM run 10; the content ids now carry over, `rung2_cids`) | rung 2 triage pre-fills `plan.txt` from rung 1's plan and the 11.x versions of `playbooks/11-to-11.md`, and stops only for modules it cannot place | M | CMM |
| 152 | Journeys take 3 h or more on a mid-size app, and a stopped journeys step restarts from the first journey (cold CMM, cold WM1) | journeys resume from the last finished journey; the cockpit shows journeys done/total | M | CMM, WM1 |
| 153 | No app code name at run start: runlog warns, and harvest and the feedback report skip until a person writes `state/app-code` (cold CMM) | the pipeline derives the code from the app name (or asks once at triage) and writes `state/app-code` for rung 1 and rung 2 | S | CMM |
| 154 | Rung 2 convert stops (exit 13) for a missing `plan-pre.txt` although no update in the plan is pre-10.21 (cold CMM run 12) | triage writes an empty `plan-pre.txt` with a comment when there is nothing to update before the convert | S | CMM |

## Done (14 Oct, remedy trial round)

Expand Down
43 changes: 42 additions & 1 deletion upgrade-toolkit/bin/cloud/upgrade-pipeline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -407,12 +407,38 @@ timing_close() { # timing_close EXITCODE
{ [ -s "$f" ] || printf 'step\tstart_utc\tend_utc\texit\n' > "$f"; printf '%s\t%s\t%s\t%s\n' "$TIMING_STEP" "$TIMING_START" "$end" "$1" >> "$f"; } 2> /dev/null || true
TIMING_STEP=""
}
# feedback (bin/feedback_report.py): every exit, a stop as much as a PASS, harvests what the run taught into
# $WORK/learnings/ and writes $WORK/FEEDBACK-REPORT.md: where it stopped, the drafts, and how to send them back (PR) or
# file a toolkit bug (issue). Rung 2 reports on the whole run folder. Never changes the exit code. UPG_HARVEST=0 skips the
# harvest; the report still comes. Before this, harvest.py ran only when someone remembered steps/report.md 3c (cold WM1).
# rung 2 localedits: modules rung 1 updated carry content id 0 in the MPR, so rung 2's localedits printed UNKNOWN ("give
# --cid Module=<id>") for each one and the run stopped for a hand-written localedits.args (cold CMM, 7 modules). The content
# ids are in rung 1's plan.txt: add a --cid=Module=ID line for each plan module the args file does not name yet.
rung2_cids() { # rung2_cids PLAN ARGSFILE
[ -f "$1" ] || return 0
local cid ver mod
while read -r cid ver mod _; do
case "$cid" in ''|\#*) continue ;; esac
[[ "$cid" =~ ^[0-9]+$ ]] && [ "$cid" != 0 ] && [ -n "$mod" ] || continue
grep -Eq -- "(^|[= ])$mod=" "$2" 2> /dev/null || printf -- '--cid=%s=%s\n' "$mod" "$cid" >> "$2"
done < "$1"
}
feedback_finish() { # feedback_finish EXITCODE STEP
[ "${DRY:-0}" != 1 ] && [ -d "${WORK:-}" ] || return 0
local fw=$WORK app="${UPG_APP_CODE:-${UPG_APP_NAME:-}}"
[ "$(basename "$WORK")" != rung2 ] || fw=$(dirname "$WORK")
[ -n "$app" ] || app=$(head -n 1 "$fw/state/app-code" 2> /dev/null || true)
python3 -I "$B/feedback_report.py" --work "$fw" --exit "$1" --step "$2" --code "$app" \
--source "${SOURCE:-}" --target "${TARGET:-}" 2>&1 | sed 's/^/ /' || true
}
cleanup() {
local a rc=$?
local a rc=$? st=${TIMING_STEP:-${STEP:-}}
[ "$rc" != 0 ] || st=""
timing_close "$rc" || true # the step that is stopping (exit 4, a judgment stop, an error) gets its row with its own exit code
# run log (bin/runlog.py): a gate step that stops for judgment still leaves its row; the exit code stays the step's own
if [ "${STEP:-}" = gate ] && [ "${DRY:-0}" != 1 ] && [ -z "${RUNLOG_DONE:-}" ]; then record_runlog || true; fi
for a in "${APPS_UP[@]:-}"; do [ -z "$a" ] || { runenv; env "${RUNENV[@]}" python3 "$B/runapp.py" stop "$a" > /dev/null 2>&1 || true; }; done
feedback_finish "$rc" "$st" || true
}
trap cleanup EXIT

Expand Down Expand Up @@ -447,6 +473,10 @@ git_identity() {
export GIT_COMMITTER_NAME="${GIT_COMMITTER_NAME:-upgrade-pipeline}" GIT_COMMITTER_EMAIL="${GIT_COMMITTER_EMAIL:-upgrade-pipeline@localhost}"
}
git_identity
# The pipeline names its containers ${UPG_NAME:-mig}-<name> but did not export the name: runapp.py, with UPG_NAME unset, then
# REFUSED to stop mig-oldjp because another run's cmm-oldjp existed (FM-9 guard), and newjp found port 8080 taken (cold WM1
# journeys, next to cold CMM). Exported, every tool it calls uses the same prefix it does.
export UPG_NAME="${UPG_NAME:-mig}"
ensure_gitignore() {
local f="$APP/.gitignore" l
for l in '.claude/' '.mxcli/' '.stepupdate/' 'deployment/'; do
Expand Down Expand Up @@ -1159,9 +1189,19 @@ step_update() {
nj_add 13 "Decision" "Update the kept module (stepupdate with a one-line plan), or put the old widget back; to accept the skew, create $WORK/widgetskew-ack.txt."
nj_park
fi
# a module update brought pages saved with a newer widget than the installed package: update-widgets (row 5) would
# drop the settings the old package lacks (cold CMM: Administration 4.5.0's Roles filter lost its association mode)
rc=0
xo "$WORK/widgetdrop.out" python3 "$B/widgetdrop.py" "$APP" || rc=$?
if [ "$rc" = 1 ] && [ ! -f "$WORK/widgetdrop-ack.txt" ]; then
nj_file 13 "Widget settings that update-widgets would drop: the installed widget package is older than the pages (steps/repair.md row 33)" "$WORK/widgetdrop.out" 10 '^WIDGET-DROP'
nj_add 13 "Decision" "Update the module that ships the widget (stepupdate with a one-line plan) before the build; to accept the loss, create $WORK/widgetdrop-ack.txt."
nj_park
fi
else
x python3 "$B/repair/jarclash.py" "$APP" --md "$WORK/jarclash.md"
x python3 "$B/widgetskew.py" "$APP" --plan "$WORK/plan.txt"
x python3 "$B/widgetdrop.py" "$APP"
fi
local rc=0 ap=()
[ ! -f "$WORK/approved.txt" ] || ap=(--approved "$WORK/approved.txt")
Expand Down Expand Up @@ -1924,6 +1964,7 @@ main() {
for s in env-values.json create-spec.json localedits.args interview-mode.txt; do
[ ! -f "$WORK/$s" ] || [ -f "$WORK/rung2/$s" ] || cp "$WORK/$s" "$WORK/rung2/$s"
done
rung2_cids "$WORK/plan.txt" "$WORK/rung2/localedits.args"
fi
exec env "RUN=$RUN" "UPG_PHASE=$(echo "$TARGET2" | cut -d. -f1-3)" "UPG_COCKPIT_OUT=$WORK/COCKPIT.html" "${r2[@]}"
fi
Expand Down
137 changes: 137 additions & 0 deletions upgrade-toolkit/bin/feedback_report.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""feedback_report.py: at the end of every run, harvest what it taught and write FEEDBACK-REPORT.md, no LLM.

The pipeline calls this from its exit trap, so a run that stops (an environment fault, a judgment stop, a crash) leaves
the same report as one that passes. Before this, bin/harvest.py only ran when a person remembered steps/report.md 3c.

<work>/learnings/ harvest.py drafts (toolkit; brain drafts in learnings/brain/ when UPG_BRAIN is set)
<work>/FEEDBACK-REPORT.md where the run stopped and why, the drafts, and how to send both back: the learnings
by PR (submit-learnings.sh), a toolkit bug as a GitHub issue with a summary to paste

The issue summary holds metadata only (versions, step, exit code, the headings of NEEDS-JUDGMENT.md), never log text,
and goes through the same denylist as harvest.py; a hit leaves the summary out and says so.

usage: feedback_report.py --work DIR --exit N [--step STEP] [--code CODE] [--source V] [--target V]
Never fails the run: exit 0 always, except 2 for a bad call. UPG_HARVEST=0 skips the harvest (the report still comes).
"""
import argparse
import os
import pathlib
import re
import subprocess
import sys

B = pathlib.Path(__file__).resolve().parent
TK = B.parent

MEANING = {
0: "gate PASS, nothing left for a person",
2: "usage or version-path error",
3: "missing tool or setup, or an mxcli of another version (an environment fault, not the app)",
4: "a fixed step failed unexpectedly (likely a toolkit or environment bug: worth an issue)",
10: "mx check / build / start errors after the upgrade: repair needed",
11: "journeys INVALID or FAIL on the old or the new app",
12: "diff or gate items need a comparison by a person",
13: "a module plan or a module decision is missing",
14: "the upgrade proposal awaits sign-off",
15: "the owner paused the run",
}


def deny_pattern():
out = subprocess.run(["bash", "-c", f'. "{B}/private_names.sh" >/dev/null 2>&1; printf %s "${{DENY:-}}"'],
capture_output=True, text=True)
return out.stdout.strip()


def toolkit_rev():
r = subprocess.run(["git", "-C", str(TK), "rev-parse", "--short", "HEAD"], capture_output=True, text=True)
return r.stdout.strip() or "unknown"


def issues_url():
r = subprocess.run(["git", "-C", str(TK), "remote", "get-url", "origin"], capture_output=True, text=True)
m = re.match(r"(?:https://|git@)github\.com[:/]([^/]+/[^/.]+?)(?:\.git)?$", r.stdout.strip())
return f"https://github.com/{m.group(1)}/issues/new" if m else "the toolkit's GitHub issues page (Issues > New issue)"


def main():
ap = argparse.ArgumentParser(usage=__doc__.split("usage: ")[1].split("\n")[0])
ap.add_argument("--work", required=True)
ap.add_argument("--exit", type=int, required=True)
ap.add_argument("--step", default="")
ap.add_argument("--code", default="")
ap.add_argument("--source", default="")
ap.add_argument("--target", default="")
a = ap.parse_args()
work = pathlib.Path(a.work)
if not work.is_dir():
print(f"feedback_report: no work folder {work}", file=sys.stderr)
return 2

learn = work / "learnings"
if os.environ.get("UPG_HARVEST", "1") == "0":
harvest = "skipped (UPG_HARVEST=0)"
elif not a.code:
harvest = "skipped: no app code name (set UPG_APP_CODE, or write it to state/app-code, then run bin/harvest.py)"
else:
cmd = [sys.executable, "-I", str(B / "harvest.py"), "--work", str(work), "--code", a.code, "--inbox", str(learn)]
if os.environ.get("UPG_BRAIN"):
cmd += ["--to", "both", "--brain-inbox", str(learn / "brain")]
else:
cmd += ["--to", "toolkit"]
r = subprocess.run(cmd, capture_output=True, text=True)
lines = [l for l in (r.stdout + r.stderr).splitlines() if l.strip()]
harvest = next((l for l in lines if l.startswith("harvest:")), lines[-1] if lines else f"harvest.py exit {r.returncode}")
if r.returncode == 1:
harvest += " (denylist hit: that place was not written)"
drafts = sorted(p.relative_to(work) for p in learn.rglob("*.md")) if learn.is_dir() else []

meaning = MEANING.get(a.exit, "unexpected failure (no exit code of the pipeline legend): worth an issue")
nj = work / "NEEDS-JUDGMENT.md"
heads = [l.strip() for l in nj.read_text(errors="replace").splitlines() if l.startswith("## ")][:12] if nj.is_file() else []
issue = [f"toolkit {toolkit_rev()}", f"upgrade {a.source or '?'} -> {a.target or '?'}",
f"stopped at step: {a.step or 'end of run'}", f"exit {a.exit}: {meaning}"]
if heads:
issue += ["needs judgment:"] + [f" {h[3:]}" for h in heads]
issue_text = "\n".join(issue)
deny = deny_pattern()
if deny and re.search(deny, issue_text, re.I):
issue_text = "(left out: it matches the private denylist; write the issue by hand)"

rel_learn = "learnings"
out = [
"# Feedback report",
"",
f"Exit {a.exit} at {a.step or 'the end of the run'}: {meaning}.",
"",
"## What the run taught",
"",
f"{harvest}",
"",
]
out += [f"- `{d}`" for d in drafts] or ["No drafts."]
out += [
"",
"## Send it back",
"",
f"1. **Learnings.** Read each draft in `{rel_learn}/`, then copy the toolkit drafts to `contrib/inbox/` of your toolkit",
" checkout and run `bin/submit-learnings.sh toolkit --dry-run`, then without `--dry-run`: one branch, one PR.",
" Brain drafts (`learnings/brain/`) go with `bin/submit-learnings.sh brain`, never to the toolkit.",
f"2. **A toolkit bug** (exit 3 or 4 that is not your machine, a wrong verdict, a crash): open {issues_url()}",
" and paste the summary below. Read it first: no customer, app, host or person names.",
"",
"## Summary for an issue",
"",
"```",
issue_text,
"```",
"",
]
(work / "FEEDBACK-REPORT.md").write_text("\n".join(out))
print(f"feedback: {work / 'FEEDBACK-REPORT.md'} ({len(drafts)} draft(s))")
return 0


if __name__ == "__main__":
sys.exit(main())
12 changes: 11 additions & 1 deletion upgrade-toolkit/bin/harvest.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@
FILE_LIKE = re.compile(r"\.(html|js|java|md|jar|mpk|mpr|css|scss|json|log|txt|py|sh|xml|tsv)$", re.I)
SHA = re.compile(r"\s*\(`[0-9a-f]{7,40}`\)")
SCRIPT_EXT = {".py", ".sh", ".js", ".mdl"}
# folders the pipeline writes itself, never the agent: the patched journey engine copy (<work>/e2e-*/engine, run_journeys.py),
# editroute's replay MDL and localedits' saved edits. Their files differ from any toolkit file by design (cold CMM: 37 of
# them were listed as scripts the agent wrote).
TOOLKIT_WRITTEN = {"replay", "saved-edits"}


def toolkit_written(rel):
parts = Path(rel).parts
return any(p in TOOLKIT_WRITTEN for p in parts[:-1]) or any(
p == "engine" and i > 0 and parts[i - 1].startswith("e2e-") for i, p in enumerate(parts[:-1]))
SKIP_DIRS = {".git", "node_modules", "__pycache__", "e2e", "playwright-report", "test-results", "vendor"}
TEXT_EXT = {".log", ".md", ".txt", ".out", ".tsv"}
MAX_TEXT = 5_000_000
Expand Down Expand Up @@ -215,7 +225,7 @@ def harvest(work, brain):
if "agent" in r.get("#", ""):
found["items"].append({"src": rel, "text": f"agent decision: {r.get('decision', '')}: "
f"{r.get('choice', '')} ({r.get('why', '')})"})
if f.suffix in SCRIPT_EXT:
if f.suffix in SCRIPT_EXT and not toolkit_written(rel):
try:
h = hashlib.sha256(f.read_bytes()).hexdigest()
except OSError:
Expand Down
9 changes: 7 additions & 2 deletions upgrade-toolkit/bin/journey-verbs.js
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,8 @@ const ROWS = ['.mx-listview-item', '.mx-templategrid-item', '.widget-gallery-ite
'[role="row"]:not(:has([role="columnheader"])) [role="gridcell"]:not(:has(.empty-placeholder))',
'[role="row"]:not(:has([role="columnheader"])) [role="cell"]:not(:has(.empty-placeholder))',
'.widget-datagrid .td:not(:has(.empty-placeholder))',
// cold CMM (9.24.2): Data grid 2.2.x cells are div.td with no role, and the grid has no widget-datagrid class
'.tr[role="row"] > .td:not(.column-selector):not(:has(.empty-placeholder))',
'[role="gridcell"]:not(:has(.empty-placeholder))'];
// what counts as a rendered row when no selector found a clickable one (instrument gap vs. empty list)
const ANY_ROW = '[role="row"]:not(:has([role="columnheader"])):not(:has(.empty-placeholder)), tbody tr, ' +
Expand Down Expand Up @@ -430,16 +432,19 @@ async function upload(page, a, name) {
// others met an Aborted workflow with the menu hidden. A row that does not exist is a fixture gap, said with the count.
const ROW_BOXES = ['.mx-listview-item', '.mx-templategrid-item', '.widget-gallery-item', 'tbody tr',
'[role="row"]:not(:has([role="columnheader"])):not(:has(.empty-placeholder))'];
const ROW_CELL = '[role="gridcell"], [role="cell"], .td:not(.column-selector)';
async function nthRow(box, n, ms) {
const tries = Math.max(1, Math.ceil((ms || 8000) / 250));
let seen = 0;
for (let t = 0; t < tries; t++) {
for (const s of ROW_BOXES) {
const loc = box.locator(s), vis = [];
// a display: contents row (Data grid 2) has no box of its own: it counts when its first cell is visible
for (let i = 0, c = await loc.count().catch(() => 0); i < c && i < 50; i++)
if (await loc.nth(i).isVisible().catch(() => false)) vis.push(loc.nth(i));
if (await loc.nth(i).isVisible().catch(() => false) ||
await loc.nth(i).locator(ROW_CELL).first().isVisible().catch(() => false)) vis.push(loc.nth(i));
if (vis.length > n) {
const cell = vis[n].locator('[role="gridcell"], [role="cell"]').first();
const cell = vis[n].locator(ROW_CELL).first();
return (await cell.count().catch(() => 0)) ? cell : vis[n];
}
seen = Math.max(seen, vis.length);
Expand Down
Loading
Loading