Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
0478a46
cockpit: a signed plan shows the current wording, and an item the sig…
MendixMau Nov 16, 2026
bfc9e76
editroute: replay the change, not the element; route again after a ro…
MendixMau Nov 16, 2026
de12f74
editroute XASSOC: associations from the original model with their mem…
MendixMau Nov 16, 2026
5900078
pipeline: a rerun of the update step starts with fresh route files (e…
MendixMau Nov 16, 2026
6d73740
editroute score: did each route work, per run and in a ledger across …
MendixMau Nov 16, 2026
081799b
editroute: brain notes on every route (brainnotes.py, hit@k measured)…
MendixMau Nov 16, 2026
a901480
pipeline: a decision file with only comments no longer fails the upda…
MendixMau Nov 16, 2026
facba73
editroute: an edited layout in a Marketplace module repoints the app'…
MendixMau Nov 16, 2026
d4b02a6
editroute: read the idcheck file the kept reason names, not the newes…
MendixMau Nov 16, 2026
b63a191
editroute: XASSOC carries dropped associations without data too
MendixMau Nov 16, 2026
293ef08
pipeline: build repair runs jardedupe with --updated-from-git
MendixMau Nov 16, 2026
9040a80
build: map module roles the updates added; after walk follows the bas…
MendixMau Nov 16, 2026
e4aa1c1
diff: approve System-module platform drops by a baseline count of 0
MendixMau Nov 16, 2026
4fba119
heal: a NEW_ENDPOINT row is a new surface, not lost data
MendixMau Nov 16, 2026
8af37a0
pipeline: platform drops step survives a work folder without db-ledge…
MendixMau Nov 16, 2026
aaf319f
controls: drop PATH entries with a space before any control runs
MendixMau Nov 16, 2026
53eeff3
controls: run the same on macOS as on Linux
MendixMau Nov 16, 2026
e13f33f
cockpitmatrix: no personal name in a comment (customer-names-tracked-…
MendixMau Nov 16, 2026
fe66a7b
controls: bt7 runs against a stand-in app server (AD-1 app_alive made…
MendixMau Nov 16, 2026
c1698e7
controls: last stale fixtures (macOS and gate drift)
MendixMau Nov 16, 2026
b56c7a1
fetch: read the zip listing once (unzip | grep -q under pipefail lost…
MendixMau Nov 16, 2026
b3eac0c
pipeline: two more grep -q pipes under pipefail
MendixMau Nov 16, 2026
bf9fe75
platform_drops: re-count editroute VENDOR approvals on the baseline
MendixMau Nov 16, 2026
c8270f5
replay: a failed replay is held, not a stop
MendixMau Nov 16, 2026
bb5e4dd
idcheck: MODULE_RECREATED is info when every element identity is kept
MendixMau Nov 16, 2026
7180efa
datamove: count gate for a run-once data move (scenario D)
MendixMau Nov 17, 2026
132323a
editroute: copy Marketplace pages the menu opens onto the company layout
MendixMau Nov 17, 2026
acdabb2
pipeline: Interview mode auto parks judgment stops and asks them in o…
MendixMau Nov 17, 2026
1879f45
editroute: the menu copy leaves out APP TITLE and APP ICON
MendixMau Nov 17, 2026
e1b677f
playbook 9-to-10: no mxcli oql on the 9.24 runtime, the read-only SQL…
MendixMau Nov 17, 2026
183d044
editroute: no menu copy of a page that holds a widget mxcli cannot write
MendixMau Nov 17, 2026
186c0f1
playbook 9-to-10: data reads on 9.24 go through the toolkit's own Pos…
MendixMau Nov 17, 2026
44d9111
data compare: a vendor module's own change and the toolkit's admin lo…
MendixMau Nov 17, 2026
5fd444b
baseline: load the package's demo data (data-snapshot.zip) with seed_…
MendixMau Nov 17, 2026
3604c25
smoke walk: a popup opened from the menu counts as landed, and microf…
MendixMau Nov 17, 2026
073e5aa
pipeline: the step scripts get the pipeline's git identity when the m…
MendixMau Nov 17, 2026
81ae926
pipeline: an explicit --db replaces the database name kept in state/d…
MendixMau Nov 17, 2026
ba76a0e
Cockpit: per-action meaning, test runs old vs new, one sign-off for r…
MendixMau Nov 17, 2026
775ddae
Cockpit: a decision opens its case (headline, tiles, old and new scre…
MendixMau Nov 17, 2026
8aa073d
editroute: a Marketplace page the menu opens gets the app's layout in…
MendixMau Nov 17, 2026
91b1098
controls: the throwaway Postgres also starts from binaries on PATH, s…
MendixMau Nov 17, 2026
68156c5
seed_snapshot: an external entity's remote key column is kept, not re…
MendixMau Nov 17, 2026
1fcd524
stepupdate: a module forced for another route that has no UI edits is…
MendixMau Nov 17, 2026
2899024
build: mx check -j names the document of each error; the stop report …
MendixMau Nov 17, 2026
51665bd
journeys: a scroll container is no landing guard
MendixMau Nov 17, 2026
33d8c2f
docs: feedback ledger, every piece of the owner's feedback with its c…
MendixMau Nov 17, 2026
09cd6c4
Module updates that carry the app's edits are not routine; regression…
MendixMau Nov 17, 2026
4fe3494
localedits: files the OS writes into a folder are no theme edit
MendixMau Nov 17, 2026
ddbc1fb
cockpit: check and theme plan cards show their evidence
MendixMau Nov 17, 2026
6d59cc9
repair row 27: CE1613 errors hide the CE0117
MendixMau Nov 17, 2026
047506d
crosscalls --own: name own documents a module update drops, before th…
MendixMau Nov 17, 2026
f3afb78
build: a crashed mx check is never 0 errors
MendixMau Nov 17, 2026
9f3331f
repair: copydoc.py copies a document a module update dropped into an …
MendixMau Nov 17, 2026
f6ebf4c
editroute: a replayed association gets member access on its other ent…
MendixMau Nov 17, 2026
47b55b0
build: a javac missing symbol names the jar that hides the class (jav…
MendixMau Nov 17, 2026
4a37eb8
update: a widget replaced under a kept module's styles parks the upda…
MendixMau Nov 17, 2026
ea991cf
ci: install pymongo, the fixture model builder needs its bson module
MendixMau Nov 17, 2026
47c4963
ci: install zsh, the toolkit wrappers exec it
MendixMau Nov 17, 2026
f7ed981
docs: ledger row for the 10 controls that fail only on Linux CI
MendixMau Nov 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/controls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@ jobs:
with:
python-version: '3.12'

- name: Python packages the controls use
# controls/lib/mkmodel.py writes fixture models as BSON (the bson module ships with pymongo)
run: pip install pymongo

- name: Shell the toolkit scripts use
# bin/*.sh wrappers exec zsh (macOS default); the Ubuntu runner has none
run: sudo apt-get install -y zsh

- name: Toolkit controls (fast)
run: bash upgrade-toolkit/controls/run-controls.sh --fast

Expand Down
123 changes: 123 additions & 0 deletions upgrade-toolkit/bin/brainnotes.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
#!/usr/bin/env python3
"""brainnotes.py: the company brain's notes that belong next to a run event (a held module, a route, a failed check).

Why (cold test WM1, 2026-11-16): the toolkit read only the brain's version tables. The brain also holds the practice and
pitfall notes for exactly the events a run stops on (never edit a Marketplace module, hide Atlas layouts in the company
UI module, PDFBox 2 jars next to PDFBox 3 after a Community Commons update), and no card showed them. The data test hit
the PDFBox case by hand while the brain already had the note.

Search, no model: every note's title, frontmatter keywords and body are scored against the query words (idf weighted;
title x3, keywords x2, body x1), plain-word stems ("layouts" = "layout"). The brain is found by brainpath.py and only read.
No brain on this machine: prints nothing, exit 0 (the toolkit runs without it).

brainnotes.py "query text" [-k 3] [--brain DIR] [--json] top notes: score, path under the brain, title
brainnotes.py --eval gold.tsv [-k 3] [--brain DIR] hit@k against a labelled set (id, slugs, why; slugs
comma-separated paths without .md, or NONE); the queries
come from a second file column 'text' or --queries FILE
"""
import argparse, json, math, os, re, sys

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
STOP = set("a an the and or of to in on for with is are be by it its at as from this that what which how should we our "
"not no do does after before into out than then when would must can there their them one two new old".split())


def stem(w):
for suf in ("ies", "es", "s"):
if len(w) > 4 and w.endswith(suf):
return w[: -len(suf)] + ("y" if suf == "ies" else "")
return w


def words(t):
# Atlas_Core is "Atlas Core" in a title; a version digit ("Atlas 4") is a word of its own
return [stem(w) for w in re.findall(r"[a-z0-9]+", (t or "").lower()) if w not in STOP and (len(w) > 1 or w.isdigit())]


def load(brain):
notes = []
for root, _, files in os.walk(os.path.join(brain, "internal")):
for f in files:
if not f.endswith(".md") or f in ("INDEX.md", "INTERNAL-ONLY.md"):
continue
p = os.path.join(root, f)
t = open(p, encoding="utf-8", errors="replace").read()
fm, body = "", t
if t.startswith("---"):
j = t.find("\n---", 3)
fm, body = t[3:j], t[j + 4:]
title = (re.search(r"^title:\s*(.+)$", fm, re.M) or [None, ""])[1].strip("'\" ")
kw = (re.search(r"^keywords:\s*(.+)$", fm, re.M) or [None, ""])[1]
notes.append({"path": os.path.relpath(p, os.path.join(brain, "internal"))[:-3], "title": title,
"f": (set(words(title)), set(words(kw)), words(body))})
return notes


def search(notes, q, k=3):
qs = set(words(q))
n = len(notes) or 1
df = {w: sum(1 for x in notes if w in x["f"][0] or w in x["f"][1] or w in x["f"][2]) for w in qs}
out = []
for x in notes:
t, kw, body = x["f"]
bset = set(body)
s = sum(math.log(1 + n / (1 + df[w])) * (3 * (w in t) + 2 * (w in kw) + (w in bset)) for w in qs)
if s > 0:
out.append((round(s, 1), x["path"], x["title"]))
return sorted(out, key=lambda r: -r[0])[:k]


def brain_dir(arg):
if arg:
return arg
try:
import brainpath
return brainpath.find()[0]
except Exception:
return None


def main():
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
ap.add_argument("query", nargs="?")
ap.add_argument("-k", type=int, default=3)
ap.add_argument("--brain"); ap.add_argument("--json", action="store_true")
ap.add_argument("--eval"); ap.add_argument("--queries")
a = ap.parse_args()
b = brain_dir(a.brain)
if not b or not os.path.isdir(os.path.join(b, "internal")):
return 0
notes = load(b)
if not a.eval:
r = search(notes, a.query or "", a.k)
print(json.dumps([{"score": s, "path": p, "title": t} for s, p, t in r]) if a.json
else "\n".join(f"{s}\t{p}\t{t}" for s, p, t in r))
return 0
q = {}
if a.queries:
for ln in open(a.queries):
if "|" in ln:
i, t = ln.split("|", 1)
q[i.strip()] = t.strip()
hit = tot = none_ok = nones = 0
for ln in open(a.eval):
f = ln.rstrip("\n").split("\t")
if len(f) < 2 or f[0] in ("id", "") or f[0] not in q:
continue
gold = {g.strip() for g in f[1].split(",") if g.strip() and g.strip() != "NONE"}
got = search(notes, q[f[0]], a.k)
paths = [p for _, p, _ in got]
if not gold:
nones += 1
print(f"{f[0]}\tNONE\t(top: {paths[:1]})")
continue
tot += 1
ok = bool(gold & set(paths))
hit += ok
print(f"{f[0]}\t{'HIT ' if ok else 'MISS'}\tgold {sorted(gold)}\tgot {paths}")
print(f"brainnotes hit@{a.k}: {hit}/{tot} situations with a gold note ({nones} with none in the brain)")
return 0 if tot and hit == tot else 1


if __name__ == "__main__":
sys.exit(main())
36 changes: 36 additions & 0 deletions upgrade-toolkit/bin/checkwhere.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""checkwhere.py: add the document of each error to an `mx check` log, from the JSON that `mx check -j` writes.

checkwhere.py CHECK.json CHECK.log

The text output of mx check names only the element ("at Action button 'actionButton1'"), and a widget name like that
occurs on many pages (cold CMM: 4 CE0117 on actionButton1/2/4/7, which sat in two Mix_New_WizardStep1 pages, not in the
login templates that also have such buttons). The JSON has the module, document and property. Appends one line per
error location to the log:

[where] CE0117 MaterialManagement / Page 'Mix_New_WizardStep1': Property 'Dynamic classes expression' of action button 'actionButton1'

The [error] lines and the summary stay as they are, so checkdiff.py and the error count read the log as before.
Exit: 0 = lines added (or none to add), 2 = the JSON is unreadable (the log is left alone).
"""
import json, sys


def where(d):
for e in d.get("errors") or []:
for loc in e.get("locations") or [{}]:
yield (f"[where] {e.get('code', '?')} {loc.get('module-name', '?')} / {loc.get('document-name', '?')}: "
f"{loc.get('element-name', '?')}")


if __name__ == "__main__":
if len(sys.argv) != 3:
sys.exit(__doc__)
try:
d = json.load(open(sys.argv[1]))
except (OSError, ValueError) as e:
print(f"checkwhere: {sys.argv[1]}: {e}", file=sys.stderr); sys.exit(2)
lines = list(where(d))
if lines:
with open(sys.argv[2], "a") as f:
f.write("Where (mx check -j):\n" + "\n".join(lines) + "\n")
Loading
Loading