Skip to content

fix(project): bundle the files a project's .meta.json records - #1566

Merged
HuiJun merged 3 commits into
developfrom
fix/bundle-recorded-members
Oct 27, 2026
Merged

HuiJun merged 3 commits into
developfrom
fix/bundle-recorded-members

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What and why

internal/workspace/project.Bundle archived every non-hidden regular file under the directory, so sysml -bundle on the Open-MBEE/OpenSysML-Extensions-Library tree gave 36 members (README, CONTRIBUTING, scripts/**, validation/, engine-contract.json, even scripts/__pycache__/*.pyc) where the project's own scripts/build-kpar.py gives 19, although the two manifests were byte-identical.

The rule now: a directory with a .meta.json is archived as the two manifests plus exactly the files the metadata records — every checksum key and every index target, at their recorded paths. A recorded file missing from the tree is an error naming it (.meta.json records LICENSE, which is not in the directory); a recorded path that escapes the directory is refused (.meta.json records "../README.md": …); a recorded entry that is not a regular file is refused. The kept .meta.json is still verified on the read-back, so a stale checksum refuses the bundle as before. A directory without a .meta.json is archived as KerML 1.1 §10.3 describes a project archive — "a model interchange file for each of the root namespaces in the project" and the two manifests — so its members are .project.json and the model files (.sysml, .kerml) outside hidden entries, library directories and nested projects, with the .meta.json generated from them; a README, __pycache__, editor backups (X.sysml~), .git/ and build scripts never travel. A project that wants its LICENSE or anything else in the archive records it in a .meta.json, as the Extensions Library does.

Consequences stated plainly: an archive another tool wrote round-trips to the files its .meta.json records. sysand's mbse4u-sysmod_5.1.4.kpar carried a README and a license its metadata does not record, so unbundling and re-bundling it gives 5 members, not 7; the round-trip test now says so. The Bundle doc comment and docs/guide/03-command-line.md document the rule.

Acceptance: sysml -bundle against build-kpar.py and the published release

Clone of OpenSysML-Extensions-Library at b36b361f85af4764de8e2817cbae62fb9e74ffdb with scripts/__pycache__/x.pyc added:

$ sysml -bundle /tmp/accept -o /tmp/accept-ours.kpar
wrote /tmp/accept-ours.kpar (OpenSysML Extensions Library 1.0.0, 19 files)
$ python3 /tmp/accept/scripts/build-kpar.py --output /tmp/accept-theirs.kpar
Wrote /tmp/accept-theirs.kpar (OpenSysML Extensions Library 1.0.0, 19 entries)
$ curl -sfLO https://github.com/Open-MBEE/OpenSysML-Extensions-Library/releases/download/v1.0.0/opensysml-extensions-library-1.0.0.kpar

unzip -l of all three, sizes and names (the develop binary gives 36 members on the same tree):

3691 .meta.json
1343 .project.json
11357 LICENSE
3699 libraries/AnalysisRecords.sysml
4909 libraries/DiagramLayout.sysml
15931 libraries/DocumentQueries.sysml
680 libraries/IdentityMetadata.sysml
21622 libraries/MOSA.sysml
2765 libraries/MigrationMetadata.sysml
20884 libraries/OOSEM.sysml
1911 libraries/OpenSysMLMathFunctions.kerml
454 libraries/OpenSysMLRenderings.sysml
1972 libraries/RandomFunctions.kerml
3613 libraries/Simulation.sysml
1681 libraries/StateActivity.kerml
2572 libraries/StateMachines.sysml
2458 libraries/StateSpaceIntegration.sysml
1556 libraries/Stochastic.sysml
14660 libraries/SysMLValidation.sysml

diff of the three listings: empty. cmp of each of the 19 members extracted from ours against build-kpar.py's and against the published opensysml-extensions-library-1.0.0.kpar: every member identical. The published archive is byte-identical to build-kpar.py's (SHA-256 1eb10ec8…); ours differs from both only in the ZIP container headers (Go's writer records a Unix mode and extended-timestamp field where Python's ZipInfo(external_attr=0, create_system=0) does not), not in any member.

Specification basis

KerML 1.1 Beta 2 §10.3 (Model Interchange Projects): "The archive shall contain a model interchange file for each of the root namespaces in the project … In addition, the archive shall contain, at its top level, exactly one file named .project.json and exactly one file named .meta.json." No row in docs/project/spec-compliance.md moves.

How it was verified

  • go test ./internal/workspace/project — TestBundleArchivesTheFilesTheMetadataRecords (recorded subset, missing recorded file named, ../ path refused), the metadata-generation test now expecting the model files alone (with an editor backup in the tree), TestBundleLeavesNestedProjectsOut, TestBundleLeavesTheLibraryDirectoryOut.
  • go test ./tests/kpar — TestBundleIsDeterministicAndUnbundleReproducesTheRecordedFiles (renamed; compareRecorded compares the recorded set, the sysmod round trip expects 5 members) and the new tests/kpar/bundle_test.go: a repository-shaped tree built at test time (two units, LICENSE, README, CONTRIBUTING, scripts/build.py with __pycache__, Widgets.sysml~, .git/HEAD, .github/) whose .meta.json records three files bundles to exactly those plus the manifests, byte for byte; LICENSE removed → cannot bundle …: .meta.json records LICENSE, which is not in the directory; .meta.json removed → the project file and the two model files, index generated.
  • The acceptance run above.
  • Full gate on this branch: go build ./... ok, go vet ./... ok, gofmt -l . empty, make lint (staticcheck + gosec, both modules) ✓, go test ./... with OPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 OPENSYSML_REQUIRE_SPEC_EXAMPLES=1 after the three download scripts — all packages ok, python3 scripts/changelog.py check ok, python3 scripts/check-doc-links.py → 0 broken links, python -m mkdocs build --strict ok, make extension-libraries-check → in sync.
  • Dry-merges: clean against origin/develop, against chore(extensions): re-pin to the project-interchange upstream and prove the vendored copy is the published project #1563 (the sibling), and textually clean against chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529's head for every file this PR touches — the bundle section of tests/kpar/kpar_test.go does not overlap chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529's OMG tier-map edits; chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529's own conflicts with develop are in docs/project/* files this PR does not change.
  • Devin Review: one finding (a recorded path through a directory link leading outside the project was archived), fixed by confining every recorded member with confinedTo as Unbundle does, thread resolved; the current head has no open finding of any severity, flags included.

Checklist

  • make test and make lint pass locally
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (see CONTRIBUTING.md)
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved (compliance rows need nothing: the census is counted at docs build)
  • No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/1e9164f57f814014a9f0aba35666e2a3
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/1e9164f57f814014a9f0aba35666e2a3?variant=devin
Requested by: @HuiJun

Bundle archived every non-hidden regular file under the directory, so the
Extensions Library tree gave 36 members where its own build-kpar.py gives 19.
A directory with a .meta.json is now archived as the two manifests plus
exactly the files the metadata records (checksum keys and index targets), a
recorded file missing from the tree an error naming it; one without is
archived as KerML 10.3 describes, the project file and the model files, with
the .meta.json generated from them. The round-trip test reproduces the
recorded files, and a new one bundles a repository-shaped tree whose
metadata records a subset of it.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 2 commits October 27, 2026 01:22
A recorded path reached through a linked directory leading outside the
project is refused, as Unbundle refuses to write through one.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 27, 2026 02:17
@HuiJun
HuiJun merged commit 0630c3d into develop Oct 27, 2026
53 checks passed
@HuiJun
HuiJun deleted the fix/bundle-recorded-members branch October 27, 2026 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant