Repository navigation
Conversation
…truct info
Replaces the generated required, enum and strings walkers with one walker driven by the
cached per-type information of libs/structs/structinfo (field offsets, kinds, required
fields, enum values, and for each rule the fields that lead to something to visit).
The generated walkers were about 29k lines and 5 MB of binary.
Also flags the maps and slices a file wrote as empty ({} or []) on the Locations tree,
so that the typed configuration can tell them from unset ones.
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Commit: Regression test reportSelected 20 changed tests (limit=20, 1 not selected) 42 tests: regression=22 unreleased=2 coverage=18 failing=0
TestAccept/bundle/python/pipeline-development-deprecated/DMS=/PYDAB_VERSION=0.266.0: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): same output as main (3aa5c96) TestAccept/bundle/python/pipeline-development-deprecated/DMS=/PYDAB_VERSION=current: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): same output as main (3aa5c96) TestAccept/bundle/python/pipeline-development-deprecated/DMS=true/PYDAB_VERSION=0.266.0: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): same output as main (3aa5c96) TestAccept/bundle/python/pipeline-development-deprecated/DMS=true/PYDAB_VERSION=current: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): same output as main (3aa5c96) TestAccept/bundle/refschema/DMS=: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): TestAccept/bundle/refschema/DMS=true: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): TestAccept/bundle/strict_profile_match/default_profile/host/DMS=: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): same output as main (3aa5c96) TestAccept/bundle/strict_profile_match/default_profile/host/DMS=true: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): same output as main (3aa5c96) TestAccept/bundle/strict_profile_match/host/DMS=: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): same output as main (3aa5c96) TestAccept/bundle/strict_profile_match/host/DMS=true: branch ✅ | main (3aa5c96) ❌ | latest (v1.24.0) ❌main (3aa5c96): latest (v1.24.0): 14 more failure details omitted, see the full report in the run. |
Collaborator
Integration test reportCommit: d2ee1fd
Top 6 slowest tests (at least 2 minutes):
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Replaces the generated required, enum and strings walkers (about 29k lines, 5 MB of binary) with one walker driven by per-type information that
libs/structs/structinfocaches: field offsets, kinds, required fields, enum values, and for each rule the fields that lead to something to visit. The defaults walker and the resource iterators stay generated.Also flags the maps and slices a file wrote as empty (
{},[]) on theLocationstree. A typed value cannot tell them from unset ones.Why
The generated walkers are most of what #7098 adds to the binary (+5 MB) and to the code (+30k generated lines). With this change the binary is +0.6 MB over main, and
bundle validateon 1000 jobs is unchanged (0.48 s against 0.51 s).Per walk on 1000 jobs: required is faster (0.85 ms against 1.25 ms), enum and strings are 1.3-1.5x slower (0.6 ms against 0.4 ms, 2.9 ms against 2.2 ms) with a quarter of the allocations. The possible optimizations are noted in
bundle/config/walk/rule_walk.go.Stacked on #7098.
Tests
Unit tests, the existing acceptance tests (
TestAccept/bundlepasses), and newTestIsEmptyandTestRootEmptiesfor the empty flag.BenchmarkRequired,BenchmarkEnumandBenchmarkStrings*inbundle/config/walk.This pull request and its description were written by Isaac.