Repository navigation
Conversation
This was referenced Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The local AlphaEvolve entrypoints currently have synthetic unit coverage, while validation against Google's actual examples is manual. Add a dedicated PR workflow that exercises the original circle-packing, TSP and signal-processing entrypoints against a single pinned upstream revision, without requiring model credentials or the Google SDK.
Each example runs seed-only and with one fixed comment-only proposal through the public CLI and real OE workers. The tests verify expected finite metrics, unchanged original/staged source hashes, cleaned import aliases and original report artifacts. Generation cases require an actual OE evolution trace containing an evaluated marked child, not merely a provider call or a successful exit. The TSP report must contain all five PNGs and its saved program; signal processing must save its evaluator archive in the run directory.
Run each case in a fresh subprocess with a small environment allowlist and deliberate dummy credential traps. Test-only startup tripwires block and immediately record socket/DNS and unscoped SDK/auth imports, including attempts caught by upstream code or made in workers that do not run
atexit. Two negative tests verify that caught attempts still fail offline validation. These are safeguards for trusted test inputs, not a security sandbox. Explicit suite execution fails on a missing checkout path, an unpinned/dirty example checkout or a path outside the OE parent workspace. Temporary outputs are cleaned up, and failures include captured diagnostics.The workflow uses a standard Ubuntu runner, Python 3.12, read-only permissions, a 15-minute timeout, cancellation of superseded runs and relevant PR path filters. It also supports manual dispatch after merging. Checkout credentials are not persisted, no account Secrets are used, and no artifacts or caches are uploaded. Scientific dependencies are pinned only for this opt-in job. The existing unit and real-model integration jobs, production code and base dependencies are unchanged.
Validation:
googlenoralpha_evolveinstalled. All six actual-example cases reported zero network/cloud attempts; each generation case recorded one evaluated proposal.Continues #468 without closing it. The upstream revision is
34cd919193945c6dc81053f9af594031b54f41a8; changing its pin requires reviewing the examples and expected metrics. Fixed proposals validate compatibility, not real-model optimization or the managed AlphaEvolve service. Local validation ran on macOS; the first GitHub-hosted Ubuntu result will come from this PR's new workflow. Complete round-trip run coverage and actual managed-service validation remain follow-ups.