test(compat): exercise pinned AlphaEvolve examples in CI

.github/workflows/alphaevolve-interop.yml

@@ -0,0 +1,65 @@
+name: AlphaEvolve upstream examples
+
+on:
+  pull_request:
+    branches: [main]
+    paths:
+      - '.github/workflows/alphaevolve-interop.yml'
+      - 'openevolve/**'
+      - 'examples/alphaevolve_interop/**'
+      - 'tests/alphaevolve_upstream/**'
+      - 'pyproject.toml'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: alphaevolve-interop-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  upstream-examples:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Checkout OpenEvolve
+        uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Read pinned upstream revision
+        run: |
+          python - <<'PY'
+          import os
+          import re
+          from pathlib import Path
+          revision = Path('tests/alphaevolve_upstream/revision.txt').read_text().strip()
+          if not re.fullmatch(r'[0-9a-f]{40}', revision):
+              raise SystemExit('Expected a full upstream commit SHA')
+          with open(os.environ['GITHUB_ENV'], 'a') as environment:
+              environment.write(f'ALPHAEVOLVE_REVISION={revision}\n')
+          PY
+
+      - name: Checkout official examples (fixed revision)
+        uses: actions/checkout@v4
+        with:
+          repository: Google-Cloud-AI/alphaevolve-on-googlecloud
+          ref: ${{ env.ALPHAEVOLVE_REVISION }}
+          path: .alphaevolve-upstream
+          persist-credentials: false
+
+      - name: Install local test dependencies (no Google SDK)
+        run: |
+          python -m pip install -e .
+          python -m pip install -r tests/alphaevolve_upstream/requirements.txt
+
+      - name: Run six original-entrypoint cases and offline tripwire tests
+        env:
+          ALPHAEVOLVE_UPSTREAM_DIR: ${{ github.workspace }}/.alphaevolve-upstream
+        run: python tests/alphaevolve_upstream/test_examples.py -v

examples/alphaevolve_interop/README.md

@@ -326,6 +326,61 @@ full SDK replacement, managed-service validation or completion of the issue.
 python -m unittest discover -s tests -p 'test_alphaevolve_entrypoint.py' -v
 ```
 
+### Pinned upstream-example CI
+
+The separate **AlphaEvolve upstream examples** workflow runs on relevant pull
+requests and can be dispatched manually after it is on the default branch. It
+uses a standard Ubuntu runner with Python 3.12, read-only repository permissions,
+no Secrets, no Google SDK and no paid provider. It has a 15-minute job limit,
+cancels superseded runs on the same PR/ref, and uploads no artifacts or caches.
+Checkout and dependency installation need Internet access; the example subprocesses
+then run with network/SDK-import tripwires and a small allowlist of environment
+variables, containing only deliberate dummy credential traps.
+
+[`tests/alphaevolve_upstream/revision.txt`](../../tests/alphaevolve_upstream/revision.txt)
+is the single upstream commit pin. Changing Google's `main` does not silently
+change these tests. The suite requires clean example trees at that revision,
+with the checkout inside OE's parent workspace (a sibling or nested directory).
+Paths are canonicalized and escapes from that workspace are rejected. The
+scientific dependencies are test-only and pinned separately; the Matplotlib pin
+retains the reviewed upstream plotting API. OE's base dependencies are unchanged.
+
+There are six original-entrypoint cases: circle packing, TSP and signal
+processing, each seed-only and with one fixed comment-only proposal. They use
+the existing public runner, unmodified upstream evaluators and real OE workers.
+The evolution cases check OE's actual trace for an evaluated child containing
+the proposal marker, instead of treating a provider call or successful process
+exit alone as proof. The cases also check expected finite seed metrics, exact
+original/staged source hashes, alias cleanup and report artifacts. TSP must
+produce all five PNG reports and its saved program; signal processing must
+retain its evaluator archive under the run directory.
+
+Python audit hooks record blocked socket/DNS attempts immediately, including in
+workers that exit without running `atexit`. Unscoped `alpha_evolve` and Google
+imports are blocked and recorded too; the explicit local aliases still work.
+Two negative tests check that even caught network/import exceptions remain
+observable and fail offline validation. These are tripwires for the trusted
+examples, not a security sandbox. Fixed proposals establish compatibility, not
+real-model optimization quality or a managed AlphaEvolve service run.
+
+To reproduce the eight opt-in checks locally from an OE checkout:
+
+```bash
+git clone https://github.com/Google-Cloud-AI/alphaevolve-on-googlecloud.git ../alphaevolve-reference
+git -C ../alphaevolve-reference checkout "$(cat tests/alphaevolve_upstream/revision.txt)"
+python -m pip install -e . -r tests/alphaevolve_upstream/requirements.txt
+ALPHAEVOLVE_UPSTREAM_DIR=../alphaevolve-reference \
+  python tests/alphaevolve_upstream/test_examples.py -v
+```
+
+Run this command in a Python 3.12 environment matching the workflow. No model
+server or Google credentials are needed. A missing path, wrong revision, dirty
+upstream checkout or offline violation makes explicit execution fail. Temporary
+run directories are cleaned up; failures include captured subprocess diagnostics.
+This suite is separate from default unit discovery and the existing
+`tests/integration` real-model job; neither implicitly downloads the upstream
+repository or requires these optional scientific packages.
+
 ## Evaluate an upstream seed locally
 
 From an installed OpenEvolve checkout:

tests/alphaevolve_upstream/requirements.txt

@@ -0,0 +1,7 @@
+# Optional scientific dependencies for the Python 3.12 upstream-example job.
+# No Google SDK, cloud credentials or model server is needed.
+numpy==2.5.4
+scipy==1.18.1
+matplotlib==3.10.9
+nest-asyncio==1.6.0
+python-dotenv==1.2.4

tests/alphaevolve_upstream/revision.txt

@@ -0,0 +1 @@
+34cd919193945c6dc81053f9af594031b54f41a8

tests/alphaevolve_upstream/run_example.py

@@ -0,0 +1,80 @@
+"""Execute one original upstream main through the public CLI, guarded offline."""
+
+import argparse
+import contextlib
+import io
+import json
+import os
+import runpy
+import sys
+from pathlib import Path
+from unittest.mock import patch
+
+
+def main():
+    guard = sys.modules.get("sitecustomize")
+    if not getattr(guard, "ACTIVE", False):
+        raise RuntimeError("Run via test_examples.py with offline tripwires enabled")
+
+    from offline_model import fixed_model
+
+    from openevolve.compat.alphaevolve import config_from_alphaevolve
+
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("example", type=Path)
+    parser.add_argument("iterations", type=int, choices=(0, 1))
+    parser.add_argument("workspace", type=Path)
+    args = parser.parse_args()
+    runner = Path(__file__).resolve().parents[2] / "examples/alphaevolve_interop/run_entrypoint.py"
+    argv = [
+        str(runner),
+        str(args.example),
+        "--api-base",
+        "http://127.0.0.1:1/v1",
+        "--model",
+        "fixed-offline-model",
+        "--output-dir",
+        str(args.workspace / "run"),
+    ]
+    if args.iterations:
+        argv += ["--evolve", "--iterations", "1"]
+
+    def configure(*values, **options):
+        config = config_from_alphaevolve(*values, **options)
+        if config.llm.api_key != "local-placeholder":
+            raise AssertionError("Runner must not use the dummy ambient key")
+        config.database.num_islands = 1
+        config.log_level = "WARNING"
+        config.evolution_trace.enabled = True
+        config.evolution_trace.include_code = True
+        config.evolution_trace.include_prompts = False
+        config.evolution_trace.buffer_size = 1
+        config.evolution_trace.output_path = str(args.workspace / "evolution.jsonl")
+        # Seed-only runs construct the real provider client but make no requests.
+        # Evolution uses real OE workers with only proposal generation replaced.
+        if args.iterations:
+            for model in config.llm.models + config.llm.evaluator_models:
+                model.init_client = fixed_model
+        return config
+
+    stdout = io.StringIO()
+    with (
+        patch.object(sys, "argv", argv),
+        patch(
+            "openevolve.compat.alpha_evolve.experiment.config_from_alphaevolve",
+            side_effect=configure,
+        ),
+        contextlib.redirect_stdout(stdout),
+    ):
+        runpy.run_path(str(runner), run_name="__main__")
+    result = json.loads(stdout.getvalue())
+    if any(name == "alpha_evolve" or name.startswith("alpha_evolve.") for name in sys.modules):
+        raise AssertionError("SDK aliases leaked after main")
+    if any(name.startswith("_openevolve_ae_entrypoint_") for name in sys.modules):
+        raise AssertionError("Temporary upstream package leaked after main")
+    (args.workspace / "result.json").write_text(json.dumps(result), encoding="utf-8")
+    (args.workspace / "process.json").write_text(json.dumps({"pid": os.getpid()}), encoding="utf-8")
+
+
+if __name__ == "__main__":
+    main()

tests/alphaevolve_upstream/support/offline_model.py

@@ -0,0 +1,28 @@
+"""Importable fixed-proposal provider for real OE multiprocessing workers."""
+
+import json
+import os
+import sys
+from pathlib import Path
+
+MARKER = "# AlphaEvolve upstream CI: evaluated fixed proposal."
+
+
+class FixedProposalModel:
+    def __init__(self, config):
+        self.model = config.name
+
+    async def generate_with_context(self, system_message, messages, **kwargs):
+        guard = sys.modules.get("sitecustomize")
+        if not getattr(guard, "ACTIVE", False):
+            raise AssertionError("Offline tripwires must also be active in OE workers")
+        with Path(os.environ["OE_INTEROP_CALL_LOG"]).open("a", encoding="utf-8") as stream:
+            stream.write(json.dumps({"model": self.model, "pid": os.getpid()}) + "\n")
+        return (
+            "<<<<<<< SEARCH\n# EVOLVE-BLOCK-START\n=======\n"
+            f"# EVOLVE-BLOCK-START\n{MARKER}\n>>>>>>> REPLACE"
+        )
+
+
+def fixed_model(config):
+    return FixedProposalModel(config)

tests/alphaevolve_upstream/support/sitecustomize.py

@@ -0,0 +1,41 @@
+"""Block and durably record network/SDK attempts in dedicated test processes.
+
+Loaded at interpreter startup via the test driver's explicit PYTHONPATH. These
+are test tripwires for trusted examples, not a sandbox for untrusted code.
+"""
+
+import importlib.abc
+import json
+import os
+import sys
+from pathlib import Path
+
+ACTIVE = False
+
+
+def record(kind):
+    # Write attempts immediately: multiprocessing workers may exit via os._exit,
+    # and upstream code can catch the exception raised by a tripwire.
+    path = Path(os.environ["OE_INTEROP_AUDIT_DIR"]) / f"{os.getpid()}.jsonl"
+    with path.open("a", encoding="utf-8") as stream:
+        stream.write(json.dumps({"kind": kind, "pid": os.getpid()}) + "\n")
+
+
+def audit(event, args):
+    if event in {"socket.connect", "socket.getaddrinfo", "socket.gethostbyname"}:
+        record(event)
+        raise AssertionError("Upstream-example test blocked network access")
+
+
+class NoCloudImports(importlib.abc.MetaPathFinder):
+    def find_spec(self, fullname, path=None, target=None):
+        if fullname.split(".", 1)[0] in {"alpha_evolve", "google"}:
+            record("cloud_import")
+            raise AssertionError("Upstream-example test blocked a cloud SDK/auth import")
+
+
+if os.environ.get("OE_INTEROP_AUDIT_DIR"):
+    sys.addaudithook(audit)
+    sys.meta_path.insert(0, NoCloudImports())
+    record("active")
+    ACTIVE = True