feat(compat): adapt local AlphaEvolve evaluators for OpenEvolve

README.md

@@ -492,6 +492,9 @@ Each call speaks newline-delimited JSON-RPC to a short-lived `codex app-server`
 
 ## Examples Gallery
 
+To use unchanged local evaluators from Google's AlphaEvolve examples, see the
+[AlphaEvolve evaluator interoperability walkthrough](examples/alphaevolve_interop/).
+
 <div align="center">
 
 ### **Showcase Projects**

examples/alphaevolve_interop/README.md

@@ -0,0 +1,96 @@
+# AlphaEvolve local evaluator interoperability
+
+This is the evaluator-contract slice of [#468](https://github.com/algorithmicsuperintelligence/openevolve/issues/468).
+It runs the **unchanged** `program.py` and `evaluate.py` from Google's
+[AlphaEvolve examples](https://github.com/Google-Cloud-AI/alphaevolve-on-googlecloud)
+through an explicit adapter. It does not replace the cloud SDK, translate cloud
+configuration, export bundles, or implement the rest of #468.
+
+## Evaluate an upstream seed locally
+
+From an installed OpenEvolve checkout:
+
+```bash
+git clone https://github.com/Google-Cloud-AI/alphaevolve-on-googlecloud.git ../alphaevolve-reference
+
+# Evaluator-only dependencies; no Google client or authentication is used.
+python -m pip install numpy scipy matplotlib pydantic
+export PYTHONPATH="$(cd ../alphaevolve-reference/src && pwd)${PYTHONPATH:+:$PYTHONPATH}"
+export MPLBACKEND=Agg
+
+python examples/alphaevolve_interop/run.py ../alphaevolve-reference/examples/circle_packing
+python examples/alphaevolve_interop/run.py ../alphaevolve-reference/examples/tsp
+python examples/alphaevolve_interop/run.py ../alphaevolve-reference/examples/signal_processing
+```
+
+Without `--config`, the runner only evaluates the seed and prints metrics and
+artifacts. It makes **no LLM or Google Cloud requests**. The `alpha_evolve.models`
+import used by circle packing resolves to the upstream source, not a fake SDK.
+Upstream example dependencies can change; inspect their requirements if an import
+fails. The signal-processing evaluator writes its normal `programs/` artifacts
+under the current directory, so run it in a disposable working directory if needed.
+
+To generate and evaluate new candidates, provide **your own OpenEvolve config**:
+
+```bash
+python examples/alphaevolve_interop/run.py ../alphaevolve-reference/examples/circle_packing \
+  --config /path/to/your-openevolve-config.yaml --iterations 1 --output-dir ./interop-output
+```
+
+This second command invokes the model/provider you configured and may incur API
+charges. It does not use or translate the Google example's `.env`, cloud client,
+`run_evolution.py`, or Google credentials. Local model endpoints work as with any
+other OpenEvolve run. Remote Cloud Run/GKE evaluators are not supported here.
+
+## Use the adapter directly
+
+```python
+from openevolve.compat.alphaevolve import adapt_evaluator
+
+# imported_evaluator is the original synchronous AlphaEvolve evaluator callable.
+evaluate = adapt_evaluator(imported_evaluator, objective_metric="sum_of_radii")
+```
+
+Use `evaluate` with `run_evolution(..., evaluator=evaluate)`, or expose it in an
+evaluator module for the CLI. The helper:
+
+- Reads the candidate's source file as UTF-8 and constructs
+  `{"content": {"files": [{"path": filename, "content": source}]}}`.
+- Accepts the nested `scores.scores` / `insights.insights` envelope as dictionaries
+  or SDK-style objects, without importing Google's package in OpenEvolve.
+- Keeps all metrics for features and maps the explicitly selected objective to
+  `combined_score`, avoiding accidental averaging of unrelated metrics.
+- Stores insights as JSON in the `alphaevolve_insights` artifact.
+- Marks exact `-1e12` sentinel, null, and non-finite scores as failures, with
+  `error: 0.0`, an error artifact, and a `combined_score` of `-1e12`. This avoids
+  giving a failed candidate zero fitness when valid objectives are negative.
+
+The sentinel is an example convention, not a universal SDK rule. Set
+`failure_sentinel=None` to disable it or choose a different sentinel. Choose a
+finite `failure_score` **below every valid score** for your problem. A failed
+secondary metric also marks the whole evaluation failed. Duplicate metrics,
+missing objectives and malformed scores raise actionable errors. The names
+`error` and `combined_score` are reserved by this mapping; select
+`objective_metric="combined_score"` if that is already your original metric.
+
+Exceptions propagate to OpenEvolve's existing retry/error handling. That generic
+handling may return zero fitness on exceptions or timeouts; this adapter does not
+change that behavior. For negative objectives, inspect error/timeout results
+before interpreting the best program.
+
+The adapter provides **no sandbox**. Like the original local evaluators, it runs
+trusted Python code with the current process's privileges. Use a separate
+execution environment for untrusted candidates.
+
+## Validation
+
+Default unit tests use synthetic envelopes and OpenEvolve's real evaluator and
+callable-serialization path, without Google's SDK, keys, or network access:
+
+```bash
+python -m unittest discover -s tests -p test_alphaevolve_compat.py
+```
+
+The walkthrough additionally permits direct validation against an upstream
+checkout. An evaluator-only run demonstrates contract compatibility, not learned
+improvement or a real-model evolution experiment.

examples/alphaevolve_interop/run.py

@@ -0,0 +1,68 @@
+"""Evaluate an unchanged AlphaEvolve example; optionally evolve it with an OE config."""
+
+import argparse
+import importlib.util
+import json
+from pathlib import Path
+
+from openevolve import run_evolution
+from openevolve.compat.alphaevolve import adapt_evaluator
+
+EXAMPLES = {
+    "circle_packing": ("circle_packing_evaluation", "sum_of_radii"),
+    "tsp": ("tsp_evaluation_function", "neg_tour_length"),
+    "signal_processing": ("signal_processing_evaluation", "overall_score"),
+}
+
+
+def load_evaluator(example_dir):
+    """Load the example without importing its cloud client or experiment entrypoint."""
+    function_name, objective = EXAMPLES[example_dir.name]
+    spec = importlib.util.spec_from_file_location(
+        "_alphaevolve_example", example_dir / "src" / "evaluate.py"
+    )
+    if spec is None or spec.loader is None:
+        raise ImportError(f"Cannot load evaluator from {example_dir}")
+    module = importlib.util.module_from_spec(spec)
+    # Deliberately not registered in sys.modules: cloudpickle transports this
+    # file-loaded callable by value to OpenEvolve's worker processes.
+    spec.loader.exec_module(module)
+    return adapt_evaluator(getattr(module, function_name), objective_metric=objective)
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("example_dir", type=Path, help="Path to the upstream example directory")
+    parser.add_argument(
+        "--config", type=Path, help="OE YAML config; omit to evaluate only (no LLM)"
+    )
+    parser.add_argument(
+        "--iterations", type=int, default=1, help="Evolution iterations with --config"
+    )
+    parser.add_argument("--output-dir", type=Path, default=Path("alphaevolve_output"))
+    args = parser.parse_args()
+    example_dir = args.example_dir.resolve()
+    if example_dir.name not in EXAMPLES:
+        parser.error(f"Supported example names: {', '.join(EXAMPLES)}")
+    if args.iterations < 1:
+        parser.error("--iterations must be positive")
+    program_path = example_dir / "src" / "program.py"
+    evaluator = load_evaluator(example_dir)
+    if args.config is None:
+        result = evaluator(program_path)
+        print(json.dumps({"metrics": result.metrics, "artifacts": result.artifacts}, indent=2))
+        return
+
+    result = run_evolution(
+        initial_program=program_path,
+        evaluator=evaluator,
+        config=args.config,
+        iterations=args.iterations,
+        output_dir=str(args.output_dir),
+        cleanup=True,
+    )
+    print(json.dumps({"metrics": result.metrics, "best_score": result.best_score}, indent=2))
+
+
+if __name__ == "__main__":
+    main()

openevolve/compat/__init__.py

@@ -0,0 +1 @@
+"""Opt-in interoperability helpers for other evolution frameworks."""

openevolve/compat/alphaevolve.py

@@ -0,0 +1,128 @@
+"""Adapt local Google Cloud AlphaEvolve evaluators without a Google SDK dependency."""
+
+import json
+import math
+from collections.abc import Mapping
+from dataclasses import dataclass
+from numbers import Real
+from pathlib import Path
+from typing import Any, Callable, Optional
+
+from openevolve.evaluation_result import EvaluationResult
+
+
+def _field(value: Any, name: str) -> Any:
+    """Read the same contract from model objects and their model_dump() dictionaries."""
+    if isinstance(value, Mapping):
+        return value.get(name)
+    return getattr(value, name, None)
+
+
+@dataclass
+class _AlphaEvolveEvaluator:
+    evaluator: Callable
+    objective_metric: str
+    failure_sentinel: Optional[float]
+    failure_score: float
+
+    def __call__(self, program_path: str) -> EvaluationResult:
+        path = Path(program_path)
+        candidate = {
+            "content": {"files": [{"path": path.name, "content": path.read_text(encoding="utf-8")}]}
+        }
+        result = self.evaluator(candidate)
+        scores = _field(_field(result, "scores"), "scores")
+        if not isinstance(scores, (list, tuple)) or not scores:
+            raise ValueError("AlphaEvolve evaluation must contain a non-empty scores.scores list")
+
+        metrics = {}
+        failed_metrics = []
+        for entry in scores:
+            metric, score = _field(entry, "metric"), _field(entry, "score")
+            if not isinstance(metric, str) or not metric.strip():
+                raise ValueError("AlphaEvolve metric names must be non-empty strings")
+            if metric in metrics or metric == "error":
+                raise ValueError(f"Duplicate or reserved AlphaEvolve metric: {metric!r}")
+            if metric == "combined_score" and self.objective_metric != "combined_score":
+                raise ValueError(
+                    "Select combined_score as objective_metric to preserve that metric"
+                )
+            if score is not None and (isinstance(score, bool) or not isinstance(score, Real)):
+                raise TypeError(f"AlphaEvolve score for {metric!r} must be a number or None")
+            if (
+                score is None
+                or not math.isfinite(score)
+                or (self.failure_sentinel is not None and score == self.failure_sentinel)
+            ):
+                failed_metrics.append(metric)
+                metrics[metric] = self.failure_score
+            else:
+                metrics[metric] = float(score)
+
+        if self.objective_metric not in metrics:
+            raise ValueError(
+                f"AlphaEvolve evaluation is missing objective {self.objective_metric!r}"
+            )
+
+        artifacts = {}
+        insights = _field(_field(result, "insights"), "insights")
+        if insights:
+            artifacts["alphaevolve_insights"] = json.dumps(
+                [
+                    {"label": _field(item, "label"), "text": _field(item, "text")}
+                    for item in insights
+                ],
+                ensure_ascii=False,
+            )
+
+        # Pick an explicit objective, rather than averaging unrelated feature metrics.
+        metrics["combined_score"] = metrics[self.objective_metric]
+        if failed_metrics:
+            # OpenEvolve error metrics alone have fitness zero, which could beat a
+            # valid negative objective (e.g. neg_tour_length). Retain a failure fitness.
+            metrics["combined_score"] = self.failure_score
+            metrics["error"] = 0.0
+            artifacts["error"] = "AlphaEvolve evaluation failed for metrics: " + ", ".join(
+                failed_metrics
+            )
+        return EvaluationResult(metrics=metrics, artifacts=artifacts)
+
+
+def adapt_evaluator(
+    evaluator: Callable,
+    *,
+    objective_metric: str,
+    failure_sentinel: Optional[float] = -1e12,
+    failure_score: float = -1e12,
+) -> Callable[[str], EvaluationResult]:
+    """Wrap a synchronous, single-file AlphaEvolve evaluator for OpenEvolve.
+
+    The callable receives ``content.files[0].content`` instead of a file path.
+    Its ``scores.scores`` envelope may be a dictionary or SDK-style object.
+    All metrics are retained; ``objective_metric`` is also used as combined_score.
+    Insights become JSON in the ``alphaevolve_insights`` artifact.
+
+    None/non-finite scores and the exact configurable failure sentinel mark the
+    evaluation failed. Choose a finite ``failure_score`` below all valid scores,
+    especially for negative objectives. Set ``failure_sentinel=None`` if -1e12
+    is a legitimate score. Exceptions and malformed contracts propagate to the
+    existing OpenEvolve evaluator error handling.
+
+    This only adapts evaluation: it does not provide cloud SDK emulation, config
+    translation, multi-file candidates, or execution isolation. The wrapped
+    evaluator executes trusted candidate code with its existing privileges.
+    """
+    if not callable(evaluator):
+        raise TypeError("evaluator must be callable")
+    if not isinstance(objective_metric, str) or not objective_metric.strip():
+        raise ValueError("objective_metric must be a non-empty string")
+    if objective_metric == "error":
+        raise ValueError("error is reserved for failure handling")
+    for name, value in (("failure_score", failure_score), ("failure_sentinel", failure_sentinel)):
+        if name == "failure_sentinel" and value is None:
+            continue
+        if isinstance(value, bool) or not isinstance(value, Real) or not math.isfinite(value):
+            raise ValueError(f"{name} must be finite")
+    return _AlphaEvolveEvaluator(
+        evaluator, objective_metric, failure_sentinel, float(failure_score)
+    )