Skip to content

Commit da5ec9a

Browse files
author
James Nightingale
authored
Merge pull request #491 from PyAutoLabs/docs/jax-pytree-registration-heuristic
docs(agents): pytree registration classification heuristic
2 parents 158db38 + 629a0aa commit da5ec9a

1 file changed

Lines changed: 39 additions & 0 deletions

File tree

‎docs/agents/jax_and_decorators.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,3 +228,42 @@ must include **both** a `jax.jit(analysis.fit_from)(instance)` round-trip
228228
`autoarray_workspace_test`; array-level JAX changes are exercised downstream in
229229
`autogalaxy_workspace_test/scripts/jax_likelihood_functions/` and the
230230
`autolens_workspace_test` equivalents.
231+
232+
---
233+
234+
## 6. Registering a new type — classification heuristic
235+
236+
When a jit trace fails on an unregistered type (a `jax.tree_util`
237+
`TypeError: unhashable type`, a `NotImplementedError`, or a tracer error whose
238+
frame above names the class), read the offending class's `__init__` /
239+
`__dict__` and classify its attributes before registering:
240+
241+
- **All attrs are `jax.Array` / `np.ndarray` / `AbstractNDArray` / primitives**
242+
→ register with `no_flatten=()` (everything dynamic).
243+
- **Known-aux patterns** — `cosmology`, `settings`, `config`, `dataset`, `psf`,
244+
`mask`, `_cache`, `_compiled`, any `scipy.spatial.*`, `Transformer*`,
245+
`PointSolver*` → register with those names in `no_flatten`; they are
246+
per-analysis constants, not traced leaves.
247+
- **A callable attribute** → do not guess. Callable state is the case that
248+
silently breaks gradients; decide aux / dynamic / split deliberately.
249+
250+
Register at the type's wiring site (e.g. `_register_fit_imaging_pytrees` in
251+
PyAutoLens) with a one-line comment naming the variant that needed it, then
252+
re-trace. Registration is iterative — each pass usually surfaces the next
253+
unregistered type one frame further in — but if a handful of passes produce no
254+
progress, the type is telling you it holds state that cannot be flattened;
255+
stop and treat it as a design question rather than adding more registrations.
256+
257+
The parity scripts under
258+
`autolens_workspace_test/scripts/jax_likelihood_functions/` are where this is
259+
exercised; `<variant>_pytree.py` asserts the round trip against the NumPy
260+
reference:
261+
262+
```python
263+
ref = analysis.fit_from(instance).log_likelihood # NumPy reference
264+
jit_ll = jax.jit(lambda i: analysis.fit_from(i).log_likelihood)(instance) # jit round-trip
265+
assert jnp.allclose(ref, jit_ll, rtol=1e-4), f"divergence: {ref} vs {jit_ll}"
266+
```
267+
268+
That round trip proves the types flatten; it does **not** prove `xp` is threaded
269+
— pair it with the `fitness._vmap(parameters)` check in §5.

0 commit comments

Comments
 (0)