prompt: record sparse-operator item 2 resolved — no incompatibility - #268
Merged
Conversation
Closes the second and last open item on the pynufft removal record. There is no incompatibility between apply_sparse_operator and TransformerNUFFT: the NotImplementedError guard was removed for nufftax by bd18a769 in May and only the workspace comment stayed stale. Verified directly — it runs at every scale tried and matches TransformerDFT to ~3e-13. The useful finding is which transformer to choose, which was undocumented and is not governed by visibility count. The DFT costs O(N_vis * N_pix) against the NUFFT's O((N_vis + N_pix) log N) plus ~2s fixed overhead, so the product decides: crossover near 1e7, which is ~5,000 visibilities at a 64x64 mask but still favours the DFT at 4,000 on a 32x32 one. Memory is the axis that actually matters. The DFT allocates with the same product and reaches ~109 GB at 1M visibilities, independently corroborating the ~123 GB bd18a769 recorded by a different route, while the NUFFT allocates nothing measurable. Past ~1e8 it is the only feasible path. Both open items on this record are now closed. The pynufft removal is complete: dependency gone from six repos, the dead use_adjoint_scaling parameter and adjoint_scaling attribute removed, and the sparse-operator question settled by measurement rather than left as a retracted claim.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the second and last open item on the pynufft removal record (
draft/maintenance/libraries/remove_pynufft_legacy_transformer.md).There is no incompatibility between
apply_sparse_operatorandTransformerNUFFT. TheNotImplementedErrorguard was removed for nufftax by PyAutoLabs/PyAutoArray#329 (bd18a769, 2026-05-22); only the workspace comment stayed stale, for three months. Verified directly: it runs at every scale tried and matchesTransformerDFTto ~3e-13 relative.My earlier pass recorded this as "unverified" rather than checking. This checks, and replaces a retracted claim with a measured one.
The useful finding: which transformer to pick
Undocumented until now, and not governed by visibility count. The DFT setup costs
O(N_vis × N_pix)against the NUFFT'sO((N_vis + N_pix) log N)plus ~2 s fixed overhead, so the product decides. Seven CPU measurements agree on a crossover near1e7:That's ~5,000 visibilities at a 64×64 mask — but still favours the DFT at 4,000 on a 32×32 one, which is why the guidance leads with the product.
Memory is the decisive axis. The DFT allocates with the same product (60 → 239 → 293 → 446 MB as the grid grows at fixed N_vis=4000) while the NUFFT allocates nothing measurable. At 10.7 bytes/element that reaches ~109 GB at 1M visibilities — independently corroborating the ~123 GB
bd18a769recorded by a different route, and confirming the NUFFT is the only feasible ALMA-scale path past ~1e8.Shipped in PyAutoLabs/PyAutoArray#479 (merged), PyAutoLabs/autolens_workspace#498 (merged), PyAutoLabs/autolens_workspace_test#263.
Both open items now closed
With item 1 (
use_adjoint_scaling) closed in #267, the pynufft removal is complete: the dependency is gone from six repos, the dead parameter and attribute are removed, and the sparse-operator question is settled by measurement rather than left as a retracted claim.Test Plan
python3 scripts/lifecycle.py check→lifecycle check: OKdashboard.md/dashboard.htmlregenerated viapyauto-brain intake --apply dashboard(149 prompts), not hand-editedGenerated by Claude Code