Repository navigation
Precompute planar trace kernels in a TraceTransformer - #605
Conversation
|
@kshyatt, any opinions about the GPU kernel work here? As a sidenote, I did some local testing and actually this breaks enzyme support, but somehow CI doesn't show it so that also might need a bit more care. I might actually try and get a correct rrule in for this as well |
I'm a bit surprised by this, I'd be interested in digging a bit to see why and see if we can avoid having to add more custom rules |
|
Because it goes through the full indexmanipulation machinery, so basically it is the same issue as |
`planartrace!` now precomputes the subblock positions and recoupling
coefficients in a cached `TraceTransformer`, and executes them with a
sector-independent kernel, analogous to the index manipulations. This
replaces the per-tree loop that built ad-hoc single-pair `FusionTreeBlock`s.
- `planar_trace` is defined for `Union{FusionTreePair, FusionTreeBlock}`,
with unique-fusion pairs returning `dst => coeff` and blocks `dst => U`,
and throws when the traced sectors are not dual to each other
- the transformer skips such (sub)blocks upfront and groups the data by
destination, such that every destination subblock is written once and
`β` is handled inside the kernel
- blocks with several destination trees are traced (and permuted) into a
buffer, recoupled with a single matmul and added into the destination
- `FusionTreeBlock`s hash their uncoupled sectors and duality flags
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GPU tensors pack the `TraceTransformer` into `isbits` descriptors, cached per device storagetype, and trace with one thread per destination element, fusing the sum over the traced entries and the recoupling into a single kernel, analogous to the generic tree transformers. Other tensor types, such as adjoints, are first converted to `TensorMap`s: tracing subblock views with cuTENSOR fails for data offsets that are not aligned (e.g. `Float64` data at odd offsets). `_linear_index` is now written as an explicit loop, since `sum` of an empty tuple, as occurs for a full trace, does not compile for GPUs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first stage sums chunks of at most `max(256, √ntraced)` traced entries per thread, so full reductions and traces with few remaining entries are still spread over many threads. The second stage recouples these partials with one thread per destination element, such that traced entries are summed once per source rather than once per destination tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Enzyme cannot differentiate through the threaded `taskforeach` in the new trace kernel, so `planartrace!` now has forward and reverse rules, as for the index manipulations. The pullback `planartrace_pullback_ΔA!` reuses the cached `TraceTransformer`: it adds `ΔC` to the traced diagonal of every source subblock with coefficients `conj(α * U[i, j]) * dim(cᵢ) / dim(cⱼ)`, going through the subblocks such that adjoint tensors are supported, and falls back to `trace_permute_pullback_ΔA!` for bosonic sectors. `treetracer` is marked inactive. The Enzyme `planartrace!` test skipped every draw without a remaining leg or traced pair, which in practice skipped (almost) all of them. It now always draws at least one of each, retries draws whose multifusion spaces have mismatched units, and also compares the pullback of adjoint tensors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…around The workaround for nested `@index` in the CPU backend of KernelAbstractions is no longer needed since KernelAbstractions v0.9.45 (JuliaGPU/KernelAbstractions.jl#877). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This replaces the per-tree loop in
planartrace!, which used the fusion tree internals through ad-hoc single-pairFusionTreeBlocks, with the same precompute/execute split as the index manipulations (treetransformers.jl/indexmanipulations.jl).Fusion trees
planar_trace(src::Union{FusionTreePair, FusionTreeBlock}, p, q)follows the conventions oftranspose/repartition: unique-fusion pairs map todst => coeff, blocks todst => Uwith a dense recoupling matrix. It repartitions all legs to the outgoing side, traces the splitting trees into the full block of remaining trees, and transposes that block.trace_sectors_matchinstead, which in the generic case avoids computing repartitions/transposes of blocks that vanish anyway.FusionTreeBlocks now hash their uncoupled sectors and duality flags, since the ad-hoc single-pair blocks are gone.Precompute:
TraceTransformer@cached treetracer(A, Vdst, Vsrc, p, q), keyed on the destination storagetype like the other transformers.(U, inds_dst, inds_src)entries, grouped by destination block: several source blocks (one per traced sector) contribute to the same destination. Every destination subblock appears in exactly one entry, so threaded tasks never write the same subblock, and subblocks without contributions get an entry that only scales byβ.Execute:
trace_transform_kernel!add_transform_kernel!, including the per-task workspaces.βon the first contribution._recouple!), and add into the destinations.GPU kernel
GPU tensors pack the
TraceTransformerintoisbitsdescriptors (cached per device storagetype, like the braid/transpose transformers) and trace in two stages:max(256, √ntraced)traced entries (one summed stride per traced pair) into a scratch buffer fromallocator, so full reductions and traces with few remaining entries are still spread over many threads;Other tensor types (e.g. adjoints) are converted to
TensorMaps first. Tracing subblock views with cuTENSOR, as the previous implementation did for all GPU tensors, fails withCUDA error: misaligned addressfor data at unaligned offsets (reproduced forFloat64fℤ₂ ⊠ SU₂ data)._linear_indexis now an explicit loop, sincesum(())(a full trace) does not compile for GPUs.Tested on an RTX A6000 against the CPU results for the same sectors and traces, in
Float64andComplexF64, including adjoint sources,β = 0with NaN-filled output, and full traces split into several chunks.test/cuda/planar.jlgains a symmetricplanartracetest for every space in the spacelist and for large traces;cuda/planarandcuda/tensorspass locally.GPU timings for one-stage (one thread per destination element) vs. two-stage kernel, for degeneracies 40 (fℤ₂), 30 (Fibonacci), 12/10/8 (fℤ₂ ⊠ SU₂):
Enzyme rules
taskforeachin the new trace kernel (IllegalTypeAnalysisExceptioninsync_end), soplanartrace!now has forward and reverse Enzyme rules, structured like thepermute!/transpose!rules.treetraceris marked inactive, liketreetransposer/treebraider.planartrace_pullback_ΔA!reuses the cachedTraceTransformer: it addsΔCto the traced diagonal of every source subblock (aStridedViewwith summed strides, no identity tensor), with coefficientsconj(α * U[i, j]) * dim(cᵢ) / dim(cⱼ)sinceinnerweighs every block withdim(c). It goes through_transform_subblocks, so adjoint tensors are supported, and falls back totrace_permute_pullback_ΔA!for bosonic sectors.planartrace!test skipped every draw without a remaining leg or traced pair, which in practice skipped (almost) all of them, so this was never tested. It now always draws at least one of each, retries multifusion draws with mismatched units, and also compares the pullback of adjoint tensors.⟨ΔC, tr(dA)⟩ = ⟨pullback(ΔC), dA⟩for fℤ₂, Fibonacci, fℤ₂ ⊠ SU₂, Fibonacci ⊠ fℤ₂ and SU₂ with partial, nested, full and domain traces;enzyme/tracepasses locally withCI=true(16145 tests, ~28 min locally, previously theplanartrace!part was effectively empty).Testing
tensors/planar,tensors/contractionsandsymmetries/doubletreepass locally; the planar trace test indoubletree.jlnow uses the block method for generic fusion.Not covered: JET was not run locally (it passes in CI); the GPU kernels were only tested on CUDA, not AMDGPU.
🤖 Generated with Claude Code