You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Loading BlockTensorKit triggers severe gradient compilation slowdown for ordinary TensorMaps #86
Loading BlockTensorKit makes the first gradient below fail to finish within 60 seconds.
Removing only the BlockTensorKit import, with the same project and input seed, avoids the slowdown.
The inputs contain no block tensors.
using LinearAlgebra, Random, TensorKit, BlockTensorKit, Zygote
BLAS.set_num_threads(1)
Random.seed!(123)
V =ComplexSpace(2)
x = (
randn(ComplexF64, V ⊗ V ⊗ V ← V),
randn(ComplexF64, V ← V ⊗ V),
)
functionloss(x)
a, b =adjoint.(x)
returnreal(dot(a, a) +dot(b, b))
end@time g =only(Zygote.gradient(loss, x))
@assertall(isapprox.(g, 2.* x))
Tested on Julia 1.13.2, macOS arm64 / Apple M5, using AppleAccelerate and one Julia/BLAS thread.
Versions: TensorKit 0.17.2, BlockTensorKit 0.3.20, Zygote 0.7.13, ChainRules 1.73.0, ChainRulesCore 1.26.1.
Each measurement uses a fresh process and excludes package loading:
Case
First gradient
Original example
>60 s, interrupted
Without importing BlockTensorKit
2.83 s
Replace broadcast with a, b = x[1]', x[2]'
2.42 s
TensorKit experiment below
3.35 s
ChainRules experiment below
3.10 s
All completed cases produced the expected gradients.
These are individual first-call measurements, not benchmark averages.
Sampling shows compiler inference activity.
The broadcast pullback produces an abstractly typed cotangent vector, and inference visits block/sparse-block tensor paths despite the ordinary TensorMap inputs.
Loading BlockTensorKit is therefore a confirmed trigger, but I haven’t isolated a faulty method or established the root cause.
Two changes independently avoid the slowdown:
TensorKit: In generic VectorInterface.add, replace the chained mutation return with:
- val = if N == length(dx)+ val = if dx isa AbstractVector || N == length(dx)
Both are experimental candidates; I’m unsure whether either is the appropriate general fix.
The ChainRules change also seems worth discussing upstream.
Thanks for reporting! I'm not entirely sure what's going on, but it sounds like some invalidations must be the cause. I'm not at my desk this week and the next so I won't have time to inspect, but it would also be interesting to know if it is specifically the last update that caused this or if it was already the case before
The TensorKit change is interesting to evaluate as well, it sounds like inference works a lot better if we make mutating functions return the original argument, rather than depending on the inplace functions in the body returning the destination. I'm not sure if this would be a measurable improvement if we just do this in general, because it is a code pattern that comes up quite often
Activity
lkdvos commentedon Oct 17, 2026
Thanks for reporting! I'm not entirely sure what's going on, but it sounds like some invalidations must be the cause. I'm not at my desk this week and the next so I won't have time to inspect, but it would also be interesting to know if it is specifically the last update that caused this or if it was already the case before
lkdvos commentedon Oct 17, 2026
The TensorKit change is interesting to evaluate as well, it sounds like inference works a lot better if we make mutating functions return the original argument, rather than depending on the inplace functions in the body returning the destination. I'm not sure if this would be a measurable improvement if we just do this in general, because it is a code pattern that comes up quite often