Add UnitVector tensor refinement

[connortsui20]

Rationale for this change

Normalized currently stores normalized Vector values with the same dtype as arbitrary vectors, so the unit-norm invariant is not represented in the type.

What changes are included in this PR?

Adds the UnitVector extension dtype for vectors whose rows have norm 1.0 within tolerance or are exactly zero. Adds the L2Normalize scalar function and uses it in the Normalized encoding, where vector directions become UnitVector while fixed-shape tensor directions retain their dtype.

What APIs are changed? Are there any user-facing changes?

Adds the public UnitVector dtype and L2Normalize scalar function. InnerProduct and CosineSimilarity accept compatible Vector and UnitVector pairs.

[codspeed-hq]

Merging this PR will improve performance by 31.34%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 8 improved benchmarks
✅ 1948 untouched benchmarks
⏩ 442 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	encode_nullable[256]	253 µs	152.3 µs	+66.14%
⚡	Simulation	encode_nullable[32]	253.1 µs	165.8 µs	+52.66%
⚡	Simulation	encode_nullable[2]	622.5 µs	433.7 µs	+43.54%
⚡	Simulation	encode_non_nullable[2]	481.7 µs	370.9 µs	+29.89%
⚡	Simulation	encode_non_nullable[32]	193.6 µs	151.7 µs	+27.61%
⚡	Simulation	take[duplicates/repeated/primitive/nonnull/chunks=16/indices=1000]	262.8 µs	228.9 µs	+14.81%
⚡	WallTime	words_gather_scalar[65536]	9.4 µs	8.2 µs	+14.02%
⚡	Simulation	compact[(2048, 90)]	1.8 µs	1.6 µs	+12.07%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/unit-vector (b831dfc) with develop (cb77d9a)}

Footnotes

1. 442 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[connortsui20]

whoops accidentally brought unrelated changes in

[connortsui20]

Closing this draft because the UnitVector refinement puts approximate normalization evidence in the logical dtype. I’m replacing it with an explicit scalar-function compute mode: Vector remains the logical type, Normalized remains a physical encoding, and callers choose whether cosine similarity and L2 norm may trust its normalized-direction claim.

[connortsui20]

Replacement draft: #9513 keeps Vector as the logical dtype and moves permission to trust Normalized evidence onto the norm-based scalar functions.