Skip to content

fix(deps): update dependency sentence-transformers to v6 - #374

Open
dreadnode-renovate-bot[bot] wants to merge 1 commit into
mainfrom
renovate/sentence-transformers-6.x
Open

dreadnode-renovate-bot[bot] wants to merge 1 commit into
mainfrom
renovate/sentence-transformers-6.x

Conversation

@dreadnode-renovate-bot

@dreadnode-renovate-bot dreadnode-renovate-bot Bot commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package Change Age Confidence
sentence-transformers >=5.1.0,<6.0.0 → >=6.3.0,<6.4.0 age confidence

Release Notes

huggingface/sentence-transformers (sentence-transformers)

v6.3.0: - Token-pooling-aware training and kernels support

Compare Source

Train multi-vector models for smaller document representations with token-pooling-aware and Matryoshka training. This release also adds optional kernels for MaxSim, Ward clustering, and transformer backbones, native AWS Neuron inference, faster utilities, and fixes across encoding, training, and evaluation.

[!TIP]
I share much more about training multi-vector models with token-pooling-aware losses in my new Token Pooling-Aware Training for Multi-Vector Embedding Models blogpost. Give it a read!

Install with:

# Inference only, use one of:
pip install "sentence-transformers==6.3.0"
pip install "sentence-transformers[kernels]==6.3.0"
pip install "sentence-transformers[openvino]==6.3.0"

# Multimodal dependencies:
pip install "sentence-transformers[image]==6.3.0"
pip install "sentence-transformers[audio]==6.3.0"
pip install "sentence-transformers[video]==6.3.0"

# Training and merging support:
pip install "sentence-transformers[train]==6.3.0"
pip install "sentence-transformers[merge]==6.3.0"

# Or combine as needed:
pip install "sentence-transformers[train,image,kernels]==6.3.0"

Token-pooling-aware training (#​4286)

Train for fewer document embeddings

Token pooling for multi-vector models merges similar document token embeddings to reduce the embeddings stored and scored during retrieval. With MultiVectorTokenPoolingLoss, you can now train the encoder for this compression so its document embeddings retain more useful information after pooling.

Wrap a retrieval loss with the pooling configurations you intend to deploy:

from sentence_transformers import MultiVectorEncoder
from sentence_transformers.multi_vector_encoder.losses import (
    CachedMultiVectorMultipleNegativesRankingLoss,
    MultiVectorTokenPoolingLoss,
)
from sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling

model = MultiVectorEncoder("answerdotai/ModernBERT-base")
pooling = HierarchicalTokenPooling(pool_factor=4)
base_loss = CachedMultiVectorMultipleNegativesRankingLoss(model, mini_batch_size=64)
loss = MultiVectorTokenPoolingLoss(
    model,
    base_loss,
    token_poolings=[None, pooling],
    weights=[0.5, 0.5],
)

Pass loss to MultiVectorEncoderTrainer to train on the average of the unpooled and factor-4 retrieval losses. Because both variants reuse the same encoder embeddings, training adds clustering and scoring work without another encoder pass.

A matched experiment compared two ModernBERT-base models with fresh 128-dimensional projections, trained on the same 120,000 MS MARCO triplets with seed 42:

Training Unpooled nDCG@​10 Factor-4 nDCG@​10 Unpooled score retained
Ordinary 51.84 49.03 94.6%
Pooling-aware 52.39 51.23 97.8%

Factor-4 pooling reduces the average document representation from 193.22 to 48.68 embeddings, while the pooling-aware model retains 97.8% of its unpooled score.

To compress documents with the trained example model, pass a pooler to encode_document or set the runtime default with MultiVectorEncoder(..., token_pooling=...). By default, pooling leaves query embeddings unchanged:

from sentence_transformers import MultiVectorEncoder
from sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling

model = MultiVectorEncoder("tomaarsen/mve-ModernBERT-base-token-pooling")
queries = model.encode_query(["Which planet is known as the Red Planet?"])
documents = model.encode_document(
    ["Mars has a reddish surface.", "Jupiter is the largest planet."],
    token_pooling=HierarchicalTokenPooling(pool_factor=4),
)
scores = model.similarity(queries, documents)

See the blogpost for the method and experiments, and the training guide for complete scripts. (#​4286)

Adaptive token budgets

Choose one of three HierarchicalTokenPooling controls to match the document budget you want to deploy:

Control Effect
pool_factor=4 Keep approximately one quarter of each document's vectors.
max_tokens=16 Keep at most 16 vectors per document, including protected tokens.
distance_threshold=... Allocate vectors according to embedding geometry using a shared Ward distance threshold.

For an adaptive budget, calibrate a threshold from representative document embeddings so the pooler targets an average count while allocating vectors according to each document's content:

from sentence_transformers import MultiVectorEncoder
from sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling

model = MultiVectorEncoder("tomaarsen/mve-ModernBERT-base-token-pooling")
calibration_documents = [
    "Mars is the fourth planet from the Sun and has a reddish surface.",
    "Jupiter is a gas giant with many moons and a prominent Great Red Spot.",
]
embeddings = model.encode_document(calibration_documents)
threshold = HierarchicalTokenPooling.calibrate_distance_threshold(embeddings, target_tokens=8)
pooling = HierarchicalTokenPooling(distance_threshold=threshold) if threshold is not None else None
compressed = model.encode_document(calibration_documents, token_pooling=pooling)

The threshold targets an average on the calibration sample, so individual documents can receive more or fewer vectors and another corpus can have a different average. If the sample already fits the budget, calibration returns None and no pooling is needed.

These controls also work without pooling-aware training, so you can try compression on an existing model and evaluate it by passing the poolers to multi-vector evaluators.

Matryoshka training for multi-vector models

MultiVectorMatryoshkaLoss trains a model to retain retrieval quality when token embeddings are truncated. Wrap your retrieval loss with the dimensions you want to support:

from sentence_transformers import MultiVectorEncoder
from sentence_transformers.multi_vector_encoder.losses import (
    CachedMultiVectorMultipleNegativesRankingLoss,
    MultiVectorMatryoshkaLoss,
)

model = MultiVectorEncoder("answerdotai/ModernBERT-base")
base_loss = CachedMultiVectorMultipleNegativesRankingLoss(model, mini_batch_size=64)
loss = MultiVectorMatryoshkaLoss(
    model,
    base_loss,
    matryoshka_dims=[128, 64, 32],
    matryoshka_weights=[1 / 3] * 3,
)

After training, use the shorter vectors by passing truncate_dim=64 to both encode_query and encode_document, or set that width as the model constructor default.

In a separate matched experiment with 30,000 MS MARCO triplets, Matryoshka training preserves more retrieval quality at 64 dimensions, measured by mean nDCG@​10 (%) across 13 NanoBEIR datasets:

Training 128-dimensional nDCG@​10 64-dimensional nDCG@​10 Full-width score retained
Ordinary 48.45 45.46 93.8%
Matryoshka 48.95 47.11 96.2%

Using 64 dimensions halves vector storage at unchanged precision, with retention measured against each model's own 128-dimensional score.

To train for fewer vectors and shorter dimensions together, wrap MultiVectorMatryoshkaLoss with MultiVectorTokenPoolingLoss so pooling happens before truncation. See the Matryoshka guide for complete training and evaluation scripts.

Optional kernels for MaxSim, token pooling, and transformer backbones (#​4344)

With use_kernels=True, all four model types can use compatible Hub kernels to accelerate operations in their transformer backbones. For multi-vector models, the same setting also enables kernels for MaxSim and MeanMaxSim scoring, as well as the Ward clustering used in token pooling:

from sentence_transformers import MultiVectorEncoder
from sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling

model = MultiVectorEncoder("lightonai/mLateOn", use_kernels=True)
queries = model.encode_query(["How can I reduce retrieval storage?"])
documents = model.encode_document(
    ["Token pooling merges similar document vectors before indexing."],
    token_pooling=HierarchicalTokenPooling(pool_factor=2),
)
scores = model.similarity(queries, documents)

Install the optional dependencies with pip install "sentence-transformers[kernels]" before enabling kernels. The default multi-vector providers target CUDA and require Triton for MaxSim acceleration, falling back to reference implementations when a compatible device or provider build is unavailable.

In the Ward benchmark, the kernel reduces pooling time from 7.00 seconds with fastcluster to 1.02 seconds, about 6.9x faster, for 5,000 SciFact titles and abstracts. Including encoding, the improvement is smaller: 23.14 to 18.91 seconds, an 18% reduction with mLateOn, BF16, batch size 32, factor 2, and an RTX 3090.

Kernels are opt-in runtime settings, so enable them again when reloading a saved model. For finer control, pass a Transformers KernelConfig to choose providers separately for the backbone, scoring, and pooling operations, as described in the kernel guide. (#​4344)

Native AWS Neuron inference (#​4293)

After installing TorchNeuron in a supported AWS environment, set device="neuron" to run dense, cross-encoder, and multi-vector inference through the existing torch backend:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="neuron")
embeddings = model.encode(["Native inference on AWS Neuron."])

For multi-process encoding, load the model on CPU in a fresh parent process so workers can use separate Neuron cores. See the PR for setup and device-allocation details.

Smaller Features

  • More efficient community detection: a shared batched implementation reduces GPU memory use on large embedding collections and speeds up CPU clustering. (#​4342)
  • Faster utilities on Apple Silicon: community detection and hard-negative mining spend less time synchronizing with the device, improving runtime on MPS. (#​4265, #​4290)
  • Faster Hugging Face Dataset inputs: batch conversion of Dataset columns reduces data preparation overhead in encoding, prediction, mining, and selected sampler and evaluator workflows. (#​4347)
  • Clearer model merging diagnostics: configuration errors explain incompatible inputs more clearly, while new benchmark examples help you compare merged checkpoints. (#​4315)

Compatibility Notes

  • Multi-vector loss scoring: the four built-in contrastive and distillation losses now follow the model's similarity even with kernels disabled, so MeanMaxSim models use mean scores instead of the previous MaxSim sums. Check your contrastive scale and teacher-score convention when upgrading. To retain the old MaxSim behavior, set similarity_fct to colbert_scores for ranking losses, colbert_scores_pairwise for margin MSE, or colbert_kd_scores for KL distillation. (#​4344)
  • Evaluation scores: historical scores can change because reranking and paraphrase-mining now follow the model's similarity, while CrossEncoder and RRF normalize MAP/nDCG against all known positives. Unretrieved positives receive no credit, so scores reflect retrieval failures. (#​4252, #​4268, #​4313, #​4054, #​4114)
  • Checkpointing names: use activation_checkpointing_enable, activation_checkpointing, and activation_checkpointing_kwargs across supported Transformers versions, or keep the legacy gradient-checkpointing names in existing scripts. (#​4328)
  • Saved compression settings: a constructor truncate_dim is saved with the model, but token_pooling is a runtime default, so supply the pooler again after reload. (#​4286)
  • Pooler normalization: newly constructed hierarchical poolers normalize their output vectors by default. Older saved configurations without normalize_embeddings retain their unnormalized behavior to preserve compatibility with existing embeddings. (#​4286)

Bug Fixes

Evaluation and ranking metrics
  • Use the model's configured similarity in reranking and paraphrase-mining evaluation, fixing dot-product and sparse-model scoring. (#​4252, #​4268, #​4313)
  • Correct CrossEncoder and RRF MAP/nDCG normalization when retrieval misses positives, and handle empty CrossEncoder retrieval results. (#​4054, #​4114, #​4264)
Encoding and low-precision models
  • Restore low-precision workflows by casting Router modules to their route's dtype and aligning distillation inputs and targets with the projection-weight dtype. (#​4314, #​4348)
  • Make CSR sparse autoencoders picklable so they work with multi-process encoding and DataLoader workers. (#​4306)
  • Prevent StaticEmbedding from sharing and modifying its source tokenizer's backend. (#​4253)
  • Accept pathlib.Path model paths in SentenceTransformer. (#​4326)
Training
  • Preserve positional CrossEncoder constructor arguments after device. (#​4350)
  • Stack tensor-formatted labels in the CrossEncoder data collator. (#​4257)
  • Include evaluation loss components in logs when using custom metric prefixes. (#​4237)
  • Exclude self-pairs when GIST guide scores tie. (#​4284)
Hard-negative mining and retrieval examples
  • Keep anchors out of their own mined negatives and apply CrossEncoder rescoring even when no score filters are configured. (#​4267, #​4262)
  • Handle mining inputs and edge cases that previously failed, including a separate corpus supplied as a Dataset column and filters that exclude every candidate. (#​4318, #​4292)
  • Honor rescoring when reusing float USearch indexes. (#​4251)

All Changes

New Contributors

Full Changelog: huggingface/sentence-transformers@v6.2.0...v6.3.0

v6.2.0: - Model merging, faster loading, and distributed evaluation

Compare Source

This release adds model merging for all four model types, reduces redundant Hub requests when loading models, and distributes evaluator inference across DDP ranks. It also includes lower-memory training losses, more flexible multi-vector compression, and fixes throughout inference, training, and evaluation.

Install this version with

# Inference only, use one of:
pip install sentence-transformers==6.2.0
pip install sentence-transformers[openvino]==6.2.0

# Multimodal dependencies (optional):
pip install sentence-transformers[image]==6.2.0
pip install sentence-transformers[audio]==6.2.0
pip install sentence-transformers[video]==6.2.0

# Training + Inference
pip install sentence-transformers[train]==6.2.0
pip install sentence-transformers[merge]==6.2.0

# Or combine as needed:
pip install sentence-transformers[train,image]==6.2.0

Model merging for all four model types (#​3776)

Combine compatible fine-tuned checkpoints without another training run using SentenceTransformer.merge, SparseEncoder.merge, CrossEncoder.merge, or MultiVectorEncoder.merge. The integration uses Arcee AI's mergekit and supports well over a dozen methods such as Linear, SLERP, TIES, and DARE.

Install the merge extra with pip install "sentence-transformers[merge]". For example, take a weighted average of two compatible MiniLM models:

from sentence_transformers import SentenceTransformer

models = [
    "sentence-transformers/all-MiniLM-L6-v2",
    "sentence-transformers/multi-qa-MiniLM-L6-cos-v1",
]

merged = SentenceTransformer.merge(
    models,
    method="linear",
    parameters={"weight": [0.6, 0.4]},
)
merged.save_pretrained("merged-MiniLM-L6")

embeddings = merged.encode(["Combining compatible model weights."])

Or interpolate between the same inputs with SLERP:

merged = SentenceTransformer.merge(
    models,
    method="slerp",
    parameters={"t": 0.5},
)

The method returns a model that can be used, trained, and saved normally. Configuration, prompts, and processors come from the first input. Inputs must have compatible architectures, module configurations, and tokenizers. Merging uses FP32 on CPU by default and holds the inputs and result in memory.

See the merging guides for Sentence Transformers, Cross Encoders, Sparse Encoders (plus an example evaluation!), and Multi-Vector Encoders. Be sure to evaluate the merged model on your own tasks, since combining checkpoints can improve some tasks while regressing others.

Faster Hub model loading (#​3890)

Combined with huggingface_hub>=1.26.0, model loading now resolves the requested branch or tag to a commit once and reuses that result across component downloads, reducing repeated Hub requests. Older Hub versions retain the previous loading behavior.

The loading benchmarks in #​3890 measured constructor time in fresh processes with a warm cache and online Hub access. Values are medians of seven alternating runs per configuration:

Model Before After HTTP requests before / after
sentence-transformers-testing/stsb-bert-tiny-safetensors 5.78 s 2.26 s 34 / 8
sentence-transformers/all-MiniLM-L6-v2 6.39 s 2.66 s 35 / 9
sparse-encoder-testing/splade-bert-tiny-nq 5.77 s 2.25 s 34 / 8
cross-encoder/ms-marco-MiniLM-L6-v2 4.38 s 2.48 s 23 / 9

Constructor time fell by 43 to 61 percent in these workloads. Loading models from local directories is unchanged and unaffected by this improvement.

Distributed evaluator inference during DDP training (#​4004)

During DDP training, evaluators now share their encode() and predict() work across the existing training processes for all four model types, with each rank processing a shard of the inputs. Rank zero combines predictions in their original order, computes metrics over the full dataset, and writes evaluation output once. Metrics are then broadcast to every rank for checkpoint selection and early stopping.

This now happens automatically when using an evaluator through the trainer. It also covers encode_query(), encode_document(), and rank(), which use those inference methods internally.

Explicit device and pool arguments retain their usual behavior, and calls with arguments that cannot be serialized fall back to rank zero. Predictions are gathered through CPU memory, so the speedup depends on inference cost and communication overhead. Evaluator support for FSDP and DeepSpeed is unchanged.

Smaller Features

Experimental FP32 layers for CrossEncoders (#​4230)

CrossEncoders can compute selected linear layers in FP32 to reduce BF16 score ties while keeping a reduced-precision backbone:

from sentence_transformers import CrossEncoder

model = CrossEncoder(
    "cross-encoder/ms-marco-MiniLM-L6-v2",
    device="cuda",
    model_kwargs={"dtype": "bfloat16"},
    keep_in_fp32_modules=["0.model.classifier"],
)

This uses the exact model.named_modules() paths. The selection is saved with the model, works under autocast during training and inference, and leaves stored parameter dtypes unchanged. In my experiments, this doesn't improve NDCG@​10, but solely helps reduce ties caused by the low BF16/FP16 output precision.

Lower memory training losses (#​4026, #​4029, #​4030)
  • BatchSemiHardTripletLoss uses quadratic intermediate memory instead of constructing cubic distance and mask tensors. It also corrects a gradient issue when selected negative distances tie (#​4026).
  • BatchAllTripletLoss removes invalid anchor-positive pairs before constructing triplets. Memory is quadratic for paired labels, though large label groups can still require cubic storage (#​4029).
  • CoSENTLoss, AnglELoss, and their sparse counterparts use sorted suffix reductions with linear intermediate storage. MPS execution requires PyTorch 2.9 or newer (#​4030).
Multi-vector compression and scoring (#​3965, #​3941, #​4008, #​4153)
  • HierarchicalTokenPooling accepts fractional pool_factor values for finer control over compression (#​3965) and can use optional fastcluster linkage for faster clustering (#​3941).
  • HierarchicalTokenPooling now normalizes output tokens by default, including protected tokens. Older saved configurations retain their previous behavior. Set normalize_embeddings=False on the pooling module to opt out (#​4008).
  • MaxSim scoring budgets score blocks using both query and document token counts, reducing memory use for long query batches. A single query-document pair can still exceed chunk_elements (#​4153).

Compatibility Notes

  • Python and Hub versions: Added Python 3.14 and huggingface_hub 2.x support, retaining Python 3.10 support and the Hub 1.3.0 minimum (#​4162, #​4003).
  • Scalar quantization: Inferring int8 or uint8 calibration ranges from a single vector now raises ValueError. Encode floats first, then call quantize_embeddings with representative calibration_embeddings or explicit ranges (#​4150).
  • Checkpoint paths: Module paths must be relative and cannot contain .. path components, even with trust_remote_code=True. Keep custom modules inside the checkpoint tree (#​4175).

Bug Fixes

Pooling precision and normalization
  • Improved FP16/BF16 mean and weighted-mean pooling arithmetic, corrected weighted positions under left padding, and fixed prompt exclusion in flattened CLS fallback selection (#​4106, #​4104, #​4159).
  • Corrected normalization of single sparse vectors and sparse matrices with non-default dtypes or low-precision values (#​4149, #​4045).
Multi-process inference
  • Worker exceptions are reported to the caller, and terminated workers are detected while polling for results (#​3920, #​4146).
  • Worker tensor outputs are moved to CPU before queueing, including tensors nested in lists and dictionaries. Hard-negative mining also stops its pool if mining fails (#​3971, #​4086).
Model placement and memory
  • Improved device handling for quantized models and models using Accelerate device maps or offloading, and reduced repeated placement work. Call clear_placement_cache() after changing dispatch hooks or adding modules following the first inference call (#​3986, #​4035).
  • Disabled Transformer KV caching by default when supported, reducing unnecessary inference memory use. Explicit use_cache overrides in config_kwargs remain honored ([#​3992](https://redirect.github.com/huggingface/sentence-t

[!IMPORTANT]
✂ PR body was truncated to here.


Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate CLI.

@dreadnode-renovate-bot

dreadnode-renovate-bot Bot commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor Author

⚠️ Artifact update problem

Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.

♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

  • any of the package files in this branch needs updating, or
  • the branch becomes conflicted, or
  • you click the rebase/retry checkbox if found above, or
  • you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:

File name: uv.lock
Using CPython 3.13.18 interpreter at: /opt/containerbase/tools/python/3.13.18/bin/python3
error: No solution found when resolving dependencies for split (markers: python_full_version >= '3.12' and python_full_version < '3.14')
  cause: Because sentence-transformers>=6.3.0 depends on transformers>=5.0.0,<6.0.0 and dreadnode[all] depends on sentence-transformers>=6.3.0, we can conclude that dreadnode[all] depends on transformers>=5.0.0,<6.0.0.
         And because dreadnode[all] depends on transformers>=4.41.0,<5.0.0 and your project requires dreadnode[all], we can conclude that your project's requirements are unsatisfiable.

@dreadnode-renovate-bot dreadnode-renovate-bot Bot added area/python Changes to Python package configuration and dependencies type/digest Dependency digest updates labels Aug 19, 2026
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot force-pushed the renovate/sentence-transformers-6.x branch from ecb95ca to 8afcff8 Compare September 2, 2026 00:29
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot force-pushed the renovate/sentence-transformers-6.x branch from 8afcff8 to 7052c0a Compare September 20, 2026 00:31
@dreadnode-renovate-bot dreadnode-renovate-bot Bot changed the title fix(deps): update dependency sentence-transformers to v6 fix(deps): update dependency sentence-transformers to v6 - autoclosed Oct 4, 2026
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot deleted the renovate/sentence-transformers-6.x branch October 4, 2026 01:09
@dreadnode-renovate-bot dreadnode-renovate-bot Bot changed the title fix(deps): update dependency sentence-transformers to v6 - autoclosed fix(deps): update dependency sentence-transformers to v6 Oct 7, 2026
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot force-pushed the renovate/sentence-transformers-6.x branch 2 times, most recently from 7052c0a to 6ec1418 Compare October 7, 2026 00:31
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot force-pushed the renovate/sentence-transformers-6.x branch from 6ec1418 to d2c89cb Compare October 25, 2026 00:29
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot force-pushed the renovate/sentence-transformers-6.x branch 3 times, most recently from aa76785 to dd906a4 Compare November 15, 2026 00:21
| datasource | package               | from  | to    |
| ---------- | --------------------- | ----- | ----- |
| pypi       | sentence-transformers | 5.7.0 | 6.3.0 |
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot force-pushed the renovate/sentence-transformers-6.x branch from dd906a4 to 353b0c0 Compare November 18, 2026 00:21

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/python Changes to Python package configuration and dependencies type/digest Dependency digest updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants