Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
2fbfe05
Only show top N terms when they are non-zero
x-tabdeveloping Oct 9, 2026
8fc32db
Save merge history in SensTopic
x-tabdeveloping Oct 9, 2026
08405e0
Added basic implementation of training for VSLDA
x-tabdeveloping Oct 15, 2026
70178ec
Added implementation for VSLDA
x-tabdeveloping Oct 16, 2026
a5652c7
Extended the Late interaction model API
x-tabdeveloping Oct 16, 2026
04934de
Added lexical-semantic feature importance from Topeax"
x-tabdeveloping Oct 16, 2026
e472b6e
Added term importance and new late API to vSLDA
x-tabdeveloping Oct 16, 2026
3b6a2d2
Bugfixes
x-tabdeveloping Oct 16, 2026
6d0f6ca
added vSLDA to init"
x-tabdeveloping Oct 16, 2026
f92fda6
Fixed issues with encoding
x-tabdeveloping Oct 16, 2026
4def95f
Added NMF and LDA
x-tabdeveloping Oct 16, 2026
f62f77f
Bumped sbert dependency
x-tabdeveloping Oct 16, 2026
3c334c1
Fixed transform API for vSDLA
x-tabdeveloping Oct 20, 2026
09c9bf1
Added gliformer-based concept vectorizer
x-tabdeveloping Oct 21, 2026
eb0d53d
bugfix
x-tabdeveloping Oct 21, 2026
77b8d98
Extensions and bugfixes to hierarchical API
x-tabdeveloping Oct 21, 2026
357c028
Added multilevel model to SensTopic
x-tabdeveloping Oct 21, 2026
2d55887
Added __init__ file to bow module
x-tabdeveloping Oct 27, 2026
bc36ec1
Updated documentation
x-tabdeveloping Oct 27, 2026
1b319e6
Added new models to tests
x-tabdeveloping Oct 27, 2026
cd44cdb
Added n_components to tests
x-tabdeveloping Oct 27, 2026
db71247
Fixed some issues with encode_kwargs in late models
x-tabdeveloping Oct 28, 2026
c7bb365
Fixed encoding when encode_kwargs is None
x-tabdeveloping Oct 28, 2026
d484cf4
Bugfix in vSLDA
x-tabdeveloping Oct 28, 2026
2d57bc6
Fixed encode_kwargs not referring to anything
x-tabdeveloping Oct 28, 2026
51dbb7c
Fixed even more errors with late API
x-tabdeveloping Oct 28, 2026
7754ef5
Added missing comma
x-tabdeveloping Oct 28, 2026
1c75749
Attempted fixing late wrapper
x-tabdeveloping Oct 28, 2026
2d27875
Version bump
x-tabdeveloping Oct 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions docs/LDA.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Latent Dirichlet Allocation

LDA is one of the most well known and widely used topic models.
It is a bag-of-words topic model that models topics as word distributions drawn from a Dirichlet prior, and documents as mixtures of topics, with the mixture weights also having a Dirichlet prior.

<figure>
<img src="../images/lda_plate.png", title="", style="width:600px;padding:0px;border:none;"></img>
<figcaption> Figure 1: Plate notation of LDA..<br> <i>Figure from Blei et al. (2003)</i> </figcaption>
</figure>


!!! note
The Turftopic implementation of LDA is a thin wrapper around the scikit-learn implementation mainly for convenience and benchmarking.
We recommend that you refer to [their documentation](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.LatentDirichletAllocation.html) for further details.

## Usage

```python
from turftopic import LDA

documents = [...]

model = LDA(10)
doc_topic_matrix = model.fit_transform(documents)
model.print_topics()
```

!!! tip
Since LDA is a BoW topic model, it does not handle stop words particularly well.
We recommend that you use a vectorizer that removes stop words or all too frequent words:
```python
from turftopic import LDA
from sklearn.feature_extraction.text import CountVectorizer

model = LDA(10, vectorizer=CountVectorizer(min_df=10, stop_words="english", max_df=0.8))
```

## Citation

Please cite the authors of the paper, and Turftopic when using LDA:

```bibtex
@article{Blei2003,
author = {Blei, David M. and Ng, Andrew Y. and Jordan, Michael I.},
title = {Latent dirichlet allocation},
year = {2003},
issue_date = {3/1/2003},
publisher = {JMLR.org},
volume = {3},
number = {null},
issn = {1532-4435},
abstract = {We describe latent Dirichlet allocation (LDA), a generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities. In the context of text modeling, the topic probabilities provide an explicit representation of a document. We present efficient approximate inference techniques based on variational methods and an EM algorithm for empirical Bayes parameter estimation. We report results in document modeling, text classification, and collaborative filtering, comparing to a mixture of unigrams model and the probabilistic LSI model.},
journal = {J. Mach. Learn. Res.},
month = mar,
pages = {993–1022},
numpages = {30}
}

@article{
Kardos2025,
title = {Turftopic: Topic Modelling with Contextual Representations from Sentence Transformers},
doi = {10.21105/joss.08183},
url = {https://doi.org/10.21105/joss.08183},
year = {2025},
publisher = {The Open Journal},
volume = {10},
number = {111},
pages = {8183},
author = {Kardos, Márton and Enevoldsen, Kenneth C. and Kostkan, Jan and Kristensen-McLachlan, Ross Deans and Rocca, Roberta},
journal = {Journal of Open Source Software}
}
```


## API Reference

::: turftopic.models.bow.lda.LDA
79 changes: 79 additions & 0 deletions docs/NMF.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Nonnegative Matrix Factorization

NMF is a factor-based topic model of bag-of-words or tf-idf representations.
It treats topics as latent factors that explain variance in word counts/importances in documents.
Factors are strictly positive, hence the name.

!!! tip
NMF is an older topic model with substantially less interpretable topics than modern approaches, and with no contextual understanding.
If you want to use a model that is more powerful but similar to NMF, we recommend you try out [KeyNMF](KeyNMF.md) or [SensTopic](SensTopic.md).

<figure>
<img src="https://upload.wikimedia.org/wikipedia/commons/f/f9/NMF.png?utm_source=en.wikipedia.org&utm_campaign=imageinfo&utm_content=thumbnail_unscaled", title="", style="width:600px;padding:0px;border:none;"></img>
<figcaption> Figure 1: Depiction of Nonnegative Matrix Factorization.<br> <i>By Qwertyus - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=29114677</i> </figcaption>
</figure>



## Usage

```python
from turftopic import NMF

documents = [...]

model = NMF(10)
doc_topic_matrix = model.fit_transform(documents)
model.print_topics()
```

!!! tip
Since NMF is a BoW topic model, it does not handle stop words particularly well.
We recommend that you use a vectorizer that removes stop words or all too frequent words, potentially with tf-idf weighting
```python
from turftopic import LDA
from sklearn.feature_extraction.text import TfidfVectorizer

model = NMF(10, vectorizer=TfidfVectorizer(min_df=10, stop_words="english", max_df=0.8))
```

## Citation

Please cite the authors of the implementation, and Turftopic when using NMF:

```bibtex
@article{Cedric2011,
author = {Févotte, Cédric and Idier, Jérôme},
title = {Algorithms for Nonnegative Matrix Factorization with the β-Divergence},
journal = {Neural Computation},
volume = {23},
number = {9},
pages = {2421-2456},
year = {2011},
month = {09},
abstract = {This letter describes algorithms for nonnegative matrix factorization (NMF) with the β-divergence (β-NMF). The β-divergence is a family of cost functions parameterized by a single shape parameter β that takes the Euclidean distance, the Kullback-Leibler divergence, and the Itakura-Saito divergence as special cases (β = 2, 1, 0 respectively). The proposed algorithms are based on a surrogate auxiliary function (a local majorization of the criterion function). We first describe a majorization-minimization algorithm that leads to multiplicative updates, which differ from standard heuristic multiplicative updates by a β-dependent power exponent. The monotonicity of the heuristic algorithm can, however, be proven for β ∈ (0, 1) using the proposed auxiliary function. Then we introduce the concept of the majorization-equalization (ME) algorithm, which produces updates that move along constant level sets of the auxiliary function and lead to larger steps than MM. Simulations on synthetic and real data illustrate the faster convergence of the ME approach. The letter also describes how the proposed algorithms can be adapted to two common variants of NMF: penalized NMF (when a penalty function of the factors is added to the criterion function) and convex NMF (when the dictionary is assumed to belong to a known subspace).},
issn = {0899-7667},
doi = {10.1162/NECO_a_00168},
url = {https://doi.org/10.1162/NECO_a_00168},
eprint = {https://direct.mit.edu/neco/article-pdf/23/9/2421/852282/neco_a_00168.pdf},
}

@article{
Kardos2025,
title = {Turftopic: Topic Modelling with Contextual Representations from Sentence Transformers},
doi = {10.21105/joss.08183},
url = {https://doi.org/10.21105/joss.08183},
year = {2025},
publisher = {The Open Journal},
volume = {10},
number = {111},
pages = {8183},
author = {Kardos, Márton and Enevoldsen, Kenneth C. and Kostkan, Jan and Kristensen-McLachlan, Ross Deans and Rocca, Roberta},
journal = {Journal of Open Source Software}
}
```


## API Reference

::: turftopic.models.bow.lda.LDA
4 changes: 4 additions & 0 deletions docs/benchmark.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Model Leaderboard

!!! note
The leaderboard is currently out of date, as multiple models have been implemented since the benchmark was last run, and we are currently working on new evaluation methods.
We are doing our best to keep up with the influx of models, and will update the leaderboard as soon as possible.

To aid you in choosing the best model for your use case, we have made a topic model benchmark and leaderboard.
The benchmark consists of all English P2P clustering tasks from the most recent version of [MTEB](https://huggingface.co/spaces/mteb/leaderboard), plus a tweet and a news dataset, as these are not present in MTEB.

Expand Down
18 changes: 18 additions & 0 deletions docs/c_top2vec.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,24 @@ doc_topic_matrix = model.fit_transform(corpus)
model.print_topics()
```


| Topic ID | Highest Ranking |
| - | - |
| -1 | caused atheism organization, genocide caused atheism, atheism organization, atheism, subject political atheists, alt atheism, caused atheism, political atheists organization, subject amusing atheists, amusing atheists |
| 166 | atheists organization, political atheists organization, christian morality organization, caused atheism organization, morality organization, atheism organization, atheists organization california, subject amusing atheists, cwru edu article, alt atheism |
| 172 | biblical, read bible, caused atheism, agnostics, caused atheism organization, atheists agnostics, christianity, alt atheism, atheism, christian morality organization |
| 173 | objective morality, morality, subject christian morality, christian morality, natural morality, say christian morality, morality organization, christian morality organization, behavior moral, moral |
| 175 | atheism, atheism organization, caused atheism organization, atheists agnostics, caused atheism, subject political atheists, alt atheism, genocide caused atheism, subject amusing atheists, amusing atheists |
| 176 | rushdie islamic law, subject rushdie islamic, islamic genocide, islamic law, genocide caused atheism, subject islamic, islamic law organization, islamic genocide organization, rushdie islamic, islamic authority |

You might also observe that the output of this model is a regular document-topic matrix, and isn't ragged.
```python
print(doc_topic_matrix.shape)
# (1024, 6)
```

This is because this way the model has the same API, as other Turftopic models, and works the same way as the top2vec package, making migration easier.

## Citation

Please cite Angelov and Inkpen (2024) and Turftopic when using C-Top2Vec in publications:
Expand Down
Binary file added docs/images/lda_plate.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/vslda_plate.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
45 changes: 5 additions & 40 deletions docs/late_interaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ Late interaction, or multi-vector models use token representations from a Senten
This can be particularly useful for clustering models, as they, by default assign one topic to a single document, but when accessing token representations, can assign topics on a per-token basis.

!!! info
There are currently no native late-interaction models in Turftopic, meaning models that explicitly model token representations in the context of a document.
We are currently working on implementing such models, but for the time being, wrappers are included, that can force regular models to use embeddings of higher granularity.
The only model natively supporting multi-vector embedding models right now is [vSLDA](vSLDA.md).
We are currently working on implementing multiple such models, but for the time being, wrappers are included, that can force regular models to use embeddings of higher granularity.
**Visualization utilities** are also on the way.

## Encoding Tokens, and Ragged Array Manipulation
Expand Down Expand Up @@ -163,46 +163,11 @@ enough and\nlistened to the arguments, I could ... 0.878
> ******************************* 0.647793 5.664651e-17 ... 1.805073e-19 4.612731e-04
```

## C-Top2Vec

Contextual Top2Vec [(Angelov and Inkpen, 2024)](https://aclanthology.org/2024.findings-emnlp.790/) is a late-interaction topic model, that uses windowed representations.
The model is essentially the same as wrapping a regular Top2vec model in `LateWrapper`, but we provide a convenience class in Turftopic, so that it's easy for you to initialize this model.
It comes pre-loaded with the following features:

- Same hyperparameters as in Angelov and Inkpen (2024)
- Phrase-vectorizer that finds regular phrases based on PMI
- `LateSentenceTransformer` by default, you can specify any model.

Our implementation is much more flexible than the original top2vec package, and you might be able to use much more powerful or novel embedding models.

```python
from turftopic import CTop2Vec

model = CTop2Vec(n_reduce_to=5)
doc_topic_matrix = model.fit_transform(corpus)

model.print_topics()
```


| Topic ID | Highest Ranking |
| - | - |
| -1 | caused atheism organization, genocide caused atheism, atheism organization, atheism, subject political atheists, alt atheism, caused atheism, political atheists organization, subject amusing atheists, amusing atheists |
| 166 | atheists organization, political atheists organization, christian morality organization, caused atheism organization, morality organization, atheism organization, atheists organization california, subject amusing atheists, cwru edu article, alt atheism |
| 172 | biblical, read bible, caused atheism, agnostics, caused atheism organization, atheists agnostics, christianity, alt atheism, atheism, christian morality organization |
| 173 | objective morality, morality, subject christian morality, christian morality, natural morality, say christian morality, morality organization, christian morality organization, behavior moral, moral |
| 175 | atheism, atheism organization, caused atheism organization, atheists agnostics, caused atheism, subject political atheists, alt atheism, genocide caused atheism, subject amusing atheists, amusing atheists |
| 176 | rushdie islamic law, subject rushdie islamic, islamic genocide, islamic law, genocide caused atheism, subject islamic, islamic law organization, islamic genocide organization, rushdie islamic, islamic authority |

You might also observe that the output of this model is a regular document-topic matrix, and isn't ragged.
```python
print(doc_topic_matrix.shape)
# (1024, 6)
```
## API Reference

This is because this way the model has the same API, as other Turftopic models, and works the same way as the top2vec package, making migration easier.
### Abstract Base Class

## API Reference
::: turftopic.late.LateModel

### Encoder

Expand Down
23 changes: 19 additions & 4 deletions docs/model_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,31 @@ It is quite important that you choose the right topic model for your use case.

| Model | Summary | Strengths | Weaknesses |
| - | - | - | - |
| [Topeax](Topeax.md) | Density peak detection + Gaussian mixture approximation | Cluster quality, Topic quality, Stability, Automatic n-topics | Underestimates N topics, Slower, No inference for new documents |
| *Factorization Models* | | | |
| [KeyNMF](KeyNMF.md) | Keyword importance estimation + matrix factorization | Reliability, Topic quality, Scalability to large corpora and long documents | Automatic topic number detection, Multilingual performance, Sometimes includes stop words |
| [SensTopic(BETA)](SensTopic.md) | Regularized Semi-nonnegative matrix factorization in embedding space | Very fast, High quality topics and clusters, Can assign multiple soft clusters to documents, GPU support | Automatic n-topics is not too good |
| [$S^3$](s3.md) | Semantic axis discovery in embedding space | Fastest, Human-readable topics | Axes can be very unintuitive, Manual n-topics |
| *Probabilistic Models* | | | |
| [Topeax](Topeax.md) | Density peak detection + Gaussian mixture approximation | Cluster quality, Topic quality, Stability, Automatic n-topics | Underestimates N topics, Slower, No inference for new documents |
| [GMM](GMM.md) | Soft clustering with Gaussian Mixtures and soft-cTF-IDF | Reliability, Speed, Cluster quality | Manual n-topics, Lower quality keywords, [Curse of dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality) |
| [vSLDA](vSLDA.md) | Reconstruct token/window/sentence representations using von-Mises components. | Contextual understanding, theoretical correctness | Manual n-topics, Can be slow with too granular representations |
| *Neural Models* | | | |
| [FASTopic](FASTopic.md) | Neural topic modelling with Dual Semantic-relation Reconstruction | High quality topics and clusters, GPU support | Very slow, Memory hungry, Manual n-topics |
| [$S^3$](s3.md) | Semantic axis discovery in embedding space | Fastest, Human-readable topics | Axes can be very unintuitive, Manual n-topics |
| [BERTopic and Top2Vec](clustering.md) | Embed -> Reduce -> Cluster | Flexible, Feature rich | Slow, Unreliable and unstable, Wildly overestimates number of clusters, Low topic and cluster quality |
| [AutoEncodingTopicModel](ctm.md) | Discover topics by generating BoW with a variational autoencoder | GPU-support | Slow, Sometimes low quality topics |
| *Clustering Models* | | | |
| [ClusteringTopicModel](clustering.md) | Embed -> Reduce -> Cluster | Flexible, many featuress | *Highly depends on construction* |
| [BERTopic](clustering.md) | UMAP+HDBSCAN+c-TF-IDF | Flexible, many features | Slow, Unstable, Wildly overestimates number of clusters, Low topic and cluster quality |
| [Top2Vec](clustering.md) | UMAP+HDBSCAN+semantic term importance | Flexible, many features, high quality topics | Slow, Unstable, Wildly overestimates number of clusters, Low cluster quality |
| [C-Top2Vec](clustering.md) | Top2Vec on contextual windows | High topic quality, contextual understanding | Slow, Unstable, Wildly overestimates number of clusters |
| *Bag-of-words models* | | | |
| [LDA](LDA.md) | Generative model of bag-of-words representations. | Reliable, well known limitations | Slow, low quality |
| [NMF](NMF.md) | Decomposes BoW representations using Non-negative Matrix Factorization. | Reliable, fast, well known limitations | Low quality |

</div>

Different models will naturally be good at different things, because they conceptualize topics differently for instance:

- `BERTopic`, `Top2Vec`, `GMM` and `Topeax` find **clusters** of documents and treats those as topics
- `BERTopic`, `Top2Vec`, `GMM` and `Topeax` find **clusters/components** of documents and treats those as topics
- `KeyNMF`, `SensTopic`, `FASTopic` and `AutoEncodingTopicModel` conceptualize topics as latent nonnegative **factors** that generate the documents.
- `SemanticSignalSeparation`($S^3$) conceptualizes topics as **semantic axes**, along which topics are distributed.

Expand All @@ -52,6 +63,10 @@ Some models are also capable of being used in a dynamic context, some can be fit
| **[GMM](GMM.md)** | :heavy_check_mark: | :x: | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
| **[AutoEncodingTopicModel](ctm.md)** | :heavy_check_mark: | :x: | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
| **[FASTopic](fastopic.md)** | :heavy_check_mark: | :x: | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
| [C-Top2Vec](clustering.md) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | :heavy_check_mark: | :x: |
| **[LDA](LDA.md)** | :heavy_check_mark: | :x: | :x: | :x: | :heavy_check_mark: | :x: | :x: |
| **[NMF](NMF.md)** | :heavy_check_mark: | :x: | :x: | :x: | :heavy_check_mark: | :x: | :x: |
| **[vSLDA](vSLDA.md)** | :heavy_check_mark: | :x: | :x: | :x: | :heavy_check_mark: | :x: | :x: |


## Model API Reference
Expand Down
Loading
Loading