About

Lexora is a GPT-style language model built from scratch using PyTorch. It features a custom BPE tokenizer, a 15.8M-parameter decoder-only Transformer, GPU training, autoregressive text generation, and temperature, Top-K, and Top-P sampling. Trained on a multi-book literary corpus with 11.41 validation perplexity.

Resources