attention-mechanism

LLMs-from-scratch

+406

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

RuVector

+28

RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.

how-to-train-your-gpt

+29

Build a modern LLM from scratch. Every line commented. Explained like we are five.

nndl/nndl

+51

邱锡鹏《神经网络与深度学习》第二版与通识版：电子书、章节目录、学习资源与勘误。

RWKV-LM

+7

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

llm-internals

+5

Learn LLM internals step by step - from tokenization to attention to inference optimization.

transformers-explained

+3

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

vit-pytorch

+2

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

ComfyUI-VDN-H3

+2

Video Delta Net hybrid attention for MiniMax-H3 as a native ComfyUI node.