Awesome LLM Long Context Modeling

Report Abuse

Basic Information

This repository is a curated, community-maintained collection of papers, blogs, technical reports and project links focused on long-context modeling for large language models. It aggregates recent and historical research across many subtopics that enable LLMs to handle extremely long inputs and outputs, including efficient attention patterns, KV-cache management and compression, length extrapolation, recurrent and state-space models, long-term memory, retrieval-augmented generation, context compression, long-chain-of-thought methods, long-text and long-video/image modeling, agentic systems, speculative decoding and benchmarks. The README is organized as a structured bibliography with topical sections, weekly and monthly paper updates, highlighted surveys and must-read items, and pointers to related repositories and evaluation suites. The repo serves as a centralized discovery hub and ongoing index of scholarly and engineering work around extending and evaluating LLM context windows.

Links

Categorization

App Details

Features
Comprehensive topical organization that groups papers and blog posts into clear areas such as efficient attention, recurrent transformers, state-space models, length extrapolation, long-term memory, RAG and ICL, agents, compression, long CoT, multimodal long-video/image work and benchmarks. Regularly updated news sections list weekly and monthly paper highlights. Includes annotated lists of survey papers, arXiv entries, technical reports, toolkits and example implementations where available. Provides many evaluation and benchmark collections for long-context LLMs and multimodal LLMs, curated reading lists and recommended citations, contributor and acknowledgement metadata, and a detailed table of contents for quick navigation. The README links to code projects, datasets and leaderboards referenced in the literature to support reproducibility and follow-up.
Use Cases
The repository helps researchers, engineers and students quickly find key literature and signal new directions in long-context LLM research. It saves time by collecting papers, surveys, benchmarks and blogs in one place and by grouping work by method and problem (e.g., KV cache compression, sparse/linear attention, state-space hybrids, long CoT, RAG). Maintainers highlight must-read surveys and weekly updates so users can stay current. The curated benchmarks and evaluation entries facilitate fair comparisons and help identify datasets, metrics and open challenges. Practitioners can discover implementation repositories and toolkits referenced in the literature, while authors and students can use the collection for literature reviews, citations and to identify gaps for new research.

Please fill the required fields*