AI research trends - Revision history

KevinYager: /* Context Length */

2026-03-23T17:11:22Z

‎Context Length

2026-03-23T17:08:40Z

‎Context Length

2026-03-02T16:13:41Z

‎Daydreaming, brainstorming, pre-generation

2026-02-28T19:59:19Z

‎Updating Weights at Inference-time

2026-02-28T19:59:10Z

‎LLM Weights Memory

2026-02-18T15:25:22Z

‎Reviews

2026-02-16T16:09:56Z

‎Continual Learning

2026-02-15T20:45:27Z

‎Context Remaking

2026-02-15T18:32:01Z

‎Retrieval beyond RAG

2026-02-12T14:30:34Z

‎Memory

@@ Line 49: / Line 49: @@
 * 2025-12-04: Google [https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/ Titans/Miras] 10M
 * 2025-12-13: [https://arxiv.org/abs/2512.12167 Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings]
-* 2026-03-18: [https://github.com/EverMind-AI/MSA MSA: Memory Sparse Attention] 100M
+* 2026-03-18: [https://github.com/EverMind-AI/MSA/blob/main/paper/MSA__Memory_Sparse_Attention_for_Efficient_End_to_End_Memory_Model_Scaling_to_100M_Tokens.pdf MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokens] ([https://github.com/EverMind-AI/MSA code]) 100M
 ==Extended Context==

@@ Line 49: / Line 49: @@
 * 2025-12-04: Google [https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/ Titans/Miras] 10M
 * 2025-12-13: [https://arxiv.org/abs/2512.12167 Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings]
 ==Extended Context==

@@ Line 166: / Line 166: @@
 =Daydreaming, brainstorming, pre-generation=
-* Gwern: [https://gwern.net/ai-daydreaming Daydreaming]
+* 2025-04: [https://arxiv.org/abs/2504.13171 Sleep-time Compute: Beyond Inference Scaling at Test-time]
 * 2026-02: [https://arxiv.org/abs/2602.01689 What LLMs Think When You Don't Tell Them What to Think About?]

@@ Line 102: / Line 102: @@
 * 2025-01: [https://arxiv.org/abs/2501.06252 Transformer<sup>2</sup>: Self-adaptive LLMs]
 * 2025-08: [https://arxiv.org/abs/2508.14143 Beyond Turing: Memory-Amortized Inference as a Foundation for Cognitive Computation]
 ==Parameters as Tokens==

@@ Line 15: / Line 15: @@
 * 2026-01: [https://developer.nvidia.com/blog/reimagining-llm-memory-using-context-as-training-data-unlocks-models-that-learn-at-test-time/ Reimagining LLM Memory: Using Context as Training Data Unlocks Models That Learn at Test-Time] (Nvidia)
 * 2026-01: [https://arxiv.org/abs/2601.02151 Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting]
 ==Context Length==

@@ Line 4: / Line 4: @@
 =Memory=
 ==Reviews==
 * 2026-01: [https://arxiv.org/abs/2601.09113 The AI Hippocampus: How Far are We From Human Memory?]

@@ Line 92: / Line 92: @@
 * 2025-11: [https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/ Introducing Nested Learning: A new ML paradigm for continual learning]
 * 2026-01: [https://arxiv.org/abs/2601.16175 Learning to Discover at Test Time]
 * 2026-02: [https://arxiv.org/abs/2602.07755 Learning to Continually Learn via Meta-learning Agentic Memory Designs]

@@ Line 50: / Line 50: @@
 ==Context Remaking==
 * 2025-08: [https://blog.plasticlabs.ai/blog/Memory-as-Reasoning Memory as Reasoning (Memory is Prediction)]
 * 2025-09: [https://arxiv.org/abs/2509.25140 ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory]

@@ Line 63: / Line 63: @@
 * 2024-12: [https://arxiv.org/abs/2412.11919 RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation]
 * 2025-03: Microsoft: [https://www.microsoft.com/en-us/research/blog/introducing-kblam-bringing-plug-and-play-external-knowledge-to-llms/ Introducing KBLaM: Bringing plug-and-play external knowledge to LLMs]
 * 2025-07: [https://arxiv.org/pdf/2507.07957 MIRIX: Multi-Agent Memory System for LLM-Based Agents] ([https://mirix.io/ mirix])
 * 2025-08: [https://arxiv.org/abs/2508.16153 Memento: Fine-tuning LLM Agents without Fine-tuning LLMs]