ResearchPapers.md

RESEARCH PAPERS

Theoretical:

• Neural Networks are Decision Trees
• Cross-Validation Bias due to Unsupervised Preprocessing
• The Forward-Forward Algorithm: Some Preliminary Investigations
• LoRA: Low-Rank Adaptation of Large Language Models (included here as it has applications beyond LLMs)
• Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Image:

• ViT related:

  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)

  • Emerging Properties in Self-Supervised Vision Transformers

  • Training data-efficient image transformers & distillation through attention

  • Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

  • A ConvNet for the 2020s (a CNN that implements several key components that contribute to the performance of Vision Transformers)

  • (CLIP) Learning Transferable Visual Models From Natural Language Supervision

• Diffusion related:

  • High-Resolution Image Synthesis with Latent Diffusion Models

  • Denoising Diffusion Probabilistic Models (DDPM)

  • Classifier-Free Diffusion Guidance

• Taming Transformers for High-Resolution Image Synthesis (VQGAN)

• Segment Anything (SAM)

• DINOv2: Learning Robust Visual Features without Supervision

• Bayesian Flow Networks

NLP

• Language Models are Few-Shot Learners (GPT-3)

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

• Training language models to follow instructions with human feedback

• Training Compute-Optimal Large Language Models (Chinchilla)

• The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

• LLaMA: Open and Efficient Foundation Language Models

• Toolformer: Language Models Can Teach Themselves to Use Tools

3D Rendering:

• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

• Highly accurate protein structure prediction with AlphaFold

Misc:

• Human-level play in the game of Diplomacy by combining language models with strategic reasoning

Contributing

Contributions are very welcome, please share back with the wider community (and get credited for it)!

Please have a look at the CONTRIBUTING guidelines, also have a read about our licensing policy.

---

Back to main page (table of contents)