nn_architecture.md

Content

Optimizing Neural Network Architectures (Growing)

1. LIFELONG LEARNING WITH DYNAMICALLY EXPANDABLE NETWORKS paper
2. Increasing Depth of Neural Networks for Life-long Learning paper
3. GROWN: GRow Only When Necessary for Continual Learning paper
4. Self-Expanding Neural Networks paper
5. Progressive Neural Network paper
6. Non-Parametric Neural Network paper
7. LEARNING TO GROW PRETRAINED MODELS FOR EFFICIENT TRANSFORMER TRAINING paper
8. Progressive Growing of GANs paper

Modular Growing or PEFT

1. LORA
2. Adapters
3. QLora
4. Laser
5. DDPM

Recurrence vs. deep neural networks

1. RWKV
2. SSMs
3. Mamba
4. Recurrent Interface Networks

Course on Continual Learning

1. https://owll-lab.com/teaching/cl_lecture/