Applied Machine Learning

This course covers the applied side of algorithmics in machine learning, with some deep learning and evolutionary algorithms thrown in as well.

Note: I update this repo on a fairly regular basis (the course is now in its third iteration).

Moshe Sipper’s Cat-a-log of Writings

Some Pros and Cons of Basic ML Algorithms, in 2 Minutes

Additional Resources (Cheat Sheets, Vids, Reads, Books, Software, Datasets)

---

Syllabus

❖ Python ❖ Artificial Intelligence ❖ Date Science ❖ Machine Learning Intro ❖ Scikit-learn ❖ ML Models ❖ Decision Trees ❖ Random Forest ❖ Linear Regression ❖ Logistic Regression ❖ Linear Models ❖ Regularization: Ridge & Lasso ❖ AdaBoost ❖ Gradient Boosting ❖ AddGBoost ❖ Ensembles ❖ XGBoost ❖ Comparing ML algorithms ❖ Gradient Descent ❖ SVM ❖ Bayesian ❖ Metrics ❖ Data Leakage ❖ Dimensionality Reduction ❖ Clustering ❖ Hyperparameters ❖ Some Topics in Probability ❖ Feature Importances ❖ Semi-Supervised Learning ❖ Neural Networks ❖ Deep Learning ❖ DL and AI ❖ Evolutionary Algorithms: Basics ❖ Evolutionary Algorithms: Advanced ❖ Large Language Models

---

Topics (according to order of instruction)

(: my colab notebooks, : my medium articles)

• Python

  • Python Programming
  • Python
  • PyCharm
  • Pandas
  • NumPy
  • NumPy
  • Numba
  • np.dot vs loop example

• Artificial Intelligence

  • Computing Machinery and Intelligence
  • Four Takeaways on the Race to Amass Data for A.I.

• Date Science

  • Data Science Landscape
  • Building the Machine Learning Model
  • 21 Most Important (and Must-know) Mathematical Equations in Data Science

• Machine Learning Intro

  • Machine Learning: history, applications, recent successes
  • How to avoid machine learning pitfalls
  • Top 10 machine learning algorithms with their use-cases
  • Introduction to machine learning
  • Train/Val/Test
  • Simple weather example
  • Iris knn (map)
  • kfold

• Scikit-learn

  • Machine learning with scikit-learn
  • A Minimal Example of Machine Learning (with scikit-learn)
  • 19 Most Elegant Sklearn Tricks I Found After 3 Years of Use
  • Toy datasets (sklearn)

• ML Models

  • Machine learning models

• Decision Trees

  • Decision trees
  • Decision trees

• Random Forest

  • Random Forests
  • Random Forests

• Linear Regression

  • Linear Regression
  • LinearReg
  • A Quick Summary of Linear Regression

• Logistic Regression

  • Logistic Regression
  • Logistic Regression
  • Cross-Entropy Loss
  • Linear Regression vs Logistic Regression in a Nutshell
  • LinVsLog
  • PolynomialFeatures

• Linear Models

  • Optimization of linear models

• Regularization: Ridge & Lasso

  • Ridge vs. Lasso
  • notes on regularization in ML

• AdaBoost

  • Adaptive Boosting

• Gradient Boosting

  • Gradient Boosting
  • Why tree gradients give you a boost

• AddGBoost

  • AddGBoost
  • Strong(er) Gradient Boosting

• Ensembles

  • Two’s Company, Three’s an Ensemble

• XGBoost

  • XGBoost

• Comparing ML algorithms

  • Comparing supervised learning algorithms
  • How to find the best performing Machine Learning algorithm
  • load_boston removal
  • Choosing the right estimator

• Gradient Descent

  • Gradient Descent
  • Least Squares
  • Least Squares
  • Stochastic Gradient Descent
  • Stochastic Gradient Descent Algorithm With Python and NumPy
  • Gradient Descent With Momentum

• SVM

  • Support Vector Machine
  • Plot different SVM classifiers in the iris dataset

• Bayesian

  • Multinomial Naive Bayes

• Metrics

  • sklearn.metrics
  • Confusion Matrix
  • Confusion Matrix & Metrics in a minute
  • Sensitivity and Specificity
  • Sensitivity and specificity
  • ROC and AUC
  • balanced accuracy
  • balanced accuracy
  • various metrics from CM

• Data Leakage

  • Data Leakage Basics, with Examples in Scikit-Learn
  • Could machine learning fuel a reproducibility crisis in science?

• Dimensionality Reduction

  • 11 Different Uses of Dimensionality Reduction
  • PCA
  • PCA
  • PCA vs LR
  • pca
  • t-SNE
  • tsne

• Clustering

  • Clustering
  • Hierarchical clustering
  • K-means clustering
  • kmeans
  • From Data to Clusters: When is Your Clustering Good Enough?

• Hyperparameters

  • Model Parameter vs. Hyperparameter
  • Hyperparameter tuning
  • Hyperparameter tuning
  • Optuna
  • optuna
  • Evaluating Hyperparameters in Machine Learning

• Some Topics in Probability

  • p-values
  • how to calculate p-values
  • p-hacking
  • Probability is not Likelihood
  • t-test
  • t-test
  • t-test vs p-value
  • scpipy ttest
  • chi-square
  • 17 Statistical Hypothesis Tests in Python
  • permutation test (AddGBoost + code)
  • The Permutation Test: A Data Scientist’s BFF

• Feature Importances

  • Decision Tree
  • Feature importances with a forest of trees
  • Explainable AI explained! | #4 SHAP
  • SHAP Values Explained

• Semi-Supervised Learning

  • Boosting vs. semi-supervised learning
  • LabelPropagation

• Neural Networks

  • Neural networks
  • Neural networks
  • Tinker With a Neural Network in Your Browser

• Deep Learning

  • Neural Networks with À La Carte Selection of Activation Functions
  • PyTorch
  • PyTorch
  • Double Descent
  • Overparameterization, Backpropagation, Alimentation: Them and Us
  • No, Kernels & Filters Are Not The Same
  • conv demo
  • convolution
  • A simple image convolution
  • Implementing Image Processing Kernels from scratch using Convolution in Python
  • Introduction to image generation (diffusion)
  • Loss is Boss and other articles in the DL section

• DL and AI

  • Growth of AI computing
  • AI move from Academia
  • Artificial General Intelligence Is Not as Imminent as You Might Think

• Evolutionary Algorithms: Basics

  • Evolutionary Computation
  • How to Build a Genetic Algorithm from Scratch in Python with Just 33 Lines of Code
  • Evolutionary Algorithms, Genetic Programming, and Learning
  • Tiny GA
  • EC-KitY
  • Genetic Programming slides
  • Tiny GP
  • GP Tutorial
  • GP vids
  • GP slides
  • Natural Computing

• Evolutionary Algorithms: Advanced

  • Multi-Objective Optimization
  • Schema theorem
  • Linear GP
  • Cartesian GP
  • Grammatical Evolution
  • Coevolutionary Computation
  • New Pathways in Coevolutionary Computation
  • Novelty search
  • Humies
  • Evolutionary Art
  • Building Activation Functions for Deep Networks
  • Evolutionary Adversarial Attacks on Deep Networks

• Large Language Models

  • Introduction to large language models
  • A Tiny Large Language Model (LLM), Coded, and Hallucinating
  • Large Language Models from scratch
  • Large Language Models: Part 2
  • Scikit-LLM
  • Coding a ChatGPT Like Transformer From Scratch in PyTorch
  • Word Embeddings: Encoding Lexical Semantics
  • The Magic Behind Embedding Models
  • What are the query, key, and value vectors?
  • Unpacking the Query, Key, and Value of Transformers: An Analogy to Database Operations