Machine learning by project

This project is not aim to get fully access of ML, it has a low target. I want to have a sense of ML by this project for the following requirements:

1. Knowing every components of the learning process.
2. Knowing how to use and there is no need to figure out every formulas.
3. A minimal implement. That means I will use the most simple way to solve problems instead of the best way.

Includes

optimizer

The most easy one should be Gradient Descent: $$ θ_j​:=θ_j​−α \frac{∂​J(θ_0​,θ_1,...​)}{\partial \theta_j}$$ At each iteration j, one should simultaneously update the parameters $\theta_1, \theta_2,...,\theta_n$. Updating a specific parameter prior to calculating another one on the $j^{(th)}$ iteration would yield to a wrong implementation. And use batch gradient descent to train data( This method looks at every example in the entire training set on every step).

linear regression

The model is: $$ h_\theta(x^i)=\theta_0+\sum_j \theta_j x_j^i$$ Where $x^i$ is one sample of input from dataset. It's maybe a vector, so $x^i_j$ represents a scalar.
The cost function is: $$ J(θ_0​,θ_1,...,θ_i​)= \frac{1}{2m} \sum_{i=1}^{m}​(h_θ(x^i)−y^i​)^2 $$ Then calculate the gradient: $$ \frac{\partial J}{\partial \theta_j}=\frac{1}{m} \sum_{i=1}^{m}​(h_θ(x^i)−y^i​)*x^i_j$$

Data

Generated by function($y=\vec kx^i+b$).

实验下来发现很难用简单梯度下降算法在实际场景中应用。

1. 当样本变化，有些数据的方差大，那么它的梯度也就比较大。这就导致同样的a在一个样本中是合适的，另一个却不合适。

logistic regression

neural network