This page contains all the info you need to develop your models using Shkyera Grad.
+
+Types
+
Almost all of the classes in Shkyera Grad are implemented using templates. To simplify creation of these objects, we introduced a standard way to instantiate objects with floating-point template parameters, i.e.
+
Linear32 = Linear<float>
+
Optimizer32 = Optimizer<Type::float32>>
+
Loss::MSE64 = Loss::MSE<double>
+
Adam64 = Adam<Type::f64>
+
+
{Class}32 = {Class}<Type::float32> = {Class}<float>
+
{Class}64 = {Class}<Type::float64> = {Class}<double>
+
+Layers
+
Here's a full list of available layers:
+
auto linear = Linear32::create(inputSize, outputSize);
+
auto dropout = Dropout32::create(inputSize, outputSize, dropoutRate);
+
+Optimizers
+
These are all implemented optimizers:
+
auto simple = Optimizer32(network->parameters(), learningRate);
+
auto sgdWithMomentum = SGD32(network->parameters(), learningRate, momentum = 0.9);
+
auto adam = Adam32(network->parameters(), learningRate, beta1 = 0.9, beta2=0.999, epsilon=1e-8);
+
+Loss functions
+
Optimization can be performed according to these predefined loss functions:
+
auto L1 = Loss::MAE32;
+
auto L2 = Loss::MSE32;
+
auto crossEntropy = Loss::CrossEntropy32;
+
+Generic Training Loop
+
Simply copy-pase this code to quickly train your network:
+
using T = Type::float32;
+
+
auto optimizer = Adam<T>(network->parameters(), 0.05);
+
auto lossFunction = Loss::MSE<T>;
+
+
for (size_t epoch = 0; epoch < 100; epoch++) {
+
auto epochLoss = Value<T>::create(0);
+
+
optimizer.reset();
+
for (size_t sample = 0; sample < xs.size(); ++sample) {
+
Vector<T> pred = network->forward(xs[sample]);
+
auto loss = lossFunction(pred, ys[sample]);
+
+
epochLoss = epochLoss + loss;
+
}
+
optimizer.step();
+
+
auto averageLoss = epochLoss / Value<T>::create(xs.size());
+
std::cout << "Epoch: " << epoch + 1 << " Loss: " << averageLoss->getValue() << std::endl;
+
}
+
Hello! Let's get right into it. By the end of this guide, you will have created and trained your first neural in Shkyera Grad!
-
+
Setup
This is easy, Shkyera Grad is a header-only library, so simply clone the repositoryu into your project:
git clone https://github.com/fszewczyk/shkyera-grad.git
@@ -98,7 +98,7 @@
#include "shkyera-grad/include/ShkyeraGrad.hpp"
Now, you can use all the features of this small engine.
- Note
- Shkyera Grad is tested in C++17. Make sure your compiler supports this version.
-
+
Scalars
Internally, Shkyera Grad always operates on individual scalars. For most purposes, you do not need to deal with them directly, but it's nice to understand how they work. Each scalar is wrapped inside a Value class. However, you should never instantiate objects of this type yourself. Instead, you should use the provided interface in the following way.
@@ -138,7 +138,7 @@
a->getGradient();
b->getGradient();
If you want some refreshment on derivatives, check out this wonderful video.
-
+
Vector
Multiple scalars can be grouped together in a Vector to simplify operating on them. Input to any Module (more on them later) is a Vector. This abstraction provides some functionality that allows you to compute, for example a dot product.
@@ -157,7 +157,7 @@
auto d = a.dot(b)
d->backward();
Vectors are very useful since this is the way both the input and output data is represented. Each sample consits of an input Vector and a target output Vector.
-
+
Sequential
Nice! You got the basics! Let's build a network. The best way to create a model is through the use of the Sequential interface. Each function that transforms an input Vector into some output Vector is implemented as a Module. This includes neural layers as well as activation functions. Hey, even Sequential is a Module. This allows for creating complex strctures while using a common, simple interface.
You can create your first neural network using SequentialBuilder in the following way.
@@ -170,7 +170,7 @@
.build();
@warn Remember that subsequent layers have to have matching input and output sizes.
- Note
- For the full list of available layers and activation functions, check out the Cheat Sheet.
-
+
Training
To train our network, we need to define an Optimizer that will optimizer the parameters as well as the Loss function that we will minimize. Shkyera Grad comes with a set of well-known optimizers and loss functions. Again, check out the Cheat Sheet for a complete list.
@@ -192,7 +192,7 @@
Loss::MSE<T>
Loss::CrossEntropy<T>
They are implemented as lambda functions, not as objects, so they do not need to be instantiated.
-
+
Learning XOR
XOR (Exclusive OR) is a simple Boolean function that maps two values two one:
@@ -207,7 +207,7 @@
| 1 | 1 | 0 |
-
+
Let's define our dataset.
Here, we basically pase the table above into Vectors.
std::vector<Vec32> xs;
@@ -218,7 +218,7 @@
xs.push_back(Vec32::of({1, 0})); ys.push_back(Vec32::of({1}));
xs.push_back(Vec32::of({0, 1})); ys.push_back(Vec32::of({1}));
xs.push_back(Vec32::of({1, 1})); ys.push_back(Vec32::of({0}));
-
+
Neural Network
We define a simple neural network to predict this function. Our network has a total of three layers. It is a bit of an overkill for this task, but we will use it for learning purposes.
auto network = SequentialBuilder<Type::float32>::begin()
@@ -229,7 +229,7 @@
.add(Linear32::create(5, 1))
.add(Sigmoid32::create())
.build();
-
+
After the training, let's inspect how our network behaves.
Nice! After compiling and running this code (make sure to use C++17), you should see something like this:
Epoch: 1 Loss: 0.263062
diff --git a/navtreedata.js b/navtreedata.js
index e9dd728..18a8e2a 100644
--- a/navtreedata.js
+++ b/navtreedata.js
@@ -25,20 +25,26 @@
var NAVTREE =
[
[ "Shkyera Grad", "index.html", [
- [ "Cheatsheet", "md_docs_tutorials_Cheatsheet.html", null ],
- [ "Get Started", "md_docs_tutorials_GetStarted.html", [
+ [ "Cheatsheet", "md_docs_tutorials_Cheatsheet.html", [
[ "Usage", "index.html#autotoc_md0", null ],
- [ "Setup", "md_docs_tutorials_GetStarted.html#autotoc_md3", null ],
- [ "Scalars", "md_docs_tutorials_GetStarted.html#autotoc_md4", null ],
- [ "Vector", "md_docs_tutorials_GetStarted.html#autotoc_md5", null ],
- [ "Sequential", "md_docs_tutorials_GetStarted.html#autotoc_md6", null ],
- [ "Training", "md_docs_tutorials_GetStarted.html#autotoc_md7", null ],
- [ "Learning XOR", "md_docs_tutorials_GetStarted.html#autotoc_md8", [
- [ "Let's define our dataset.", "md_docs_tutorials_GetStarted.html#autotoc_md9", null ],
- [ "Neural Network", "md_docs_tutorials_GetStarted.html#autotoc_md10", null ],
- [ "Training Loop", "md_docs_tutorials_GetStarted.html#autotoc_md11", null ],
- [ "Verifying the results", "md_docs_tutorials_GetStarted.html#autotoc_md12", null ],
- [ "Results", "md_docs_tutorials_GetStarted.html#autotoc_md13", null ]
+ [ "Types", "md_docs_tutorials_Cheatsheet.html#autotoc_md2", null ],
+ [ "Layers", "md_docs_tutorials_Cheatsheet.html#autotoc_md3", null ],
+ [ "Optimizers", "md_docs_tutorials_Cheatsheet.html#autotoc_md4", null ],
+ [ "Loss functions", "md_docs_tutorials_Cheatsheet.html#autotoc_md5", null ],
+ [ "Generic Training Loop", "md_docs_tutorials_Cheatsheet.html#autotoc_md6", null ]
+ ] ],
+ [ "Get Started", "md_docs_tutorials_GetStarted.html", [
+ [ "Setup", "md_docs_tutorials_GetStarted.html#autotoc_md8", null ],
+ [ "Scalars", "md_docs_tutorials_GetStarted.html#autotoc_md9", null ],
+ [ "Vector", "md_docs_tutorials_GetStarted.html#autotoc_md10", null ],
+ [ "Sequential", "md_docs_tutorials_GetStarted.html#autotoc_md11", null ],
+ [ "Training", "md_docs_tutorials_GetStarted.html#autotoc_md12", null ],
+ [ "Learning XOR", "md_docs_tutorials_GetStarted.html#autotoc_md13", [
+ [ "Let's define our dataset.", "md_docs_tutorials_GetStarted.html#autotoc_md14", null ],
+ [ "Neural Network", "md_docs_tutorials_GetStarted.html#autotoc_md15", null ],
+ [ "Training Loop", "md_docs_tutorials_GetStarted.html#autotoc_md16", null ],
+ [ "Verifying the results", "md_docs_tutorials_GetStarted.html#autotoc_md17", null ],
+ [ "Results", "md_docs_tutorials_GetStarted.html#autotoc_md18", null ]
] ]
] ]
] ]
diff --git a/navtreeindex0.js b/navtreeindex0.js
index d3abc05..ee3001c 100644
--- a/navtreeindex0.js
+++ b/navtreeindex0.js
@@ -3,17 +3,22 @@ var NAVTREEINDEX0 =
"index.html":[],
"index.html#autotoc_md0":[0],
"md_docs_tutorials_Cheatsheet.html":[0],
+"md_docs_tutorials_Cheatsheet.html#autotoc_md2":[0,1],
+"md_docs_tutorials_Cheatsheet.html#autotoc_md3":[0,2],
+"md_docs_tutorials_Cheatsheet.html#autotoc_md4":[0,3],
+"md_docs_tutorials_Cheatsheet.html#autotoc_md5":[0,4],
+"md_docs_tutorials_Cheatsheet.html#autotoc_md6":[0,5],
"md_docs_tutorials_GetStarted.html":[1],
-"md_docs_tutorials_GetStarted.html#autotoc_md10":[1,6,1],
-"md_docs_tutorials_GetStarted.html#autotoc_md11":[1,6,2],
-"md_docs_tutorials_GetStarted.html#autotoc_md12":[1,6,3],
-"md_docs_tutorials_GetStarted.html#autotoc_md13":[1,6,4],
-"md_docs_tutorials_GetStarted.html#autotoc_md3":[1,1],
-"md_docs_tutorials_GetStarted.html#autotoc_md4":[1,2],
-"md_docs_tutorials_GetStarted.html#autotoc_md5":[1,3],
-"md_docs_tutorials_GetStarted.html#autotoc_md6":[1,4],
-"md_docs_tutorials_GetStarted.html#autotoc_md7":[1,5],
-"md_docs_tutorials_GetStarted.html#autotoc_md8":[1,6],
-"md_docs_tutorials_GetStarted.html#autotoc_md9":[1,6,0],
+"md_docs_tutorials_GetStarted.html#autotoc_md10":[1,2],
+"md_docs_tutorials_GetStarted.html#autotoc_md11":[1,3],
+"md_docs_tutorials_GetStarted.html#autotoc_md12":[1,4],
+"md_docs_tutorials_GetStarted.html#autotoc_md13":[1,5],
+"md_docs_tutorials_GetStarted.html#autotoc_md14":[1,5,0],
+"md_docs_tutorials_GetStarted.html#autotoc_md15":[1,5,1],
+"md_docs_tutorials_GetStarted.html#autotoc_md16":[1,5,2],
+"md_docs_tutorials_GetStarted.html#autotoc_md17":[1,5,3],
+"md_docs_tutorials_GetStarted.html#autotoc_md18":[1,5,4],
+"md_docs_tutorials_GetStarted.html#autotoc_md8":[1,0],
+"md_docs_tutorials_GetStarted.html#autotoc_md9":[1,1],
"pages.html":[]
};