Dropout Layer and Moduliarizing Activations

[fszewczyk]

Dropout Layer

Introduced a Linear Dropout layer:

layerf32 = Dropout32::create(15, 5, 0.2);
layerf32 = Dropout<Type::float32>(15, 5, 0.2);
layerf32 = Dropout<float>(15, 5, 0.2);

layerf64 = Dropout64::create(15, 5, 0.2);
layerf64 = Dropout<Type::float64>(15, 5, 0.2);
layerf64 = Dropout<double>(15, 5, 0.2);

Modularizing Activations

Now each Activation is an insstance of a Module class which makes it easier to create sequential models, e.g.

auto mlp = SequentialBuilder<Type::float32>::begin()
                .add(Linear32::create(2, 15))
                .add(ReLU32::create())
                .add(Dropout32::create(15, 5, 0.2))
                .add(ReLU32::create())
                .add(Linear32::create(5, 1))
                .add(Sigmoid32::create())
                .build();