Neural Network in C++ From Scratch and Backprop-Free Optimizers

Introduction

1. Features of the framework

2. Neural networks from scratch

A graph representing a neural network

3. Framework Design

Computation example

Second iteration reproduces the computation on mattmazur’s website

Building a Neural Network

Predicting a value

Backpropagation

4. Exotic backpropagation-free optimizer: ShakingTree optimizer

Theory

In practice

Loss on test set = f(seconds) on a dedicated machine
Best loss on test set = f(seconds) on a dedicated machine

Limitations

5. What I learned doing this

6. Github repository / code

We work on statistics, machine learning, AI and we want to bring Artificial General Intelligence into the real world — Follow us https://twitter.com/HyugenAI