Neural Network in C++ From Scratch and Backprop-Free Optimizers

Image for post
Image for post

Introduction

1. Features of the framework

2. Neural networks from scratch

Image for post
Image for post
A graph representing a neural network

3. Framework Design

Computation example

Image for post
Image for post
Second iteration reproduces the computation on mattmazur’s website

Building a Neural Network

Image for post
Image for post
Image for post
Image for post

Predicting a value

Image for post
Image for post

Backpropagation

Image for post
Image for post

4. Exotic backpropagation-free optimizer: ShakingTree optimizer

Theory

In practice

Image for post
Image for post
Loss on test set = f(seconds) on a dedicated machine
Image for post
Image for post
Best loss on test set = f(seconds) on a dedicated machine

Limitations

5. What I learned doing this

6. Github repository / code

Support us 💙

We work on statistics, machine learning, AI and we want to bring Artificial General Intelligence into the real world — Website: https://www.hyugen.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store