A more parameter-efficient bottleneck for EfficientNet!

Linear Bottleneck with Efficient Channel Attention instead of Squeeze Excitation

Introduction

Linear Bottleneck

Linear Bottleneck (cf MobileNetV2 paper)

EfficientNet: Linear Bottleneck + Squeeze Excitation

EfficientNet solving the “attention problem”: linearbottleneck_se. A better representation with hard-sigmoid can be found in MobileNetV3 paper (page 3)
Our reproduction of EfficientNetB0 after ~50 epochs (source for red line)

EfficientNetECA: Linear Bottleneck + Efficient Channel Attention

ECA layer : https://arxiv.org/pdf/1910.03151.pdf
Validation Accuracy we got with EfficientNetECA

Tl;dr

Conclusion / Further works

More plots

Partial Top1Acc = f(n_images_seen) — with curve smoothing
Partial Top1Acc = f(time) — with curve smoothing

Addendums

Code

We work on statistics, machine learning, AI and we want to bring Artificial General Intelligence into the real world — Follow us https://twitter.com/HyugenAI