In this paper review, we’ll talk about a data augmentation technique that generalizes the well-known and highly successful Mixup technique to nonlinear interpolation schemes.
Overview
Year Published: 2019 (ICML)
Topics covered: data augmentation, deep learning, autoencoders, generalization
Research Gap Filled: Improved the generative capabilities of neural network and autoencoder models through: (i) Randomized Mixup interpolation, (ii) Provable generalization guarantees using principles of Vicinal Risk Minimization (VRM).
Links:
Abridged Summary
The authors apply layer-randomized Mixup [2], a stochastic linear interpolation technique that improves the generalization of supervised machine learning models through the principle of Vicinal Risk Minimization (VRM) [2], to the hidden state representations of inputs in neural networks.
Concretely, Mixup is applied at a randomly chosen layer of a neural network k by combining two minibatches at the kth layer of the network. By applying Mixup at different layers throughout the network, learning smooth manifolds (a proven measure of generalization [1]) at differing levels of…
