Thoughts and Theory

Gaussian Process Regression is a remarkably powerful class of machine learning algorithms. Here, we introduce them from first principles.

Gaussian Process Regression (GPR) is a remarkably powerful class of machine learning algorithms that, in contrast to many of today’s state-of-the-art machine learning models, relies on few parameters to make predictions. Because GPR is (almost) non-parametric, it can be applied effectively to solve a wide variety of supervised learning problems, even when little data is available. With state-of-the-art automatic differentiation frameworks such as PyTorch and TensorFlow, it’s easier than ever to learn and apply GPR to a multitude of complex supervised learning tasks.


Getting Started

In this article, we explore how we can, and do, regularize and control the complexity of the models we learn through Bayesian prior beliefs.

I’m currently reading “How We Learn” by Stanislas Dehaene. First off, I cannot recommend this book enough to anyone interested in learning, teaching, or AI.

One of the main themes of this book is explaining the neurological and psychological bases of why humans are so good at learning things quickly and with great sample-efficiency, i.e. given only a limited amount of experience¹. One of Dehaene’s main arguments of why humans can learn so effectively is because we are able to reduce the complexity of models we formulate of the world. In accordance with the principle of Occam’s Razor², we find…


Thoughts and Theory

Gaussian Process Regression can be used to learn a multitude of periodic and aperiodic signals, such as those depicted in this figure. Photo by Ryan Stone on Unsplash

Unlimited Model Expression + Modern Computing

Ever wonder how you can create non-parametric supervised learning models with unlimited expressive power? Look no further than Gaussian Process Regression (GPR), an algorithm that learns to make predictions almost entirely from the data itself (with a little help from hyperparameters). Combining this algorithm with recent advances in computing, such as automatic differentiation, allows for applying GPRs to solve a variety of supervised machine learning problems in near-real-time.

In this article, we’ll discuss:

  1. A brief overview/recap of the theory behind GPR
  2. The types of problems we can use GPR to solve, and some examples
  3. How GPR compares to other supervised…


Thoughts and Theory

Photo by Ben White on Unsplash

Teaching and learning are two of the most important skills we can cultivate to better ourselves and those around us. While we may think of these skills as ones that only apply while we’re in school, in this article, I hope to illustrate how important it is to actively use these skills every day for the rest of your life.

Like it or not, we are always teaching and learning — but the impacts, positive or negative, we leave on ourselves and others through these activities depend largely on our skills in these abilities [1]. …


Great article - thank you for sharing and for thoughtfully motivating this exciting topic.

Thought I'd share another paper on neural network pruning from MIT CSAIL: https://arxiv.org/abs/1911.07412


Photo by Living Smarter on Unsplash

What is Data Fusion?

Data fusion, in the abstract sense, refers to combining different sources of information in intelligent and efficient ways such that the system processing the data performs better than had it just been given a single data source.

In this article, we will discuss how and why data fusion is leveraged for a variety of intelligent applications, specifically for self-driving cars. Then, we’ll dive into a specific case study on “sparse” data fusion for self-driving cars to see how data fusion is used in action.

High-Level Idea of Data Fusion: If I have two or more sources of data, and each…


Making Sense of Big Data

Photo by Jason Leung on Unsplash

Despite recent incredible algorithmic advances in the field, deep reinforcement learning (DRL) remains notorious for being computationally expensive, prone to “silent bugs”, and difficult to tune hyperparameters. These phenomena make running high-fidelity, scientifically-rigorous reinforcement learning experiments paramount.

In this article, I will discuss a few tips and lessons I’ve learned to mitigate the effects of these difficulties in DRL — tips I never would have learned from a reinforcement learning class. …


Hands-on Tutorials

Kriging [1], more generally known as Gaussian Process Regression (GPR), is a powerful, non-parametric Bayesian regression technique that can be used for applications ranging from time series forecasting to interpolation.


Recent advances in TensorFlow and reinforcement learning environments, such as those available through OpenAI Gym and the DeepMind Control Suite, have allowed for rapid prototyping, experimentation, and deployment of reinforcement learning applications across many domains.

TensorFlow-Agents, a TensorFlow-2-based reinforcement learning framework, is a high-level API for training and evaluating a multitude of reinforcement learning policies and agents. It enables fast code iteration, with good test integration and benchmarking¹.

Image Source

This article illustrates the application of tf_agents to Multi-Agent Reinforcement Learning (MARL) problems. In this article, we apply tf_agents to our novel, multi-agent variant of OpenAI Gym’s CarRacing-v0 environment. Our implementation of…


Background and Motivation

OpenAI Gym¹ environments allow for powerful performance benchmarking of reinforcement learning agents. In this article, we introduce a novel multi-agent Gym environment, MultiCarRacing-v0, that augments the original Gym CarRacing-v0 environment. This augmented environment can be used for evaluating any deep multi-agent reinforcement learning agent that learns from pixels.

Our implementation of this MultiCarRacing-v0 environment can be found here, and is used for evaluation in our recent paper “Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space” (see BibTex citation below).

Race segments from trained agents in our MultiCarRacing-v0 environment.

This environment augments the original GymCarRacing-v0 environment through the following features:

  1. Multi-car races: Our environment supports…

Ryan Sander

Graduate Research Assistant @ MIT CSAIL, Tutor, Dark Roast Coffee Drinker, GitHub: https://github.com/rmsander/, LinkedIn: https://www.linkedin.com/in/rmsander

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store