Probably Approximately Correct

It's all a simulation anyway.


The cover of Probably Approximately Correct by Leslie Valiant

It was Charles Darwin who taught us the foundational and the now widely accepted concept of evolution. Stating that all life has, over time, descended from a common ancestor through a process of natural selection, much like how we selectively bred wolves to produce hundreds of the domesticated dog breeds we see today. The struggle of existence has only allowed the best of us to produce offspring, resulting in a natural process of selection, where only the fittest among us will survive. This evolutionary process tells us why, over millennia, the Giraffes got taller, Sloths got slower, Alligators got scarier and we doubly wise men became (kinda) smarter.

As life continues to evolve, it also continues to grow in its complexity. If you want to succeed in an interview, or choose a life partner, or (if you're like my wife) decide on a restaurant, you can be sure there is no equation that could guarantee you success. Even if we could collect all the relevant information we needed to answer these questions (assuming we even knew what that relevant information was) there's still no sure-fire way to combine the information in a way to yield an answer for us. Yet, every day, billions of us answer these questions as we go about our lives. It's these "guesses," "hunches," and "gut instincts" that Dr. Leslie Valiant, a world-renowned theoretical computer scientist, and his book are interested in.

Historically, computers were only suited for solving the theory-full questions in our lives, like modeling fluid flows or calculating the orbits of satellites using Newton's laws. These theory-full questions, typically mathematical or scientific theories, like Einstein's famous equation E=mc2, have clearly defined inputs and instructions telling us how they operate to accurately calculate a solution to our questions. More recently though, computers have been growing "softer" skills, ones that require them to answer more theory-less questions, which have no well-defined formula to calculate, like deciding on a movie we'd like to watch, or how to safely drive us home from the pub.

Computers today learn to answer these theory-less questions much like how we learned about our world in our infancy (or adolescence for some), by drawing general lessons from a particular experience to better answer the next question life throws. Unlike the theory-full algorithms, expertise is not given from the designer of the equation to the student or computer, but extracted from the experiences gained from experimentation or through evolution by the student. These are the equations for learning, rather than answer using E=mc2 absolutes, these algorithms produce probably approximately correct answers.

Probably Approximately Correct aims to explain the current state and future goals of our attempts to find a mathematical definition of the learning algorithms that we have, through evolution, used to prosper in this (sometimes excessively) complex world. With the minuscule small hint of our current understanding of these learning algorithms, we have revolutionized and transformed our world (mostly) for the better. Through the power of learning algorithms that can organize the endless supply of websites, the least educated person today has immeasurably more knowledge at their fingertips than the most educated had just a few decades ago. With a more thorough understanding of these learning algorithms, we stand to reshape our civilization even more dramatically in the decades to come.