Learning how to make a thing learnable is harder than learning it

Here’s something pretty crazy: there are around 10^28 atoms in the human body. An astromically tiny percentage of arrangements of 10^28 atoms in 3D space correspond to a viable human. Most arrangements of atoms would just melt into a puddle!

More

Learning without a gradient (part 1)

Learning using gradient-based methods is a lot like trying to find the top of Mount Everest starting from a random point on earth, while blindfolded, by repeatedly sampling only the altitude and slope of the ground you’re standing on, and then using that information to decide where to teleport next.

More

What is really the root cause of the event-stream vulnerability?

I was trying to make sense of the event-stream vulnerability. My take:

More

Evolution is a much more interesting learning algorithm than most people realize

Is evolution just another hill-climbing-like algorithm where the gradient is inefficiently estimated via sampling rather than direct differentiation? No. The usual formulations of evolutionary computation (EC) capture little of the algorithmically interesting aspects of evolution, which has led many researchers to wonder (perhaps correctly) whether there’s really any point to EC compared to more rationalized and efficient approaches to ML. This reaction is a bit like deeming human powered-flight a fruitless endeavor after seeing the first ineffectual ornithopers. Let’s not throw the baby out with the bathwater!

More

The Fermi paradox, and is "intelligence" the inevitable byproduct of any long-running evolutionary process?

I read a fascinating book recently on the Fermi Paradox, which asks the question: if there are supposedly millions of other civilizations in the galaxy, why haven’t we detected any real evidence of their existence? One class of solutions says: what happened on Earth is a fluke. We’re alone in the galaxy. Gosh, that would be disappointing! Let’s have a closer look at one piece of the puzzle.

More