Facebook creates a "mirror world" for learning artificial intelligence

Dreams of a robot-butler seem close -it seems, here, stretch out your hand, and he rushes to open the door in front of you and wash your socks. But first we need to train the robots to move in the human environment. For this, Facebook has created an incredibly realistic simulator for teaching artificial intelligence any kind of indoor action.

Virtual environments - from Atari games to simulatorsself-driving cars have become a leading tool for teaching cars how to interact with their environment. One of the main reasons is that such environments give us complete control over the environment of the agent. The real world is too crowded with a huge number of variables that need to be considered, but in the simulation we not only know all the variables - we can customize them at will.

It costs a lot. The controlled world of simulation is very different from the environment in which the physical robot will subsequently function, so agents have to be practiced thousands or millions of times to get it right. Virtual environments can be scrolled hundreds of times faster than in real time, and the parallel operation of hundreds of experiments requires only a few computer chips, not hundreds of new robots and test environments.

A new solution to this problem is thatto create an increasingly realistic digital environment. The bottom line is that the closer we reproduce the real world in a simulation, the easier it will be for the AI ​​to transfer what it studies to the real world. This is exactly the logic behind Facebook's new Replica database set: a photorealistic 3D mirror for various interior spaces, including an apartment and a retail store.

Facebook's Replica: How to Teach Artificial Intelligence

Data for Replica was extracted directly.from the real world using a specially designed video camera that combines HD video, high-precision depth data and a simultaneous localization and mapping system (SLAM) that records where all these measurements were taken from.

This approach allows you to reflect even reflectivesurfaces and complex textures that are often difficult to reproduce. In addition, Facebook engineers carefully tagged everything in the data set, including the color coding of various categories of objects. After that, all data can be downloaded to the Habitat simulator platform to create a three-dimensional environment that the AI ​​can explore.

The company not only helps AI with navigation, italso hopes that the data set can improve the technology of augmented reality (AR). If AI can better understand the physical spaces that people occupy, it would be better to combine digital projections with the real world.

Although Replica is not perfect, this is the most idealthe mirror of the human world that has ever been modeled. And Facebook almost certainly has plans to expand and enrich the data set. Hopefully this will bring us closer to the dreams of the robots-butlers.

Or not? Let's discuss in our chat in Telegram.