What is reinforcement learning? It is about to let machines learn by experimenting. As it is explained in the magazine, reinforcement learning copies a very simple principle from nature. The psychologist Edward Thorndike documented it more than 100 years ago. Thorndike placed cats inside boxes from which they could escape only by pressing a lever. After a considerable amount of pacing around and meowing, the animals would eventually step on the lever by chance. After they learned to associate this behavior with the desired outcome, they eventually escaped with increasing speed.

Reinforcement learning algorithms can help, for example, to improve the "driving skills" of self-driving cars. Today’s driverless vehicles often falter in complex situations that involve interacting with human drivers, such as traffic circles or four-way stops. AI engineers can collaborate with simulation engineers to integrate a digital model of the driverless car in a simulated environment, replicating the most complex traffic situations without any risk for people and real traffic. In the virtual space, the control software can perform the maneuvers over and over altering its instructions a little in each attempt. Applying deep learning techniques, the system can extract the patterns from the best performed maneuvers, learning from experimentation.

We see then that high fidelity simulation is going to be a key lever to reinforcement learning but, what kind of simulation we need to do an effective learning? In a former article, I have already discussed about the many applications that simulation brings to the development of smart and connected devices, including the virtual experimentation and design of IoT's products. In this article, I was also discussing about the kind of simulation that it is needed to integrate in an effective way digital models of the new products in simulated environments: we need to work with Net-Centric and interoperable models and simulations. In the same way that the physical product evolves to the Internet of the Things, its digital model will need to evolve to an Internet of the Simulations or IoS, in which heterogeneous simulations of the diverse physical and cyber subsystems of the smart product can interoperate without restrictions.

But to evolve actual simulation products and solutions to the Internet of Simulations, several challenges needs to be addressed, especially in the technologies and architectures to integrate all kind of simulations and real system in a common virtual space. My company, Simware Solutions, has been investing in new simulation technologies for the IoS and the result is our Simware platform. We are also collaborating with lead research groups as the Distributed Systems and Services Research group of the University of Leeds in the development of new simulation architectures and technologies for the Internet of Simulations. Staff and researchers in DSS group are doing very interesting projects related to the application of the Internet of Simulations to the virtual design and experimentation of new vehicles, included self-driving vehicles.

If you are interested in collaborate with us to improve the integrability of cyber-physical systems in simulated environments, send me an email to jmlopez@simware.es or contact with my marketing people at marketing@simware.es.