Deep Reinforcement Learning (DRL) techniques have allowed deep neural networks (DNN) based AI to move from theoretical concepts in the research lab to real-life, business solutions. Although tools such as the Microsoft Project Bonsai toolchain have tremendously streamlined, accelerated, and simplified the design, training, and deployment of Autonomous Systems AI agents, it does not mean they suddenly became simple click-and-deploy solutions.  

In this article, we will review Neal Analytics’ key learnings from helping customers successfully deploy DRL-trained AI agents with Project Bonsai across multiple industries and use cases such as production yield optimization, robotics, or logistics, to name a few. If you are not yet familiar with the core concepts behind DRL, please refer to this previous article. 

The 5–step process of Autonomous Systems AI agent projects 

To design, train, and deploy successful Autonomous Systems AI agents, project teams need to step through five defined, mostly consecutive, and always iterative steps. 

1. 1. Machine Teaching methods to help define the key solution characteristics and limits, such as the agent inputs and outputs or the reward function parameters 
   2. AI agent (aka the Project Bonsai “brain”) design 
   3. Simulation design and, in the case of an AI-based simulation, training 
   4. Brain training and testing (with the simulation) 
   5. And, finally, brain deployment and real-life testing

This process is obviously linear and numerous iterations will happen at each step and across several of those. For instance, the first training might show that the reward function is not effective and therefore that the team needs to go back – at least partly – to the Machine Teaching drawing board.  

Each of these steps will require attention to different considerations and therefore will require different skill sets. 

Machine Teaching considerations and skills

During this step, key considerations will be to build consensus among the subject matter experts (SMEs: process engineers, R&D, operators) on which parameters and real-life experience-based heuristics will be relevant for the AI agent.  

To achieve this, the project leadership needs to use a proven assessment method (along with the obvious consensus-building skills) and apply it, with the help of the SMEs, to their particular process.  

Brain design

From architecture selection to reward function building and curriculum design, in-depth knowledge of the Project Bonsai toolchain will be needed to design an effective brain.  

Simulation design

In addition to skills on the chosen platform or approach (custom model, simulation platform, AI-based simulation, etc.), the team will need to carefully define the simulation scope, or “operational universe”, and devise a strategy to ensure appropriate scaling. This is critical to ensure the simulations are fast enough for brain training to remain within acceptable ranges (i.e., not weeks!).  

Brain training

Much like the brain design step, in-depth Project Bonsai experience will be required to ensure the right potential training approaches are evaluated, training is scaled up (speed) and out (accuracy) as needed… but not too much. A “right-sized” approach will be more cost-efficient while ensuring the brain’s overall performance is appropriate to the business case.  

Deployment

Great data engineering skills are required to ensure a successful deployment. These skills will help the project team across performance validation, results measurement, and edge deployment. 

Key elements for a successful Autonomous System project 

As the project advance through those five steps, Neal Analytics has identified four key elements required for overall project success: 

1. 1. A well-defined development process (as depicted in the previous section) 
   2. Deep collaboration among all the appropriate stakeholders: process engineers, R&D, operators, system integration partner, and additional teams depending on the business case (IT, operations, etc.) 
   3. Integration of all the levels of business process expertise from the most theoretical levels (R&D) to the most pragmatic ones (operators) 
   4. A well-rounded team that is equipped with all the required skills for the project:

• • • Data engineering 
    • Process expertise 
    • Traditional AI/ML data science 
    • DRL expertise (in particular, using Microsoft Project Bonsai) 
    • Simulation development 
    • Edge & IoT 
    • And potentially additional skills depending on the project 

Getting started with an Autonomous Systems project 

With the right tool, such as Microsoft Project Bonsai, the right process (i.e, the 5 steps mentioned above), and the right set of skills and stakeholders, it is now possible to bring AI out of the research labs shadows and into real-life processes to bring tangible business benefits to a multitude of business processes across most industries. 

Intrigued to see if Autonomous Systems could help your business process? Neal Analytics can help! Contact us here to learn more. 

Reference material:

• More about Autonomous Systems
• Learn how DRL helps solve complex business problems (with examples)
• Learn more about Production Yield Optimization