Unlocking the potential of AI in manufacturing with machine teaching and deep reinforcement learning

Machine teaching and deep reinforcement learning offer new opportunities to manufacturers looking to optimize complex processes.

Unlocking the potential of AI in manufacturing with machine teaching and deep reinforcement learning

In most continuous manufacturing scenarios, equipment is typically controlled by low level controllers whose setpoints are given either by human operators or through Advanced Process Control (APC) systems in real time. This standard approach comes with inherent limitations that can keep a manufacturing line from reaching maximum efficiency and throughput while maintaining standard of quality.

Much of the challenge in optimizing control methods is often due to overall process complexity.

When operating conditions have a wide variation, APC systems can struggle to account for all scenarios. Additionally, human operators who rely on expertise to make quick decisions in real time may struggle to account for more than a handful of potentially dozens of control variables at any given moment. Further potential issues can be introduced as product standards change or additional rules and constraints are placed on a manufacturing process.

Fortunately, recent developments in a specific type of AI – deep reinforcement learning – now opens new opportunities to optimize production processes using self-learning strategies for AI. Machine teaching is one approach that makes reinforcement learning more accessible by leveraging the knowledge and insights of subject matter experts to help train the model.  

Deep reinforcement learning as potential solution for process control optimization

Deep reinforcement learning (DRL) is a machine learning method that involves the training of a software agent to learn how to act in an optimized way. To do this, the agent is trained via trial and error interactions with an environment. The learning is “reinforced” as it begins to correlate certain actions and calibrations with desired results. Reinforcement learning is a powerful tool, but it’s often difficult to effectively use as a solution for complex manufacturing processes.

machine learning vs machine teaching with project bonsai diagram

The key challenge is creating an accurate simulation that can be used for proper AI training. After all, you don’t want to train your software agent on the actual machinery. So, you need a robust simulator that can provide the same rules and processes from which the model can learn.

The sheer complexity of the physical and chemical reactions involved in some manufacturing processes are extremely difficult to simulate, especially through traditional approaches.

Traditional simulation approaches typically leverage the physics and chemical equations that govern a system to simulate the process and environment. For a complex system, using this approach can quickly become insurmountable for most without the support of scientific and modeling experts. Not only are experts required to build these models, but the models themselves can become too complex to run in a cost and time-effective way.

That’s hardly optimal.

Limitations of reinforcement learning and the need for accurate simulator

Historically, the challenge of creating an accurate simulator for reinforcement learning has created a barrier to entry for many manufacturers. While relying too heavily on human operators can lead to errors and unoptimized processes, training AI with reinforcement learning often required experts in physics and chemistry (and a good chunk of the budget).

However, through a novel, innovative approach and the new capabilities in computer vision technology, many of these longstanding issues are quickly becoming a thing of the past.

Instead of relying heavily on the traditional approach of building a simulator purely through a physics-based method, we can incorporate a more data-driven approach. With data, constructing a simulation that can effectively enable reinforcement learning for complex manufacturing processes is now possible – both technically and economically.

In other words, instead of modeling the physics of a systems through equations, we model its behavior with advanced AI models trained on real-life data.

This data-driven approach has already proven its capabilities with domains such as image tagging, speech recognition, and text translation where the switch from rule-based to AI models enables these systems to reach human-parity levels in the late 2010s.

Computer vision AI: Collecting data points to build a simulator

A key component of the data-driven approach is the availability of validated data in sufficient quantity that is representative of a varied set of operating conditions. Up until recently, validated data meant human-verified data. This model could not economically scale to the numbers (from hundreds of thousands to millions of data points) needed to train a deep learning model.

However, recent advancements in computer vision AI technology allows for cameras to examine manufacturing processes at extremely high resolution and use trained AI models to process the images for precise and accurate measurement in real time.

Once these camera models are tuned and validated against current standards of measurement, they can be used to build a large, accurate, and varied dataset. This dataset can be used to develop a robust simulator that is able to accurately represent a wide training environment for reinforcement learning to take place.

simulator training with ai vision at the edge

Building a simulator from data

Once the data has been collected and validated, a model is constructed to simulate the manufacturing process itself.

The model typically uses a traditional supervised machine learning approach with labelled training data. The specific modeling approach can vary depending on the specifics and unique dependencies of the manufacturing process.

If necessary, multiple models can even be trained and ensembled for use in simulating multi-step processes. Upon completion, the simulator’s performance is then validated against machine data in the real world.

Using a simulator for reinforcement learning

Reinforcement learning can be used effectively with a validated simulator that’s capable of providing an adequately wide training environment. Microsoft Project Bonsai is perfectly suited for the application of deep reinforcement learning in this case, by providing an Azure-based automated reinforcement learning platform.

Project Bonsai automates many of the tasks that traditional reinforcement learning models require, such as manual building, testing, and tuning of individual models and frameworks. The Bonsai framework also allows for the knowledge and insights from subject matter experts on the manufacturing process to be incorporated into the machine teaching strategy. By involving experts, operators, and stakeholders in the early development process, the simulation and reinforcement learning models can be more effectively built around the desired optimization goal. 

“Machine teaching in Bonsai allows engineers and subject matter experts to use advanced reinforcement learning techniques like curriculum learning and hierarchical decomposition to solve key problems in manufacturing. Examples include speeding up polymer production, ensuring high-quality extrusion despite unobserved variability in process inputs, and many others.” 

 Victor Shnayder, Principal PM, Autonomous Systems 

Benefits of autonomous systems

Manufacturers looking to solve longstanding process efficiency issues now have more options than ever to address them, and novel data-driven approaches with computer vision are opening the door to achieve what was once thought to be impossible. 

As systems become more autonomous, we can improve human operator efficiency. Quality standards can be better maintained and optimized while reducing the potential for human error. Processes become safer, more reliable, and ultimately more effective. 

Our greatest strengths as humans are based around creativity, the ability to adapt to unforeseen circumstances, and use any known pattern in any domain to derive new solutions. With autonomous systems, we can leverage what AI is good at – building models based on a defined set of data – and focus on higher-level, value-adding activities. That frees up our time (and brains) to more effectively handle out-of-scope situations, spot elements that may impact the control system in advance, validate using real-life data and personal experience, evaluate options proposed by the machine, and more.

Investing in the technology to enable these process innovations quickly returns value through decreased product waste, lower production costs, and higher quality product. AI in manufacturing is here to stay and it will eventually become the standard. Early adoption of these cutting-edge solutions now will help a manufacturer stay ahead of the curve.