AI and machine learning are giving manufacturers a remarkable ability to boost throughput, optimize production processes and supply chains, reduce costs, and more. According to a survey conducted by MIT, nearly 60% of manufacturing companies are using AI capabilities to improve their business processes. 

In most continuous manufacturing scenarios, the equipment is typically controlled by low-level controllers whose setpoints are given either by human operators or through Advanced Process Control (APC) systems in real-time. This standard approach comes with inherent limitations that can keep a manufacturing line from reaching maximum efficiency and throughput while maintaining a standard of quality.

Many of the challenges in optimizing control methods are often due to overall process complexity.

When operating conditions have a wide variation, APC systems can struggle to account for all scenarios. Additionally, human operators who rely on expertise to make quick decisions in real-time may struggle to account for more than a handful of potentially dozens of control variables at any given moment. Further potential issues can be introduced as product standards change or additional rules and constraints are placed on a manufacturing process.

Fortunately, recent developments in a specific type of AI – deep reinforcement learning – now opens new opportunities to optimize production processes using self-learning strategies for AI. Machine teaching is one approach that makes reinforcement learning more accessible by leveraging the knowledge and insights of subject matter experts to help train the model.  

Deep reinforcement learning as a potential solution for process control optimization

Deep reinforcement learning (DRL) is a machine learning method that involves the training of a software agent to learn how to act in an optimized way. To do this, the agent is trained via trial and error interactions with an environment. The learning is “reinforced” as it begins to correlate certain actions and calibrations with desired results. Reinforcement learning is a powerful tool, but it’s often difficult to effectively use as a solution for complex manufacturing processes.

The key challenge is creating an accurate simulation that can be used for proper AI training. After all, you don’t want to train your software agent on the actual machinery. So, you need a robust simulator that can provide the same rules and processes from which the model can learn.

The sheer complexity of the physical and chemical reactions involved in some manufacturing processes is extremely difficult to simulate, especially through traditional approaches.

Traditional simulation approaches typically leverage the physics and chemical equations that govern a system to simulate the process and environment. For a complex system, using this approach can quickly become insurmountable for most without the support of scientific and modeling experts. Not only are experts required to build these models, but the models themselves can become too complex to run in a cost and time-effective way.

That’s hardly optimal.

Limitations of reinforcement learning and the need for an accurate simulator

Historically, the challenge of creating an accurate simulator for reinforcement learning has created a barrier to entry for many manufacturers. While relying too heavily on human operators can lead to errors and unoptimized processes, training AI with reinforcement learning often required experts in physics and chemistry (and a good chunk of the budget).

However, through a novel, innovative approach and the new capabilities in computer vision technology, many of these longstanding issues are quickly becoming a thing of the past.

Instead of relying heavily on the traditional approach of building a simulator purely through a physics-based method, we can incorporate a more data-driven approach. With data, constructing a simulation that can effectively enable reinforcement learning for complex manufacturing processes is now possible – both technically and economically. 

In other words, instead of modeling the physics of a system through equations, we model its behavior with advanced AI models trained on real-life data.

This data-driven approach has already proven its capabilities with domains such as image tagging, speech recognition, and text translation where the switch from rule-based to AI models enables these systems to reach human-parity levels in the late 2010s.

Computer vision AI: Collecting data points to build a simulator

A key component of the data-driven approach is the availability of validated data in sufficient quantity that is representative of a varied set of operating conditions. Up until recently, validated data meant human-verified data. This model could not economically scale to the numbers (from hundreds of thousands to millions of data points) needed to train a deep learning model.

However, recent advancements in computer vision AI technology allows for cameras to examine manufacturing processes at extremely high resolution and use trained AI models to process the images for precise and accurate measurement in real-time.

Once these camera models are tuned and validated against current standards of measurement, they can be used to build a large, accurate, and varied dataset. This dataset can be used to develop a robust simulator that is able to accurately represent a wide training environment for reinforcement learning to take place.

Building a simulator from data

Once the data has been collected and validated, a model is constructed to simulate the manufacturing process itself.

The model typically uses a traditional supervised machine learning approach with labeled training data. The specific modeling approach can vary depending on the specifics and unique dependencies of the manufacturing process.

If necessary, multiple models can even be trained and ensembled for use in simulating multi-step processes. Upon completion, the simulator’s performance is then validated against machine data in the real world.

Using a simulator for reinforcement learning

Reinforcement learning can be used effectively with a validated simulator that’s capable of providing an adequately wide training environment. Microsoft Project Bonsai is perfectly suited for the application of deep reinforcement learning in this case, by providing an Azure-based automated reinforcement learning platform.

Project Bonsai automates many of the tasks that traditional reinforcement learning models require, such as manual building, testing, and tuning of individual models and frameworks. The Bonsai framework also allows for the knowledge and insights from subject matter experts on the manufacturing process to be incorporated into the machine teaching strategy. By involving experts, operators, and stakeholders in the early development process, the simulation and reinforcement learning models can be more effectively built around the desired optimization goal. 

“Machine teaching in Bonsai allows engineers and subject matter experts to use advanced reinforcement learning techniques like curriculum learning and hierarchical decomposition to solve key problems in manufacturing. Examples include speeding up polymer production, ensuring high-quality extrusion despite unobserved variability in process inputs, and many others.” 

– Victor Shnayder, Principal PM, Autonomous Systems 

Benefits of autonomous systems

Manufacturers looking to solve longstanding process efficiency issues now have more options than ever to address them, and novel data-driven approaches with computer vision are opening the door to achieve what was once thought to be impossible. 

As systems become more autonomous, we can improve human operator efficiency. Quality standards can be better maintained and optimized while reducing the potential for human error. Processes become safer, more reliable, and ultimately more effective. 

Our greatest strengths as humans are based around creativity, the ability to adapt to unforeseen circumstances and use any known pattern in any domain to derive new solutions. With autonomous systems, we can leverage what AI is good at – building models based on a defined set of data – and focus on higher-level, value-adding activities. That frees up our time (and brains) to more effectively handle out-of-scope situations, spot elements that may impact the control system in advance, validate using real-life data and personal experience, evaluate options proposed by the machine, and more.

Investing in the technology to enable these process innovations quickly returns value through decreased product waste, lower production costs, and higher quality products. AI in manufacturing is here to stay and it will eventually become the standard. Early adoption of these cutting-edge solutions now will help a manufacturer stay ahead of the curve.

Additional resources: 

• • Learn about Autonomous systems 
  • Transform your manufacturing with data, from production to delivery 
  • Key elements for designing and deploying a successful reinforcement learning-trained AI solution with Microsoft Project Bonsai 
  • Advanced simulations, the key to successful Deep Reinforcement Learning-based AI deployments 

This article was also published on LinkedIn.

This article was originally published 7/16/2020.