Introduction:

‍

It is impossible to overestimate the significance of actuators in today’s industries. Automation and robotics are made possible by actuators, which leads to faster production as well as increased precision and safety.Actuators are used in a wide range of industries, such as manufacturing, aerospace, automotive, robotics, and home automation, to perform important functions, such as opening and closing valves, moving robotic arms, adjusting control surfaces, or actuating brakes.

‍

A device that moves or operates something is called an actuator. When an actuator receives a portion of the input energy as a feedback control signal, the actuator starts moving the machine part. In other terms, an actuator transforms energy into mechanical or physical motion. An actuator’s primary function is to regulate a machine’s internal motions. It is a critical component in many systems and machines, providing the force and motion necessary for various operations. 

‍

Actuators can be categorized into 2 categories based on their source of energy and their range of motion. The 1st category includes Electrical, Hydraulic, Pneumatic,etc and the 2nd category includes linear and rotary actuators.

‍

Section 1: Types of Actuators based on their range of motion:

‍

As mentioned before, actuators can be grouped according to their range of motion, for as whether they provide linear or rotary motion. Rotating actuators produce rotation around an axis, whereas linear actuators produce motion in a straight line. Engineers and designers can choose the most suitable actuator for their unique needs by being aware of the differences between various types of actuators.

‍

Linear Actuators:

‍

Linear actuators consist of several components, including a motor, lead screw, and guide rod. The motor provides the energy needed to move the actuator, while the lead screw translates rotational motion into linear motion. The guide rod ensures that the actuator moves in a straight line and prevents it from rotating. 

‍

Rotary Actuators:

‍

On the other hand, Rotary actuators as mentioned above rotate around their axis or simply create a circular motion, Rotary actuators generally consist of a shaft, housing, and internal mechanism. The shaft is the central part of the actuator that rotates around an axis. The housing encloses the internal mechanism, which may include gears, pistons, or other mechanisms that produce rotational motion. Rotary actuators have sub-sections that will be discussed on a blog specified for rotary actuators.

‍

Section 2: Types of Actuators Based on Their Energy source:

The 3 most popular types of actuators based on their energy source are Electrical, Pneumatic and Hydraulic actuators – There are other types but not as popular so they will not be discussed, for extra information you can check the resources provided at the end of the blog- each type has its own advantages and disadvantages that will be discussed in this section. 

‍

Electrical Actuators:

‍

Electrical actuators, as it’s suggested from the name, which gives them some distinct advantages. They are known for being quieter than their counterparts, offering high levels of accuracy and precision. They also provide complete control over motion profiles and are easily programmable, making them suitable for a wide range of applications.

‍

Despite their benefits, electric actuators also have some drawbacks. They can overheat, and their parameters such as speed and torque are fixed. Additionally, they tend to be more expensive than pneumatic or hydraulic actuators

‍

Acrome’s Stewart platform integrates electrical actuators in their systems such as stewart platforms so it offers high precision alongside being easy to program making it perfect for use in research labs and universities.

‍

‍

Hydraulic Actuators:

‍

To create force and motion, hydraulic actuators employ pressurized fluid. They are renowned for having a high power density and the capacity to produce significant forces, which makes them perfect for use in heavy-duty machinery and equipment for the construction and industrial sectors. Moreover, hydraulic systems can offer exact control over motion’s speed and direction, making them appropriate for uses where precise placement is important.

‍

Yet, there are certain disadvantages to using hydraulic actuators. Leaks can be challenging to find and fix, and they need regular maintenance to keep the hydraulic fluid clean and clear of impurities. Due to the necessity for pumps, hoses, and other components, hydraulic systems can be expensive to build and maintain. Additionally, because the hydraulic fluid can become contaminated with particles over time, hydraulic systems might not be appropriate for situations where cleanliness is crucial.

‍

‍

Pneumatic Actuators:

‍

Pneumatic actuators, on the other hand, use compressed air to generate force and motion. They are commonly used in industries where electrical power sources may not be available or not suitable due to safety concerns. Pneumatic actuators are lightweight, relatively inexpensive, and offer fast and precise operation. They are often used in applications where speed is critical, such as in the automation of assembly lines. 

‍

However, pneumatic systems require a steady supply of compressed air, which can be costly to generate and maintain. They are also not as powerful as hydraulic or electrical actuators, which limits their use in heavy-duty applications. Despite these limitations, pneumatic actuators are still a popular choice for many industrial applications due to their reliability and efficiency.

‍

Conclusion:

In conclusion, actuators play a crucial role in various industries, enabling automation and robotics to improve production speed, precision, and safety. With a wide range of applications in manufacturing, aerospace, automotive, robotics, and home automation, the choice of actuator depends on factors such as the energy source and the range of motion required.

‍

We have explored the different types of actuators based on their range of motion, as well as their energy sources. Each type has its own unique advantages and disadvantages, making it essential for engineers and designers to select the most appropriate actuator for their specific needs.

‍

In this blog series we will take a deep look into each type of actuator, providing a comprehensive understanding of their operation and potential applications. Stay tuned to learn more about the fascinating world of actuators and how they continue to shape our technological advancements.

‍

To learn more about actuators and their applications, consider exploring the following resources:

‍

• Sclater, N., & Chironis, N. P. (2001). Mechanisms and Mechanical Devices Sourcebook. New York: McGraw-Hill. [A comprehensive guide to various types of actuators and their applications]

‍

• Rajput, R. K. (2018). A Textbook of Fluid Mechanics and Hydraulic Machines. New Delhi: Laxmi Publications. [A resource for understanding hydraulic actuators]

‍

• Pippenger, J. (2007). Pneumatic Actuators: For Industrial Automation. New York: Momentum Press. [A detailed introduction to pneumatic actuators]

‍

• Kuo, B. C. (2003). Automatic Control Systems. New Jersey: Prentice Hall. [A book covering various types of control systems, including those using actuators]

‍

• Uicker, J. J., Pennock, G. R., & Shigley, J. E. (2003). Theory of Machines and Mechanisms. New York: Oxford University Press. [A textbook providing an in-depth understanding of machines and mechanisms, including actuators]

‍

• International Society for Automation (ISA) [A professional organization with resources on automation, including articles and research papers on actuators]

‍

• Motion Control & Motor Association (MCMA)

‍

Robotics and Automation News

The Ball Balancing Table (BBT) is a great place to start if you want to learn and gain experience with control theory firsthand. The BBT consolidates high-grade accuracy with open-source accessibility, providing students, engineers and researchers with a ecosystem to test and improve control algorithms.

This blog is about a project where we use The Ball Balancing Table to make a PID controller and Q-Learning so that it can solve a maze. In this post, we’ll be looking at the underlying principles of both the hardware and algorithm, how maze is encoded into a matrix and what kind of real-time tweaks could break beyond current-systems capabilities. Let’s dive in!

‍

Introduction to the Ball Balancing Table (BBT)

‍

The Ball Balancing Table (Figure 1) is a classic experiment in control systems that bridges industrial processes and DIY projects. It consists of a flat surface (the table) where a ball is placed, and the objective is to control the tilt of the table to guide the ball to specific locations. Students can learn essential control concepts, such as feedback systems, by experimenting with different types of controllers, such as PID, and adaptive control. The open-source software integration allows users to modify and test advanced control algorithms, making it a versatile tool for both academic and real-world applications.

‍

‍

‍

Control System Design

‍

In this project, we employ a PID controller (Proportional-Integral-Derivative), one of the most used control mechanisms in automation systems. The PID controller helps maintain the desired trajectory by adjusting the angles of the BBT’s platform based on feedback from the ball’s position. Here’s how it works:

• Proportional (P): Reacts to the current error between the target and the actual position of the ball.
• Integral (I): Accounts for accumulated past errors to reduce steady-state error.
• Derivative (D): Predicts future error based on the current rate of change.

Together, these terms allow the table to adjust its tilt dynamically, keeping the ball within a defined path, and ultimately solving the maze.

‍

Encoding the Maze

‍

The maze is represented as a matrix of 0s and 1s in Python, where:

• 0 represents open spaces.
• 1 represents walls or obstacles.

This matrix forms the environment within which the ball must move. The goal is to guide the ball from the start position to the maze’s exit. Here’s an example matrix representation:

maze = [

[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],

[1, 0, 0, 0, 1, 0, 0, 0, 0, 1],

[1, 0, 1, 0, 1, 0, 1, 1, 0, 1],

[1, 0, 1, 0, 0, 0, 0, 1, 0, 1],

[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

]

‍

The maze to be encoded is illustrated in the following figure.

‍

‍

This image is converted to a matrix using image processing. In the following step obtained grid is mapped to the coordinates of the BBT. As the ball moves, the system interprets the matrix and sends the ball to the corresponding open spaces.

‍

Reinforcement Learning and Q-Learning

‍

Reinforcement learning is learning what to do—how to map situations to actions—so as to maximize a numerical reward signal. The learner is not told which actions to take, but instead must discover which actions yield the most reward by trying them. In the most interesting and challenging cases, actions may affect not only the immediate reward but also the next situation and, through that, all subsequent rewards. These two characteristics—trial-and-error search and delayed reward—are the two most important distinguishing features of reinforcement learning.

One of the most common RL algorithms in areas like shortest path is Q-learning. Here’s how it works in the context of solving the maze:

• The ball (agent) is placed at the start of the maze.
• For each step, the agent decides which direction to move (up, down, left, or right).
• If the move leads to a valid position, it gets a reward; if it hits a wall, it receives a penalty.
• Over time, the agent learns which actions maximize the cumulative reward, allowing it to find the optimal path through the maze.

The Q-learning algorithm uses the following equation to update the “quality” (Q-value) of each state-action pair:

‍

Q(state,action)=Q(state,action)+alpha (reward+gamma max(Q(newstate,allactions) )−Q(state,action)) 

‍

Where:

• alpha is the learning rate, determining how much new information overrides old information.
• gamma is the discount factor, which determines the importance of future rewards.
• reward is the feedback received from the environment after performing an action.

In our project, Q-learning enables the ball to learn and solve the maze. Once the agent (the ball) has learned the optimal path, it starts sending commands to the BBT to follow the learned trajectory.

The solution to the maze (figure 2) is illustrated in the figure below, showcasing the optimal path determined by the Q-learning algorithm. This path represents the sequence of moves that successfully navigate the maze.

In addition, the heat map visualizes the learned Q-values for each state within the maze. The map highlights the desirability of each position based on the cumulative rewards, with warmer colors indicating higher Q-values and cooler colors representing lower values.

‍

‍

‍

‍

Working Mechanism of the BBT Maze Solver

‍

Here’s how the maze-solving mechanism works:

1. Initialize the environment: The maze is encoded into a matrix, and the Q-learning agent is initialized.
2. Learn the path: The agent iterates through the maze, exploring different paths. Over time, it learns which paths lead to the goal (exiting the maze) and which result in dead ends.
3. Send instructions to the BBT: Once the path is learned, the coordinates of each step are converted to BBT coordinates using the function maze_to_bbt_coords().
4. Move the ball: The ball follows the learned path, controlled by the PID algorithm, which adjusts the tilt of the table based on real-time feedback from the ball’s position.

Here’s a simplified pseudocode snippet showing how the BBT receives commands to move to specific points:

#pseudocode starts here 
FUNCTION move_bbt_to_position(setpointx, setpointy):  positionx, positiony = GET current_ball_position()  # Retrieve the current position of the ball  errorx = setpointx - positionx  # Calculate the error in x-axis  errory = setpointy - positiony  # Calculate the error in y-axis 
 outputx = APPLY_PID_controller_x(errorx)  # Calculate new position for x using PID control  outputy = APPLY_PID_controller_y(errory)  # Calculate new position for y using PID control 
 SET_servo(outputx, outputy)  # Send servo commands to adjust BBT position 
    UPDATE_device()  # Update the BBT with the new position 
#End of the pseudocode 

‍

The application source code is available in this Github repository.

The image bellow shows the Ball Balancing Table (BBT) in conjunction with the maze.

‍

‍

In the video below, a trial example outcome of the BBT maze solver with the reinforcement learning algorithm can be seen. Please note, the video is accelerated by 10x, because the RL algorithm is not optimized for speed in this example.

‍

‍

Real-World Applications Examples

‍

The BBT maze solver can be seen as a scaled-down simulation of complex industrial control systems, offering several potential applications:

• Robotics: Autonomous navigation systems, like those used in robotic vacuum cleaners, could employ similar algorithms to navigate around obstacles.
• Game AI: The same principles can be applied in video games where non-player characters (NPCs) need to navigate complex environments.
• Real-Time Traffic Management: In a future where AI drives vehicles, managing traffic could resemble solving a maze, with controllers needing to adapt dynamically to real-time conditions, much like the adjustments made in the BBT Maze Solver.

‍

Future Improvements and Directions

‍

This project offers several avenues for further development:

1. Real-Time Maze Recalculation: By adding a camera to the top of the BBT, the system could take snapshots of the maze and dynamically adjust the path if obstacles are moved or removed in real-time.
2. Adaptive Control Algorithms: Implementing more advanced control algorithms, like Autonomous PID tuning, controllers itself can continuously adapt their parameters with RL, allowing the system to automatically fine-tune its response to environmental changes and disturbances.
3. Deep Reinforcement Learning: Transitioning from Q-learning to deep reinforcement learning (using neural networks) could enable the system to solve more complex mazes with greater accuracy and flexibility such as moving in diagonals.

Conclusion

‍

The Ball Balancing Table and Q-learning provide an exciting mix of hardware and software where classic control theory meets cutting-edge machine learning techniques. Through projects like this, we can deepen our understanding of control systems, reinforcement learning, and their potential real- world applications. With continuous improvements, these algorithms can drive self-regulating traffic networks, control autonomous robots, and advance the development of intelligent gaming systems.

By exploring these concepts and implementing them in hands-on projects, we unlock new opportunities for innovation and understanding. Whether it is for a student learning control theory or a researcher experimenting with advanced machine learning algorithms, the BBT offers a fantastic platform to bring these ideas to life.

‍

References

1. Acrome. Ball Balancing Table.
2. Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press.
3. K. J. Astrom, & T. Hagglund. (2006). Advanced PID Control. ISA – Instrumentation, Systems, and Automation Society.