[Retracted] Intelligent Optimization Design of Automatic Sorting Robot Process
Abstract
In order to further improve the efficiency of sorting robots, an algorithm based on artificial intelligence design is evaluated. On the basis of determining the design requirements of belt conveyor and motor selection, the reverse device and return device and camera bracket sorting car are redesigned, and the sorting car is further analyzed by finite element method. All programs are written based on PLC. The results show that the recognition and sorting rate of 100 express surface orders with the same specifications reaches 100% and the sorting rate reaches 94% by analyzing the data recorded in the database, and the average time consumed in each recognition process is 0.0493097 s and 0.047133 s, respectively, in basic and simulated recognition. The difference did not exceed 0.01 s, and the time was basically the same. Therefore, the scheme verifies the rationality of mechanical structure design, visual scheme design, and control system design.
1. Introduction
In March 2016, AlphaGo, a robot based on Google’s Go software, beat Lee Se-dol in a game of Go. Many scientists and technologists thus believed that artificial intelligence had ushered in the third wave of its development, and we began to expect the progress of artificial intelligence to make human life better. Since the concept of artificial intelligence was put forward in 1956, it had entered the first stage of its development. By the 1960s, the programs of artificial intelligence had been able to prove many mathematical principles automatically [1]. Later, the introduction of knowledge made artificial intelligence enter the second period of rapid development. In this stage, the rapid inference of machines and the successful application of “expert system” caused another heat wave in the field of artificial intelligence. Until the 1990s, the emergence of machine learning enabled us to extract knowledge automatically from the data. Now, the third development heat wave of artificial intelligence has hit the world [2].
Machine learning is advancing rapidly as fields such as statistics, mathematics, computer science, and neuroscience continue to improve. As a branch of artificial intelligence, the progress in machine learning will further promote the development of artificial intelligence [3]. Machine learning has been a crucial “catalyst” in the 60 years of development. Machine learning is to make the computer learn repeatedly to improve its performance and intelligently identify new samples through relevant algorithms, so that the computer can make correct responses and behaviors without explicit programming control [4]. In the past decade, machine learning has been widely applied in various fields such as autonomous driving, speech recognition, data mining, medical diagnosis, weather forecast, and industrial control. The manufacturing method of intelligent robot sorting system is shown in Figure 1.
Recently, the behavior of a group of robots that can organize themselves into a group to perform complex tasks has attracted extensive attention from the academic community [5]. More and more perfect intelligent algorithms are used to better control the robot population [6]. In the past few years, many learning methods have been developed, including supervised and unsupervised learning algorithms. Swarm intelligence algorithms were first proposed in the early 1990s in the research on a cellular robotic system. Since then, many kinds of swarm intelligence algorithms have been proposed. For example, ant optimization algorithm is inspired by the behavior of ants looking for the shortest distance between the nest and food. Particle swarm optimization algorithm simulates the flight of birds and the movement of searching for food, and artificial bee colony algorithm is inspired by the process of bees searching for food [7].
2. Literature Review
The diversity of unstructured environments and products requires new robotic solutions that combine intelligent machine design with advanced artificial intelligence techniques to address the challenges of object recognition, dexterous grasping, or path planning [8]. The essence of robot visual recognition is to capture images for detection and classification. Traditional target recognition technology is mainly divided into three stages: image preprocessing, feature representation, and region classification, which often requires the consistency of target scene structure. Irregular shape, color disunity, occlusion, and disorderly placement will bring great challenges to traditional visual recognition methods.
In 2006, the neural network as a visual recognition model was proposed for the visual recognition technology, which started a period of research upsurge. AlexNet network designed by Wu et al. won the championship in ImageNet image recognition competition with 10.9% lower test error rate than the second place, which promoted the development of deep neural network in the field of computer vision [9]. Soares et al. proposed r-CNN, the target detection algorithm. In the second year, fastR-CNN and FasterR-CNN detection algorithms were proposed, and the detection effect was constantly improved. In the ILSVRC and COCO2015 competition, a regional proposal network (RPN) was introduced. With detection speed of 7 FPS and detection accuracy of 73.2%, it ranked first [10].
At present, for target positioning methods, in order to better grasp objects, the application of depth camera was becoming more and more popular. Point cloud registration target estimation methods occupied the mainstream place. NDT algorithm was proposed by Z. Lv et al. This algorithm was used for point cloud registration when the initial value was not required, and it was relatively stable [11]. Z. Lv et al. designed the SUPER4PCS algorithm with fast operation speed but low registration [12]. Yang et al. proposed LINEMOD, a 6D pose estimation algorithm for objects in a chaotic environment, on ACCV in 2012, and good results were achieved [13].
When the robot is sorting the goods placed on the shelf, the geometric constraints of the shelf and goods make the robot pass through the allowable pose change which is very small, so high requirements are put forward for the robot trajectory planning and control under multiple constraints. Yang et al. proposed the covariant Hamiltonian optimization (CHOMP) algorithm for motion planning, which was a parameter invariant trajectory optimization method and could be used to locally optimize the feasible trajectory of a 7-DOF manipulator [14]. Zhou et al. designed an accurate robust fuzzy control algorithm (PRFC) suitable for electric manipulator, adjusted the fuzzy rules for controlling tracking spatial origin, optimized the traditional fuzzy controller, and improved the robustness of robot system operation [15].
At present, most storage enterprises at home and abroad are doing more and more researches on mobile robots and scheduling algorithms, but a lot of labor is still needed for cargo picking. Therefore, in terms of the identification and positioning method of multitarget goods in the warehouse environment, the detection and positioning of shelf objects to improve the degree of automatic warehouse sorting were mainly investigated in the research.
3. Methods
3.1. Design Requirements of the Belt Conveyor
In the selection process of the belt conveyor, the main factors to consider were belt bandwidth and belt speed. For the belt design, the bandwidth was required to be greater than the width of the car base to ensure a good margin to avoid collisions between the car and the inner wall of the conveyor when turning. The adjustable range of the speed provided by the belt was required to be between 0.5 m/s and 1.5 m/s to ensure that the camera could still shoot clear and high-contrast images under the movement of the belt [16, 17].
3.2. Motor Selection
The maximum weight of express items was required to not exceed 5 kg in the research, and the requirement for the friction of the belt was not strict, so the flat belt with simple structure could meet the requirements. In addition, the transmission structure of flat belt was very simple, with high transmission efficiency, and the pulley was also very easy to manufacture, so flat belt was the ideal choice for the research [18].
T2.0 mm PVC belt was selected in the research. The simple choice of motor was as follows.
According to the requirements of the research, the total weight of the car and the object was not more than 10 kg, and the number of cars arranged on the whole ring belt machine was 6. In order to meet the requirements of the photo shooting, the belt speed was required to be between 0.5 m/s and 1.5 m/s.
In the two formulas, F is the friction force, u is the friction coefficient, m is the total weight of car and express, g is the gravitational acceleration, P is the output power of p-motor, and V is the velocity of belt running.
Then, the following could be obtained from formulas (1) and (2):
The minimum power of motor: P = 0.5 × 6 × 10 × 9.8 × 0.5 = 147 W
The maximum power of motor: P = 0.5 × 6 × 10 × 98 × 15 = 441 W
Based on the overall consideration, the motor model YB2-801-2 was selected here, with a rated power of 0.75 kW, which met the requirements.
3.3. Design of the Reversing Device and the Returning Device
In order to save space to the utmost extent, the design of the sorter should be in line with the requirements of not only completing the outer ring belt machine sorting but also completing the inner belt machine sorting. So it is a test for the car in the process of motion on the package action and reversing action. The reversing device and the returning device in the car were designed in the research.
Among them, the reversing device is installed between the first sorting mouth and the orbit of the visual scanning system. The cylinder of the reversing device could not move by default. When the visual system determined that the sorting port corresponding to the express loaded in the car was located inside the belt, the cylinder started to move. The cylinder jacked up the directional dial. In the process of the car passing through this section of curve path, its orientation would change from outward to inward, so that the express could be sorted out from the sorting mouth inside the belt machine. As for the returning device, its principle was the same as that of the reversing device, which was to reverse the carriage by changing its path [19]. Because the car circulated on the circular belt machine, the car carriage faced different directions before loading the bag. In order to facilitate loading, the car carriage faced the outside of the belt, and this action was completed by the returning device.
3.4. Design of Camera Bracket
Considering the spatial structure and convenient handling of the whole sorting machine, it was necessary to set up a camera bracket, which could not only maintain stability in the operation of the sorting machine but also made the height and angle of the camera and the light source convenient and adjustable, so as to adapt to the movement of the sorting machine caused by some external factors [20].
The industrial camera and the light source are fixed on the horizontal profile above the bracket by bolts and nuts. The angle of the industrial camera and the light source could be controlled by adjusting the bolts and nuts. The upper horizontal profile was fixed to the vertical profile by angle code, so that the profile height could be adjusted and so as to ensure that the height of the industrial camera and the light source could be adjusted. At the same time, in order to ensure the stability of the whole camera bracket, a vertical profile was added under the bracket. According to the parameters of the camera and lens, the size of the car, and the width of the belt machine, the height of the whole camera bracket was 140 cm with the adjustable range of 30 cm in the height direction and the width was 80 cm that did not interfere with the movement of the car and cylinder and other components.
3.5. Structure Design of Sorting Car
For the design of the sorting car, high stability and reliability were required. During the movement of the main ring, it was necessary to keep the car running smoothly when it was not sorting, and the movement was timely and smooth when it was sorting [21, 22]. Sorting car was mainly composed of base, support, platform plate, door plate, spring rod, bevel gear, and so on.
Sorting car was the core mechanical structure of the sorting machine. Whether express could accurately fall in the corresponding sorting mouth completely depended on whether the car door could be accurately opened or not, and the completion of the action of the door opening mainly depended on the car transmission mechanism, so the design of the car transmission mechanism was particularly important.
3.5.1. Calculation of Gear Torque
In order to ensure that the rotation range of the lever was enough to open the door when the car lever was in contact with the pushing mechanism and avoid the express stuck in the door, according to the calculation of the maximum elongation of the reset spring constraining the door opening action and its elastic coefficient, the minimum torque of the input shaft required by the bevel gear was 5 N·m.
In the formulas, nII was the speed of axis II (r/min), and nI was the speed of axis I (r/min).
The above formulas could be calculated as follows:
TIIIenter = 3.918 N∙m, TIIout = 8.163 N∙m, TIIenter = 8.278 N∙m, TIout = 8.723 N∙m, TIenter = 8.856 N∙m, and then, TIenter > 5 N∙m; then, it met the requirements of the transmission mechanism of the car.
3.5.2. Straight Bevel Gear Design
The gear type was standard straight bevel gear transmission, with 7-level gear precision and pressure angle of 20°. The material of two gears was 40Cr (quenching and tempering). The hardness of tooth surface was 280 HBS. The number of teeth of pinion zi = 21, z2 = 21, and u = 1.
According to the design principle of gear contact fatigue strength and the design principle of tooth root bending fatigue strength, the following design conclusions could be obtained: tooth number Z1 = 41 and Z2 = 42, pressure angle a = 20°, displacement coefficient X1 = 0 and X2 = 0, split cone angle δ = 45° and δ2 = 45, and tooth width b1 = b2 = 11 mm. 40Cr (quenching and tempering) was used by both gears. The gears were designed with 7-level precision.
3.5.3. Design of Spur Gear
Straight bevel gear transmission was selected, with the pressure angle of 20° and 7-level precision 7. The material of big gear was 45Cr (quenching and tempering). The hardness of tooth surface was 240 HBS. The material of small gear was 40Cr (quenching and tempering). The hardness of tooth surface was 280 HBS. The number of teeth of pinion u = 0.5 and z = 61.
According to the design principle of gear contact fatigue strength and the design principle of tooth root bending fatigue strength, the following design conclusions could be obtained: tooth number Z1 = 81 and Z2 = 43, modulus m = 0.5, pressure angle a = 20°, displacement coefficient X1 = 0 and X2 = 0, center distance a = 31 mm, tooth width b1 = 40.5 mm and b2 = 21.5 mm. 40Cr (quenching and tempering) was used by small gear. 45Cr (quenching and tempering) was used by big gear. The gears were designed with 7-level precision.
3.6. Finite Element Analysis of the Sorting Car
Since the tilting car and the external cylinder worked through relative motion. The impact occurred when they contacted. So the finite element model was established to analyze the impact [23]. ANAYS software was used for display dynamics analysis. Firstly, the 3D model established with SOLIDWORKS was imported into the finite element analysis software ANAYSWorkbench18.1, and the push rod material was set as 45 steel and L-shaped rod material as stainless steel. Material properties were edited through EngineeringData module of Workbench. Through the size function, the grid size was defined and the total number of grids was 85557. Then, the analysis parameters were modified. The end time was set as 0.15 s. The fixed cylinder was added, then the speed limit was added, and the speed of the whole car was set as 1 m/s. Finally, the equivalent stress and equivalent strain were added into the solution, and the analysis was conducted by clicking solve.
By finite element calculation, when the equal effect of impact contact became 0.32 mm/mm, the equivalent stress of impact was 74.201 MPa, the yield strength of 45 steel was 355 MPa, and the yield strength of 304 stainless steel was 205 MPa. The experimental results were far less than the yield strength, and the strength check was safe.
3.7. PLC Programming
Before the preparation of PLC program, first of all, the functional requirements of the control system were analyzed. The tray sorting machine control system was required to meet the functional requirements. When each sorting port photoelectric was triggered, the number of photoelectric trigger and sorting number were judged to meet the trigger conditions in the counting principle. Solenoid valve was electrified for a second, so that the cylinder piston rod out of about a second and reset after a second. The belt motor could be controlled to start and stop [24]. The data registered from D1 to D7 would be reset when switched on. The sixth step was the motor start-stop procedure. X001 was responsible for starting, and X002 was responsible for disconnecting. X003 was the photoelectric switch, which was triggered once, the value of data register D1 plus 1 and to represent the value of the photoelectric switch. When the photoelectric switch was triggered, read the value of the corresponding register in PLC through the serial port (here was the value of D1). If the value of D1 and the sorting port number met the trigger conditions in the counting principle, then M0 was made forcibly closed through communication with PLC, so that the solenoid valve Y001 was energized for one second. That is, the cylinder was pushed out for about one second; the power was off and reset after one second. PLC I/O address allocation table is shown in Table 1.
| Input | Switch | Output | Element |
|---|---|---|---|
| X0 | Start button | Y0 | Cylinder 1 |
| X1 | Stop button | Y1 | Cylinder 2 |
| X2 | Sensor 1 | Y2 | Cylinder 3 |
| X3 | Sensor 2 | Y3 | Cylinder 4 |
| X4 | Sensor 3 | Y4 | Cylinder 5 |
| X5 | Sensor 4 | Y5 | Cylinder 6 |
| X6 | Sensor 5 | ||
| X7 | Sensor 6 | ||
| X10 | Sensor 7 | ||
3.7.1. PLC Communication
PLC communication refers to the data communication between PLC and PLC, between PLC and PC, and between PLC and other intelligent equipment. This function was conducive to PLC monitoring, and data reading and writing between PLC and PLC could be more convenient for control [25, 26].
PLC communication interface was mainly divided into RS232, RS485, and Ethernet. The RS232 interface was adopted in the research, which was connected to the touch screen serial port through the USB adapter, namely, serial communication. Here, there were two main types of serial port communication way. The first was that according to the communication between PLC and agreement, a small amount of code to do simple communication could be used through the SerialPort class provided by the NET, but when the content of communication with PLC was large, the code needed to be done in a large number of data hexadecimal conversion and validation, etc., which seemed more complex. The second way was to use MXComponent software officially provided by Mitsubishi. By calling its dynamic link libraries ActUtlType.dll and ActUWzd.dll, data interaction with PLC could be carried out conveniently without considering problems such as base conversion verification, but the software must be downloaded, which was about 500 M in size. Through the analysis of PLC ladder diagram, the purpose of communication with PLC was just to control the forced closure of switch MO and read the value of data register D1. The content was very simple. The first way and communicating with PLC were chosen. The message format of setting and resetting of bit elements used in communication is shown in Tables 2 and 3. The message format read by word elements is shown in Table 4.
| Start | Command | Address | Finish | Checksum |
|---|---|---|---|---|
| STX | CMD | ADDRESS | ETX | SUM |
| 02 h | 37 h | Address | 03 h | Sum |
| Start | Command | Address | Finish | Checksum |
|---|---|---|---|---|
| STX | CMD | ADDRESS | ETX | SUM |
| 02 h | 38 h | Address | 03 h | Sum |
- Note that the command (CMD) 37 represents enforcing ON and 38 represents enforcing OFF. Sum checksum: SUM = CMD + ADDRESS + ETX.
| Start | Command | The first address | Digit | Finish | Checksum |
|---|---|---|---|---|---|
| STX | CMD | GROUPADDRESS | BYTES | ETX | Sum |
- Note that the command (CMD) 30 represents reading. First address algorithm: Address = Address∗2 + 1000 h and then converted to hexadecimal.
3.7.2. Design of the Software Running Interface
Before designing the interface, what functions the interface contain should be first analyzed. In the research, the interface included the following functions: serial port number detection, belt start and stop, serial port open and close, serial port parameter setting, displaying the number of sorted express, displaying the identified to be sorted express, and displaying the real-time update table in the database.
Sorting machine operation interface was mainly composed of three parts, which were the serial port debugging interface, the sorting interface, and the simulation experiment interface.
3.7.3. Design of the Serial Port Debugging Interface
As the lower machine in the research was PLC, so communication debugging with PLC was necessary. The serial port debugging interface mainly included the parameter configuration area, the data receiving area, and the data sending area. The parameter configuration area was to ensure the serial port number, baud rate, data bit, stop bit, and parity bit of the parameter port to adjust. Due to the data receiving area and data sending area, the corresponding PLC soft components could be chosen according to the specified message format in the data receiving area to send relevant commands to PLC. The execution of this command by PLC also was displayed in the data receiving area in the format of messages. The design of the interface was beneficial to the debugging between the upper computer and the lower computer, which also provided a basis for troubleshooting.
3.7.4. Design of the Sorting Interface
The sorting interface was mainly composed of three control buttons and three parts of visual interface. The three control buttons were create template, start match, and stop match. Create template-supported manual interception of any area in the express order for modeling, which was convenient to change the template. The whole device and program would be run when the match started, and the whole device and program would be stopped when the match stopped. The visual interface included the camera recognition interface, the data interface, and the chart interface. Among them, the camera recognition interface showed the image of each express order captured by the camera and the three-segment code area located. The data interface was used to display the number of fractional two-dimensional code matched by each express order template and the three-segment code identified. The chart interface mainly showed the sorting quantity of express order numbers at different addresses in the form of charts. The sorting interface made it easy to control the entire equipment and program, as well as visually observe the sorting process.
3.7.5. Design of the Simulation Experiment Interface
The simulation experiment interface was mainly to simulate the whole process of sorting machine and verified the sorting principle. After the sorting principle was changed, the simulation experiment interface could be verified at any time, and the feasibility of sorting principle could be guaranteed through the interface.
4. Results and Discussions
On the basis of the testing through simulation system, the actual prototype was also needed to test the reliability and stability of the sorting machine. The test process was divided into the following parts.
4.1. Testing Capturing Pictures
Due to the limitations of the site and equipment, the design of the camera’s support mechanism was relatively simple, which was easy to cause the camera position offset, leading to the failure to take the express delivery sheet. Before the sorting, the computer was used to connect to the camera to capture pictures. If the clear surface sheet could be captured, the next step was proceeded. Otherwise, the camera position was adjusted until the clear surface sheet was captured.
4.2. Turning on the Power
The whole ring belt machine was composed of two semicircular belt machines and two linear belt units. Through the change of the circuit, the four belt machine starting and stopping at the same time could be controlled.
4.3. Placing Package
After the power was turned on, the car moved circularly with the belt machine. The package was placed on the car manually under the condition of ensuring the smooth movement of the car. According to the counting principle described earlier in this research, the timing of placing package was not restricted, and package could be placed in the upper bag area at any time.
4.4. Sorting Package
The sorting process of package was a fully automatic process basically. In the whole process, on the one hand, the data changes of the control interface of the industrial computer needed to be detected. On the other hand, the running condition of the car on the belt should be observed, so as to turn off the power in case of accidents.
In order to combine the actual scene of package sorting to the maximum extent, the physical prototype was transported to Huainan Datong Logistics Park for further testing.
The test results are shown in Table 5.
| Test item | Test effect | Action time | Whether meets the requirements or not |
|---|---|---|---|
| The visual program | Information extraction | 45 ms | Yes |
| Interface program | The interface runs smoothly | Fast | Yes |
| Belt conveyor | Synchronous start, consistent speed | Faster | Yes |
| Sorting car | The door can be opened and repositioned. | Slow | Yes |
| Pneumatic circuit | Stable operation | 150 ms | |
From the experimental records, the belt conveyor could realize synchronous start. The tilting car could realize the reservation function. The door and cylinder cooperated reliably, and the door could be opened normally. The car platform and the turning device cooperated reliably and could realize the turning of the platform in real time. The cylinder responded quickly, and the scheduled action with the car could be realized through the PLC program. HALCON software program ran stably after adjusting the parameters. After adjusting the interface program parameters, the interface ran smoothly and responded quickly. PLC could normally complete the action information provided by the interface program. After the test was completed, the parameter comparison with the existing sorter is shown in Figure 2.
It could be seen from Figure 2 that on the premise of meeting the sorting efficiency, the tilting-pan automatic sorting machine could greatly save the cost of equipment and the floor area and could meet the sorting needs of the third and fourth tier cities.
The average time was calculated using the function-count_seconds (seconds) in HALCON, as shown in Table 6.
| Item | Correct identification | Incorrect identification | Average time |
|---|---|---|---|
| Actual situation | 94 pieces | 6 pieces | 0.0493097 s |
| Simulation | 100 pieces | 0 piece | 0.0471338 s |
5. Conclusion
Based on the establishment of the experimental prototype in the research, the recognition scheme based on shape template matching was adopted to identify and sort 100 express surface sheets of the same specifications. By analyzing the data recorded in the database, the recognition rate reached 100% and the sorting rate reached 94%, and the average time consumed by each recognition process was 0.0493097 s and 0.0471338 s, respectively, and the difference was not more than 0.01 s. The time was basically the same, which verified the reasonableness of the mechanical structure design, visual scheme design, and control system design. Vision-based robot sorting system is a multidisciplinary field. Although the work of vision was mainly investigated in the research, there were still a lot of work to be further studied and explored.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This study was supported by the Hebei Province “333 Talents Project” Funding Project (Project No. A202101033) and the Project of Hebei Provincial Department of Science and Technology (No. 22E50212D).
Open Research
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
