1. Introduction

The control of manipulator has always been an important and hot issue in mechanical control. At present, the control theory and method of manipulator have been deeply studied [1–3]. Machine vision technology takes the photosensitive element as the processing core and uses the analog signal transmitted by the optical sensor to convert and collect the electrical signal and optical signal. With the development of camera and imaging processing technology, the data information processed by machine vision processing technology can be transformed from two-dimensional to three-dimensional. Under the influence of automatic machinery production mode, industrial manipulator has become an important equipment to support modern production. How to realize the automatic control process of industrial manipulator has also become a current research hotspot.

The early vision-based manipulator control system is to process the image and obtain the target position from the image, so as to achieve the effect of intelligent control and closed-loop control [4, 5]. For example, Li y et al. used the closed-loop vision algorithm developed based on monocular and binocular vision to realize the step of pose estimation from target recognition, which is completely automatic [6–8]. However, the manipulator control system relying only on vision cannot meet the needs of modern industry because of its limited accuracy, the resulting error is also large, and the prior knowledge of the environment is required [9]. Literature [10] developed the controller of the manipulator based on the behavior data of the double rotating joint manipulator and the reinforcement learning algorithm, but there are some defects, such as limited data points and the need to configure a large amount of prior knowledge for the manipulator model, so it is impossible to control the manipulator with different structures after training one controller. The visual sensor can collect rich information in the image. By combining with the reinforcement learning algorithm, the intelligent controller can learn information from the image independently, and the control strategy is optimized by the reward and punishment mechanism to realize the manipulator control after the manipulator with different structures is trained by the intelligent controller without any prior knowledge of the manipulator structure model [10–12].

The above shows that the use of machine learning combined manipulator alone cannot meet the high-precision requirements of today’s manipulator [13]. Therefore, many scholars turn their research objectives to the six-dimensional force sensor with high-precision performance. The core technology of the development of the six-dimensional force sensor is the structural design of the sensor elastomer, and its structural form directly affects the performance of the six-dimensional force sensor [14]. With the development of robot application towards high speed and high precision, the problem of dynamic force measurement is becoming more and more prominent. The six-dimensional force sensor not only requires high sensitivity and small cross interference in each axis, but also requires a certain working bandwidth to meet the needs of dynamic force measurement. In order to adapt to different working environments, the structures of six-dimensional force sensor elastomers developed by experts and scholars are also diverse. Among them, Korean scholars have designed a six-dimensional force/torque sensor to detect the full force information of the foot of humanoid robot. The sensor is small and has adjustable and independent sensitivity to different force components [15, 16]. Ge Yu of the Institute of intelligent machinery, Chinese Academy of Sciences, and others invented an ultrathin six-dimensional force/torque sensor based on cross beam and double E-shaped film, which is used to obtain force sensing information of underwater robot wrist [17–19]. In addition, because the cross beam sensor has the advantages of symmetrical structure, high sensitivity, and small-dimensional coupling, more and more scholars improve it in order to obtain a six-dimensional force sensor with better performance [20–22]. The sensor has the characteristics of high sensitivity and small-dimensional coupling. Based on the traditional cross beam sensor, Wu C et al. [23–25] proposed a new six axis wrist force sensor by setting through holes on each sensitive beam. The sensor has good static and dynamic characteristics.

To sum up, machine vision can better meet the intelligent recognition requirements of the manipulator system, while the six-dimensional force sensor can better meet the sensitivity and accuracy requirements of the manipulator in the manipulation process. Therefore, this paper applies machine vision and six-dimensional force sensor to the manipulator control system with high accuracy requirements, in order to realize the use requirements of the manipulator control system in the environment with high intelligence and high accuracy requirements.

2. Basic Theory

2.1. Design of Controller

In order to realize the autonomous control of the manipulator, an intelligent controller based on machine control is designed. By learning step by step, the intelligent controller finally learns to control the movement of the manipulator, so that the end of the manipulator is controlled from the initial position to the target position. In each step, the intelligent controller collects the image of the environment through the visual sensor and obtains the next control quantity through online learning.

As shown in Figure 1, the intelligent controller includes a vision sensor, an image feature processor, a control actuator, and a learning arithmetic unit.

2.1.1. Vision Sensor

The visual sensor collects the actual information of the current environment in the form of color image, and the image matrix is the three-dimensional image matrix of RGB mode. The image matrix contains the current state and target state of the manipulator, as well as environmental information.

2.1.2. Image Feature Processor

After graying and thresholding the received image matrix I, the image feature processor adjusts the size and deformation and then outputs the state matrix s_t to the core of the reinforcement learning algorithm. Figure 2 describes in detail the process of rr types of manipulator, xr types of manipulator, and xy types of manipulator acquiring image matrix I by visual sensor and inputting it to image feature processor for processing into state matrix s_t.

2.1.3. Control Actuator

The reinforcement learning algorithm operator generates a control amount a_t(a₁, a₂) according to the state matrix s_t and reward value r_t generated by the image feature processor. The control actuator calculates the feed increment required to control the rotating joint or slider of the manipulator through the control amount a_t and then maps the rotation angle increment directly to the joint torque M_t(m₁, m₂).

$$ (1)

where x_i and y_i represent the coordinate position of the manipulator control system.

2.1.4. Learning Arithmetic Unit

The learning arithmetic unit calculates the state matrix and reward value transmitted by the current image feature sensor, generates the control quantity, and inputs it into the manipulator through the control actuator, resulting in changes in the environment.

According to Equation (1), the principle in learning is that if the learning process deteriorates, the distance between the end position of the manipulator and the target position increases, and the reward value is reduced to punish. If the learning process is improved, the distance between the end position of the manipulator and the target position decreases, and the reward is carried out by increasing the reward value.

2.2. Machine Vision Learning Model

2.2.1. Camera Selection

When the industrial manipulator completes its work, it needs to obtain the visual information in the environment and locate the workpiece position or target position. Therefore, it is necessary to map the position information in the image to the real-world coordinate system, so it is necessary to calibrate the camera and establish the relationship between the coordinate systems.

When collecting information, we generally need to select an appropriate camera to collect the required information. The vision system in this paper is mainly composed of two cameras. The remote camera is fixed in front of the manipulator to shoot the target point, and the end camera is fixed at the end of the manipulator to form a hand eye system, which is used to complete the tasks of defect detection and so on. Therefore, the camera needs to have appropriate focal length, good definition, and resolution. The detailed parameters of the camera are shown in Table 1.

1. Detailed parameters of camera.

Index	Camera parameters
Camera model	MindVision MV-U300
Effective pixel	2048H × 1536 V (3,000,000)
Pixel size	3.2 μm × 3.2 μm
Signal to noise ratio	43 dB
Optics	1/2”CMOS color
Data interface	USB2.0 480 Mb/s
Transmission distance	5 m
Power supply mode	USB power supply

2.2.2. Camera Calibration

The process of camera calibration is to determine the projection matrix P converted from the world coordinate system to the image coordinate system. Camera calibration is generally divided into two parts. One is to convert from the world coordinate system to the camera coordinate system. In this step, the camera external parameters R, T (rotation matrix, translation matrix) and other parameters are determined. The second part is to convert from the camera coordinate system to the image coordinate system. In this step, the camera internal parameters K and other parameters are determined. The camera imaging model is divided into linear model and nonlinear model. The perspective projection model of the camera is shown in Figure 3 without considering the lens distortion.

The perspective projection model contains four coordinate systems. World coordinate system O_w − X_wY_wZ_w, that is, a real coordinate system defined by the user in the real space according to the needs, reflecting the actual spatial position of the camera and the measured object. The camera coordinate system 2, its origin is the optical center of the camera, and the z-axis coincides with the camera optical axis, and the X_c axis and Y_c axis are parallel to the xy axis of the imaging plane coordinate system. It is defined to describe the object position from the perspective of the camera. Like the plane coordinate system o^′ − xy, this coordinate system is the plane imaged by the photosensitive element of the camera and is the coordinate system obtained by translating the camera focal length f distance along the z-axis of the camera coordinate system. Image coordinate system, that is, the coordinate system o − uv of the image we see. The origin is in the upper left corner of the image, and the unit is pixel. The optical signal collected by the camera through the sensor is converted into digital signal, that is, the image we see. An image can be regarded as a matrix of M × N, and each value in the matrix is the gray value of each pixel.

3. Six-Dimensional Force Sensor Model

In the manipulator control system, the dynamic performance of the force sensor directly affects the practical application performance of the manipulator. Force sensor is a kind of electrical component that converts force signal into relevant electrical signal. Among them, the six-dimensional force sensor can measure the force and torque information about the three coordinate axis in space at the same time. It is the most complete form of force sensor.

3.2. Mathematical Model of Orthogonal Parallel Six-Dimensional Force Sensor

Figure 4 is the structural diagram of orthogonal parallel six-dimensional force sensor.

b₁b₂b₃ represents the force measuring platform, and B₁B₂B₃ represents the fixed platform. b₁B₁, B₂b₂, B₃b₃ is three force measuring branches arranged vertically, and the included angle between any two points of b₁, b₂, b₃ and the origin o is 120°. b₄B₄, b₅B₅, b₆B₆ is three horizontally arranged force measuring branches, their axes are tangent to a circle with a circle center of o^′, the three tangent points are the midpoint of the horizontal force measuring branch, and the included angle between any two tangent points and the origin o^′ is also 120°.

The rectangular coordinate system o − xyz is the measurement reference coordinate system of the six-dimensional force sensor, and the origin o is determined as the geometric center of the lower surface of the force measuring platform, wherein, the x-axis is perpendicular to the horizontal force measuring branch b₄B₄, and the z-axis is perpendicular to the fixed platform B₁B₂B₃ and upward.

4. Design of Manipulator Control System

4.1. Design of Collision Point Detection System for Manipulator Control

In order to test the feasibility of the application of the above six-dimensional force sensor model in the manipulator control system, this paper proposes a manipulator control system based on machine vision and six-dimensional force sensor. Different from the traditional application of six-dimensional force sensor, in order to realize the sensitivity of the manipulator control system, the collision detection method is used for the system design. The system design is shown in Figure 5.

4.2. Collision Point Detection Model

According to the different impact points and external forces, body collision can be roughly divided into single point single external force collision, single point external force/external torque mixed collision, multipoint external force collision, and multipoint external force/external torque mixed collision. The collision of multipoint and multiexternal force/external torque can be regarded as a special combination of single point and single external force. In order to achieve high-precision collision detection, the installation position diagram of the sensor in this paper is shown in Figure 6.

The six-dimensional force sensor can measure the component information of force and torque on three axes. Assuming that the sensor at the base collects F(F_x, F_y, F_z) and M(M_x, M_y, M_z), the position information of the collision point is expressed as:

$$ (3)

where P(P_x, P_y, P_z) is the position vector of the collision point relative to the sensor coordinate system. Equation (3) can be rewritten as:

$$ (4)

Because |F| = 0, the position solution of the collision point is not unique. At this time, other constraints need to be used to determine the position of the collision point. Substituting the data collected by the sensor to a frame into Equation (2) can obtain an external force action vector line L_c(p), which is expressed as:

$$ (5)

Different parameters collected by each frame of the sensor determine different external force vector lines. In practice, most of the directions of the external force of the collision have a certain angle with the motion direction of the robot. Therefore, the external force vector lines calculated by using different frames of the sensor will intersect at a certain point P_c in space, which is the collision point. The specific calculation method is as follows:

$$ (6)

Due to the measurement error of the sensor, the calculated spatial force vector lines will not intersect at the collision point in space, but the projection in a certain plane will intersect near the projection of the formal collision point in the plane. Therefore, the intersection of the external force vector line in the plane can be obtained by the projection method, and then, the intersection coordinates can be substituted into the original equation to obtain the collision point information, as shown in Figure 7.

Suppose $$ is the projection line of L_c(p) in XOY plane, $$ in XOZ plane, and $$ in YOZ plane. The calculation formula is:

$$ (7)

In Figure 7, P^′(x, y) is the intersection of the projection $$ and $$ of two external force vector lines, P₁ and P₂ are the corresponding points of the projection intersection on the external force vector line, and the calculation formulas of P₁ and P₂ are:

$$ (8)

where η−_is the selection factor for determining the best projection plane, and the selection rule of water projection plane is:

$$ (9)

$$ (10)

where θ_XOY, θ_XOZ, θ_YOZ− is the included angle of $$ and $$ projected by the external force vector line on the XOY, XOZ, YOZ plane.

If XOY plane is the optimal projection plane, the calculation formulas of P₁(x₁, y₁, z₁) and P₂(x₂, y₂, z₂) are:

$$ (11)

$$ (12)

The real collision point coordinates are near two points, but the exact position cannot be calculated. Therefore, it is preliminarily determined that the collision point P is the midpoint of two points, that is:

$$ (13)

At the moment of robot collision, the sensor can collect multiple groups of data, and the schematic diagram of the solved spatial external force vector line is shown in Figure 8. In order to search the optimal solution closest to the real collision point in multiple groups of calculation results, it is assumed that $$ is the error factor, and P^″(x^″, y^″, z^″) is the preliminarily calculated collision point coordinate. Available:

$$ (14)

Among the preliminarily determined collision points, search the point that minimizes $$; that is, it is considered to be the optimal solution closest to the real collision point. The optimal search results are shown in Figure 9.

5. Simulation Experiment

In order to verify the effectiveness and accuracy of the collision point detection model proposed in this paper, a 3-DOF manipulator model is constructed for simulation experiments, as shown in Figure 10.

In order to reduce the amount of calculation, the connecting rod and joint center of the manipulator are symmetrical, and the center of gravity of each connecting rod is located on its own central axis.

Table 1 shows the structural parameters of each connecting rod of the 3-DOF robot in this paper. The material is alloy, ρ = 2.7 × 10³kg/m³, In Figure 10, l₁ = 120mm, l₂ = 150mm, l₃ = 200mm, l₄ = 200mm, h = 30mm. In Table 2, the member parameters are shown as follows.

The data collected by the sensor at the base changes with time. After dynamic compensation processing, the data collected by the sensor is used as the input of Equation (4) to calculate the collision point. Import the manipulator model into Adams to verify the constructed manipulator model. In the experiment, the magnitude, direction, and position of the simulated impact force are known, and the experimental result data are based on the sensor coordinate system.

In the experiment:θ₁ = −π/6, θ₂ = −π/6, θ₂ = −π/4, $$, $$, $$, and $$. The robot moves repeatedly for 6 times according to the set parameters and carries out collision test at different points with the same collision force (100 N) when running to the third second. The experimental results are shown in Table 3.

The calculation results are shown in Figures 11 and 12.

As shown in Figure 11, Figure 11(a) shows the error value of mechanical arm experimental resultant force, Figure 11(b) shows the error value of formula computer mechanical arm resultant force, Figure 11(c) shows the relative error value of mechanical arm experimental resultant force, and Figure 11(d) shows the relative error value of formula computer mechanical arm resultant force. Compared with Figures 11(a) and 11(b), it can be found that the fluctuation of mechanical arm resultant force experimental test value is significantly larger than that calculated by formula, and there is no obvious law. The difference between the maximum error value and the minimum error value is large, up to 2.26 N, and the minimum value is only 0.49 N. The change of the calculated value of the formula shows the law that it increases with the increase of the distance from the collision point, and reaches the maximum value at the farthest position from the force sensor coordinate system, which is close to 1.5 N. Comparing Figures 11(c) and 11(d), it is found that their performance rules are roughly the same as those in Figures 11(a) and 11(b). The difference is that in the relative error, the maximum values of the two are close, and the relative error of the measured value is less than the formula calculated value. Therefore, the formula calculation can reflect the relative error of the measured computer arm control system to some extent.

When the relative distance error of the mechanical arm (a) is the collision distance of the three points, as shown in the figure, the calculation error of the mechanical arm (b) is the relative distance between the collision points, and the calculation error of the mechanical arm (a) is the basic distance of the collision point, as shown in the formula of the collision point (b), and they are all within the allowable range of error. However, it changes with the distance of the collision point. The collision error first increases with the increase of the distance, and the maximum error is 16 mm, but the relative displacement error in all directions fluctuates in the range of about 5%. Compared with the formula calculation, the experimental error value is basically the same, and the farthest relative error is no more than 5%. It can be seen that this method can meet the collision accuracy requirements of the farthest point of the manipulator.

6. Conclusion

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.