A 4DOF trained in simulation using reinforcement learning to reach a target, and transferred via serial to the physical arm. Task: reach — moving the tip of the arm (tool) towards a red dot generated within the arm's real workspace, using PPO (Proximal Policy Optimization, Stable-Baselines3) on a PyBullet model built from the real URDF/meshes of the mechanism. 



Wiring (-> motorshield pins)

| Servo | Funzione | Pin |

| 1 | Base | A1 |

| 2 | Shoulder | A0 |

| 3 | Elbow | 6 |

| 4 | Gripper | 9 |

| 5 | BT Module | v,g,rx,tx |

Neural network

An input layer with 17 neurons (one for each observation value: 4 (positions) + 4 (velocities) + 3 (ee_pos) + 3 (target_pos) + 3 (delta_pos)) 

An output layer with 4 neurons (one for each motor). 

SOFTWARE

Mearm_real_env.py 

Gymnasium environment for the MeArm/Keyestudio KS0198X robotic arm, simulated in PyBullet from the real URDF (URDF and mashes available on GitHub). 

Serial_bridge.py 

Sim-to-real bridge: reads a trained policy and sends the corresponding servo angles to the physical arm via serial. 

Train.py 

PPO training (with linear reward) of the 'reach' task on the real MeArm model.Run on an M1 mac for 1M steps across 4 parallel environments (you can use even more steps if necessary for your specific case). PPO takes the data from all 4, averages it, and updates the weights much more stably.

To monitor training progress with TensorBoard, open the terminal:tensorboard --logdir ./tb_logs/" 

Evaluate.py 

Loads the trained model and shows it in action within the PyBullet GUI.