Autonomous EMG-Controlled Bionic Arm

An autonomous bionic arm that uses muscle signals and computer vision to detect, approach, and grasp objects with minimal user input. The patient initiates the system with a single muscle twitch and triggers closure with a double twitch.

How It Works

1. Muscle Activation: An EMG sensor connected to an ESP32-S3 detects muscle activity and classifies the user's intended gesture.
2. Autonomous Initialization: A single twitch initializes the system and starts object detection.
3. Object Detection: YOLO detects bottles in the camera feed, while MediaPipe tracks hand landmarks.
4. Distance Measurement: A TFMini-S LiDAR connected to an Arduino Uno measures the distance to the target.
5. Target Locking: The system locks onto a bottle detected in the center region when the measured distance remains within the configured range for 1.5 seconds.
6. Grasp Trigger: Once the target is locked and the LiDAR distance drops below the configured threshold, the grab action activates and remains latched.
7. Double-Twitch Control: A double muscle twitch is intended to trigger the closure command.

Tech Stack

• Python — System integration and control logic
• OpenCV — Camera processing and visualization
• YOLO (Ultralytics) — Bottle detection
• MediaPipe — Hand landmark tracking
• NumPy & Pandas — EMG signal processing and dataset management
• Scikit-learn — Random Forest EMG gesture classification
• Joblib — Saving and loading the trained model
• PySerial — Serial communication with microcontrollers
• ESP32-S3 N16R8 — EMG signal acquisition via GPIO 4
• Arduino Uno — LiDAR data acquisition
• TFMini-S LiDAR — Distance measurement

Control Modes

• EMG Mode: Collect muscle-signal data, train the classifier, and perform live gesture classification.
• Keyboard Fallback: A single Spacebar tap starts the detection system, and a double tap stops it.

Project Goal

Build an intuitive, autonomous assistive bionic arm that reduces the need for continuous manual control by combining EMG-based user intent, computer vision, and distance sensing.

Note: The current software implements the detection, target-locking, and grab-trigger logic. Physical arm actuation and double-twitch integration should be described as implemented only if they are connected and working in the current build.