PAROL6 Headless Control System

A Research-Grade Robotic Platform for Advanced Manipulation

Status: September 2025 snapshot, no longer maintained. This repository is now quite outdated. The Python API moved into the official repository, PCrnjak/PAROL6-python-API, where it has since been developed much further together with other community contributors, so use that for anything new. I'm now working on a second version of the vision system for the PAR6, the PAROL6's successor.

Demo

Gemini Live search-and-pick: one pick-and-place cycle, sped up (no audio).

Full demo with voice commands (1:45):

PAROL6.Vision.Demo.with.sound.web.mp4

Overview

This project presents a comprehensive control system for the PAROL6 robotic arm, transforming it from a GUI-dependent platform into a fully programmable research tool capable of vision-guided manipulation and natural language control. Originally developed to meet the requirements of a personal robotics project, the system has since been adopted by the PAROL6 community and officially recognized by the robot's creator.

I built this during my BEng in Electrical and Mechanical Engineering at the University of Edinburgh (I'm now doing an MSc in Robotics at TU Delft). I identified the need for programmatic control of the PAROL6 arm to enable advanced experimentation beyond what the existing GUI interface allowed. This led to the development of a complete client-server architecture that decouples robot control from user interface dependencies, enabling remote operation, automated tasks, and integration with modern perception systems.

Technical Architecture

The system employs a UDP-based client-server architecture that separates the computational and control layers, allowing for distributed processing and flexible deployment. The server (headless_commander.py) maintains a high-frequency control loop (100Hz) for real-time robot operation, while clients can send commands from any networked device using the lightweight Python API.

Core Components

• Control Server: Handles serial communication with the robot, inverse kinematics calculations, and trajectory planning. Maintains command queue with acknowledgment tracking for reliable operation.

• Client API: Provides 30+ functions for robot control, from basic joint movements to complex smooth trajectories. Designed with minimal dependencies to enable deployment on resource-constrained devices.

• Vision Pipeline: Integrates Intel RealSense D435I depth camera through a custom 3D-printed mount attached to the electric gripper. Complete hand-eye calibration system enables accurate transformation between camera and robot coordinate frames.

• Trajectory Generation: Advanced smooth motion capabilities including circular paths, parametric arcs, splines, and helical movements. These features were specifically developed in response to community requests, demonstrating the system's extensibility.

Engineering Challenges Solved

Hand-Eye Calibration

One of the most significant technical challenges was achieving accurate hand-eye calibration between the RealSense camera and the robot's end-effector. The solution involved developing a complete calibration pipeline using ChArUco markers, with iterative refinement to achieve millimeter-level accuracy in 3D positioning. The calibration system accounts for the camera's mounting geometry and provides validated transformations for vision-guided manipulation.

Real-Time Coordination

Synchronizing vision processing, trajectory planning, and robot control required careful system design. The architecture ensures that vision data is properly transformed and validated before commanding robot movements, with safety checks at multiple levels to prevent collisions or invalid movements.

Motion Synchronization

The system implements sophisticated speed synchronization across all six joints, ensuring smooth coordinated motion even when joints have different ranges and speed limits. This was particularly important for the community-requested smooth motion features, where maintaining constant end-effector velocity is crucial.

Community Impact & Development

The project has evolved significantly through community interaction. After sharing the initial headless control system on the PAROL6 Discord, it gained immediate traction, leading to:

• Official Recognition: The robot's creator incorporated the system into the official repository and promoted it as a recommended control method
• Feature Expansion: Based on user requests, I developed additional capabilities including smooth trajectory generation, despite having no personal need for these features
• Continued Development: The API has since been carried forward in the official repository by other community contributors, well beyond this snapshot

This community-driven approach has transformed the project from a personal tool into a platform that enables other researchers and enthusiasts to pursue their own robotics experiments.

System Capabilities

The platform demonstrates several advanced capabilities:

Precise Motion Control: Full kinematic control with multiple input modes - joint angles, Cartesian poses, and smooth trajectories. The system validates all movements through inverse kinematics before execution.

Vision-Guided Manipulation: Integration with depth sensing enables 3D object localization and approach planning. The system uses modern vision-language models for flexible object detection, allowing natural descriptions rather than predefined object classes.

Safety Systems: Multiple layers of safety including E-stop monitoring, joint limit checking, and speed synchronization. The system maintains safe operation even during experimental procedures.

Extensible Framework: Modular design allows easy integration of new sensors, control algorithms, or AI models. The UDP communication protocol enables control from various programming languages and platforms.

Technical Implementation Details

The codebase is organized into distinct modules:

• headless_commander.py (161KB): Main robot controller handling all serial communication and command execution
• robot_api.py (57KB): Comprehensive client API with all control functions
• smooth_motion.py (32KB): Advanced trajectory generation algorithms using scipy interpolation
• PAROL6_ROBOT.py: Robot configuration including DH parameters and joint limits
• Vision subsystem: Camera calibration, hand-eye calibration, and 3D position calculation modules

The system requires specific library versions for the core robot control (roboticstoolbox-python 1.0.3, numpy 1.23.4) to ensure compatibility with the kinematic calculations, while peripheral modules can use current library versions.

Applications & Research Potential

The platform has been successfully demonstrated for pick-and-place tasks with natural language commands, showcasing the feasibility of intuitive human-robot interaction. While currently at the proof-of-concept stage for complex manipulation, the system provides a solid foundation for research in:

• Human-Robot Interaction: Natural language interfaces for robot control reduce the barrier to entry for non-technical users
• Manipulation Strategies: The vision system and precise control enable experimentation with different grasping and placement techniques
• Multi-Modal Control: The framework supports various input modalities, from direct joint control to high-level task descriptions

Future Directions

The platform is designed for continuous evolution. Potential areas for expansion include force-feedback integration for delicate manipulation tasks, multi-robot coordination using the distributed architecture, and advanced planning algorithms for complex assembly operations. The open nature of the system encourages experimentation and modification for specific research needs.

Conclusion

This project demonstrates the transformation of a hobbyist robotic platform into a research-grade system through systematic engineering and community collaboration. By addressing real needs and responding to user feedback, the system has grown beyond its original scope to become a valuable tool for the robotics community. The combination of robust engineering, practical problem-solving, and collaborative development exemplifies the approach needed for advancing robotics research.

Credits & Origins

The controller is built on Petar Crnjak's PAROL commander software (experimental_kinematics branch, starting from Commander_minimal_version.py). From it I kept the robot's kinematics, which are unmodified, and the core run loop and serial communication that talk to the controller firmware. Almost everything else was transformed or newly written for this project: the UDP client-server architecture and command acknowledgement system, the motion planning, the smooth trajectory generation, the vision and hand-eye calibration pipeline, and the Gemini integration.

Running It

This is a snapshot, so expect some setup friction. The official API is the better starting point for new projects.

Robot control

1. Install the base software and dependencies as described in API-Specific-README.md, which is also the full API reference.
2. Start the controller from the repository root. It homes the robot on startup unless you pass --no-auto-home, and it remembers the port in com_port.txt:

   python Headless/headless_commander.py --com-port COM5

3. Control the robot from any script with the client API, e.g. from Headless.robot_api import move_robot_joints (run with the repository root on PYTHONPATH).

Vision and Gemini

1. Extra dependencies: pip install pyrealsense2 opencv-python google-genai python-dotenv pillow, plus pyaudio for voice input.
2. Put your Gemini API key in a GEMINI_API_KEY environment variable, or in a .env file in the repository root (GEMINI_API_KEY=...).
3. Calibrate your own camera-gripper setup: print a board with Headless/Vision/generate_charuco_board.py, run calibrate_hand_eye.py (and optionally calibration_refinement.py), then point DEFAULT_CALIBRATION_FILE in Headless/Vision/vision_controller.py at your result. The included calibration file only fits my custom camera-gripper mount, and it contains manual offset corrections because the ChArUco board I used was slightly warped.
4. With the controller running, start the Gemini controller from the repository root:

   PYTHONPATH=. python Headless/Gemini/gemini.py --input audio   # push-to-talk (space); omit --input for text mode

   On Windows PowerShell, set the path with $env:PYTHONPATH="." first. The Gemini model names in gemini.py date from September 2025 and may need updating.

License

GPL-3.0, matching the upstream PAROL software this builds on. See LICENSE.