JackM323/JackBot
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects 2 Releases Wiki Activity

unfinished Readme changes

Browse Source
This commit is contained in:
JackM323
2026-08-13 09:11:16 +02:00
parent 12b8f80002
commit f5c07edc0a
1 changed files with 43 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+43 -53
View File
@@ -9,40 +9,38 @@ JackBot is a modular 3D hexapod robot control and machine learning framework bui
JackBot is a Python-based hexapod robot project that combines: JackBot is a Python-based hexapod robot project that combines:
* a robot control stack for a six-legged walking robot, * a robot control stack for a six-legged walking robot,
* a physics simulator for testing in software before using real hardware, and * a physics simulator (`simulation.py`) for testing in software before using real hardware, and
* a reinforcement learning pipeline that teaches the robot how to move through trial and error. * a reinforcement learning pipeline that teaches the robot how to move through trial and error or pretraining from kinematics.
The project is not just one script or one model. It is a full control system that can: The project is a full control system that can:
* run the robot in a PyBullet simulation, * run the robot in a PyBullet simulation,
* accept human commands from a GUI or gamepad, * accept human commands from a GUI or gamepad,
* stream target joint positions to physical hardware, and * stream target joint positions to physical hardware (ESP32 / Arduino), and
* train a policy using PPO so the robot can learn locomotion automatically. * train and evaluate policy checkpoints using PPO and Behavioral Cloning (BC).
## Why the Project Exists ## Why the Project Exists
A hexapod is hard to control manually because each leg has several joints and the robot has to maintain balance while moving. Instead of hard-coding every motion rule, this project uses the robot model and physics simulation as a testbed to learn walking behavior. A hexapod is hard to control manually because each leg has several joints and the robot has to maintain balance while moving. Instead of hard-coding every motion rule, this project uses the robot model and physics simulation as a testbed to learn or refine walking behavior.
In practice, the workflow looks like this: In practice, the RL workflow looks like this:
1. The robot starts from a standing pose. 1. The robot starts from a standing pose.
2. The policy receives sensory information from the simulation. 2. The policy receives sensory information (joint angles + command vector) from the simulation.
3. The policy chooses new joint target motions. 3. The policy chooses continuous target joint positions.
4. The simulation updates the physics. 4. The simulation steps forward and computes physics.
5. The policy receives a reward for surviving and moving in the right direction. 5. The policy receives shaped rewards for surviving, following commands, and remaining stable.
6. Over many iterations, PPO improves the controller. 6. Over many iterations, PPO improves the locomotion policy.
--- ---
## Key Components ## Key Components
* **Robot abstraction (`Robot.py`)**: wraps the robot body, robot state, and joint control API. * **Robot abstraction (`Robot.py`)**: Wraps the robot body, state machine execution, joint control API, and backend selection.
* **Kinematics (`kinematics.py`)**: turns target leg poses into actual joint angles through inverse kinematics. * **Kinematics (`kinematics.py`)**: Turns target leg positions into actual joint angles through inverse kinematics.
* **Simulation (`simulation.py`)**: creates and updates the PyBullet world where the robot can be tested safely. * **Simulation (`simulation.py`)**: Consolidates PyBullet scene management, motor positioning, physics queries, and environment stepping into a single manager.
* **Inputs (`inputs/`)**: lets the robot be controlled from either a GUI, a gamepad, or a generated target command. * **Inputs (`inputs/`)**: Interface for receiving control commands from a GUI, gamepad, or procedural random generators.
* **ML environment (`ml/env.py`)**: provides the Gymnasium environment that the policy interacts with. * **ML Subsystem (`ml/`)**: Contains Gymnasium environments (`env.py`), custom callbacks (`callbacks.py`), behavioral cloning pretraining (`pretrain_bc.py`), PPO training (`run_train.py`), and multi-phase evaluation scripts (`run_eval.py`, `run_eval_training.py`).
* **PPO training (`ml/train.py`)**: trains a policy using stable-baselines3.
* **Evaluation (`ml/evaluate.py`)**: loads a saved model and runs policy rollouts to inspect performance.
--- ---
@@ -50,40 +48,40 @@ In practice, the workflow looks like this:
```text ```text
JackBot/ JackBot/
├── main.py # manual control and hardware streaming entry point ├── main.py # Manual control, GUI, and hardware streaming entry point
├── Robot.py # robot wrapper and backend abstraction ├── Robot.py # Robot wrapper, gait execution, and backend abstraction
├── config.py # global settings and communication configuration ├── config.py # Global settings and hardware/backend configuration
├── kinematics.py # inverse kinematics used to map motion commands to joints ├── kinematics.py # Inverse kinematics mapping cartesian targets to joint angles
├── simulation.py # PyBullet simulation shell for the robot ├── simulation.py # PyBullet simulation manager (handles engine, URDF loading, & physics)
├── robot_init.py # neutral standing pose and initial joint values ├── robot_init.py # Neutral standing pose and initial joint definitions
├── DataTypes.py # typed data structures for positions and joint angles ├── DataTypes.py # Typed numpy structures for positions and joint angles
│ │
├── states/ # gait/state-machine behavior logic ├── states/ # Gait and state-machine behavior logic
│ ├── State.py │ ├── State.py
│ ├── IdleState.py │ ├── IdleState.py
│ ├── WalkingState.py │ ├── WalkingState.py
│ └── WaveState.py │ └── WaveState.py
│ │
├── inputs/ # command sources for the robot ├── inputs/ # Command input sources
│ ├── InputProvider.py │ ├── InputProvider.py
│ ├── PygameController.py │ ├── PygameController.py
│ └── RandomeInputProvider.py │ └── RandomeInputProvider.py
│ │
├── gui/ # visual control frontend ├── gui/ # Visual control frontend (Tkinter/CustomTkinter)
│ └── MainWindow.py │ └── MainWindow.py
│ │
├── EspCommunication.py # ESP32 communication layer ├── EspCommunication.py # ESP32 socket communication layer
├── ArduinoCommunication.py # Arduino serial communication layer ├── ArduinoCommunication.py # Arduino serial communication layer
├── JackBotUrdf.urdf # robot mesh and joint definition ├── JackBotUrdf.urdf # Robot URDF mesh and joint axis definition
│ │
└── ml/ # reinforcement learning subsystem └── ml/ # Reinforcement Learning Subsystem
├── env.py # Gymnasium environment used by PPO ├── env.py # Gymnasium environment wrapping PyBullet & Robot control
├── SimManager.py # PyBullet scene setup and stepping ├── callbacks.py # Custom SB3 callbacks for TensorBoard logging & Curriculum progression
├── MetricsOverlay.py # HUD and visual overlays ├── pretrain_bc.py # Behavioral Cloning (BC) script to pre-train policy from IK teacher
├── train.py # PPO training entry point ├── run_train.py # Main training entry point (runs parallelized PPO via SubprocVecEnv)
├── run_train.py # command-line training launcher ├── run_eval.py # Phase-by-phase policy evaluator with CLI reports
├── evaluate.py # policy evaluation loop ├── run_eval_training.py # Benchmark reward system across phases using pure kinematics
└── run_eval.py # CLI wrapper for evaluation └── MetricsOverlay.py # PyBullet HUD debug text and visual overlays
``` ```
--- ---
@@ -91,7 +89,7 @@ JackBot/
## System Requirements ## System Requirements
* **Python 3.12** (Recommended) * **Python 3.12** (Recommended)
* **OS:** Linux (Ubuntu/Debian) or Windows 10/11 * **OS:** Windows 10/11 or Linux
* **Dependencies:** `pybullet`, `gymnasium`, `stable-baselines3`, `torch`, `numpy`, `pygame`, `ikpy`, `pyserial`, `matplotlib` * **Dependencies:** `pybullet`, `gymnasium`, `stable-baselines3`, `torch`, `numpy`, `pygame`, `ikpy`, `pyserial`, `matplotlib`
--- ---
@@ -118,9 +116,9 @@ pip install -r requirements.txt
--- ---
## Usage Guide ### Usage Guide
### 1. Manual Control & Hardware Streaming (`main.py`) ## Manual Control & Hardware Streaming (`main.py`)
`main.py` is the operational entry point for driving the robot manually via GUI sliders or a gamepad, running either in 3D PyBullet simulation or connected to physical hardware. `main.py` is the operational entry point for driving the robot manually via GUI sliders or a gamepad, running either in 3D PyBullet simulation or connected to physical hardware.
@@ -139,19 +137,11 @@ Edit `config.py` prior to launching `main.py` to configure execution mode and co
--- ---
### 2. How the Machine Learning System Works ### Machine Learning Pipeline (`ml/`)
The machine learning subsystem teaches the robot to move by interacting with a simulation environment instead of relying on a manually written gait controller. The machine learning system trains the hexapod to move using a combination of Behavioral Cloning (BC) (optional) and Proximal Policy Optimization (PPO) driven by a multi-phase curriculum.
At a high level: ### ML Workflow Steps
* The policy receives a vector of observations from the simulator.
* The policy outputs a set of continuous joint deltas.
* Those outputs are applied to the robot in the physics simulation.
* The environment computes a reward based on survival, movement, stability, and command-following quality.
* PPO updates the policy over many training iterations to improve the controller.
#### What the policy sees and does
The policy input contains: The policy input contains: