unfinished Readme changes
This commit is contained in:
@@ -9,40 +9,38 @@ JackBot is a modular 3D hexapod robot control and machine learning framework bui
|
||||
JackBot is a Python-based hexapod robot project that combines:
|
||||
|
||||
* a robot control stack for a six-legged walking robot,
|
||||
* a physics simulator for testing in software before using real hardware, and
|
||||
* a reinforcement learning pipeline that teaches the robot how to move through trial and error.
|
||||
* a physics simulator (`simulation.py`) for testing in software before using real hardware, and
|
||||
* a reinforcement learning pipeline that teaches the robot how to move through trial and error or pretraining from kinematics.
|
||||
|
||||
The project is not just one script or one model. It is a full control system that can:
|
||||
The project is a full control system that can:
|
||||
|
||||
* run the robot in a PyBullet simulation,
|
||||
* accept human commands from a GUI or gamepad,
|
||||
* stream target joint positions to physical hardware, and
|
||||
* train a policy using PPO so the robot can learn locomotion automatically.
|
||||
* stream target joint positions to physical hardware (ESP32 / Arduino), and
|
||||
* train and evaluate policy checkpoints using PPO and Behavioral Cloning (BC).
|
||||
|
||||
## Why the Project Exists
|
||||
|
||||
A hexapod is hard to control manually because each leg has several joints and the robot has to maintain balance while moving. Instead of hard-coding every motion rule, this project uses the robot model and physics simulation as a testbed to learn walking behavior.
|
||||
A hexapod is hard to control manually because each leg has several joints and the robot has to maintain balance while moving. Instead of hard-coding every motion rule, this project uses the robot model and physics simulation as a testbed to learn or refine walking behavior.
|
||||
|
||||
In practice, the workflow looks like this:
|
||||
In practice, the RL workflow looks like this:
|
||||
|
||||
1. The robot starts from a standing pose.
|
||||
2. The policy receives sensory information from the simulation.
|
||||
3. The policy chooses new joint target motions.
|
||||
4. The simulation updates the physics.
|
||||
5. The policy receives a reward for surviving and moving in the right direction.
|
||||
6. Over many iterations, PPO improves the controller.
|
||||
2. The policy receives sensory information (joint angles + command vector) from the simulation.
|
||||
3. The policy chooses continuous target joint positions.
|
||||
4. The simulation steps forward and computes physics.
|
||||
5. The policy receives shaped rewards for surviving, following commands, and remaining stable.
|
||||
6. Over many iterations, PPO improves the locomotion policy.
|
||||
|
||||
---
|
||||
|
||||
## Key Components
|
||||
|
||||
* **Robot abstraction (`Robot.py`)**: wraps the robot body, robot state, and joint control API.
|
||||
* **Kinematics (`kinematics.py`)**: turns target leg poses into actual joint angles through inverse kinematics.
|
||||
* **Simulation (`simulation.py`)**: creates and updates the PyBullet world where the robot can be tested safely.
|
||||
* **Inputs (`inputs/`)**: lets the robot be controlled from either a GUI, a gamepad, or a generated target command.
|
||||
* **ML environment (`ml/env.py`)**: provides the Gymnasium environment that the policy interacts with.
|
||||
* **PPO training (`ml/train.py`)**: trains a policy using stable-baselines3.
|
||||
* **Evaluation (`ml/evaluate.py`)**: loads a saved model and runs policy rollouts to inspect performance.
|
||||
* **Robot abstraction (`Robot.py`)**: Wraps the robot body, state machine execution, joint control API, and backend selection.
|
||||
* **Kinematics (`kinematics.py`)**: Turns target leg positions into actual joint angles through inverse kinematics.
|
||||
* **Simulation (`simulation.py`)**: Consolidates PyBullet scene management, motor positioning, physics queries, and environment stepping into a single manager.
|
||||
* **Inputs (`inputs/`)**: Interface for receiving control commands from a GUI, gamepad, or procedural random generators.
|
||||
* **ML Subsystem (`ml/`)**: Contains Gymnasium environments (`env.py`), custom callbacks (`callbacks.py`), behavioral cloning pretraining (`pretrain_bc.py`), PPO training (`run_train.py`), and multi-phase evaluation scripts (`run_eval.py`, `run_eval_training.py`).
|
||||
|
||||
---
|
||||
|
||||
@@ -50,40 +48,40 @@ In practice, the workflow looks like this:
|
||||
|
||||
```text
|
||||
JackBot/
|
||||
├── main.py # manual control and hardware streaming entry point
|
||||
├── Robot.py # robot wrapper and backend abstraction
|
||||
├── config.py # global settings and communication configuration
|
||||
├── kinematics.py # inverse kinematics used to map motion commands to joints
|
||||
├── simulation.py # PyBullet simulation shell for the robot
|
||||
├── robot_init.py # neutral standing pose and initial joint values
|
||||
├── DataTypes.py # typed data structures for positions and joint angles
|
||||
├── main.py # Manual control, GUI, and hardware streaming entry point
|
||||
├── Robot.py # Robot wrapper, gait execution, and backend abstraction
|
||||
├── config.py # Global settings and hardware/backend configuration
|
||||
├── kinematics.py # Inverse kinematics mapping cartesian targets to joint angles
|
||||
├── simulation.py # PyBullet simulation manager (handles engine, URDF loading, & physics)
|
||||
├── robot_init.py # Neutral standing pose and initial joint definitions
|
||||
├── DataTypes.py # Typed numpy structures for positions and joint angles
|
||||
│
|
||||
├── states/ # gait/state-machine behavior logic
|
||||
├── states/ # Gait and state-machine behavior logic
|
||||
│ ├── State.py
|
||||
│ ├── IdleState.py
|
||||
│ ├── WalkingState.py
|
||||
│ └── WaveState.py
|
||||
│
|
||||
├── inputs/ # command sources for the robot
|
||||
├── inputs/ # Command input sources
|
||||
│ ├── InputProvider.py
|
||||
│ ├── PygameController.py
|
||||
│ └── RandomeInputProvider.py
|
||||
│
|
||||
├── gui/ # visual control frontend
|
||||
├── gui/ # Visual control frontend (Tkinter/CustomTkinter)
|
||||
│ └── MainWindow.py
|
||||
│
|
||||
├── EspCommunication.py # ESP32 communication layer
|
||||
├── EspCommunication.py # ESP32 socket communication layer
|
||||
├── ArduinoCommunication.py # Arduino serial communication layer
|
||||
├── JackBotUrdf.urdf # robot mesh and joint definition
|
||||
├── JackBotUrdf.urdf # Robot URDF mesh and joint axis definition
|
||||
│
|
||||
└── ml/ # reinforcement learning subsystem
|
||||
├── env.py # Gymnasium environment used by PPO
|
||||
├── SimManager.py # PyBullet scene setup and stepping
|
||||
├── MetricsOverlay.py # HUD and visual overlays
|
||||
├── train.py # PPO training entry point
|
||||
├── run_train.py # command-line training launcher
|
||||
├── evaluate.py # policy evaluation loop
|
||||
└── run_eval.py # CLI wrapper for evaluation
|
||||
└── ml/ # Reinforcement Learning Subsystem
|
||||
├── env.py # Gymnasium environment wrapping PyBullet & Robot control
|
||||
├── callbacks.py # Custom SB3 callbacks for TensorBoard logging & Curriculum progression
|
||||
├── pretrain_bc.py # Behavioral Cloning (BC) script to pre-train policy from IK teacher
|
||||
├── run_train.py # Main training entry point (runs parallelized PPO via SubprocVecEnv)
|
||||
├── run_eval.py # Phase-by-phase policy evaluator with CLI reports
|
||||
├── run_eval_training.py # Benchmark reward system across phases using pure kinematics
|
||||
└── MetricsOverlay.py # PyBullet HUD debug text and visual overlays
|
||||
```
|
||||
|
||||
---
|
||||
@@ -91,7 +89,7 @@ JackBot/
|
||||
## System Requirements
|
||||
|
||||
* **Python 3.12** (Recommended)
|
||||
* **OS:** Linux (Ubuntu/Debian) or Windows 10/11
|
||||
* **OS:** Windows 10/11 or Linux
|
||||
* **Dependencies:** `pybullet`, `gymnasium`, `stable-baselines3`, `torch`, `numpy`, `pygame`, `ikpy`, `pyserial`, `matplotlib`
|
||||
|
||||
---
|
||||
@@ -118,9 +116,9 @@ pip install -r requirements.txt
|
||||
|
||||
---
|
||||
|
||||
## Usage Guide
|
||||
### Usage Guide
|
||||
|
||||
### 1. Manual Control & Hardware Streaming (`main.py`)
|
||||
## Manual Control & Hardware Streaming (`main.py`)
|
||||
|
||||
`main.py` is the operational entry point for driving the robot manually via GUI sliders or a gamepad, running either in 3D PyBullet simulation or connected to physical hardware.
|
||||
|
||||
@@ -139,19 +137,11 @@ Edit `config.py` prior to launching `main.py` to configure execution mode and co
|
||||
|
||||
---
|
||||
|
||||
### 2. How the Machine Learning System Works
|
||||
### Machine Learning Pipeline (`ml/`)
|
||||
|
||||
The machine learning subsystem teaches the robot to move by interacting with a simulation environment instead of relying on a manually written gait controller.
|
||||
The machine learning system trains the hexapod to move using a combination of Behavioral Cloning (BC) (optional) and Proximal Policy Optimization (PPO) driven by a multi-phase curriculum.
|
||||
|
||||
At a high level:
|
||||
|
||||
* The policy receives a vector of observations from the simulator.
|
||||
* The policy outputs a set of continuous joint deltas.
|
||||
* Those outputs are applied to the robot in the physics simulation.
|
||||
* The environment computes a reward based on survival, movement, stability, and command-following quality.
|
||||
* PPO updates the policy over many training iterations to improve the controller.
|
||||
|
||||
#### What the policy sees and does
|
||||
### ML Workflow Steps
|
||||
|
||||
The policy input contains:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user