Machine Learning Trainer

Training environment to make a walk model for the hexapod
generated code that will be checked

ml/README.md

@@ -0,0 +1,59 @@
+JackBot ML training and evaluation
+
+Quick-start
+
+1. Create a Python virtualenv and activate it:
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+```
+
+2. Install dependencies (GPU users should install `torch` appropriate for their CUDA):
+
+```bash
+pip install -r requirements.txt
+```
+
+3. Quick training (short, for smoke test):
+
+```bash
+python ml/run_train.py --timesteps 50000 --model ml/checkpoints/ppo_joint_command
+```
+
+Use multi-robot training with `--num-robots` and `--robot-spacing`:
+
+```bash
+python ml/run_train.py --timesteps 50000 --model ml/checkpoints/ppo_joint_command --num-robots 3 --robot-spacing 0.75
+```
+
+Choose the start pose at reset with `--start-pose`:
+
+```bash
+python ml/run_train.py --timesteps 50000 --model ml/checkpoints/ppo_joint_command --start-pose init_deg
+```
+
+If you want to watch the agent train in the PyBullet window, add `--gui`:
+
+```bash
+python ml/run_train.py --timesteps 50000 --model ml/checkpoints/ppo_joint_command --gui
+```
+
+4. Evaluate the trained model in GUI mode:
+
+```bash
+python ml/run_eval.py --model ml/checkpoints/ppo_joint_command.zip --episodes 3 --gui
+```
+
+For evaluation with multiple robots and start pose:
+
+```bash
+python ml/run_eval.py --model ml/checkpoints/ppo_joint_command.zip --episodes 3 --gui --num-robots 2 --robot-spacing 0.6 --start-pose init_deg
+```
+
+Notes
+
+- `ml/env.py` exposes a `JackBotEnv` gym environment that uses your `JackBotUrdf.urdf`.
+- The observation is `[joint_angles..., vx, vy, vz, omega]` and the action is per-joint delta in [-1,1].
+- Start with small timesteps and in `use_gui=False` to speed up iteration. Increase timesteps and tune the reward for better walking quality.
+- Keep the trained models off hardware until they behave well in simulation.