Training from the CLI - Innate Docs

Most people should train from the apps. Use the training_client CLI to script training, work headless, or train with your own fork of the ACT code. It ships with Innate OS; run it on the robot.

Setup

source ~/innate-os/ros2_ws/install/setup.zsh
export INNATE_SERVICE_KEY="<your-service-key>"   # from ~/innate-os/.env

Every command takes a SKILL_DIR, the skill’s folder in custom_skills/. It defaults to the current folder.

Workflow

cd ~/innate-os/workspace/custom_skills/my-skill
alias tc="python -m training_client.cli"

tc submit .                           # register the skill, writes its ID to metadata.json
tc upload .                           # upload the episodes
tc run . --preset act-default         # start training
tc watch . <run-id>                   # wait until it finishes
tc download . <run-id>                # fetch the checkpoint
tc activate . <run-id>                # make the robot use it

Commands

Command	Description
`submit [SKILL_DIR]`	Create (or reuse) a cloud skill from the dataset dir; writes `training_skill_id` to `metadata.json`
`upload [SKILL_DIR]`	Upload the skill’s data files
`run [SKILL_DIR]`	Launch a training run (options below)
`status [SKILL_DIR] RUN_ID`	Print a run’s current status
`watch [SKILL_DIR] RUN_ID`	Poll a run until it completes (`--interval` seconds, default 20)
`runs [SKILL_DIR]`	List all runs for the skill
`skills`	List all your skills
`download [SKILL_DIR] RUN_ID`	Download a completed run’s results (`--dest` for the output dir)
`fetch-data [SKILL_DIR]`	Download the skill’s input training data (`--dest`)
`activate [SKILL_DIR] RUN_ID`	Set the trained checkpoint + stats file into `metadata.json` so the robot loads it

Run options

Start from a preset and override what you need, or set everything yourself:

Option	Description
`--preset`	Server-side preset (e.g. `act-default`) — pre-fills everything below
`--repo`	GitHub `owner/repo` of a custom ACT fork to train with
`--ref`	Branch, tag, or commit in that repo
`--command`, `-c`	Training command parts (repeatable)
`--env`, `-e`	Environment variables as `KEY=VALUE` (repeatable)
`--gpu-type`	GPU type (e.g. `H100`)
`--min-gpus` / `--max-gpus`	GPU count bounds
`--hours`	Estimated duration
`--budget`	Max total cost in USD
`--checkpoint-patterns`, `-p`	Globs for which output files to upload (e.g. `checkpoints/*/.pt`)

python -m training_client.cli run ./my-skill --preset act-default --max-gpus 4 --budget 500

Train with your own ACT fork

Change the model or loss in your fork of the ACT code, then point a run at it. No server changes needed.

python -m training_client.cli run ./my-skill \
    --repo innate-inc/ACT-test \
    --ref lambda_refactor \
    -c "pip install -r requirements.txt; python3 -m act_test.train_dist --data-dir /data/dataset" \
    --hours 8 --gpu-type H100 --min-gpus 1 --max-gpus 2 \
    --budget 200