Policy-Defined Skills

A policy-defined skill has no code. It’s a folder with a metadata.json and a model or recording, created for you by the apps.

Type	What it is	Use it when	How to make one
Learned	An ACT policy that reacts to what the cameras see	Object positions vary: picking, placing, grasping	Train it from 30+ demonstrations
Replay	A recorded motion, played back exactly	The motion is always the same: a wave, a gesture	Record it once in the app and save it as a replay skill

Replay skills also record head movement, so the robot looks where you looked.

How a learned skill runs

The arm moves to the start pose saved at training time.
Every 40 ms (25 Hz), the policy reads both cameras and the joint positions.
It outputs 6 arm joint commands and 2 base velocity commands.
It stops when it predicts the task is done, or when the time limit runs out.

On the robot the model runs as an optimized TensorRT engine, built automatically when the model downloads.

Call it from code

Import a policy like any skill class. The robot generates the class from the skill’s folder:

from innate_skills.pick_socks import PickSocks

def get_skills(self) -> list[SkillRef]:
    return [PickSocks]

Inside another skill, declare it and call it with a timeout: pick: PickSocks, then self.pick(timeout=60). See Call other skills. If the policy is still training and not on this robot yet, there is no class to import. Declare it by name instead: pick = PhysicalSkill("pick_socks").

Metadata file

You never write this by hand. Training and recording create it.

{
  "name": "pick_socks",
  "type": "learned",
  "guidelines": "Use when you need to pick up socks from the floor",
  "execution": {
    "model_type": "act_policy",
    "checkpoint": "run_abc123/act_policy_step_135000.pth",
    "stats_file": "run_abc123/dataset_stats.pt",
    "action_dim": 10,
    "duration": 45.0,
    "start_pose": [-0.015, -0.399, 1.456, -1.135, -0.023, 0.833]
  }
}

{
  "name": "wave",
  "type": "replay",
  "guidelines": "Use when greeting someone or saying hello",
  "execution": {
    "model_type": "replay",
    "replay_file": "episode_0.h5",
    "replay_frequency": 50.0,
    "start_pose": [1.577, -0.6, 1.477, -0.738, 0.0, 0.0],
    "end_pose": [1.577, -0.6, 1.477, -0.738, 0.0, 0.0]
  }
}

Edit guidelines to change when the agent uses the skill.

Get Started

Simulator

Build Agents and Skills

MARS

Train the Arm

Go Deeper

Help

Hackathon

Policy-Defined Skills

How a learned skill runs

Call it from code

Metadata file

​How a learned skill runs

​Call it from code

​Metadata file

How a learned skill runs

Call it from code

Metadata file