Skip to main content
Learn to create your first agent, train your first manipulation model, give MARS the ability to read emails, and put the pieces together

​
Create your first agent

Now that you know the basics, you can create your first agent for MARS using the SDK. On the app, go to Configuration -> WiFi and read the IP of the robot. All your code lives and runs on the robot, so the comfortable way to work is to open your editor (VSCode, Cursor, Windsurf, …) directly on MARS over Remote SSH — full file tree, search, and terminal, as if the robot’s filesystem were local. Development Setup gets you there in a few minutes. In a hurry? A plain terminal works too: 
ssh jetson1@<YOUR-ROBOT-IP>
Either way, go to ~/innate-os/workspace/custom_agents/ and create a hello_world.py agent file: 
from typing import List
from brain_client.agent_types import Agent

class HelloWorld(Agent):
    @property
    def id(self) -> str:
        return "hello_world"

    @property
    def display_name(self) -> str:
        return "Hello World"

    def get_skills(self) -> List[str]:
        return [
            "innate-os/navigate_to_position",
            "innate-os/wave",
        ]
    def get_inputs(self) -> List[str]:
        return ["micro"]

    def get_prompt(self) -> str:
        return """
You are a friendly greeting robot whose sole purpose is to say hello world to the user!

Your personality:
- You are a nice and cheerful robot.

Instructions:
- When you see a user in front of you, say "hello world" and wave at the user.
- Don't navigate, just turn around if you don't see the user.
"""
“wave” and “navigate_to_position” are basic skills that come already created for the robot. This agent makes use of them to act autonomously. Save the file — the runtime watches this directory and hot-reloads it within a second or two (the same goes for any later edits). Open the app, and your agent appears on the Home screen (pull down to refresh if needed): start it, sit in front of the robot, and observe! If it ever doesn’t show up, innate service restart is the reliable fallback.
This is the same agent structure dissected method-by-method in the reference docs:

Anatomy of an Agent

Every method explained, plus a copy-paste template for new agents.

​
Train your first manipulation model for a skill

Innate Robots arms can be trained using state-of-the-art manipulation AI models, running straight on the onboard computer. For MARS, we developed an improved version of ACT (Action Chunking with Transformers) with a reward model — see the training overview for details. To train it, you can use the app to collect episodes of data for imitation learning. I.E. you will be repeatedly performing the task with the robot for a given amount of repetitions to make sure it learns it the way you want. In the app, go to Skills -> Physical, create a new skill, name it, and press “Add Episodes”. You demonstrate the task with the leader arm — a teleoperation controller that MARS mirrors; if you’re unsure what it is or why it has no camera and a trigger instead of a gripper, see What the leader arm is. Then, arm the arm and press record to collect an episode. Ideally, all episodes should start in a similar position and end in a similar position, following roughly the same movement. Start with very similar trajectories to accomplish the goal while making sure that the arm camera has the objective of motion relatively in sight. More guidelines on training can be found in Data Collection. Below, an example of training the arm to pick up a cherry.
Once you collected around 50 episodes, you can start considering stopping data collection. We can easily train your dataset for you if you go to the Training tab and press Train with whole dataset. You can also use the episodes for yourself by ssh-ing in the robot and getting them there. Once the model is trained (which takes up to 4 hours), you can get it back on your robot and then trigger it from Manual Control Screen!

​
Create your first digital skill

Innate robots can also run any kind of code in the embodied Innate agent, which can be used to query APIs online or run custom routines onboard. As an example, let’s give MARS a skill that reads your latest Gmail messages. Grab the complete RetrieveEmails implementation from the Digital Skills worked example and save it as ~/innate-os/workspace/custom_skills/retrieve_emails.py on the robot — it’s copy-paste-ready, you only need to fill in your Gmail address and an app password. Then run the skill from an agent that can query it: 
from typing import List
from brain_client.agent_types import Agent

class EmailAssistant(Agent):
    @property
    def id(self) -> str:
        return "email_assistant"

    @property
    def display_name(self) -> str:
        return "Email Assistant"

    def get_skills(self) -> List[str]:
        # Custom skills are namespaced "local/", shipped skills "innate-os/"
        return [
            "local/retrieve_emails"
        ]

    def get_prompt(self) -> str:
        return """
You are an email assistant. 
When the user sits in front of you, you should tell them what their last email is
"""
To learn more about the Skills SDK and go further:

Skills

See policy-defined and code-defined skills with interface references.

Advanced Development

Modify ROS2 packages, recompile, and take full control of MARS.