Manipulation

Overview

The manipulation system consists of a compact 5 degree-of-freedom (DOF) robotic arm designed for research and development in robotic manipulation. The system combines precise motor control, integrated vision capabilities, and a modular end-effector design to support a wide range of manipulation tasks.

Users can develop manipulation strategies through multiple approaches:

Learning-based policies trained through teleoperation demonstrations
Hardcoded motion sequences for repeatable tasks
Recorded trajectories that can be played back for specific operations

The arm's integration with the innate SDK enables straightforward development of both learned and programmatic manipulation policies. Through the included leader arm interface, users can easily demonstrate desired behaviors, which can then be used to train learning-based policies or recorded for direct playback.

Hardware

Specifications

The robotic arm is a 5 degree-of-freedom (DOF) manipulator with a modular end effector. The arm's movement is powered by a combination of high-quality Dynamixel servo motors:

3 x Dynamixel XL430 motors
- Typically used for major joints requiring higher torque
- Enhanced positioning accuracy
- Built-in PID control
3 x Dynamixel XL330 motors
- Used for lighter-load movements and end effector
- Optimized for speed and precision
- Energy-efficient operation

Vision System

Integrated arm camera
- 150-degree field of view
- Provides visual feedback during teleoperation
- Enables vision-based manipulation tasks

Performance Characteristics

Maximum reach: 10 inches
Payload capacity: 200 grams at full extension
Working envelope: Spherical segment defined by maximum reach

Joint Configuration

The 5 DOF configuration provides:

Base rotation
Shoulder movement
Elbow articulation
Wrist pitch
Wrist roll

End Effector System

The arm features a modular end effector mount that supports:

User-designed custom end effectors
Swappable tool attachments
Additional end effector designs (coming soon)

This modularity allows users to adapt the arm for various applications by designing and implementing their own end effector solutions. Future releases will include new end effector designs to expand the arm's capabilities.

Teleoperation

Leader Arm Setup

To teleoperate the robot using the included leader arm:

Prerequisites:
- Ensure no other manipulation tasks are running
- Use command: innate manipulation pause
- Have access to a workstation with available USB-C port
Hardware Connection:
- Connect leader arm to workstation via USB-C
- Note: Leader arm can draw up to 1 Amp of current
- Ensure stable power supply to workstation
Initialization:
- Enter command: innate manipulation teleop
- Wait for initialization (takes a few seconds)
- System will initialize both leader and follower arms
Operation:
- Follower arm will mirror leader arm's joint configuration
- Camera feed window will automatically display
- Real-time visual feedback provided through arm camera

Recorded Behaviors

Setup

Connect leader arm to workstation via USB-C
Verify no other manipulation tasks are running using command line
Run initialization command:

innate manipulation train

Task Configuration

When prompted, provide the following information:

Task Type Selection
- Enter 'r' for recorded policy
Task Name
- Provide a descriptive name for the task
Task Description
- Enter a concise description (1-2 sentences)
- Include:
  - Objects to be manipulated
  - Task objectives
  - Key guidelines
- Note: This description will be used when calling the task via the agent

Recording Process

Once teleoperation mode initializes, use the following controls:

Spacebar: Multiple functions
- Start recording the trajectory
- Save recorded trajectory
X: Cancel and delete recording
Escape: Save and exit

Storage Location

Recorded behaviors are stored in ~/behaviors directory on the robot

Note: Unlike learned policies, recorded behaviors:

Only require one successful demonstration of the desired trajectory
Will replay the exact recorded trajectory when executed
Provide no autonomous adaptation to environment changes

Learned Policies

Maurice provides a streamlined process for developing learned manipulation policies through demonstration. Users can create new manipulation behaviors by demonstrating tasks via teleoperation, without requiring expertise in machine learning. The system handles the underlying complexities of policy training and deployment.

Development Process

Data Collection
- Demonstrate tasks using the leader arm teleoperation interface
- Capture multiple demonstrations to provide task variations
- System automatically logs relevant state and action data
- No manual data formatting required
Data Upload
- Upload demonstration data to Maurice console
- System validates data integrity automatically
- Access demonstration playback for verification
- Organize demonstrations by task type
Policy Configuration
- Select neural network architecture for the policy
- Choose from available base models as starting points
- Configure model structure and parameters
- Adjust training parameters such as:
  - Learning rate
  - Epochs
- Default configurations provided for common use cases
Training Execution
- Initiate training through Maurice console
- Monitor training status via progress dashboard
- System automatically handles optimization process
- Training typically takes 1-6 hours depending on task complexity
Deployment
- Download trained policy files
- Load policy onto Maurice system
- Verify behavior matches demonstrations
- Deploy to production environment

The process enables technical users to develop manipulation policies based on practical task knowledge while maintaining control over the underlying model architecture and training process.

Data Collection Process

Setup

Connect leader arm to workstation via USB-C
Verify no other manipulation tasks are running using command line
Run initialization command:


innate manipulation train

Task Configuration

When prompted, provide the following information:

Task Type Selection
- Enter 'l' for learned policy
- Enter 'r' for recorded policy
Task Name
- Provide a descriptive name for the task
Task Description
- Enter a concise description (1-2 sentences)
- Include:
  - Objects to be manipulated
  - Task objectives
  - Key guidelines
- Note: This description will be used when calling the task via the agent

Recording Demonstrations

Once teleoperation mode initializes, use the following controls:

Spacebar: Multiple functions
- Start recording a new example
- Save current example
X: Cancel and delete current example
Escape: Save all episodes and exit

Data Collection Tips

Vary Task Settings
- Change object positions between demonstrations
- Vary robot's initial position
- Modify environmental conditions when applicable
- These variations help policy generalize to new situations
Maintain Consistency
- Use the same strategy across all demonstrations
- Keep movement patterns similar
- Maintain consistent grasp points
- Uniform approach angles when possible
Handle Failures
- When a demonstration fails, continue to completion
- Do not cancel failed attempts
- Retry the task with the same configuration
- Failed attempts provide valuable learning data

Data Access

Recorded demonstrations are stored in ~/data directory
Access data via SSH connection to robot
Data is automatically formatted for training

Data Management Commands

List All Tasks

innate data list

This command displays:

Task names
Data size
Task specifics

View Task Details

innate data status task_name

This command shows:

Task type (learned/recorded)
Task description
Number of episodes
Data statistics:
- Episode lengths
- Other relevant metrics

Adding More Data

To add additional demonstrations to an existing task:

Run the training command again:


innate manipulation train

Enter the same task name as before
New demonstrations will be appended to existing data
Verify updated data status using data status command

Data Upload

Upload Command

innate data upload task_name

Requirements

Robot must remain powered on
Stable internet connection required
Upload time varies with internet speed (up to 45 minutes)

Monitor Progress Check upload status using:

innate data upload task_name -p

Verification in Maurice Console

Visit Maurice data console
Navigate to "Data Sets" section
Locate your uploaded task
Verify:
- All episodes are present
- Playback episodes to check upload quality

Best Practices

Ensure stable power supply during upload
Maintain consistent network connection
Verify upload completion before powering down
Monitor progress periodically for large datasets

Policy Training

Policy Configuration

Navigate to "Policies" section in Maurice console
Click "Add New Policy"
Select architecture and base model
- Currently supports ACT and Random Weights
- More improved base models coming soon
Configure training parameters
- Learning rate
- Number of epochs
- Default values work well for most tasks
Select training datasets in dataset tab
Click execute to begin training

Training Execution

Training runs in background and can be monitored via the policy console, where you can track key parameters like train and validation loss. Policy training typically takes between 1-6 hours depending on dataset size and number of epochs.

Download Policy

Download Command

innate policy download task_name

Storage Location

Model weights are stored in ~/policy directory on the robot

Policy Evaluation

To test the downloaded policy, use:

innate policy run task_name -t num_seconds

This will run the policy for the specified number of seconds.

Control Via SDK

Setup

from innate import manipulation
manipulation.init()

Joint Control

# Get current joint positions# Returns numpy array of length 6 representing joint angles
joint_pose = manipulation.get_joint_pose()

# Set joint positions# joint_pose: numpy array of length 6 containing joint angles
manipulation.set_joint_pose(joint_pose)

# Execute joint trajectory# joint_poses: list of numpy arrays, each of length 6# times: list of corresponding timestamps
manipulation.set_joint_trajectory(joint_poses, times)

End Effector Control

# Get end effector pose# Returns manipulation.Pose object with position and quaternion
ee_pose = manipulation.get_ee_pose()

# Set end effector pose# ee_pose: manipulation.Pose object with:#   - position: [x, y, z]#   - quaternion: [x, y, z, w]
manipulation.set_ee_pose(ee_pose)

# Execute end effector trajectory# ee_poses: list of manipulation.Pose objects# times: list of corresponding timestamps
manipulation.set_ee_trajectory(ee_poses, times)

Gripper Control

# Get current gripper pressure# Returns float value between 0 and 1
pressure = manipulation.get_gripper_pressure()

# Set gripper pressure# pressure: float value between 0 and 1
manipulation.set_gripper_pressure(pressure)

Behavior and Policy Execution

# Run a recorded behavior
manipulation.run_behavior(task_name)

# Run a learned policy for specified duration# time: duration in seconds
manipulation.run_policy(task_name, time)

# Interrupt current execution
manipulation.interrupt()

Example Usage

from innate import manipulation
import time
import numpy as np

# Initialize the SDK
manipulation.init()

# Example 1: Basic joint control
print("Moving to initial position...")
# Small joint angles for compact arm
initial_joints = np.array([0.0, -0.2, 0.3, 0.0, -0.2, 0.0])  # radians
manipulation.set_joint_pose(initial_joints)
time.sleep(2)  # Wait for movement

# Example 2: Gripper control
print("Testing gripper...")
manipulation.set_gripper_pressure(0.5)  # 50% pressure for delicate grip
time.sleep(1)
manipulation.set_gripper_pressure(0.0)  # Open gripper
time.sleep(1)

# Example 3: End effector trajectory
poses = []
times = []
# Create a small square motion (1cm movements)
base_pose = manipulation.get_ee_pose()  # Get current pose
for i in range(4):
    new_pose = base_pose
    if i == 0:
        new_pose.position[0] += 0.01  # Move 1cm in x
    elif i == 1:
        new_pose.position[2] -= 0.01  # Move 1cm down
    elif i == 2:
        new_pose.position[0] -= 0.01  # Move back in x
    elif i == 3:
        new_pose.position[2] += 0.01  # Move back up
    poses.append(new_pose)
    times.append(i * 1.0)  # 1 second per movement

print("Executing small square trajectory...")
manipulation.set_ee_trajectory(poses, times)

# Example 4: Run a learned policy
print("Running pick and place policy...")
manipulation.run_policy("pick_and_place", 5)  # Run for 5 seconds

# Example 5: Run a recorded behavior
print("Running wave behavior...")
manipulation.run_behavior("wave_hello")

Control Via Command Line

Get joint positions

innate manipulation get-joint-pose
# Returns: Six space-separated values representing joint angles in radians

Set joint positions

innate manipulation set-joint-pose j1 j2 j3 j4 j5 j6
# Input: 6 space-separated float values representing joint angles in radians

Get end effector pose

innate manipulation get-ee-pose
# Returns: Seven space-separated values (position x y z and quaternion x y z w)

Set end effector pose

innate manipulation set-ee-pose x y z qx qy qz qw
# Input:#   - Position: 3 space-separated floats (x y z)#   - Quaternion: 4 space-separated floats (qx qy qz qw)

Get gripper pressure

innate manipulation get-gripper-pressure
# Returns: Float value between 0 and 1

Set gripper pressure

innate manipulation set-gripper-pressure pressure
# Input: Float value between 0 and 1

Run behavior

innate manipulation run-behavior task_name
# Input: String name of the recorded behavior

Run policy

innate manipulation run-policy task_name -t time
# Input:#   - task_name: String name of the policy#   - time: Integer number of seconds to run

Interrupt execution

innate manipulation interrupt

Example Usage

#!/bin/bash

echo "Testing various manipulation commands..."

# Get current joint positions
echo "Current joint positions:"
innate manipulation get-joint-pose

# Move to a specific joint configuration (small angles)
echo "Moving to new position..."
innate manipulation set-joint-pose 0.0 -0.2 0.3 0.0 -0.2 0.0
sleep 2

# Get end effector pose
echo "Current end effector pose:"
innate manipulation get-ee-pose

# Move end effector to new pose (small movements)
echo "Moving end effector..."
# Moving to position slightly forward and up (1cm increments)
innate manipulation set-ee-pose 0.01 0.0 0.01 0.0 0.0 0.0 1.0
sleep 2

# Test gripper
echo "Testing gripper..."
innate manipulation set-gripper-pressure 0.5
sleep 1
innate manipulation set-gripper-pressure 0.0
sleep 1

# Run a recorded behavior
echo "Running wave behavior..."
innate manipulation run-behavior wave_hello
sleep 3

# Run a policy for 5 seconds
echo "Running pick and place policy..."
innate manipulation run-policy pick_and_place -t 5

echo "Done!"

PreviousWorkstation Setup NextNavigation

Last updated 6 months ago