Spatial Memory - Innate Docs

While MARS drives, it keeps useful views tied to their place on the map. A skill can search these in plain language, like “the kitchen” or “my keys”, and get back the best match with a photo and coordinates. Requires OS 0.7.0 or newer.

Remembered views on the map, in the web app's Nav page

Search and drive there

from innate import Skill, SpatialMemory
from innate_skills.navigate_to_position import NavigateToPosition


class GoToRemembered(Skill):
    memory: SpatialMemory
    navigate: NavigateToPosition

    def guidelines(self) -> str:
        return "Find something the robot has seen before, and drive there."

    def execute(self, query: str):
        verdict = self.wait_for(self.memory.begin(query), timeout=150.0)
        if verdict is None or verdict.error:
            self.fail("Memory search failed")
        if not verdict.found:
            return f"I don't remember seeing {query}."

        self.say(verdict.message)
        self.navigate(x=verdict.x, y=verdict.y, local_frame=False)
        return f"Went to where I last saw {query}"

A search can take a couple of minutes on a big map, so begin() returns a reader you wait on. Waiting with self.wait_for() keeps Stop working.

Result

Field	Contents
`found`	Whether anything matched. `False` is a valid answer, not an error.
`message`	The answer, ready to say out loud
`image`	JPEG bytes of the remembered view
`x`, `y`, `theta`	Where the robot stood, in the map frame. `theta` is in radians.
`seen_stamp`	When it was seen
`explanation`	Why the model picked this view
`error`	Set only if the search itself broke

The shipped search_memory skill does the search and returns SkillOutput(verdict.message, image=verdict.image), so the agent can look at the photo. Add it to an agent and the robot will check its memory before saying it hasn’t seen something.

Memories are stored per map on the robot. Browse or clear them on the web app’s Nav page.