Docs / Running tests

Connect your stack

Invariant runs your software as a container with inputs and outputs. Adapters handle the details for ROS 2, plain processes and Python, so you don't change your code to test it.

Design preview. These docs describe how Invariant is being built. Commands, output and limits are illustrative until release, and stage tags mark what ships first. See the roadmap.

The container contract MVP

Every adapter is built on one contract. If your software can follow it, Invariant can test it.

	What Invariant provides or expects
Inputs	`/invariant/in` holds the log slice or scenario files. `INVARIANT_SCENARIO` holds scenario parameters as JSON, `INVARIANT_SEED` the seed, and `INVARIANT_DEADLINE_S` the time limit.
Outputs	Write to `/invariant/out`: `run.mcap` with the topics you want scored, and optionally `metrics.json` for values computed inside your software.
Exit code	`0` means the run completed, not that it passed. Scoring happens afterwards. A non-zero exit or a timeout marks the run `failed`.
Isolation	One run per container. No network by default. Read-only inputs. CPU, memory and GPU limits from the resource profile.

ROS 2 adapter MVP

For ROS 2 stacks on Humble, Jazzy or Lyrical. The adapter starts your launch file, waits until it's ready, plays the input (log slice or simulator bridge) with sim time, records the topics you list to MCAP, and shuts everything down cleanly.

invariant.yamlsystem:
  image: ghcr.io/acme/nav_stack:${GIT_SHA}
  adapter: ros2
  ros2:
    distro: jazzy
    launch: nav_stack bringup.launch.py
    args: { use_sim_time: true }
    play: { rate: 1.0, topics: [/points, /imu, /odom] }   # from the log slice
    record: [/localization/pose, /cmd_vel, /tf]
    ready: { topic: /localization/pose, timeout_s: 30 }
    rmw: rmw_cyclonedds_cpp

Playback speed. Replay can run faster than real time (rate: 2.0) if your stack keeps up. Invariant records dropped messages and processing lag. If the candidate can't keep up, that's reported as a failure, not hidden as an infrastructure error. A run that saw different data is never scored as a pass.
Middleware. Pin the RMW in the spec. Results are only compared between runs with the same RMW, because a middleware change alone can change timing.
ROS 1. ROS 1 Noetic reached end of life on May 31, 2025 (ROS blog). It can run under the process adapter in its own container, but isn't a supported adapter.

Process adapter MVP

For anything else: a C++ binary, a Python script, a non-ROS middleware stack. You give a command, and Invariant handles inputs, outputs and limits.

invariant.yamlsystem:
  image: ghcr.io/acme/perception:${GIT_SHA}
  adapter: process
  process:
    command: ["/app/detect", "--in", "/invariant/in", "--out", "/invariant/out/run.mcap"]

Python adapter Next

For models and policies you call from Python. Invariant imports your entry point and calls it per frame or per step, which avoids container start-up per run for fast model evaluations.

eval_entry.pyfrom invariant import system

@system
class Detector:
    def setup(self, ctx):
        self.model = load_model(ctx.artifact("weights"))
    def on_frame(self, frame):
        return self.model(frame.image)   # list of boxes

Log formats

MCAP is the native format (MCAP). rosbag2 recordings in MCAP or SQLite storage are read directly. Other formats need a converter to MCAP, which you can run as a one-time step or as part of the scenario.

Next: Simulators for closed-loop scenarios.

← PreviousVerdicts and statistics Next →Simulators