Skip to content

realsense stuff - #3449

Open
jeff-hykin wants to merge 8 commits into
mainfrom
jeff/feat/stereo_prep
Open

realsense stuff#3449
jeff-hykin wants to merge 8 commits into
mainfrom
jeff/feat/stereo_prep

Conversation

@jeff-hykin

@jeff-hykin jeff-hykin commented Aug 13, 2026

Copy link
Copy Markdown
Member

realsense stuff for stereo vslam

…ntrols

The camera publishes its own mount tree read off the device (factory
extrinsics) instead of xacro-transcribed constants; frames are named after
the camera model actually plugged in. Every sensor is pinned to the host
clock, the IMU runs at one negotiated rate reported back, and color
auto-exposure priority is exposed. Extracted from jeff/feat/cuvslam.
@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
4260 4 4256 69
View the top 2 failed test(s) by shortest run time
dimos.e2e_tests.test_dimsim_path_replaning::test_path_replanning
Stack Traces | 117s run time
lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155b8799730>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8873600>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7155b8b53ec0>
direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b8b2bf80>
spawn_wall_on_pose = <function spawn_wall_on_pose.<locals>.spawn at 0x7155b88504a0>

    @pytest.mark.self_hosted_large
    def test_path_replanning(
        lcm_spy, start_blueprint, dim_sim, direct_cmd_vel_explorer, spawn_wall_on_pose
    ) -> None:
        start_blueprint(
            "--dimsim-scene=empty",
            "run",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        # robot spawns at (3, 2)
    
        # side wall
        dim_sim.add_wall(2, -2.5, 12, -2.5)
        # other side wall
        dim_sim.add_wall(2, 3.5, 12, 3.5)
        # back wall (behind robot)
        dim_sim.add_wall(2, -2.5, 2, 3.5)
        # forward wall (far end)
        dim_sim.add_wall(12, -2.5, 12, 3.5)
        # dividing wall at x=7 with doors at y=[-1.5,-0.5] and y=[1.5,2.5]
        dim_sim.add_wall(7, -2.5, 7, -1.5)
        dim_sim.add_wall(7, -0.5, 7, 1.5)
        dim_sim.add_wall(7, 2.5, 7, 3.5)
    
        direct_cmd_vel_explorer.linear_speed = 0.8
>       direct_cmd_vel_explorer.follow_points([(10, 2), (2.5, 2), (3, 2)])

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7155b8b53ec0>
direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b8b2bf80>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155b8799730>
spawn_wall_on_pose = <function spawn_wall_on_pose.<locals>.spawn at 0x7155b88504a0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8873600>

dimos/e2e_tests/test_dimsim_path_replaning.py:47: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../simulation/mujoco/direct_cmd_vel_explorer.py:105: in follow_points
    self._wait_for_pose()
        self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b8b2bf80>
        waypoints  = [(10, 2), (2.5, 2), (3, 2)]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b8b2bf80>

    def _wait_for_pose(self) -> PoseStamped:
        self._new_pose.clear()
        self._new_pose.wait(timeout=5.0)
>       assert self._pose is not None, "No odom received"
E       AssertionError: No odom received

self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b8b2bf80>

.../simulation/mujoco/direct_cmd_vel_explorer.py:65: AssertionError
dimos.e2e_tests.test_dimsim_walk_forward::test_walk_forward
Stack Traces | 201s run time
lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155ba578c50>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8851300>
human_input = <function human_input.<locals>.send_human_input at 0x7155b88513a0>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x715612ba3050>

    @pytest.mark.self_hosted_large
    def test_walk_forward(lcm_spy, start_blueprint, human_input, dim_sim) -> None:
        start_blueprint(
            "run",
            "--disable",
            "spatial-memory",
            "--disable",
            "security-module",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        origin_x, origin_y = 1, 2
        dim_sim.set_agent_position(origin_x, origin_y)
    
        human_input("move forward 3 meter")
    
>       lcm_spy.wait_until_odom_position(origin_x + 3, origin_y, threshold=0.4, timeout=120)

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x715612ba3050>
human_input = <function human_input.<locals>.send_human_input at 0x7155b88513a0>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155ba578c50>
origin_x   = 1
origin_y   = 2
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8851300>

dimos/e2e_tests/test_dimsim_walk_forward.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/lcm_spy.py:167: in wait_until_odom_position
    self.wait_for_message_result(
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7155b88520c0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155ba578c50>
        threshold  = 0.4
        timeout    = 120
        x          = 4
        y          = 2
dimos/e2e_tests/lcm_spy.py:153: in wait_for_message_result
    wait_until(
        event      = <threading.Event at 0x7156137226f0: unset>
        fail_message = 'Failed to get to position x=4, y=2'
        listener   = <function LcmSpy.wait_for_message_result.<locals>.listener at 0x7155b8852480>
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7155b88520c0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155ba578c50>
        timeout    = 120
        topic      = '/odom#geometry_msgs.PoseStamped'
        type       = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

predicate = <bound method Event.is_set of <threading.Event at 0x7156137226f0: unset>>

    def wait_until(
        predicate: Callable[[], bool],
        *,
        timeout: float,
        interval: float = 0.1,
        message: str | None = None,
    ) -> None:
        """Poll ``predicate`` until it returns truthy or ``timeout`` elapses."""
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            if predicate():
                return
            time.sleep(interval)
>       raise TimeoutError(message or f"Timed out after {timeout}s waiting for condition")
E       TimeoutError: Failed to get to position x=4, y=2

deadline   = 3318934.472226195
interval   = 0.1
message    = 'Failed to get to position x=4, y=2'
predicate  = <bound method Event.is_set of <threading.Event at 0x7156137226f0: unset>>
timeout    = 120

.../utils/testing/waiting.py:35: TimeoutError
View the full list of 2 ❄️ flaky test(s)
dimos.e2e_tests.test_dimsim_spatial_memory::test_go_to_the_bed

Flake rate in main: 16.67% (Passed 125 times, Failed 25 times)

Stack Traces | 108s run time
lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155ba1203b0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8850ea0>
human_input = <function human_input.<locals>.send_human_input at 0x7155b8851080>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7155b9e5f0b0>
explore_house = <function explore_house.<locals>.explore at 0x7155b8851760>

    @pytest.mark.self_hosted_large
    def test_go_to_the_bed(lcm_spy, start_blueprint, human_input, dim_sim, explore_house) -> None:
        start_blueprint(
            "run",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
>       explore_house()

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7155b9e5f0b0>
explore_house = <function explore_house.<locals>.explore at 0x7155b8851760>
human_input = <function human_input.<locals>.send_human_input at 0x7155b8851080>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7155ba1203b0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8850ea0>

dimos/e2e_tests/test_dimsim_spatial_memory.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/conftest.py:249: in explore
    direct_cmd_vel_explorer.follow_points(points)
        direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b9fb8410>
        points     = [(3.881, 4.803), (4.16, 1.615), (1.596, 1.505), (1.649, 0.137), (-3.644, -0.064), (-3.759, -2.661), ...]
.../simulation/mujoco/direct_cmd_vel_explorer.py:105: in follow_points
    self._wait_for_pose()
        self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b9fb8410>
        waypoints  = [(3.881, 4.803), (4.16, 1.615), (1.596, 1.505), (1.649, 0.137), (-3.644, -0.064), (-3.759, -2.661), ...]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b9fb8410>

    def _wait_for_pose(self) -> PoseStamped:
        self._new_pose.clear()
        self._new_pose.wait(timeout=5.0)
>       assert self._pose is not None, "No odom received"
E       AssertionError: No odom received

self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7155b9fb8410>

.../simulation/mujoco/direct_cmd_vel_explorer.py:65: AssertionError
dimos.e2e_tests.test_manipulation_planning_groups::test_single_arm_plans_and_executes_through_control_coordinator

Flake rate in main: 100.00% (Passed 0 times, Failed 1 times)

Stack Traces | 139s run time
lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x715612b8dbe0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8852340>

    def test_single_arm_plans_and_executes_through_control_coordinator(
        lcm_spy: LcmSpy,
        start_blueprint: Callable[..., DimosCliCall],
    ) -> None:
        """Plan with one arm and execute through its trajectory task."""
        _start_openarm_mock_planner(start_blueprint, lcm_spy)
    
        client = RPCClient(None, ManipulationModule)
        coordinator_client = RPCClient(None, ControlCoordinator)
        try:
>           [left] = _wait_for_groups(client, 1)

client     = <dimos.core.rpc_client.RPCClient object at 0x7155ba38db80>
coordinator_client = <dimos.core.rpc_client.RPCClient object at 0x715612394590>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x715612b8dbe0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7155b8852340>

dimos/e2e_tests/test_manipulation_planning_groups.py:158: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

client = <dimos.core.rpc_client.RPCClient object at 0x7155ba38db80>, count = 1

    def _wait_for_groups(
        client: RPCClient,
        count: int,
        *,
        timeout: float = 120.0,
    ) -> tuple[PlanningGroupInfo, ...]:
        deadline = time.time() + timeout
        last_error: BaseException | None = None
        while time.time() < deadline:
            try:
                groups = client.list_planning_groups()
                if len(groups) == count:
                    return groups
            except Exception as exc:
                last_error = exc
            time.sleep(0.5)
>       raise TimeoutError(f"Timed out waiting for {count} planning groups") from last_error
E       TimeoutError: Timed out waiting for 1 planning groups

client     = <dimos.core.rpc_client.RPCClient object at 0x7155ba38db80>
count      = 1
deadline   = 1786849990.1175327
groups     = (PlanningGroupInfo('left_arm/manipulator', joints=('left_arm/openarm_left_joint1', 'left_arm/openarm_left_joint2', 'le...right_joint6', 'right_arm/openarm_right_joint7'), base='openarm_body_link0', tip='openarm_right_link7', gripper=False))
last_error = None
timeout    = 120.0

dimos/e2e_tests/test_manipulation_planning_groups.py:64: TimeoutError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Aug 13, 2026
@jeff-hykin jeff-hykin closed this Aug 13, 2026
auto-merge was automatically disabled August 13, 2026 01:00

Pull request was closed

@jeff-hykin jeff-hykin reopened this Aug 14, 2026
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Aug 14, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Aug 14, 2026
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Aug 14, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Aug 15, 2026
@github-actions github-actions Bot added ready-to-merge Required CI checks have passed on this PR and removed ready-to-merge Required CI checks have passed on this PR labels Aug 15, 2026
@jeff-hykin
jeff-hykin enabled auto-merge August 16, 2026 02:50
@github-actions github-actions Bot added ready-to-merge Required CI checks have passed on this PR and removed ready-to-merge Required CI checks have passed on this PR labels Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PlzReview ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant