Selecting AI TYPE for instance Point Goal using remote (PC/desktop) controller aka PYTHON/NPM PACKAGES....

[vbot1983]

Is it possible to Select an AI type for instance the Point Goal Navigation using the Desktop Controllers aka PYTHON or NPM(I know currently it is not possible)

However, in the block programming codes I can see that one can select AI...

I need to do the same that is give the robot command dynamically and through script running on say Raspberry Pi -> so the pi that runs a script gives the 2D waypoint vector ...rather than the user programming the coded statically...

I can see the firbase code available in the opencode directory...so will appreciate if one of the developement contributors can explain the way forward...

[DKrishna007]

To dynamically control a robot’s navigation using a Raspberry Pi, you can set up a communication system (e.g., ROS, MQTT) where the Pi sends 2D waypoint data to the robot.

Setup Communication:

Use ROS if the robot supports it. The Pi sends waypoint data via a ROS topic, like this in Python:

rospy.init_node('waypoint_publisher')
pub = rospy.Publisher('/robot/goal', Pose2D, queue_size=10)
goal = Pose2D(x=2.0, y=3.0, theta=0.0)
pub.publish(goal)

Alternatively, use MQTT for lightweight messaging between the Pi and the robot.

Dynamic Waypoint Control:

The Raspberry Pi script can compute or receive waypoint updates dynamically (e.g., from sensors or Firebase) and publish them to the robot.

Adjust AI Parameters Dynamically:

If needed, modify AI navigation parameters (e.g., goal thresholds) via the same communication setup, allowing the robot to adapt to new commands in real-time.

If Firebase is part of your setup, you could read waypoint data from it and send it to the robot. Reaching out to the project's developers may provide more guidance on how to integrate this dynamically within their system.

Let me know if you'd like help with any part of this!

[thias15]

Hi @vbot1983, sorry for the delay. The previous answer mentions ROS and MQTT, but OpenBot doesn't actually use either. Let me try to map out what's possible today.

The robot phone exposes two control surfaces:

1. The WebSocket protocol used by controller/node-js and the Flutter controller. This protocol is geared toward joystick-style control (drive commands, indicator toggles, mode switches), not toward "navigate to coordinate (x, y)". Fragment switching is partially supported (see the work in PR Flutter controller v2 #464), but Point Goal Navigation specifically expects an AR-anchored goal that the user picks in the phone UI, not an external (x, y).
2. Block Code playground at open-code/. This is the path you've already noticed. Scripts are authored visually, compiled to JS/Python, then pushed to the phone via Firebase. Today the goal selection happens inside the script at compile time, so it isn't dynamic.

If you want a Raspberry Pi to drive the bot to dynamic waypoints, the realistic options today are:

• (a) Extend the WebSocket protocol with a new message type like {"command": "GOTO", "x": ..., "y": ...} and handle it in the pointGoalNavigation fragment so it accepts an external goal instead of an AR-anchored one. Non-trivial because the existing fragment is built around AR.
• (b) Use Firebase as the bus the Pi writes to, then have the robot poll Firebase in the existing Block Code runtime. Cheaper to prototype, but carries a Firebase dependency.
• (c) Skip Point Goal Navigation entirely and treat your Pi as the autopilot: feed direct drive commands over the existing WebSocket protocol based on whatever logic the Pi runs. This loses AR-based goal anchoring but is the path of least resistance.

None of this is shipping today, so this issue is effectively a feature request. Are you still interested in pursuing it? If so, which of (a) / (b) / (c) is closest to what you actually need?