Status Update: 2023-12-19

[haixuanTao]

🤖

[haixuanTao]

Dora ROS2 Bridge

• Updated our RustDDS and ros2-client forks to include the latest changes
  • Required rewriting and redesigning our changes because the APIs changed upstream.
• Submit RustDDS pull request to upstream our new DeserializerAdapter design with DeserializeSeed support: Redesign DeserializerAdapter trait to support deserializing using DeserializeSeed jhelovuo/RustDDS#313
• Submit draft PR for updating ros2-client crate for DeserializeSeed support: Update to RustDDS with DeserializeSeed support jhelovuo/ros2-client#20

[haixuanTao]

dora-llm

• adding python sound device to capture audio from microphone
• Adding Whisper API to dora-llm
• Adding argo translate to dora-llm to translate text from different language
• (CS Team) ChatGPT API investigation for code modification.
  • Prototyping code generation and code modification.
  • Next step: LLM Agents ( Critic Agent, Memory Agent, Managing Agent (sub task), Coding Agent )

[haixuanTao]

Relevant: https://arxiv.org/pdf/2310.06770.pdf

[haixuanTao]

dora-minecraft

• Minecraft Voyager investigation: https://github.com/MineDojo/MineDojo and https://github.com/MineDojo/Voyager and now: https://github.com/dora-rs/dora-mineflayer/tree/main
• This also makes it possible to use dora with multiple player, run LLM competition.
• Rewriting Voyager in Python which is going to be harder but more reusable

[haixuanTao]

dora-FOSDEM

• Got accepted to FOSDEM ( Feb 2024 in Bruxelle ) in the embedded, mobile and automotive track
• Going to work on polishing dora-rs and some demos.

[haixuanTao]

dora-logging and output

• Working on making sending logs, stdout and stderr from nodes easier to make it simpler to build llm retroaction loop.

[haixuanTao]

Maybe unprioritize it for now for ROS2 bridge