RobotecAI · maciejmajek · Nov 15, 2025 · Oct 6, 2025 · Oct 6, 2025 · Oct 6, 2025
diff --git a/.github/workflows/poetry-test.yml b/.github/workflows/poetry-test.yml
@@ -46,7 +46,7 @@ jobs:
           jackd -d dummy -r 44100 &
 
       - name: Install python dependencies
-        run: poetry install --with openset,nomad,s2s,simbench
+        run: poetry install --with perception,nomad,s2s,simbench
 
       - name: Update rosdep
         shell: bash

diff --git a/README.md b/README.md
@@ -50,7 +50,7 @@ guide and tutorials. 📚
 -   [x] rai_tts: Text-to-speech models and tools.
 -   [x] rai_sim: Package for connecting RAI to simulation environments.
 -   [x] rai_bench: Benchmarking suite for RAI. Test agents, models, tools, simulators, etc.
--   [x] rai_openset: Openset detection models and tools.
+-   [x] rai_perception: Object detection tools based on open-set models and machine learning techniques.
 -   [x] rai_nomad: Integration with NoMaD for navigation.
 -   [ ] rai_finetune: Finetune LLMs on your embodied data.
 

diff --git a/docker/Dockerfile b/docker/Dockerfile
@@ -25,7 +25,7 @@ RUN apt-get update && apt-get install -y \
     wget
 
 # Install Poetry
-RUN curl -sSL https://install.python-poetry.org | python3 - --version 1.8.4
+RUN curl -sSL https://install.python-poetry.org | python3 - --version 2.1.1
 ENV PATH="/root/.local/bin:$PATH"
 
 # Clone and setup RAI

diff --git a/docs/ROS_2/ros_packages.md b/docs/ROS_2/ros_packages.md
@@ -2,9 +2,9 @@
 
 RAI includes multiple configurable ROS 2 packages.
 
-| Package                 | Description                                                                                                                                                                                           | Documentation                                   |
-| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
-| **rai_open_set_vision** | Package enabling use of [GroundingDINO](https://github.com/IDEA-Research/GroundingDINO) and [GroundedSAM](https://github.com/IDEA-Research/Grounded-SAM-2) -- an open-set detection model with ROS 2. | [rai_open_set_vision](../extensions/openset.md) |
-| **rai_nomad**           | Package integrating [NoMaD](https://general-navigation-models.github.io/nomad/index.html) -- an exploration model with ROS2.                                                                          | [rai_nomad](../extensions/nomad.md)             |
-| **rai_interfaces**      | Definition of custom messages and services used in RAI.                                                                                                                                               |                                                 |
-| **rai_bringup**         | Launch files to run RAI.                                                                                                                                                                              |                                                 |
+| Package            | Description                                                                                                                                                                                                                                | Documentation                                 |
+| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------- |
+| **rai_perception** | Object detection tools based on open-set models and machine learning techniques. Integrates [GroundingDINO](https://github.com/IDEA-Research/GroundingDINO) and [GroundedSAM](https://github.com/IDEA-Research/Grounded-SAM-2) with ROS 2. | [rai_perception](../extensions/perception.md) |
+| **rai_nomad**      | Package integrating [NoMaD](https://general-navigation-models.github.io/nomad/index.html) -- an exploration model with ROS2.                                                                                                               | [rai_nomad](../extensions/nomad.md)           |
+| **rai_interfaces** | Definition of custom messages and services used in RAI.                                                                                                                                                                                    |                                               |
+| **rai_bringup**    | Launch files to run RAI.                                                                                                                                                                                                                   |                                               |
diff --git a/docs/demos/manipulation.md b/docs/demos/manipulation.md
@@ -22,7 +22,7 @@ manipulation techniques.
 2. Download additional dependencies:
 
     ```shell
-    poetry install --with openset
+    poetry install --with perception
     vcs import < demos.repos
     rosdep install --from-paths src/examples/rai-manipulation-demo/ros2_ws/src --ignore-src -r -y
     ```

diff --git a/docs/demos/rosbot_xl.md b/docs/demos/rosbot_xl.md
@@ -30,7 +30,7 @@ platform in a nice apartment.
     sudo apt install ros-${ROS_DISTRO}-navigation2 ros-${ROS_DISTRO}-nav2-bringup
     vcs import < demos.repos
     rosdep install --from-paths src --ignore-src -r -y
-    poetry install --with openset
+    poetry install --with perception
     ```
 
 !!! tip "Alternative: Demo source build"
@@ -70,7 +70,7 @@ platform in a nice apartment.
 
 The rosbot demo utilizes several components:
 
-1. Vision processing using Grounded SAM 2 and Grounding DINO for object detection and segmentation. See [RAI OpenSet Vision](../extensions/openset.md).
+1. Vision processing using Grounded SAM 2 and Grounding DINO for object detection and segmentation. See [RAI perception](../extensions/perception.md).
 2. RAI agent to process the request and interact with environment via [tool-calling](https://python.langchain.com/docs/concepts/tool_calling/) mechanism.
 3. Navigation is enabled via [nav2 toolkit](../API_documentation/langchain_integration/ROS_2_tools.md#nav2), which interacts with [ROS 2 nav2](https://docs.nav2.org/) asynchronously by calling [ros2 actions](https://docs.ros.org/en/jazzy/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Actions/Understanding-ROS2-Actions.html).
 4. Embodiment of the Rosbot is achieved using [RAI Whoami](../tutorials/create_robots_whoami.md) module. This makes RAI agent aware of the hardware platform and its capabilities.

diff --git a/docs/extensions/openset.md → docs/extensions/perception.md b/docs/extensions/openset.md → docs/extensions/perception.md
@@ -1,4 +1,4 @@
---8<-- "src/rai_extensions/rai_open_set_vision/README.md:sec1"
+--8<-- "src/rai_extensions/rai_perception/README.md:sec1"
 Agents create two ROS 2 Nodes: `grounding_dino` and `grounded_sam` using [ROS2Connector](../API_documentation/connectors/ROS_2_Connectors.md).
 These agents can be triggered by ROS2 services:
 
@@ -15,15 +15,15 @@ These agents can be triggered by ROS2 services:
 
 ## RAI Tools
 
-`rai_open_set_vision` package contains tools that can be used by [RAI LLM agents](../tutorials/walkthrough.md)
+`rai_perception` package contains tools that can be used by [RAI LLM agents](../tutorials/walkthrough.md)
 enhance their perception capabilities. For more information on RAI Tools see
 [Tool use and development](../tutorials/tools.md) tutorial.
 
---8<-- "src/rai_extensions/rai_open_set_vision/README.md:sec3"
+--8<-- "src/rai_extensions/rai_perception/README.md:sec3"
 
 > [!TIP]
 >
 > you can try example below with [rosbotxl demo](../demos/rosbot_xl.md) binary.
 > The binary exposes `/camera/camera/color/image_raw` and `/camera/camera/depth/image_raw` topics.
 
---8<-- "src/rai_extensions/rai_open_set_vision/README.md:sec4"
+--8<-- "src/rai_extensions/rai_perception/README.md:sec4"
diff --git a/docs/setup/install.md b/docs/setup/install.md
@@ -43,7 +43,7 @@ There are two ways to start using RAI:
 
 !!! important "Package availability"
 
-    `rai_openset` and `rai_nomad` are not yet available through pip. If your workflow relies on openset detection or NoMaD integration, please refer to the
+    `rai_perception` and `rai_nomad` are not yet available through pip. If your workflow relies on openset detection or NoMaD integration, please refer to the
     [developer environment instructions](#setting-up-developer-environment) setup.
 
     `rai_interfaces` is available as `apt` package. However, due to package distribution delays, the latest version may not be immediately available. If you encounter missing imports, please build `rai_interfaces` from [source](https://github.com/RobotecAI/rai_interfaces).
@@ -113,14 +113,14 @@ rosdep install --from-paths src --ignore-src -r -y
     install additional dependencies:
 
     ```bash
-    poetry install --with openset,nomad,s2s,simbench # or `--all-groups` for full setup
+    poetry install --with perception,nomad,s2s,simbench # or `--all-groups` for full setup
     ```
 
     | Group Name | Description | Dependencies |
     |------------|-------------|--------------|
     | [s2s][s2s] | Speech-to-Speech functionality | rai_asr, rai_tts |
     | [simbench][simbench] | Simulation and benchmarking tools | rai_sim, rai_bench |
-    | [openset][openset] | Open-set detection capabilities | groundingdino, groundedsam |
+    | [perception][perception] | Open-set detection capabilities | groundingdino, groundedsam |
     | [nomad][nomad] | Visual Navigation - NoMaD integration | visualnav_transformer |
     | docs | Documentation-related dependencies | mkdocs, mkdocs-material, pymdown-extensions |
 
@@ -168,5 +168,5 @@ Pick your local solution or service provider and follow one of these guides:
 
 [s2s]: ../tutorials/voice_interface.md
 [simbench]: ../simulation_and_benchmarking/overview.md
-[openset]: ../extensions/openset.md
+[perception]: ../extensions/perception.md
 [nomad]: ../extensions/nomad.md
diff --git a/examples/manipulation-demo.py b/examples/manipulation-demo.py
@@ -30,7 +30,7 @@
     ResetArmTool,
 )
 from rai.tools.ros2.simple import GetROS2ImageConfiguredTool
-from rai_open_set_vision.tools import GetGrabbingPointTool
+from rai_perception.tools import GetGrabbingPointTool
 
 from rai_whoami.models import EmbodimentInfo
 

diff --git a/examples/rosbot-xl-demo.py b/examples/rosbot-xl-demo.py
@@ -32,7 +32,7 @@
     Nav2Toolkit,
 )
 from rai.tools.time import WaitForSecondsTool
-from rai_open_set_vision.tools import GetGrabbingPointTool
+from rai_perception.tools import GetGrabbingPointTool
 
 from rai_whoami import EmbodimentInfo
 

diff --git a/mkdocs.yml b/mkdocs.yml
@@ -128,7 +128,7 @@ nav:
     - RAI Bench: simulation_and_benchmarking/rai_bench.md
     - Simulators: simulation_and_benchmarking/simulators.md
   - Extensions:
-    - OpenSet Vision: extensions/openset.md
+    - Perception: extensions/perception.md
     - NoMaD Visual Navigation: extensions/nomad.md
   - FAQ:
     - Overview: faq/faq.md