Find this example on our SD-image

TensorFlow_Lite_Pose_RPi_64-bits

TensorFlow Lite Posenet running at 9.4 FPS on bare Raspberry Pi 4 with Ubuntu

A fast C++ implementation of TensorFlow Lite Posenet on a bare Raspberry Pi 4 64-bit OS.
Once overclocked to 1825 MHz, the app runs at 9.4 FPS without any hardware accelerator.
Special made for a Raspberry Pi 4 see Q-engineering deep learning examples

---

Papers: https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5

---

Benchmark.

Frame rate Pose Lite : 9.4 FPS (RPi 4 @ 1825 MHz - 64 bits OS)
Frame rate Pose Lite : 5.0 FPS (RPi 4 @ 2000 MHz - 32 bits OS) see 32-OS

---

Dependencies.

To run the application, you have to:

• A raspberry Pi 4 with a 64-bit operating system. It can be the Raspberry 64-bit OS, or Ubuntu 18.04 / 20.04. Install 64-bit OS
  
• TensorFlow Lite framework installed. Install TensorFlow Lite
  
• OpenCV 64 bit installed. Install OpenCV 4.5
  
• Code::Blocks installed. ($ sudo apt-get install codeblocks)

---

Installing the app.

To extract and run the network in Code::Blocks
$ mkdir MyDir
$ cd MyDir
$ wget https://github.com/Qengineering/TensorFlow_Lite_Pose_RPi_64-bits/archive/refs/heads/master.zip
$ unzip -j master.zip
Remove master.zip and README.md as they are no longer needed.
$ rm master.zip
$ rm README.md

Your MyDir folder must now look like this:
Dance.mp4
posenet_mobilenet_v1_100_257x257_multi_kpt_stripped.tflite
TestTensorFlow_Lite_Pose.cpb
Pose_single.cpp

---

Running the app.

Run TestTensorFlow_Lite.cpb with Code::Blocks. More info or
If you want to connect a camera to the app, follow the instructions at Hands-On.
I fact, you can run this example on any aarch64 Linux system.

See the movie at: https://www.youtube.com/watch?v=LxSR5JJRBoI

---

CLI Version

No code blocks. Only run it with CLI.

---

1. Install opencv

Required package:

sudo apt install libopencv-dev

Verify the installation:

pkg-config --modversion opencv4

To see the CMake configuration file:

cat /usr/lib/aarch64-linux-gnu/cmake/opencv4/OpenCVConfig.cmake

---

2. TensorFlow Lite

I recommend operating the installation on Debian Bookworm with GCC 12. In my case, building TensorFlow Lite 2.6.0 failed on Debian Trixie with both GCC 14 and GCC 12.

---

3. cmake

Assuming the repo is at ~/. We can build it:

cd ~/TensorFlow_Lite_Pose_RPi_64-bits
mkdir build
cd build/
cmake -DOpenCV_DIR=/usr/lib/aarch64-linux-gnu/cmake/opencv4 ..
make -j4

After the build:

mv posenet_cli ../
cd ..
rm -r build

---

Camera Issues

If you want to change the camera source (or the video source), change VideoCapture cap("Dance.mp4"); in Pose_single.cpp at line 184.

For example, if you have one camera connected, the node is /dev/video0. Use VideoCapture cap("/dev/video0");
If you have two cameras, for instance, a CSI RaspiCam and a USB webcam, and you want to use the USB camera, it lives at location /dev/video1, change it to VideoCapture cap("/dev/video1");

Some tricks for checking the cameras' information.

Check Camera Input Source

Install the package:

sudo apt install v4l-utils

List all video-related devices:

v4l2-ctl --list-devices

You can check if the name of a device contains "camera", and it's probably your destination device. To check:

ffplay /dev/video1

---

CSI Camera

For a CSI camera, you might see the device listed but not working for ffplay. To see what CSI camera is available and attached to the system:

rpicam-vid --list-cameras

You will also see the available resolutions and its tested framerates. Like my device, a Raspberry Pi Camera Module V2:

Available cameras
-----------------
0 : imx219 [3280x2464 10-bit RGGB] (/base/soc/i2c0mux/i2c@1/imx219@10)
    Modes: 'SRGGB10_CSI2P' : 640x480 [103.33 fps - (1000, 752)/1280x960 crop]
                            1640x1232 [41.85 fps - (0, 0)/3280x2464 crop]
                            1920x1080 [47.57 fps - (680, 692)/1920x1080 crop]
                            3280x2464 [21.19 fps - (0, 0)/3280x2464 crop]
        'SRGGB8' : 640x480 [103.33 fps - (1000, 752)/1280x960 crop]
                    1640x1232 [41.85 fps - (0, 0)/3280x2464 crop]
                    1920x1080 [47.57 fps - (680, 692)/1920x1080 crop]
                    3280x2464 [21.19 fps - (0, 0)/3280x2464 crop]

To test the camera:

rpicam-hello

Or specify the camera index

rpicam-hello --camera 2

To access a CSI camera in Pose_single.cpp, modify the capture pipeline as follows. Assuming using the resolution 1640x1232 with framerate 15fps, and the camera path /base/soc/i2c0mux/i2c@1/imx219@10 (you checked this from rpicam-vid --list-cameras) :

const std::string gst =
    "libcamerasrc camera-name=/base/soc/i2c0mux/i2c@1/imx219@10 ! "
    "video/x-raw,width=1640,height=1232,framerate=15/1,format=NV12 ! "
    "videoconvert ! video/x-raw,format=BGR ! "
    "appsink drop=true max-buffers=1 sync=false";

VideoCapture cap( gst, cv::CAP_GSTREAMER );

---

---