This project is a robot using the Mbot structure as a base, with its 2 DC motors and a custom Arduino Uno board.
To control the structure, a maixcam AI camera is added to the top of the bot and connected to its RX/TX serial port.
The camera detects a single object (here a dual-color orange/grey Benchy) and will attempt to follow it.
The RX/TX and GND pins from the camera (A19 => UART1_TX, A18 => UART1_RX, and GND) are used to connect to the Mbot's TX/RX and GND pins from the Bluetooth module (which needs to be removed).
The Maixcam power supply can be provided by any 5V power bank. In our example, we are using 16340 lithium-ion batteries:
A lithium-ion cell is plugged into a special shield equipped with a step-up converter that converts 3.7V-4.2V from the cells into a 3.3V or 5V output.
It also has an On/Off switch.
The 16340 cells have enough power to provide 1 or 2 hours of continuous power to the Maixcam / Maixcam Lite AI camera.
For this project, we used the internal battery that comes with the Mbot to power the Arduino and the DC motors.
To make things easier, the Mbot Arduino firmware has been updated (see Mbot motor simplified).
You will need to follow the Mbot beginner's guide to see how to set up your Arduino IDE in order to download the proper set of libraries (see Makeblock libraries).
Once downloaded, just load the Mbot motor simplified ino file and upload it to your Mbot.
Maixcam uses MaixVision IDE to connect to the camera and execute Python scripts.
You can find many useful examples on how to use all the features of the camera in the example section of MaixVision.
The already trained model can be found here: yolo11n_cv181x_int8_labboite.cvimodel and yolo11n_cv181x_int8_labboite.mud
You will need to upload both files to /root/models/ on the camera file system.
You can then execute main.py from MaixVision IDE and the application should run properly.
To train the model, follow these steps:
- WSL 2
- WSL Ubuntu 24.04
- Nvidia GPU (here RTX 4060 Ti 16GB)
- Docker for Windows
Once you have your Ubuntu 24.04 image set up under WSL:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo apt install nvidia-utils-550
sudo apt install nvidia-smi
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install -y cuda-toolkit
sudo apt install docker.io
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu22.04/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt update && sudo apt install -y nvidia-docker2
sudo docker pull ultralytics/ultralytics:latest
Create a shared folder inside the Ubuntu image:
mkdir /data_1
sudo chmod -R 777 /data_1
Run docker using the GPU:
sudo systemctl restart docker
docker run --name YoloV11_1 -it --gpus all --ipc=host -v /data_1:/data ultralytics/ultralytics:latest b8d63d34bff7
Once the container is created, you should be able to use this command to run your docker image:
docker start -ai YoloV11_1
Configure the shared folder
mkdir /data/my_dataset
mkdir /data/my_yolo11n
chmod 777 /data/my_dataset/
chmod 777 /data/my_yolo11n/
Paste your yolo dataset inside the folder '/data/my_dataset/'
cd /data/my_yolo11n
Copy train_yolo_v11n.py (from train_yolo_v11n.py) into your WSL folder '/data_1/my_yolo11n'
python train_yolo_v11n.py
Wait a few hours for the training to finish. It should then generate a file 'best.pt' that you will reuse to quantize the model for use by the Maixcam. You may need to note down somewhere where the .pt is generated for future use.
If the previous script has worked, you will need to convert your Pytorch file into a .onnx file before quantizing the model into a .cvimodel Int8 format.
yolo export model=/ultralytics/runs/detect/train/weights/best.pt imgsz=640 format=onnx opset=14
chmod 777 -R /data/my_yolo11n/
If you get an error regarding 'Double Free()', you may want to exit your current docker container (just type exit) and do the following once back in your WSL Ubuntu image:
cd /data_1/my_yolo11n
docker run --rm -v $PWD:/workspace ultralytics/ultralytics yolo export model=/workspace/train/weights/best.pt imgsz=640 format=onnx opset=14
Make sure to check or update the path to your best.pt file.
then
chmod 777 -R /data_1/my_yolo11n/
Inside your WSL Ubuntu 24.04 image:
docker pull sophgo/tpuc_dev:v3.1
docker run --privileged --name maixcamDocker -v /data_1:/data -it sophgo/tpuc_dev:v3.1
You should now be inside your new maixcamDocker docker image
pip install tpu_mlir==1.7
pip install tpu_mlir[onnx]==1.7
pip install tpu_mlir[torch]==1.7
pip install tpu_mlir[all]==1.7
cd /data
Once the container is created, you can run for future use:
docker start -ai maixcamDocker
cd /data
Here we use version 1.14 but if it fails try version 1.7
git clone -b v1.14 --depth 1 https://github.com/sophgo/tpu-mlir.git
cd tpu-mlir
source ./envsetup.sh
mkdir model_yolo11n && cd model_yolo11n
chmod 777 -R /data/tpu-mlir/model_yolo11n/
For model calibration purposes, you will need to copy 1 image from your original dataset into: '\wsl.localhost\Ubuntu-24.04\data_1\tpu-mlir\model_yolo11n\image\image.jpg'
cd /data/tpu-mlir/model_yolo11n/Workspace
Clean up your working folder from your docker image:
rm -rf *
Copy the onnx file:
cp /data/my_yolo11n/train/weights/best.onnx ./yolo11n.onnx
Copy the script export_to_cvi_maixcam.py to '/data_1/my_yolo11n/'
python /data/my_yolo11n/export_to_cvi_maixcam.py /data/tpu-mlir/model_yolo11n/Workspace/yolo11n --pixel_format rgb --mean "0,0,0" --scale "0.00392156862745098,0.00392156862745098,0.00392156862745098" --output_names "/model.23/dfl/conv/Conv_output_0,/model.23/Sigmoid_output_0" --dataset /data/my_dataset/valid/images/ --test_input ../image/image.jpg --work_dir ./maixcam/
mkdir /data/my_yolo11n/cvi_model_maixcam_output
cp /data/tpu-mlir/model_yolo11n/Workspace/maixcam/yolo11n/yolo11n_cv181x_int8.cvimodel /data/my_yolo11n/cvi_model_maixcam_output
chmod -R 777 /data/my_yolo11n/cvi_model_maixcam_output
Your quantized .cvimodel should be available inside your docker folder '/data/my_yolo11n/cvi_model_maixcam_output' or your Ubuntu 24.04 folder '/data_1/my_yolo11n/cvi_model_maixcam_output'
Mud files are text files used by MaixPy to determine the input format of the model as well as the path to the cvimodel file and the different class names trained inside your model.
Here is an example: (see yolo11n_cv181x_int8_labboite.mud)
[basic]
type = cvimodel
model = yolo11n_cv181x_int8_labboite.cvimodel
[extra]
model_type = yolo11
input_type = rgb
mean = 0, 0, 0
scale = 0.00392156862745098, 0.00392156862745098, 0.00392156862745098
labels = Benchy
You will need it for your Maixcam Python script to perform inference on your objects.
Follow the project on:
 |
 |