Today we explore the impressive performance of Kendryte K210 and MicroPython in image segmentation and recognition tasks. The K210 delivers impressive speeds, hitting around 10 fps (without a attached display) and 2-3 fps with a display on the MNIST digit recognition task, running from a smaller than credit size board. The code accompanying this article leverages the K210 to seamlessly segment images captured by the camera, identify individual digits, and redraw them on the display. The K210 boards are strong competitors to the popular OpenMV boards, offering similar capabilities at a much lower cost.
Here is the link to the code and instructions on using it:
https://github.com/code2k13/maix_dock_mnist
The Hardware
If you have been following my work, you must have noticed that in the past few months I have been conducting experiments on running TinyML on the RP2040 microcontroller. From implementing Markov chains, SVC-based classifiers, dense neural networks and Convolutional Neural Networks, I have explored a variety of things.
A common feature in all these experiments was my choice of microcontroller, the RP2040, which lacks specialized support for accelerated neural network execution. This led me to seek low-cost alternatives. I discovered a range of boards from Sipeed, including the Maix Dock, featuring the Kendryte K210 based neural network accelerator. Initially skeptical, I decided to order a board called the ‘Maix M1 Dock’, which I found to be an excellent value.
The board comes with a camera and a 2.4-inch LCD for around USD 40 (in India), presenting a more affordable option compared to OpenMV boards. Whats more, these boards like the OpenMV come with a version of micropython called “MaixPy“, which has plethora of image processing functions. A tool called “nncase“ makes it possible to convert TFlite micro models to a format understood by the K210. The K210 is able to execute these models natively and fast.
The Software
While documentation for these boards exists, it could benefit from better organization. With some assistance from Google, I managed to upload the latest firmware and a converted MNIST classification model using the kflash-gui tool . Then I installed the MaixPy IDE on my PC.
One issue I encountered with the MaixPy IDE was that the “Frame Buffer” view did not display images captured by “sensor.snapshot()”. With guidance from OpenMV’s site, I was able to get it working in the “Serial Terminal” window using “print(img2.compressed_for_ide())”.
The MaixPy IDE is essentially a clone of the OpenMV IDE, which I find to be well-designed. For instance, it allows me to experimentally determine the optimal threshold values for blob detection.
Once everything was set up, experimenting with the board was enjoyable. I was impressed by the performance of the Kendryte K210. Image classification using the RP2040 with a basic Convolutional Neural Network takes around 3-4 seconds per image. However, the K210 operates on a different level entirely, comfortably running the Micropython code at around 3-4 frames per second (fps), when connected to serial terminal for display (its twice as fast when a display is not connected). Remember, the code isn’t just performing classification. It handles multiple tasks such as:
- Capturing an image using the sensor and inverting its colors.
- Detecting blobs in the captured image and extracting the region of interest (tile) for each detected blob.
- Resizing each tile and running it through the AI model to get a prediction.
- Displaying the processed tiles and predictions on the screen in a grid.
Conclusion
In conclusion, the Kendryte K210, with its affordability and powerful performance, offers a compelling option for TinyML projects. Despite some initial setup challenges, the board’s ability to handle complex tasks at impressive speeds makes it a valuable tool for hobbyists and professionals alike. Future explorations could focus on further optimizing performance and expanding the range of applications for the K210.
