Computer Vision with the Raspberry Pi

Computer Vision with the Raspberry Pi

A recent development fresh out of CAROBOT is the involvement of computer vision with a Raspberry Pi and Python. Using the open-source library for computer vision – OpenCV – and a Pi Camera attached, the beginnings of a program to help the Pi understand what it’s seeing is taking shape.

So far, the Pi – using the camera attached – can read in the feed of what it’s seeing at a specified resolution. What our program intends to do is to analyze what it’s seeing in every single frame before moving onto the next. This clearly gates high FPS (Frames Per Second) if there are more computationally intensive operations going on with each analysis, so the resolution should remain small and the operations be kept simple.

As a part of the analysis, it reads in the 2D array of pixels of each frame, with each pixel assigned a specific RGB (Red-Green-Blue) value. Using functions from the OpenCV library, we can create contour plots of a specified range in colour. Seeing that the intention is to follow a line (namely a black one on a white background), we would want to filter out all the pixels that aren’t extremely dark and keep a sort of ‘map’ of what is dark.

When we filter this out, we create an outline of areas that are black – these are contours! Using other functions such as erode() and dilate(), we can clear up ‘noise’ which can be seen by the computer and only interpret the spots of contour that are large and dark. Through use of more functions readily available in OpenCV, we can set up ‘rectangles’ around the contours, such that we can figure out the orientation it has with respect to a vertical line in the center.

Using the rectangles previously used, we can set up motion of the rover based on the orientation and distance to the center, where the motion is based on the need to correct itself and have the line be in the middle of the screen. What’s next is to start incorporating motion and colour detection!

With more to come, the program will be only more polished and well-performing in the future!

CAROBOT’s new Raspberry Pi HAT Beta Test

CAROBOT's new Raspberry Pi HAT Beta Test

The newest HAT to come out from CARobot is the upcoming prototype SwissCHEESE HAT for the Raspberry Pi. Like the Arduino SwissCHEESE HAT, the Pi HAT is outfitted with a handful of Integrated Circuits such as the MCP3008 (Analog to Digital Converter), the TB6612 FNG (Motor Driver), the PCA9685 (PWM Driver), an EEPROM (Electronically Erasable Programmable Read-Only Memory), and of course, the J8 GPIO Expansion featured on most Raspberry Pi’s. Specifically, the ideal Raspberry Pi model is the 3-B. All those circuits work in tandem to help direct input and output from the specially crafted input and output ports for various modules, which includes but not limited to: LEDs, buttons, buzzers, sensors, motors, and servos.

The backend of the SwissCHEESE HAT will be coming from a Python library custom-made just for the HAT. The library itself will be a composition of multiple other libraries pertaining to the different circuits available on the HAT as well as other publicly available libraries. A large chunk of the libraries happen to be Adafruit’s open source Python libraries for the PCA9685 and the MCP3008!

The inner machinations of how the HAT controls input and output are thanks to the MCP3008 and PCA9685 respectively. For input, the MCP3008 can convert analog signals (a total of 8 different channels with 10-bit resolution, or from values of 0 to 1023) into digital values. The chip uses the SPI (Serial Peripheral Interface) to communicate with the Pi itself, using various communication lines such as the clock, Master Input/Slave Output (MISO), & Master Output / Slave Input (MOSI). On the other end, we have the PCA9685 controlling the outputs using I2C, or Inter-Integrated Circuit. The circuit produces PWM (Pulse-Width Modulation) signals that are sent with a specific frequency and duration of when it is ‘high’, also known as duty-cycle. The chip has a total of 16 different channels, featuring capabilities of 12-bit resolution, but is also able to go beyond up to even 16-bit resolution. Using a system of two signal lines, the SDA and SCK line, for data and the clock respectively, can communicate with the Pi.

The library is made up of several classes, functions inside these classes, and a couple hierarchies that branch over the two major types of modules that will be used with the Pi – input and output. The style that it emulates is Object-Oriented Programming, for the sake of specializing modules individually. To put it quickly for the sake of brevity, the individual module classes inherit the parent class for initialization of the pin and determine the signal that is receiving or is sending out based on whether it is an input or output. The existence of the individual classes is for specialized functions that only work with the module itself, such as the flickering of an LED, or the spinning a servo. Specialized classes are also made for unique modules such as the motor and ultrasonic sensor, which operate on more than one signal line. Ultimately, the library should feature a comprehensive class hierarchy of every module that is available to CARobot’s SwissCHEESE disposal, as well as unique functions for these modules. The library draws inspiration for some functions and their purpose from already available Python libraries such as the RPi.GPIO, gpiozero, and Adafruit libraries. As the prototype develops further and further into a polished piece of hardware, the Python library too will also grow.