Computer Vision with the Raspberry Pi

Computer Vision with the Raspberry Pi

A recent development fresh out of CAROBOT is the involvement of computer vision with a Raspberry Pi and Python. Using the open-source library for computer vision – OpenCV – and a Pi Camera attached, the beginnings of a program to help the Pi understand what it’s seeing is taking shape.

So far, the Pi – using the camera attached – can read in the feed of what it’s seeing at a specified resolution. What our program intends to do is to analyze what it’s seeing in every single frame before moving onto the next. This clearly gates high FPS (Frames Per Second) if there are more computationally intensive operations going on with each analysis, so the resolution should remain small and the operations be kept simple.

As a part of the analysis, it reads in the 2D array of pixels of each frame, with each pixel assigned a specific RGB (Red-Green-Blue) value. Using functions from the OpenCV library, we can create contour plots of a specified range in colour. Seeing that the intention is to follow a line (namely a black one on a white background), we would want to filter out all the pixels that aren’t extremely dark and keep a sort of ‘map’ of what is dark.

When we filter this out, we create an outline of areas that are black – these are contours! Using other functions such as erode() and dilate(), we can clear up ‘noise’ which can be seen by the computer and only interpret the spots of contour that are large and dark. Through use of more functions readily available in OpenCV, we can set up ‘rectangles’ around the contours, such that we can figure out the orientation it has with respect to a vertical line in the center.

Using the rectangles previously used, we can set up motion of the rover based on the orientation and distance to the center, where the motion is based on the need to correct itself and have the line be in the middle of the screen. What’s next is to start incorporating motion and colour detection!

With more to come, the program will be only more polished and well-performing in the future!



The CarduBlock software is an intelligent software learning tool used to help kids with Arduino programming. Cardublock consists of two parts: an MIT open source library known as Openblocks, and the Ardublock component made by a team of software developers. Ardublock utilizes the Openblocks library in order to suit its software needs. Cardublock is a derivative of Ardublock but improved and modified to better teach students about robotics.

The modifications made are to help make the software more interactive and user friendly. While making it easier to program in Arduino, the software also helps to teach kids the fundamentals of Object Oriented Programming. This is a key factor in today’s software society that we live in. By introducing them to this at a young age, it opens their mind to different career paths in the future.

In order to modify CarduBlock, the Openblocks library first needs to be reviewed. Its code must be traced and further documented in order to understand the classes used to provide a GUI. In this case, a delete button was added so that Renderable blocks could be deleted without confusion. This was programmed into the Renderable blocks class as an added function. Furthermore, the delete button had to be integrated into the UI itself. In this case it was implemented into the right click popup menu.

One of the most important classes in the Openblocks library is the Page class. This is the workspace that kids would be using to put the blocks on. Before it was gray and dull, but now it is white and bright thanks to variable changes to the Page class. This was done to make it more aesthetically pleasing.

 Moving onto the Cardublock side, the top and bottom panel colour needed to be changed as well as the button colours. For the buttons, a white hexadecimal value was used, and for the panels a light pink hexadecimal value was used. This was done in the OpenblocksFrame class in the Cardublock project.

This updated project has now been put on GitHub as an opensource project with issues that still need to be addressed. Resolving these issues over time will better both the functionality and user interface for the software applet. The next steps are to update and change the block drawers. A data factory is a design tool in software engineering. Each drawer is a data factory that contains blocks. It is all managed in the Openblocks library which is where the next modification will take place.