Computer Vision with the Raspberry Pi

Computer Vision with the Raspberry Pi

A recent development fresh out of CAROBOT is the involvement of computer vision with a Raspberry Pi and Python. Using the open-source library for computer vision – OpenCV – and a Pi Camera attached, the beginnings of a program to help the Pi understand what it’s seeing is taking shape.

So far, the Pi – using the camera attached – can read in the feed of what it’s seeing at a specified resolution. What our program intends to do is to analyze what it’s seeing in every single frame before moving onto the next. This clearly gates high FPS (Frames Per Second) if there are more computationally intensive operations going on with each analysis, so the resolution should remain small and the operations be kept simple.

As a part of the analysis, it reads in the 2D array of pixels of each frame, with each pixel assigned a specific RGB (Red-Green-Blue) value. Using functions from the OpenCV library, we can create contour plots of a specified range in colour. Seeing that the intention is to follow a line (namely a black one on a white background), we would want to filter out all the pixels that aren’t extremely dark and keep a sort of ‘map’ of what is dark.

When we filter this out, we create an outline of areas that are black – these are contours! Using other functions such as erode() and dilate(), we can clear up ‘noise’ which can be seen by the computer and only interpret the spots of contour that are large and dark. Through use of more functions readily available in OpenCV, we can set up ‘rectangles’ around the contours, such that we can figure out the orientation it has with respect to a vertical line in the center.

Using the rectangles previously used, we can set up motion of the rover based on the orientation and distance to the center, where the motion is based on the need to correct itself and have the line be in the middle of the screen. What’s next is to start incorporating motion and colour detection!

With more to come, the program will be only more polished and well-performing in the future!

CarduBlock

CarduBlock

The CarduBlock software is an intelligent software learning tool used to help kids with Arduino programming. Cardublock consists of two parts: an MIT open source library known as Openblocks, and the Ardublock component made by a team of software developers. Ardublock utilizes the Openblocks library in order to suit its software needs. Cardublock is a derivative of Ardublock but improved and modified to better teach students about robotics.

The modifications made are to help make the software more interactive and user friendly. While making it easier to program in Arduino, the software also helps to teach kids the fundamentals of Object Oriented Programming. This is a key factor in today’s software society that we live in. By introducing them to this at a young age, it opens their mind to different career paths in the future.

In order to modify CarduBlock, the Openblocks library first needs to be reviewed. Its code must be traced and further documented in order to understand the classes used to provide a GUI. In this case, a delete button was added so that Renderable blocks could be deleted without confusion. This was programmed into the Renderable blocks class as an added function. Furthermore, the delete button had to be integrated into the UI itself. In this case it was implemented into the right click popup menu.

One of the most important classes in the Openblocks library is the Page class. This is the workspace that kids would be using to put the blocks on. Before it was gray and dull, but now it is white and bright thanks to variable changes to the Page class. This was done to make it more aesthetically pleasing.

 Moving onto the Cardublock side, the top and bottom panel colour needed to be changed as well as the button colours. For the buttons, a white hexadecimal value was used, and for the panels a light pink hexadecimal value was used. This was done in the OpenblocksFrame class in the Cardublock project.

This updated project has now been put on GitHub as an opensource project with issues that still need to be addressed. Resolving these issues over time will better both the functionality and user interface for the software applet. The next steps are to update and change the block drawers. A data factory is a design tool in software engineering. Each drawer is a data factory that contains blocks. It is all managed in the Openblocks library which is where the next modification will take place.

CAROBOT’s new Raspberry Pi HAT Beta Test

CAROBOT's new Raspberry Pi HAT Beta Test

The newest HAT to come out from CARobot is the upcoming prototype SwissCHEESE HAT for the Raspberry Pi. Like the Arduino SwissCHEESE HAT, the Pi HAT is outfitted with a handful of Integrated Circuits such as the MCP3008 (Analog to Digital Converter), the TB6612 FNG (Motor Driver), the PCA9685 (PWM Driver), an EEPROM (Electronically Erasable Programmable Read-Only Memory), and of course, the J8 GPIO Expansion featured on most Raspberry Pi’s. Specifically, the ideal Raspberry Pi model is the 3-B. All those circuits work in tandem to help direct input and output from the specially crafted input and output ports for various modules, which includes but not limited to: LEDs, buttons, buzzers, sensors, motors, and servos.

The backend of the SwissCHEESE HAT will be coming from a Python library custom-made just for the HAT. The library itself will be a composition of multiple other libraries pertaining to the different circuits available on the HAT as well as other publicly available libraries. A large chunk of the libraries happen to be Adafruit’s open source Python libraries for the PCA9685 and the MCP3008!

The inner machinations of how the HAT controls input and output are thanks to the MCP3008 and PCA9685 respectively. For input, the MCP3008 can convert analog signals (a total of 8 different channels with 10-bit resolution, or from values of 0 to 1023) into digital values. The chip uses the SPI (Serial Peripheral Interface) to communicate with the Pi itself, using various communication lines such as the clock, Master Input/Slave Output (MISO), & Master Output / Slave Input (MOSI). On the other end, we have the PCA9685 controlling the outputs using I2C, or Inter-Integrated Circuit. The circuit produces PWM (Pulse-Width Modulation) signals that are sent with a specific frequency and duration of when it is ‘high’, also known as duty-cycle. The chip has a total of 16 different channels, featuring capabilities of 12-bit resolution, but is also able to go beyond up to even 16-bit resolution. Using a system of two signal lines, the SDA and SCK line, for data and the clock respectively, can communicate with the Pi.

The library is made up of several classes, functions inside these classes, and a couple hierarchies that branch over the two major types of modules that will be used with the Pi – input and output. The style that it emulates is Object-Oriented Programming, for the sake of specializing modules individually. To put it quickly for the sake of brevity, the individual module classes inherit the parent class for initialization of the pin and determine the signal that is receiving or is sending out based on whether it is an input or output. The existence of the individual classes is for specialized functions that only work with the module itself, such as the flickering of an LED, or the spinning a servo. Specialized classes are also made for unique modules such as the motor and ultrasonic sensor, which operate on more than one signal line. Ultimately, the library should feature a comprehensive class hierarchy of every module that is available to CARobot’s SwissCHEESE disposal, as well as unique functions for these modules. The library draws inspiration for some functions and their purpose from already available Python libraries such as the RPi.GPIO, gpiozero, and Adafruit libraries. As the prototype develops further and further into a polished piece of hardware, the Python library too will also grow.

BotQuest Spring 2019 – “The A-MAZE-ING Race!”

BotQuest Spring 2019 – “The A-MAZE-ING Race!”

The BotQuest Spring 2019 tournament was an a-maze-ing success where bright young minds created robots that successfully navigated through a tricky maze. The maze in question had a few dead ends and multiple different paths that led to the exit. Competitors could follow the maze walls, the line paths, or a combination of both. Competitors demonstrated their problem-solving skills, creativity, and sportsmanship as they competed to see whose robot could complete the maze the fastest.

This tournament demonstrated the ever-growing STEM interest in the community. We hope to see more participants and a bigger audience in the upcoming Fall 2019 BotQuest Tournament as we cultivate the STEM interest in our community!

A special thanks to MPL and the Angus Glen library staff, CAROBOT staff, volunteers, and competitors that helped make this tournament a success.

P.S. Wondering how you can make your own line-following, maze-solving rover? Well you’re in luck – you can check out our tutorial!

AMEX Workshop 2019

AMEX Workshop 2019

We recently had the opportunity to collaborate with American Express for a special workshop. Under the direction of CAROBOT, the students learned how to use components like buttons, sensors, and motors, as well as how to have their rover make decisions. This culminated in a task where the students applied their knowledge to create a rover capable of following a preset course.

The class was full of energy, and the kids had a great time learning how to use the rovers. Thank you AMEX for giving us this opportunity to introduce robotics to all of these young people!

Tutorial: Arduino Night Light

Tutorial: Arduino Night Light

Hello everyone!

This tutorial will cover how to create lights that turn on when it gets dark. We will be using the Arduino microcontroller along with the SwissCHEESE shield to make it easier to code.

This tutorial assumes that you have a working environment setup with the Arduino IDE and Cardublock library. See here for more information.

Materials

Wiring

  1. When wiring, make sure that the Arduino USB wire is not plugged into the computer.
  2. Insert the SwissCHEESE shield onto the Arduino board.
    Tutorial: Arduino Night Light
  3. Attach a jumper wire to a LED of your choosing.  The other end of the wire should be attached to pins O0 on the SwissCHEESE board.
    Tutorial: Arduino Night Light
  4. Attach a jumper wire to a SwissCHEESE Light Sensor. The other end of the wire should be attached to pins I1 on the SwissCHEESE board.
    Tutorial: Arduino Night Light

Programming

Open up Arduino IDE, and go to Tools –> CarduBlock Edu to open up the Cardublock user interface.

  1. The light sensor takes in values from 0 to 1023. The dimmer the light, the higher the value. We will first create a variable called “darkness”, by going to Variables/Constants –> Set Integer Variable. Next we will set the variable’s value to LDR, which is found in CAROBOT SwissCHEESE –> LDR.
    Tutorial: Arduino Night Light
  2. Now we want to make a decision, so first add an if/else statement from Control –> If/Else. Add the greater than test found in Tests –> ‘>’ to the test section of the If/Else Statement. From Variables/Constants, add Standard Integer, and Standard Integer Variable, and rename like below. What we are doing is checking if the darkness is greater than 800.
    Tutorial: Arduino Night Light
  3. Now we are going to decide what we do with the If/Else statement. So if the darkness exceeds 800, then we will turn the LED on, otherwise, we will turn it off. Drag the LED blocks from CAROBOT SwissCHEESE –> LED and adjust the status sections like below. 
    Tutorial: Arduino Night Light
  4. Upload the code to the Arduino. 

You should have something that looks like this:

Tutorial: Light Controlled Robot Cars

Hello everyone! In this tutorial we will create light controlled robot cars and play soccer with them. It will be similar to Rocket League, except we will use arms attached to servo motors to hit the ball.

This tutorial assumes that you have a working environment setup with the Arduino IDE and Cardublock library. See here for more information.

https://www.instagram.com/p/BklPi-wnUrD/?utm_source=ig_web_copy_link

 

Materials

Note: we will be using our own robot car configuration including two motors and a caster. Feel free to use your own robot car as long as it is controlled with the Arduino, and has 2 DC motors included. 

Assembly:

  1. Mount the Arduino on the car chassis, and then mount the SwissCHEESE shield on the Arduino.Tutorial: Light Controlled Robot Cars
  2. Using 2 nuts and bolts, attach the servo face at the front end of your robot car. Tutorial: Light Controlled Robot Cars
  3. Using 2 nuts and bolts, attach 2 brackets to the servo to form the kicker.Tutorial: Light Controlled Robot Cars
  4. Using 2 nuts and bolts per light sensor, attach one at the front, back, left and right sides of the chassis.Tutorial: Light Controlled Robot Cars

 

Wiring

  1. When wiring, make sure that the Arduino USB wire is not plugged into the computer.
  2. Attach the wires of the two motors to the SwissCheese at M1, and M2.Tutorial: Light Controlled Robot Cars
  3. Connect a jumper wire from:
    1. I0 to the light sensor on the right.
    2. I1 to the light sensor on the front.
    3. I2 to the light sensor on the left.
    4. I3 to the light sensor on the back.
    5. O0 to servo at the front.Tutorial: Light Controlled Robot Cars

Programming

    1. Open up Arduino IDE, and go to Tools –> CarduBlock Edu to open up the Cardublock user interface.
    2. Subroutines make your program easier to read. To use subroutine blocks, or delays look in Control. To use motor blocks, or servo blocks, look in CAROBOT SwissCHEESE. Using this, assemble 6 subroutines like below.Tutorial: Light Controlled Robot Cars
    3. Next, create the following below. To use the loop, or if else statements, look in Control. To use the orange comparison blocks, look under Tests. To use the light sensor, look in CAROBOT SwissCHEESE.

Tutorial: Light Controlled Robot Cars

  1. Now connect the Arduino USB wire to the computer and hit “Upload To Arduino” *

*Make sure you have the following selected: Tools –> Boards –> Arduino Uno/Genuino,  and Tools –> Ports –> COMX (where x is a number).

SwissCHEESE Robot Car Development

SwissCHEESE Robot Car Development

Hello world! (Cheesy, huh?) As a summer student at CAROBOT Learning and Research Organization, I was tasked with following a modified engineering design process to develop a new car design for the CAROBOT SwissCHEESE Education kits.

SwissCHEESE Robot Car Development

Step 1. Identify Needs

The first step in the modified engineering design process is to find the customer’s needs. Through discussion with my superior, I had come up with the following list of needs:

SwissCHEESE Robot Car Development

Step 2. Establishing Car Specifications

The next step involved creating a list of metrics that helped us quantify our needs.

SwissCHEESE Robot Car Development

Then I came up with a list of marginally accepted and ideal values for our metrics.

SwissCHEESE Robot Car Development

By acknowledging the constraints, I could move on to the next step.

Step 3. Generate Product Concepts

Now came the fun part: producing design sketches.

Concept 1 was the idea of a modular car where certain parts could be exchanged for others. An idea with this concept was the use of slide-in feature where the base carrying the Arduino and breadboard could be slid into a frame. However, this idea was later dismissed because sliding parts are prone to wearing out.

Concept 2 was a minimal design with only the core components attached on the base. These core components included 2 motors, the Arduino and SwissCHEESE boards, a breadboard and a caster.

SwissCHEESE Robot Car Development

Concept 3 was the opposite of the first concept; instead of keeping the flexibility that comes with using multiple parts, the compartment design only would work with components of similar size. The pro of having such a design would be a more rigid model, where parts must stay in the confines of discrete sections.

Concept 4 was based on a DIY car called the Donkey Car. This design would make the car much sturdier in exchange for more material.

Concept 5 was popular with some of the board members. The concept gives the person assembling it creative freedom to place components where they want to. Furthermore, this is a multi-layer concept. The first layer holds the battery, the second layer holds the microcontroller and other electronics parts, and the last layer for customization.

SwissCHEESE Robot Car Development

A final idea was the use of having a sleek-looking car that would have hinges to open the cover. However, the costs would increase because this is a 3D printed design.

SwissCHEESE Robot Car Development

Step 4. Select Product Concepts

I chose concept 2 because of its simplicity, only the base, and two caster attachments needs a laser cutter. Concept 5 was a popular contender, but the hexagonal shape caused the breadboard and battery to jut out at odd angles.

SwissCHEESE Robot Car Development

Step 5. Test Product Concepts

For the chosen concept, I designed the parts using Solidworks and Draftsight.

Note: The caster is not shown in the diagram below as it was difficult to find a solid model of the specific caster used.

SwissCHEESE Robot Car Development

SwissCHEESE Robot Car Development

Step 6. Set Final Specifications

From this model, I noted a few weaknesses. The edges were sharp, and although the board looked aesthetically pleasing, the contours on the sides restricted where the user could place the motor attachments. For the last model, I added a servo slot to the base, rounded the edges, and removed the contour on the sides.

Step 7. Economic Analysis

The cost of the new car is much less than that of the earlier one due to less material being used. That is because the earlier model had two layers, compared to one layer for the new model.

Step 8. Assembling the Car

Some of the parts  in the earlier design were no longer needed, and this resulted in assembly time decreasing. The size of the new base board is around the same as the earlier one.

SwissCHEESE Robot Car Development SwissCHEESE Robot Car Development SwissCHEESE Robot Car Development

 

Maker Festival 2018

The robot soccer game at Maker Festival 2018.

Maker Festival 2018 was a thrilling event.  Our booth garnered a lot of attention and positive feedback! We introduced many people to the exciting field of technology and robotics through CarduBlock Edu. They learned at firsthand just how easy it is to use, and its integration potential with various projects and educational tools.

Many people were also drawn to our FIFA World Cup themed robot soccer game! By controlling the rovers with a flashlight, the teams would work together to score against the opposition. The kids were very competitive and loved to score against others. We plan on returning to next year’s Maker Festival with an even better and more interesting project to truly dazzle festival-goers!