Home

6417 Fridolins Robotik Computer-Vision Wiki

Team 6417 Fridolin Robotics has decided to get stared with Computer-Vision in Offseason 2019 – 2020. This Wiki should explain how the Computer-Vision system works to help other teams getting started with their own one. The vision uses the 2019 game rules and vision-targets.

This Vision-System requires following components:

• Nvidia Jetson TX1
• Ubuntu 16.04
• Logitech HD 3000 Lifecam (or something better)

Setting up the Jetson needs a host computer with ubuntu 16.04. We’ve used a Virtual-Machine but you have to pay attention with the USB rooting, because the flash must be made with that cable. To do so it is recommended to use Nvidia Jetpack. There you can decide which libs should be installed additionally. Definitely check on the OpenCV library.

To build the written code we’ve used Cmake for Makefile generation and GNU make for compiling. You can find an extended tutorial here

Our code is programmed in C++ and the connection to the Robot is reached over the NetworkTables.

Straight forward to the code. There are two important Classes. The FridoProcess class contains all the methods for image processing from OpenCV. The FridoCalculation class calculates the distance and the angle of the vision-target relative to the camera. It uses the contours given by the process class. The calculated distance is not the distance to the center of the camera lens. It describes the distance in a 90° angle vector from the target to the camera layer. The angle is being calculated in a 90° angle from the camera layer to the center of the target. This needs to be, because the x-offset doesn’t affect the angle.

But first the FridoProcess class. The resize method is important for the calculation afterwards. Everything is calculated with points and coordinates. So the frame resolution needs to be constant. The second method blurs the image with the median blur method from OpenCV. This method is very time expensive because of the huge calculation. The blur is useful because it neutralizes single pixels with an extraordinary color. The HSV threshold can filter the image on hue, saturation and value. With this method everything around the white targets changes into black, because of the HSV differences between the target color and the rest of the image. This color difference is reached by using retroreflective targets and a light ring around the camera. The following erode method can smooth the white area. But the area gets smaller because of this method. Eroding an image can help the findContours() method to do its job. After the contours have been found, they need to be filtered. The code filters the contours by maximum and minimum height and width as well as the maximum and minimum area. It also calculates the smallest rectangular and counts the edges. If all the conditions are true, the corners of the contour are sent to the calculation class.

The second class calculates the distance, the angle and the offset of the target. First It sorts the contours that the left one is contour 0 and the right one is contour 1. But these contours have hundreds of points. To reduce them to four we can search for a minimum Area rectangle on the contour. The result are the four corners which need to be sorted one more time. Later the corners are used to calculate the height of the target in pixel. And for that the corners must be constantly in the same order each time a new rectangle is found. In the next step the program calculates some important points like the center of the target and so on. They’re used for different operations. One of these operations is the following method which calculates the heights. There are three heights to calculate. The average height for the distance of the center of the target as well as the right and the left target height. They’re used to calculate the distance to one single target which is subtracted by the other one. The result is the difference between the two distances. It’s used as the opposite side of the angle of the target. The difference between the two targets is consistent and can be used too as the hypotenuses of the right-angled triangle. Arcus sinus does the rest!

There is also a illustrate class to draw the contours for debugging.

Because of many changes in the preseason this Wiki will be actualised when the code is on the next level for the next season!