|
The game this year presents many challenging problems in both the mechanical and programming sides. One of the key issues for us as programmers is to be able to hone in on "the Rack" during both the autonomous and teleoperated sections of the game. Since the Rack can be rotated and translated this year, it is not possible to simply use dead reckoning methods such as inertial navigation or gear tooth sensors . However, FIRST does offer us an alternative method of finding the Rack: tracking the four green lights atop it with the use of the CMUCam2.
But before we could start focusing on the camera there were a few other housekeeping and preliminary issues we needed to address. In the beginning of the season we spent time setting up a remote server which could house our code and files. We also set up a CVS on this server, which automatically updates versions of our code, and allows us to work remotely from anywhere with an internet connection.
We have used this capability to get ahead of schedule, working not only at school but also together at home using a collaborative editing program called Gobby. This allows us to work together on the code at the same time (We are actually writing this blog entry using that program).
In the past week we have not only gotten the basic driving functions of the robot completed, but we have also started work on the CMUCam2 camera. This camera communicates with the robot controller and can be configured to track a given RGB or YCrCb color. When it detects the specified color, it returns the coordinates of a large bounding box around any portion of that color in its view.
The default camera code gives the ability to track two lights at one time, however it does not actually return two separate bounding boxes for each light. Instead it sees the two lights and draws a large box, or "blob" as we call it, around both lights. If this blob is big enough, it infers that there are two lights, instead of just one.
However, soon after Kickoff, discussions about alternate methods of multiple object tracking began on the popular FIRST related forum, ChiefDelphi. After posting a potential method and receiving acknowledgment from Kevin Watson, the writer of a great deal of FIRST related code, our team began implementing that technique.
The method we have been implementing revolves around the CMUCam2 command called "virtual window". This allows the camera's view to be split up into smaller windows and re-processed. In our initial version of our code, we divided the window up into multiple rows and columns and printed a 1 or 0 through the serial port if there was any green in that section. Quickly our code grew more advanced, and we wrote a Python script to interpret the serial data and display small green rectangles in a window on our computers.
We quickly learned that this method was too slow. Each new box was processed at a rate of 26.2 milliseconds, and with a total of only 32 boxes, this took at least 838.4 milliseconds. This would be too slow for actual game play, with only 15 seconds of autonomous mode to work with. We can not afford to spend a long time scanning with the camera because there are other functions which must also be run during this period, like driving and manipulation.
Our second version was split up into three steps. First it processed the whole view of the CMUCam2. Then based on the center of the bounding box the camera returned, we split the view in half and processed the left side and then the right side. Then this code printed the coordinates for the first large bounding box and then the two smaller rectangles, and our Python script drew a side by side comparison of the initial bounding box and the re-processed views.
We have made a video which shows an example of our code at work. The position and orientation of the two lights are altered throughout the video. This can be seen in the lower right box which shows two green rectangles drawn based on the coordinates returned by the camera and our code. However, only one green box appears when using the original default code, seen in the upper right corner.
We hope that this method will provide an alternate means to track two lights in view. If you are interested in more information, or would like the source code in its beta phase, please post a comment below. Can anyone share their experience with vision algorithms with or without the CMUCam2? Also, since we hope to combine data from the CMUCam2 with other sensors, can someone share any information about multi-modal sensor fusion and adaptive filters or point out any good resources or papers on these topics and other issues of control theory and filtering?
Farewell,
Team 250 Programmers
| 
Re: Team 250: Programming CMUCam2
"Almost" Good Answers: