Overview

In this activity, you will explore simple motion detection, background subtraction, and color tracking. These allow us to find object by movement or color and to track them across the frames of a video.

The Github repository for this assignment will contain a starter code file, activ15.py. Put your code in this file, and create others, as directed below and according to the TODO comments in the file.

Simple motion detection

Simple motion detection just computes the difference between two adjacent frames in the video. Our motion detector then adds the differences found in the blue, green, and red channels, to ensure we don’t miss anything.

Try out the simpleMotion function in activ15.py. There are three sample calls, try some other videos as well.

Try this to hand in: Use threshold to convert diff2 into a black and white image, and then apply it as a mask to the original frame, and display the result. How well does it work? Optionally, try using opening or closing (morphological filters) to smooth and clean up the mask before applying it.

Background subtraction

The readings talked about three different background subtraction methods, but here we will focus on the MOG2 and KNN methods that OpenCV implements.

In activ15.py, there is a function called backgroundSubtract, which takes in a video source and a number for which model you want to run: 0 for MOG2 and 1 for KNN. Try out this program on various video feeds, and see how the MOG2 and KNN compare. When do they work well, and when do they struggle?

Try this to hand in: Pick one of these options for further exploration:

Look at the documentation for the createBackgroundSubtractor functions. They each have optional inputs, including history, as well as thresholds for each algorithm. Try varying the history and/or the threshold values… how do those affect the behavior of the program?
Use findContours to identify moving objects from the mask image, can you tune it to find interesting moving objects and not noise?
Background subtraction is one step in a process called background substitution, where we substitute a different background for the black parts of the mask. Implement this:
- Pick an image
- Resize it to match the video frame size,
- Define a second mask to be the opposite of the one from the background subtraction (0 where it has 255, and vice versa)
- Use the second mask to select just the background pixels you need
- Add together the masked foreground and the masked background
- Display the result
- (Note that this is the same general process as question 1 on Homework 3)

Color-tracking with CamShift

The camshift function in your activ15.py file takes in a reference image and then runs the Camshift algorithm to track the object and display what it has found. Try it out: the default reference image is for one of the bright blue floor hockey balls in my collection.

Try this to hand in: Create a reference image for a different colored object, and try tracking that instead.

What to hand in

Put all of your function definitions for this activity into the activ15.py file to be submitted. Make sure you format your code appropriately:

At the top of the file is a triple-quoted string describing the file
Next you include all import statements
Next you have your function definitions, visually separated by blank lines, and maybe comments with dashed or other visual horizontal lines
Each function should have a triple-quoted descriptive comment right after the def line
All calls to all functions should be in a script at the bottom of the file, ideally inside an if __name__ ... block

Use commit and push to copy your code to Github to submit this work.