ICA: Masks and Thresholds

Author

Susan Eileen Fox

Published

October 9, 2025

Overview

In this activity, we will explore how to create and apply masks to images, and how to use the threshold and inRange functions to build threshold images. You will apply these to a variety of images in the SampleImages folder, as well as the BallFinding folder, and to a live webcam.

The Github repository for this assignment will contain a starter code file, activ11.py. Put your code in this file, and create others, as directed below and according to the TODO comments in the file.

Before working on this activity, I encourage you to download the following zip files, unzip them, and move them into the Github repo for this activity.

  • BallFinding.zip
  • The updated SampleImages.zip
  • SampleVideos.zip

Masks

A mask is just a black and white image. It can be created by processing a regular image with something like the threshold operations, or you can just make a black image from scratch and then draw white shapes on it.

To apply a mask to an image, the mask and the image need to be the same size. Then we use the bitwise_and operation to combine the two. Anywhere that the mask is white, it will keep the other image’s color, and anywhere that the mask is black, it will set the pixel to black.

Below is an example where we mask all but three rectangles of an image.

def maskBuilder(image):
    """Takes in an image and draws rectangular masks on it."""
    (h, w, d) = image.shape
    print(h, w)
    maskIm = np.zeros(image.shape, image.dtype)

    cv2.rectangle(maskIm, (0, 0), (w, h // 4), (255, 255, 255), -1)
    cv2.rectangle(maskIm, (200, 2 * h // 3), (400, 5 * h // 6), (255, 255, 255), -1)
    cv2.rectangle(maskIm, (3 * w // 5, 350), (3 * w // 4, 450), (255, 255, 255), -1)

    # Apply the mask
    newMidway = cv2.bitwise_and(midway, maskIm)
    return maskIm, newMidway

midway = cv2.imread("SampleImages/mightyMidway.jpg")
mask, finalIm = maskBuilder(midway)
cv2.imshow("Final", finalIm)
cv2.waitKey()

Try this function in activ11.py. Try changing the coordinates of the rectangles, adding shapes, removing shapes.

Modifying the moving mask on video

In the reading you had a program that connected to the video camera, and showed a masked version of the camera image, where the only part of the video visible was a square that moved from one frame to the next, bouncing around. The activ11.py file contains a function-based version of this program.

  • Try the program out as is, there are sample calls that run it on different video files as well as a webcam
  • Modify the program so that it draw a circle on the mask image, rather than the square it currently draws
  • Change the size of the circle drawn
  • Change the speed and direction of the mask shape’s movement by altering deltaX and deltaY

Converting color representations

The cvtColor function in OpenCV will convert an image from one color representation to another. The key to this function is the code that we pass as its second argument. The code tells the function what the representation of the current image is, and what representation we want. Below are some typical conversion codes:

Code Description
cv2.COLOR_BGR2GRAY Convert BGR to grayscale
cv2.COLOR_BGR2HSV Converts BGR to HSV
cv2.COLOR_GRAY2BGR Converts grayscale to BGR
cv2.COLOR_BGR2RGB Converts BGR to RGB
`cv2.COLOR_BGR2

See the Color Space Conversions page in OpenCV’s documentation for a complete list.

The function below is in the activ11.py file. Try it with the sample call in the file, then try varying the image, and examine the results. Do you understand how to use this function?

def testConversions(origImage):
    """Takes in an image, and it converts it to grayscale and HSV. It also converts the grayscale back to BGR,
    displays the results, and prints the shapes of the images."""
    gray1 = cv2.cvtColor(origImage, cv2.COLOR_BGR2GRAY)
    BGRIm2 = cv2.cvtColor(gray1, cv2.COLOR_GRAY2BGR)
    HSVIm = cv2.cvtColor(origImage, cv2.COLOR_BGR2HSV)

    print(origImage.shape, gray1.shape, BGRIm2.shape, HSVIm.shape)
    cv2.imshow("Original", origImage)
    cv2.imshow("Gray1", gray1)
    cv2.imshow("BGR2", BGRIm2)
    cv2.imshow("HSV", HSVIm)
    cv2.waitKey()

Using the threshold function

The threshold function takes in a grayscale image and produces a thresholded image that is grayscale, and often black and white. The readings go over all the many different thresholding modes that this function has. We will start with just the basic cv2.THRESH_BINARY mode for today.

Note: The threshold function returns two values as a tuple. The first value is the threshold that was used, and the second is the actual resulting thresholded image.

The function below takes in an image and a threshold value, and it returns the result of calling threshold.

def binaryThresh(image, threshValue=128):
    """Takes in an image and also has an optional input, threshValue, which can be set by the function call, or
    it defaults to the value 128. This function converts the image to grayscale and performs a binary threshold
    on it. It returns the resulting image."""

    grayImg = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    tVal, threshIm = cv2.threshold(grayImg, threshValue, 255, cv2.THRESH_BINARY)
    print("Threshold value:", tVal)
    return threshIm

In the activ11.py file, red through this function, and then go to the bottom of the file where we have some sample calls. Try out the samples given to you, and examine the results.

Then, perform these experiments, and record the results either as a comment in the code file, or in a separate .txt file (this next part might be interesting to try with a partner):

  • Write code to try each of the images in the Coins folder (inside SampleImages) with this function. For each image, how well does the binaryThresh function isolate the coins from the background?
  • Can you modify the threshold value and improve the results? Try a few values and see: report on what you discovered.
  • Modify this function to use the combination of cv2.THRESH_BINARY and cv2.THRESH_OTSU or cv2.THRESH_TRIANGLE instead of using a fixed threshold value (see readings for how to combine and use these). Evaluate the results on all the coin pictures: does it help?
    • Look at the BallFinding folder, which contains images and videos of brightly colored balls. Pick one or two ball colors, and select one image for each. Try your function on these: how well does it work?

Combining thresholds with video (OPTIONAL)

Examine the functions processImage and threshVideo: these are a variation on the videoFeeder program from the last activity. Run the examples to see what it does.

Modify the processImage function so that it calls either cv2.threshold directly, or the binaryThresh function, to perform a threshold on the image. Examine the results. Test this in class using some of my bright and dark colorful balls and other objects.

Thresholding with color

The inRange function can isolate colorful objects better than the grayscale-based threshold function. To do this best, we need to work with HSV images, so that hue is a single value that won’t change greatly between well-lit and shadowed pixels of the same object. The inputs to inRange are as shown here:

threshImage = cv2.inRange(colorImage, lowBounds, highBounds)
  • It assumes that the colorImage is a three-channel color image. We will use HSV rather than BGR for this.
  • The lowBounds input is a tuple with three values: the lower boundary of acceptable values for each of the three channels
  • The highBounds input is a tuple with three values: the upper boundary of acceptable values for each of the three channels

Remember for HSV images that the three channels represent:

  • Hue: a value between 0 and 180 in OpenCV representing the hue of the pixel (given a global HSV values that ranges from 0 to 360, divide it by 2 to get the corresponding value in OpenCV’s HSV representation)
  • Saturation: a value between 0 and 255, represents the intensity of the color. Low saturation pixels tend to look gray, high saturation values have an intense version of the color
  • Value: a value between 0 and 255, represents the brightness of the color. High value pixels are very light, low value pixels are very dark

The code snippet below shows how to use inRange on one of the ball pictures:

ballImg = cv2.imread("BallFinding/Green/Green1BG1Mid.jpg")
hsvBall = cv2.cvtColor(ballImg, cv2.COLOR_BGR2HSV)
threshImg = cv2.inRange(hsvBall, (45, 10, 0), (65, 255, 255))

cv2.imshow("Original", ballImg)
cv2.imshow("inRange", threshImg)
cv2.waitKey()

This code is copied in the main script of activ11.py: try it out!

  • Experiment with varying the bounds, changing one value at a time.
    • What happens if we increase the lower bound on saturation?
    • What if we lower the upper bound on either saturation or value?
    • What if we broaden or narrow the hue range?

Suppose we wanted to try this on a different picture, with a different color of ball or one of the coin pictures.

To do that, we need to determine the correct range of hue values to pick the ball.

You can do this in one of two ways:

Option 1:

  • Bring up a picture of the object you want to track
  • Open an online color-picker that displays HSV
  • Do your best to pick a color close to your target color
  • Read its hue value, and divide it by 2, then make a range of acceptable hue values that surround it

Option 2:

  • Bring up a picture of the object you want to track (some image viewers will let you select colors from the pictures, with
  • an “eyedropper” tool, but it isn’t universal)
  • Open a tool to let you select the color from your Desktop (Digital Color Meter on the Mac, on Windows if you have Power Toys it comes with a Color Picker that can report colors from any window on the desktop)
  • Record the color, probably in RGB, of target pixels from the image
  • Use an online converter to convert RGB to HSV

Try at least one other colorful object

Combining color thresholds with video (OPTIONAL)

  • Copy the threshVideo and processImage functions, and rename them colorThreshVideo and processImage2
  • Change the colorThreshVideo to call processImage2 instead of processImage
  • Modify the processImage2 function so that it calls cv2.inRange on the input image and returns the result
  • You can hard-code the color boundaries in the function, so it only looks for one color
  • Test this on the relevant video files, or try it on the webcam with you holding the correct colorful object (I have external USB cameras if you want to try rolling a ball on the floor)

What to hand in

Put all of your function definitions for this activity into the activ11.py file to be submitted. Make sure you format your code appropriately:

  • At the top of the file is a triple-quoted string describing the file
  • Next you include all import statements
  • Next you have your function definitions, visually separated by blank lines, and maybe comments with dashed or other visual horizontal lines
  • Each function should have a triple-quoted descriptive comment right after the def line
  • All calls to all functions should be in a script at the bottom of the file, ideally inside an if __name__ ... block

Use commit and push to copy your code to Github to submit this work.