What are callbacks with OpenCV

Before we start learning how to draw on images using a mouse, we need to understand what is a callback.

Callback:

Let’s say a loved one is traveling by car somewhere, and you want to know that she gets to her destination safely. You would ask her to call you “when she gets there.” It makes no sense for her to call you right away when she’s standing in front of you. It only makes sense at the end of her journey.

A callback in programming is the same thing: call this function (the callback) when a process completes.

The same concept applies to event-oriented programming in general. When a mouse button is clicked (an event), call a function. We don’t know when the button will be clicked. All we can do is tell the button to “call me back” or call this function when the mouse button is clicked.

Mouse Callbacks:

A callback can happen when a user performs an operation using the mouse; this operation is usually known as an event.

Only one callback is present for a mouse, which is setMouseCallback() , all mouse operation will call this funtion only. We can have conditional blocks to execute something based on the event/operation performed using the mouse.

The mouse events/operations could be:

  • EVENT_MOUSEMOVE
  • EVENT_LBUTTONDOWN
  • EVENT_RBUTTONDOWN
  • EVENT_MBUTTONDOWN
  • EVENT_LBUTTONUP
  • EVENT_RBUTTONUP
  • EVENT_MBUTTONUP
  • EVENT_LBUTTONDBLCLK
  • EVENT_RBUTTONDBLCLK
  • EVENT_MBUTTONDBLCLK

Working with Mouse events in OpenCV

Before we start working, we need to set the import and images so that I will be using the below code for all the programs, and I will be skipping these in other codes of this article, but you should not.

import cv2
import matplotlib.pyplot as plt
img = cv2.imread("flower.jpg")
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
#scaled down image is used to keep the pop up window to visible arae
scaled_image = cv2.resize(rgb_img,(0,0), rgb_img, 0.4, 0.4)

By this time, you are aware of what is a callback; we need to set what we need to do when a call back occurs like whether do you want to print something or draw something or close the window popup.

Details required to perform Mouse event-based operation:

  • What to be done when a call back occurs : What we want to do when a mouse operation performed.
  • When should this callback occur : Are the call back based on the specific window, if so what is the window name.
  • Do we have event-based operations : Are we going to perform tasks based on the type of click or movement like left button down, right button down or mouse movement

First, we will start by printing some value to the console when the mouse does some event on the popup window.

When should this callback occur :

I want to have this call back only when I use my mouse on the pop-up window, which has the title as "Title of Popup Window."

cv2.namedWindow("Title of Popup Window")
What to be done when a callback occurs:

I want to call a funtion called "just_print_for_all"

cv2.setMouseCallback("Title of Popup Window", just_print_for_all)
Do we have event-based operations:

As of now, I do not wish to have any event-based operation, so I will be printing the text to console irrespective of operation.

def just_print_for_all(event, x, y, flags, param):
    print("chercher tech is my name")

Show the image to the user; when we show it, we should have control how and when are we going to close the popup. Also, we need to set the title for the image. So to show the image on the separate popup, we need to provide the title of the popup along with image data to cv2.imshow() funtion.

The complete OpenCV code for mouse event-based operations

import cv2
import matplotlib.pyplot as plt
img = cv2.imread("flower.jpg")
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
scaled_image = cv2.resize(rgb_img,(0,0), rgb_img, 0.4, 0.4)

def just_print_for_all(event, x, y, flags, param):
    print("chercher tech is my name")

# set when to have a call back
cv2.namedWindow("Title of Popup Window")

#what to happen on call back
cv2.setMouseCallback("Title of Popup Window", just_print_for_all)

#show image to user with title
cv2.imshow("Title of Popup Window", scaled_image)
cv2.waitKey()
cv2.destroyAllWindows()
Why imshow() and waitKey() are to bottom?

We need to close the window by pressing some key on the keyboard; for that reason, I have waitKey() at the bottom.

Till the user presses the key, waitKey() will hold the control of the program at that point without allowing to go further. If I put waitKey() at the top of the program, then you never can reach the next line to it. cv2.imshow() can be movable to top.

When the user presses some key then only destroyAllWindows() will be executed.
draw-shape-mouse-opencv-image

You may see the similar output when you move the cursor on the image; you get so many outputs because the mouse movement is also an operation. We will learn how to have outputs for a specific event of the mouse.

Draw Circle when we left-click on a popup with OpenCV

Lets filter and perform some useful operation on the image rather than printing text to console. In this example, we will modify the code a little bit, so that we can draw a circle on the image when we left-click it.

In the last example, I have used (event, x, y, flags, param), but I have not explained what it is.

  • event: what type of operation occurred like mouse mover, left button down, left button up.
  • x -x coordinate where the event occurred
  • y - Y coordinate where the event occurred
  • flags - Specicif condition when the event occurred, like whether SHIft/CTRL/ALT button pressed along with mouse and so on
  • param - I have not explored yet. :(

We can use the above details to draw a circle because the circle needs minimal details like x,y, and radius.

import cv2
img = cv2.imread("flower.jpg")
def draw_circle(event, x, y, flags, param):
    if event == cv2.EVENT_LBUTTONDOWN:
        print("hello")
        cv2.circle(img,(x,y),100,(0,255,0),-1)
        
cv2.namedWindow(winname= "Title of Popup Window")
cv2.setMouseCallback("Title of Popup Window", draw_circle)

while True:
    cv2.imshow("Title of Popup Window", img)
    if cv2.waitKey(10) & 0xFF == 27:
        break
cv2.destroyAllWindows()

draw-circle-on-image-opencv-with-mouse

What is wrong with cv2.waitKey(10) & 0xFF == 27

I was trying to perform the above example for almost 2 hours but no Luck. Then I search online but no luck. Finally, I started changing everything in the code to find where the issue lies, and root cause for the issue.

Problem statement:

The circle I was drawing is not reflecting on the image, before modifying the code, it was the same as print to console example on the top you can find it.

The solution to the problem:

I was using cv2.waitKey() not using any other key like 0xFF. So I have realized that cv2.waitKey() was the culprit. When modified the code to cv2.waitKey(10) & 0xFF == 27, code started working perfectly.

So to persist changes made to the image we need to use the cv2.waitKey(param), the param must be greater than 0 always to persist the changes.

Based on the value of the param(in milliseconds), the time to reflect the changes also takes time. If we provide 2 milliseconds, then changes will reflect in 2 milliseconds itself. But if you give 20000 milliseconds, then it will take 20000 milliseconds for changes to appear.

Multiple Mouse Events with OpenCV

We have seen how to draw a circle on Images with OpenCV, but in the practical application, we might be drawing more than one shape. We can draw multiple shapes to different buttons and use them on mages.

The below example shows a circle on the left mouse button down and square on the right mouse button down.

import cv2
img = cv2.imread("flower.jpg")

def just_print_for_all(event, x, y, flags, param):
    if event == cv2.EVENT_LBUTTONDOWN:
        cv2.circle(img,(x,y),100,(0,0,255),-1)
    elif event == cv2.EVENT_RBUTTONDOWN:
        cv2.rectangle(img, pt1=(x,y), pt2=(x+100,y+100),color=(0,255,255),thickness=-1)
        
cv2.namedWindow(winname= "Title of Popup Window")
cv2.setMouseCallback("Title of Popup Window", just_print_for_all)

while True:
    cv2.imshow("Title of Popup Window", img)
    if cv2.waitKey(10) == 27:
        break
cv2.destroyAllWindows()

chain-of-events-mouse-events-opencv

Drawing a rectangle by dragging in OpenCV

We can draw a rectangle by dragging and dropping; for this, the logic is going to simple.

  • The starting point is going to be the first mouse down point, ix, iy
  • Move the cursor to a rectangle for this the mouse down should be present(drawing==True) then only get the x and y at the same time draw the rectangle, so that we have a good finish
  • Mouse up point should be the final X and Y, so from (ix, iy) to (X, Y), Now we got the point so we can draw the rectangle.
import cv2
img = cv2.imread("flower.jpg")
#img2 = cv2.imread("flower.jpg")

# variables
ix = -1
iy = -1
drawing = False

def draw_reactangle_with_drag(event, x, y, flags, param):
    global ix, iy, drawing, img
    if event == cv2.EVENT_LBUTTONDOWN:
        drawing = True
        ix = x
        iy = y            
            
    elif event == cv2.EVENT_MOUSEMOVE:
        if drawing == True:
            cv2.rectangle(img, pt1=(ix,iy), pt2=(x, y),color=(0,255,255),thickness=-1)
    
    elif event == cv2.EVENT_LBUTTONUP:
        drawing = False
        cv2.rectangle(img, pt1=(ix,iy), pt2=(x, y),color=(0,255,255),thickness=-1)
        
cv2.namedWindow(winname= "Title of Popup Window")
cv2.setMouseCallback("Title of Popup Window", draw_reactangle_with_drag)

while True:
    cv2.imshow("Title of Popup Window", img)
    if cv2.waitKey(10) == 27:
        break
cv2.destroyAllWindows()

drag-drop-square

If you looking for a more elegant way of doing, then use the below code, but at a given time, you can only one rectangle.

import cv2
img = cv2.imread("flower.jpg")
#img2 = cv2.imread("flower.jpg")

# variables
ix = -1
iy = -1
drawing = False

def draw_reactangle_with_drag(event, x, y, flags, param):
    global ix, iy, drawing, img
    if event == cv2.EVENT_LBUTTONDOWN:
        drawing = True
        ix = x
        iy = y
            
            
    elif event == cv2.EVENT_MOUSEMOVE:
        if drawing == True:
            img2 = cv2.imread("flower.jpg")
            cv2.rectangle(img2, pt1=(ix,iy), pt2=(x, y),color=(0,255,255),thickness=10)
            img = img2
    
    elif event == cv2.EVENT_LBUTTONUP:
        drawing = False
        img2 = cv2.imread("flower.jpg")
        cv2.rectangle(img2, pt1=(ix,iy), pt2=(x, y),color=(0,255,255),thickness=10)
        img = img2
        
cv2.namedWindow(winname= "Title of Popup Window")
cv2.setMouseCallback("Title of Popup Window", draw_reactangle_with_drag)

while True:
    cv2.imshow("Title of Popup Window", img)
    if cv2.waitKey(10) == 27:
        break
cv2.destroyAllWindows()

drag-drop-rectangle-opencv

Comment / Suggestion Section
Point our Mistakes and Post Your Suggestions

Subscribe to See Videos

Subscribe to my Youtube channel for new videos : Subscribe Now