Home
Uni-Logo
 

cvFlap Project - Computer Vision Cat Flap With Mouse Recognition

Cat without mouse: Cat flap is supposed to open Cat with mouse: Cat flap is supposed to keep closed

After our cats brought several mice (dead or alive) through the cat flap, we decided to find a sustainable solution for that problem. The suggestion of my wife to close the cat flap forever would work of course but is uncool, and we will have to open the door every few minutes, when one of the cat wants to come in or go out.

The much simpler (and a little more nerdy) solution is a camera, an image recognition software, and a computer controlled cat flap that only opens, if the cat comes without a mouse or any other hunting trophy. The idea is old, and already about 10 years ago someone build such a cat flap called Flo control. Their setup (camera from the side, back light illumination) looks easy to analyze, but I doubt, if one really can recognize any mouse just by the silhouette. So our cat flap has an LED illumination beside the webcam and a blue screen in the background. The software will be developed in an open source project.

Phase 1: Setting up the hardware

One goal was to make the reproduction of the cvFlap as easy as possible, to acquire a large community for the Open-Source Project. The parts outside are a standard USB webcam (I use the Logitech C270 USB HD Webcam (around 22,- EUR), because it is well supported by the Linux-Kernel and provides a good image quality), and some LED illumination. As computer we took an old laptop running Ubuntu-Linux 10.04, but as the software is build based on OpenCV, it should be possible to compile the software for other operating systems (like Windows, or Android).

The cheapest and easiest way to build some computer controlled door opener with USB connection is to disassemble a gamepad with a rumble motor inside (I bought the Hama USB-Gamepad "Black Force", (around 12,- EUR) which was the cheapest with rumble motor that I found). There are ready drivers for joysticks in the linux kernel, so everything can be done with a standard Installation. I removed one motor (there are two) from the gamepad, put a longer cable between the controller and the motor and use the motor now to open the latch at the cat flap (sketch and photo of prototype will follow soon)

Cat flap with image recognition mounted in the basement window Well cover removed. Left: webcam and LED illumination. Right: Cat flap and blue screen

Phase 2: Collection of Training data

As we want to develop software that works for every cat and every cat flap, the image recognition has to be adapted with training images to each surrounding.

For the collection of the training images we use the "BackgroundSubtractorMOG"-class of OpenCV to split the image into static background and moving foreground, and save the image if we detect more that 1 percent foreground pixels. Everything is there in OpenCV, so the source code collect_training_data.cc is very short. If you have setup OpenCV properly you can compile it with

c++ `pkg-config opencv --cflags --libs` -Wall collect_training_data.cc -o collect_training_data

Don't forget to adapt the path for the stored images in the source code. That's it. Create the directory for the images (if not already there) and start the program.

  mkdir /home/ronneber/Docs/Katzenklappe/liveimages
  cd /home/ronneber/Docs/Katzenklappe/
  ./collect_training_data

[28.9.2011] This program now runs for about two weeks. The laptop acquires about 5 images per second, as soon as something moves. Our two cats enter and leave the house several times at the day. Until now we have around 9600 images, and four mouse events.

Phase 3: Development of the Image Recognition Software

[28.9.2011] The development of the image recognition software just started. If you want to participate in this project, just let me know. If you are Computer Science student at the University of Freiburg you can do a part of this project as "Team project", "Bachelor thesis" or "Master thesis".

The available work packages are

  1. Detection of cat face and cat mouth. Output should be: Center/Bounding Box of Mouth, or "no cat mouth visible". Evaluation of standard methods, Extension of standard methods, Development of own methods. Check the following methods:
    • Normalized Cross Correlation
    • Background Subtraction, Segmentation
    • Viola-Jones Detector (Haar Wavelets, Boosting)
    • HOG (Histogram of Oriented Gradients) + Linear Support Vector Machine
    • Feature Based approaches, Voting
    • Latent SVM
  2. Clean mouth Recognition: Classification into "clean mouth" (cat flap opens) and "no clean mouth" (cat flap keeps closed). Possible approaches are:
    • Normalized Cross correlation
    • HOG + SVM
    • Other Features + SVM