Where is the cat?

Experimental setup

1. 01
   Seeing. Frigate watches the camera images and reports: cat. Or dog. That is all it knows. Which cat, it cannot say.
2. 02
   Recognising. So there is a small model of our own, trained on the gaming PC with a good twelve thousand crops of our own animals, by day and at night in infrared: Asuna, Haru, Sammy, Shin, Shiro and Sora. Sora is the dog. Strictly speaking the dog would not need a model, there is only one.
3. 03
   Counting. Where an animal was last seen tells whether it is inside or outside. Home Assistant shows a picture and the place for every animal, plus two counters: this many inside, this many outside.
4. 04
   Describing. At the press of a button an AI is meant to say in one sentence what the animal is doing: “Haru is lying on the cat tree, asleep.” Only a tight crop of the animal goes out for it, without background, never the whole camera image. If there is no crop, it stays with the recognition in the house. And only at the press of a button, at most once an hour per animal, so that it costs next to nothing.

What is still missing

• A computer that runs it all around the clock. The model is trained, not yet built in.
• More night images. Haru is the hardest to catch at night: at the last count there were 48 night images out of 710.
• Nothing more as far as the rule goes: what the cameras see stays in the house (chamber 06). The only exception are these small crops for step 04. No room, no background, just the animal.

Why?

Because “where is the cat?” is asked in this house more often than any other question. And because it is a nice exercise to teach a machine something every person in the house can do at a glance.

Test result

Pending. In training the model put 96 out of 100 test images with the right animal. The test subjects were not informed about the experiment. They would not have listened anyway.

Frigate source code Ultralytics YOLO