Local Package Detection with Frigate and Home Assistant

My front-door package reminders now use a local Frigate classifier. The doorbell watches the white porch floor, Frigate decides whether a package is waiting, and Home Assistant updates the same helper my dashboards and notifications already use.

In the original package reminder experiment, LLM Vision sent camera frames to a cloud model. That gave me a way to answer a useful question: is there still something at the door? Frigate already had the camera feed, so I moved that decision into Frigate and removed the LLM Vision integration.

Train for the place your deliveries actually land

Packages in my setup always land on the white floor. That makes the camera region easy to define. I use a broad crop for the delivery area and a second crop near the doorstep edge, where a partly visible box can disappear into the rest of the scene.

Both models use two labels: package_present and clear. This is separate from the object detector behind my Frigate driveway car alert. The car detection keeps doing its job while these small models classify the porch state.

Here is the broad crop from my Frigate configuration. The coordinates are fractions of the camera’s detection frame. Draw your own delivery area in Frigate instead of copying the numbers from my porch.

classification:
  custom:
    front_door_packages:
      threshold: 0.9
      state_config:
        motion: true
        interval: 15
        cameras:
          frontdoorbell:
            crop: [0.10, 0.74, 1.0, 1.0]

Frigate’s state-classification guide covers the setup and training screens. These models run on a CPU with AVX and AVX2 support. Training downloads the base weights; the trained model then runs locally. Frigate is free software, and each package check has no cloud API charge.

Keep the helper that the house already understands

The part I wanted to preserve was the package state. A new classifier should be able to feed the existing reminders, dashboard cards, kiosk, and briefings without making each one understand a new model.

input_boolean:
  front_door_packages_present:
    name: "Front Door Packages Present"
    icon: mdi:package-variant-closed

input_text:
  front_door_package_classification:
    max: 255

input_datetime:
  front_door_package_last_sync:
    has_date: true
    has_time: true

The Frigate integration supplies a sensor for each crop. My sync automation requires valid labels from both. Either crop reporting package_present turns the helper on. Both reporting clear turn it off. The public binary sensor still exposes that helper’s state to the rest of Home Assistant.

The notification waits for the helper to stay on for one minute, then calls the shared notification script with a camera attachment. Our measured model inference was about 5–6 milliseconds per crop. That fast decision feeds the same state confirmation and notification timing we already use.

Pickup now follows the camera state too. The previous version used an August lock relock to request another cloud check. Frigate keeps watching the floor, and the helper clears once both crops confirm the delivery area is empty. A low-confidence image holds the last verified state rather than wiping out the reminder.

Let Codex do the frame collection and training work

I asked Codex to go through our own historical doorbell footage and snapshots, extract examples with packages and examples with an empty floor, label them, and train the native Frigate models. Each crop started with 46 labeled examples: 15 package-present and 31 clear.

The useful examples included bags, boxes, shadows, people, and different lighting. Some tall and partly visible boxes gave the first model trouble. Codex added those examples, trained again, and replayed separate frames. That gives you a practical way to improve a model using the things your own camera sees.

You can give Codex a task like this:

Use my existing Frigate porch camera to build local package state classification.

Inspect the current Home Assistant package alerts and every helper consumer first. Packages are always placed on the white floor; focus the crop there.

Review my authorized historical camera footage and snapshots. Extract a small, varied set of package-present and clear examples, including bags, partial boxes, shadows, people, and night scenes. Deduplicate nearby frames and keep separate events for validation.

Create and train native Frigate state classifiers with clear and package_present labels. Keep the current object detector unchanged. Measure inference time separately from full alert latency, and test uncertain frames without publishing fake states to production.

Preserve the existing package helper and public binary sensor, shared notification routing, and downstream dashboards. Require valid classifier labels before syncing, preserve state through uncertainty, and clear only when the delivery area is confidently empty.

Validate both configurations, retain a rollback backup, deploy through the established workflow, and verify live state and camera playback. Summarize the examples used, results, and retraining changes so I can keep improving the model with my own footage. Keep images, credentials, and private addresses out of public source.

Start with one camera region and two states that mean something to your house. A gate being open, bins being in place, or a delivery waiting at the door all give Home Assistant a state it can act on. Keep the helper stable, then connect it to the reminders you want.

The complete Home Assistant package and setup and training notes are on GitHub. They include both crops, the sync automation, the notification flow, and the status helpers.

Watch the local package detection walkthrough

The video shows the camera crops, helper YAML, pickup reset, and Codex training workflow together. If you are already hosting Frigate, this is a useful next project for turning a fixed camera view into an automation.

Happy Automating!