How On-Device Object Detection Works
What this detector does
The AI Object Detector scans a photo and draws a labeled box around every recognizable thing it finds -- a person, a car, a dog, a laptop, a chair. Under the hood it runs the COCO-SSD model through TensorFlow.js, a neural network trained on the COCO dataset to recognize 80 everyday object classes. Each detection gives you three pieces of information: a class name, a bounding box marking where the object sits, and a confidence score from 0 to 100% telling you how sure the model is.
Detection starts the moment an image loads, and the whole analysis happens on-device -- the model file (~10MB) downloads once and is cached for later visits.
When to reach for it
Use it to auto-tag a folder of photos, sanity-check what a dataset actually contains before training your own model, or count how many people or vehicles appear in a scene. It is handy for accessibility work too -- generating a quick text description of an image -- and for anyone prototyping computer-vision ideas without wiring up a Python environment.
Because it exports structured JSON with coordinates, it also fits into a larger pipeline: run the detector, take the boxes, and feed them into cropping, blurring, or annotation scripts.
A concrete example
Drop in a street photo. The model might return person at 92%, car at 88%, and traffic light at 71%, each wrapped in its own colored box. Raise the minimum confidence to 80% and the traffic light disappears from the results, leaving only the two high-certainty detections. Lower it to 40% and fainter, partially hidden objects start showing up -- along with the occasional wrong guess. Download the annotated image to keep the overlays, or export JSON to get the raw class names, scores, and box coordinates.
Notes and limits
COCO-SSD knows 80 classes, so anything outside that list -- a specific dog breed, a brand logo, readable text -- will not be labeled. Overlapping or very small objects are the hardest cases and often slip below the threshold. The confidence slider is your main control: higher values trade recall for precision, lower values do the reverse. If a photo returns nothing, dropping the threshold usually surfaces the objects the model saw but was not confident about.