An end-to-end, ultra-low-cost IoT computer vision system designed to automate fruit quality inspection. This project proves that highly accurate, multi-object enterprise-grade AI inspection can be achieved on budget edge hardware (a $5 camera) without heavy local GPU processing.
Fruit spoilage is a critical challenge in agricultural supply chains. Existing automated solutions often rely on expensive industrial cameras or struggle with overlapping fruits. This project provides a decentralized, low-cost system capable of multi-object detection and real-time quality classification.
The entire ecosystem is controlled via an asynchronous Telegram Bot interface, allowing users to trigger hardware captures remotely and receive immediate, annotated quality reports directly to their phones or desktops.
Initial iterations utilizing MobileNetV2 struggled to localize and separate multiple fruits in a single frame. The architecture was pivoted to a custom YOLOv8 Nano model trained on a manually cleaned dataset of 8 distinct classes of fresh and spoiled fruits.
To overcome cloud latency and timeouts, the PyTorch .pt weights were optimized into the ONNX format. This eliminated framework overhead and reduced inference latency on a free-tier cloud CPU from over 15 seconds down to just 3-4 seconds.
| Metric | Score |
|---|---|
| mAP@50 | 98.97% |
| Precision | 98.17% |
| Recall | 97.79% |
| Inference Speed | 3-4 seconds (Cloud CPU) |
| Component | Technologies Used |
|---|---|
| Hardware | ESP32-CAM Development Board, OV2640 2MP Camera Sensor |
| Deep Learning | YOLOv8 Nano (trained on Intel Arc A750 GPU), ONNX Runtime |
| Backend | Python 3.10, FastAPI, Uvicorn, OpenCV, httpx, asyncio |
| Cloud Infrastructure | Hugging Face Spaces (Free Tier CPU) |
| User Interface | Telegram Bot API |
| Edge Firmware | C++ (Arduino IDE), WiFiClientSecure, HTTPClient |
The operational flow is designed asynchronously to prevent blocking and handle hardware timeouts:
- The user sends a
/capturecommand via the Telegram Bot. - The FastAPI backend receives the command and sets a state trigger flag.
- The ESP32-CAM (polling every 2 seconds) detects the flag, wakes up, clears stale buffers, and captures a QVGA image.
- The edge device compresses the frame to JPEG and pushes the payload via HTTP POST to the cloud.
- The ONNX-optimized YOLOv8 engine processes the image and calculates confidence scores.
- OpenCV draws bounding boxes, formats a Markdown statistical summary, and pushes the final report back to the user.
To run this project on a local machine (Windows, Mac, or Linux), follow these steps:
- Download Python 3.10 or newer (ensure you check "Add python.exe to PATH" during installation on Windows).
- Download the dataset from Mendeley: Rotten Fruit Detector Dataset (v3).
- Open your terminal and create an isolated virtual environment:
# Windows
python -m venv venv
venv\Scripts\activate
# Mac/Linux
python3 -m venv venv
source venv/bin/activate- Install the required AI packages (this installs
ultralytics,opencv-python, andonnxruntime):
pip install -r requirements.txt- Test the model on a local image (the
--showflag will pop up a window with bounding boxes):
python predict.py "1.jpg" --showTo connect the physical camera to your network:
- Plug the ESP32-CAM into your computer using an ESP32-CAM-MB micro-USB shield.
- Open the Arduino IDE.
- Update the Wi-Fi credentials (SSID and Password) to your local router in the
ESP-32_Flash_File.cppfile. - Flash the code to the board.
To set up your own remote UI:
- Open Telegram and search for "BotFather".
- Send the
/newbotcommand and provide a name and username (e.g.,Rotten Fruit Detector). - Copy the generated
BOT TOKEN. - Install the specific bot libraries in your environment:
pip install python-telegram-bot inference pillow- Cloud Sleep States: The current deployment relies on the Hugging Face Free Tier, which imposes a "cold start" sleep state after 48 hours of inactivity. Future iterations will migrate to a dedicated DigitalOcean App Platform container to eliminate this delay.
- Network Dependency: The system strictly requires a 2.4GHz Wi-Fi connection, making deep-field agricultural deployment challenging. Future hardware will integrate a custom PCB with a battery management system and a GSM/LTE module for remote operation.
- Dataset Expansion: The model will be continuously trained on expanded datasets to include more regional fruit classifications.
This architecture scales directly to conveyor-belt sorting systems and cold-chain logistics monitoring at a fraction of traditional costs. None of this was possible without the dedication of the team.
Supervised by: Sayefa Arafah Arpona (Lecturer, Department of CSE, Bangladesh University of Business and Technology)
R&D by: Sohel Rana, Faysal Islam Fahad, Naushin Sultana MiMi, and Mst. Milhan Jannat Jerin.