MultiModal-Fashion

A multimodal fashion recommendation project that builds a visual embedding index from product images and serves a Streamlit chat UI powered by a local LLM (Ollama) and ChromaDB.

Features

• Image embedding with CLIP via sentence-transformers
• Vector search using ChromaDB (Docker)
• Local LLM responses via Ollama (Docker)
• Streamlit-based chat UI
• Simple indexing pipeline for large datasets

Project Structure

• main.py: Builds/updates the vector index from archive/data.csv and archive/data/
• chat_app.py: Streamlit chat UI for recommendations
• src/data_loader.py: CSV loading and image path validation
• src/embedding.py: Image/text embedding utilities
• src/vector_db.py: ChromaDB client wrapper
• src/llm_service.py: Ollama LLM wrapper
• docker-compose.yml: ChromaDB + Ollama services

Requirements

• Python 3.10+ recommended
• Docker + Docker Compose
• Sufficient disk space for dataset and vector index

Dataset Setup

This repo expects the dataset to exist locally (not committed to Git).

Dataset source:

• https://www.kaggle.com/datasets/nirmalsankalana/fashion-product-text-images-dataset

Expected structure:

archive/
  data.csv
  data/
    10000.jpg
    10001.jpg
    ...

Notes:

• archive/ is ignored by .gitignore to keep Git history small.
• If you want to store the dataset in Git, use Git LFS.

Setup (Local)

Create a virtual environment and install dependencies:

python -m venv .venv
# Windows PowerShell
.\.venv\Scripts\Activate.ps1
# macOS/Linux
# source .venv/bin/activate

pip install -r requirements.txt

Start Services

Run ChromaDB and Ollama:

docker-compose up -d

Pull required Ollama models inside the container:

docker exec -it ollama_service ollama pull llama3
docker exec -it ollama_service ollama pull llava

Build the Vector Index

This step reads the dataset and stores embeddings in ChromaDB.

python main.py

Run the App

Start the Streamlit UI:

streamlit run chat_app.py

Open the URL printed in the terminal (usually http://localhost:8501).

Configuration Notes

• ChromaDB runs on localhost:8000 by default.
• Ollama runs on localhost:11434 by default.
• If you do not have an NVIDIA GPU, remove the deploy.resources GPU block in docker-compose.yml.
• chroma_data/ stores local ChromaDB data and is ignored by Git.

Troubleshooting

• Chroma connection error: Ensure docker-compose up -d is running and port 8000 is open.
• Ollama model not found: Run the ollama pull commands inside the container.
• Very slow indexing: CPU indexing can be slow for large datasets; reduce batch size or use a GPU.

License

MIT License. See LICENSE for details.