AI-enhanced file integrity monitoring — SHA-256 cryptographic verification combined with Isolation Forest anomaly detection to catch tampering that hash mismatches alone can't classify.
Built as a final-year cybersecurity project at Nnamdi Azikiwe University. Production-grade FastAPI backend, full async PostgreSQL persistence, Docker deployment, and a 30-test pytest suite.
Traditional file integrity monitoring is binary: the hash either matches or it doesn't. That tells you that a file changed — not how suspicious the change is. A modified config file and a malware-injected binary both produce a hash mismatch. They should not be treated the same way.
HashGuard adds an AI layer that profiles your registered file corpus and flags anomalies based on statistical deviation — file size, byte entropy, and MIME characteristics — so altered files are classified by risk level, not just presence or absence.
File Uploaded
│
▼
SHA-256 Hash ──► Registry Lookup
│
┌───┴──────────────┐
│ │
Match Found No Match
│ │
CLEAN ✅ Feature Extraction
[file_size, entropy, mime]
│
Isolation Forest
(trained on registered corpus)
│
┌─────────┴──────────┐
score > 0 score < 0
│ │
ALTERED ⚠️ ANOMALY 🚨
(unregistered but (statistically
statistically unusual —
normal file) high suspicion)
The Isolation Forest trains dynamically on every registered file in the
corpus. The model activates once ≥5 files are registered — below that
threshold, unmatched files return UNKNOWN rather than a potentially
misleading verdict.
| Verdict | SHA-256 | AI Model | Meaning |
|---|---|---|---|
✅ CLEAN |
Match | — | File is exactly as registered. Integrity confirmed. |
ALTERED |
No match | Normal | File changed but statistically resembles the corpus. Likely legitimate modification. |
🚨 ANOMALY |
No match | Anomalous | File changed AND is statistically unusual. High suspicion — investigate. |
❓ UNKNOWN |
No match | Skipped | Corpus < 5 files. Insufficient data to run the model. |
- SHA-256 cryptographic hashing — any single-bit change produces a completely different hash (avalanche effect)
- Shannon entropy analysis — measures byte randomness; packed/encrypted payloads have distinctly different entropy profiles from normal files
- Isolation Forest anomaly detection — unsupervised ML; no labelled attack data required. Trains on what normal looks like, flags what doesn't
- MIME detection from magic bytes — not trusting file extensions, reading actual file signatures
- Full async FastAPI backend — non-blocking I/O throughout
- SQLModel + PostgreSQL — typed ORM with Alembic-ready schema
- Embedded web dashboard — file registration, verification, and corpus
view from a single UI at
/ - Docker + Docker Compose — one command to full running system
- 30-test pytest suite — unit tests for feature extraction and anomaly model, integration tests for all API endpoints, in-memory SQLite fixtures
Why Isolation Forest over a supervised classifier? Labelled malware/tampered-file datasets don't reflect your specific file corpus. A supervised model trained on generic attack data would perform poorly on a fleet of proprietary config files or internal documents. Isolation Forest trains on your normal, which means it detects your anomalies — without requiring a single labelled example.
Why Shannon entropy as a feature? Entropy is one of the most reliable indicators of file transformation. Plaintext documents sit around 4–5 bits/byte. Compressed or encrypted content hits 7.5–8 bits/byte. A PDF that suddenly reads as near-maximum entropy is almost certainly packed or injected. This is the same signal used in real-world malware classifiers.
Why SHA-256 AND machine learning — not one or the other? SHA-256 alone produces false confidence: an unknown file returns "not registered" with no risk context. ML alone is probabilistic and would produce false positives on legitimately modified files. The two-layer approach gives you cryptographic certainty where it exists and statistical risk classification where it doesn't.
Why async FastAPI over Flask/Django? File integrity checks in a real deployment happen in bulk — multiple files hitting the API simultaneously. Async handling means the verification endpoint never blocks while computing hashes or running the model on a previous request.
| Layer | Technology |
|---|---|
| API framework | FastAPI (async) |
| ML model | scikit-learn — Isolation Forest |
| Feature extraction | hashlib (SHA-256), SciPy (entropy), python-magic |
| ORM | SQLModel |
| Database | PostgreSQL (production), SQLite (test fixtures) |
| Dependency management | Poetry |
| Testing | pytest + pytest-asyncio |
hashguard/ ├── app/ │ ├── main.py # FastAPI app factory, lifespan, startup │ ├── router.py # All API endpoints │ ├── models.py # SQLModel table definitions │ ├── schemas.py # Pydantic request/response models │ ├── database.py # Engine, session, create_tables() │ ├── features.py # SHA-256, entropy, MIME detection │ ├── anomaly.py # Isolation Forest training & prediction │ └── ui.py # Embedded HTML/CSS/JS dashboard ├── tests/ │ ├── conftest.py # Fixtures — in-memory SQLite, TestClient │ ├── test_features.py # Unit tests: hashing, entropy, MIME │ ├── test_anomaly.py # Unit tests: model training, prediction │ └── test_api.py # Integration tests: all endpoints ├── Dockerfile ├── docker-compose.yml ├── pyproject.toml ├── poetry.lock └── pytest.ini
- Dashboard: http://localhost:8000
- Swagger docs: http://localhost:8000/docs
Prerequisites: Python 3.11+, Poetry, PostgreSQL running locally.
# Install dependencies
poetry install
# Configure environment
cp .env.example .env
# Edit DATABASE_URL in .env
# Run (tables are auto-created on startup)
poetry run uvicorn app.main:app| Method | Path | Description |
|---|---|---|
GET |
/ |
Web dashboard |
GET |
/docs |
Swagger / OpenAPI |
POST |
/register |
Register a file — store hash + features |
POST |
/verify |
Verify a file against the registry |
GET |
/files |
List all registered files |
DELETE |
/files/{id} |
Remove a record |
curl -X POST http://localhost:8000/register \
-F "file=@report.pdf" \
-F "tag=original"{
"status": "registered",
"record_id": 1,
"filename": "report.pdf",
"sha256": "2cf24dba5fb0a30e26e83b2ac5b9e29e...",
"file_size": 204800,
"entropy": 7.81,
"mime_guess": "application/pdf",
"tag": "original"
}curl -X POST http://localhost:8000/verify \
-F "file=@report_modified.pdf"{
"verdict": "ANOMALY",
"sha256": "91e9240f415223982edc345532630710...",
"file_size": 204902,
"entropy": 7.99,
"anomaly_score": -0.0842,
"matched_record": null,
"message": "No hash match. AI model flagged this file as statistically anomalous for this corpus."
}Tests use in-memory SQLite — no Postgres instance required.
poetry run pytest -vtests/test_features.py::TestSha256::test_known_value PASSED tests/test_features.py::TestSha256::test_avalanche_effect PASSED tests/test_features.py::TestEntropy::test_high_entropy PASSED tests/test_features.py::TestEntropy::test_low_entropy PASSED tests/test_anomaly.py::TestIsolationForest::test_training PASSED tests/test_anomaly.py::TestIsolationForest::test_prediction PASSED tests/test_api.py::TestRegister::test_register_file PASSED tests/test_api.py::TestVerify::test_verify_clean PASSED tests/test_api.py::TestVerify::test_verify_unknown PASSED ... ========================= 30 passed =========================
Built as a project for CYB at Nnamdi Azikiwe University, Department of Cybersecurity. The project explores the application of unsupervised machine learning to supplement traditional cryptographic integrity verification — specifically addressing the classification gap between "file changed" and "file is suspicious."
The formal academic report covers: threat modelling, feature selection rationale, Isolation Forest hyperparameter tuning, test methodology, and limitations of the approach.
Madueke Chiagoziem Prosper (King) B.Sc. Cybersecurity — Nnamdi Azikiwe University GitHub · LinkedIn