Understanding Privacy Preserving Knowledge in models via Mechanistic Interpretability

This project aims to study the privacy-preserving properties of LLMs with a real-world application in mind. The idea is to see if a model deployed in a setting where it has access to PII data and is asked to perform a task(summarization, drawing insights etc) without leaking PII data(by masking) can still potentially leak data by looking at internal activations and top k tokens generated.