Data Extraction

Data extraction attacks target the information processed or memorised by AI/ML systems. They take three main forms. First, training-data extraction: large language models can memorise verbatim spans of their training corpus, and an attacker who crafts the right prompts can pull back PII, API keys, or copyrighted text — a result demonstrated against GPT-2 by Carlini et al. and reproduced against several production models. Second, model extraction: by repeatedly querying a hosted model and observing outputs, an attacker can reconstruct enough behaviour to clone proprietary fine-tunes. Third, system-prompt and conversation leakage: indirect prompt injection or insecure logging can leak the application's instructions and other users' conversations. Multi-tenant inference platforms (vLLM, Triton, hosted APIs) and RAG systems are particularly exposed. Defenses: output filtering, differential privacy in training, rate limits, and strict tenant isolation.