AI Threat AlertAttackHIGH
TrojanPraise: Jailbreak LLMs via Benign Fine-Tuning
Zhixin Xie, Xurui Song, Jun Luo
The demand of customized large language models (LLMs) has led to commercial LLMs offering black-box fine-tuning APIs, yet this convenience introduces...
AI Security Research
AI Threat Alert indexes 4,319+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
- Adversarial attacks
- Model defenses
- Red-teaming benchmarks
- Surveys
- Security tooling
- Total4,319
- Attack1,676
- Benchmark1,229
- Defense610
- Tool432
- Survey252
Showing 3061–3080 of 4,319 papers
ToolMEDIUM
AgenTRIM: Tool Risk Mitigation for Agentic AI
Roy Betser, Shamik Bose, Amit Giloni +3 more
AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper tool...
SurveyMEDIUM
Adversarial Defense in Vision-Language Models: An Overview
Xiaowei Fu, Lei Zhang
The widespread use of Vision Language Models (VLMs, e.g. CLIP) has raised concerns about their vulnerability to sophisticated and imperceptible...
SurveyMEDIUM
De-Anonymization at Scale via Tournament-Style Attribution
Lirui Zhang, Huishuai Zhang
As LLMs rapidly advance and enter real-world use, their privacy implications are increasingly important. We study an authorship de-anonymization...
AttackHIGH
Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMs
Anirudh Sekar, Mrinal Agarwal, Rachel Sharma +4 more
Prompt injection attacks have become an increasing vulnerability for LLM applications, where adversarial prompts exploit indirect input channels such...
BenchmarkMEDIUM
Efficient Privacy-Preserving Retrieval Augmented Generation with Distance-Preserving Encryption
Huanyi Ye, Jiale Guo, Ziyao Liu +1 more
RAG has emerged as a key technique for enhancing response quality of LLMs without high computational cost. In traditional architectures, RAG services...
DefenseLOW
Cross-reality Location Privacy Protection in 6G-enabled Vehicular Metaverses: An LLM-enhanced Hybrid Generative Diffusion Model-based Approach
Xiaofeng Luo, Jiayi He, Jiawen Kang +4 more
The emergence of 6G-enabled vehicular metaverses enables Autonomous Vehicles (AVs) to operate across physical and virtual spaces through...
BenchmarkMEDIUM
Multimodal Generative Engine Optimization: Rank Manipulation for Vision-Language Model Rankers
Yixuan Du, Chenxiao Yu, Haoyan Xu +3 more
Vision-Language Models (VLMs) are rapidly replacing unimodal encoders in modern retrieval and recommendation systems. While their capabilities are...
BenchmarkMEDIUM
Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Models
Xiaomei Zhang, Zhaoxi Zhang, Leo Yu Zhang +3 more
Visual token compression is widely adopted to improve the inference efficiency of Large Vision-Language Models (LVLMs), enabling their deployment in...
ToolMEDIUM
Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework
Zimo Ji, Daoyuan Wu, Wenyuan Jiang +5 more
Large Language Model (LLM)-based agent systems are increasingly deployed for complex real-world tasks but remain vulnerable to natural language-based...
BenchmarkMEDIUM
Gradient Structure Estimation under Label-Only Oracles via Spectral Sensitivity
Jun Liu, Leo Yu Zhang, Fengpeng Li +2 more
Hard-label black-box settings, where only top-1 predicted labels are observable, pose a fundamentally constrained yet practically important feedback...
BenchmarkMEDIUM
Gradient Structure Estimation under Label-Only Oracles via Spectral Sensitivity
Jun Liu, Leo Yu Zhang, Fengpeng Li +2 more
Hard-label black-box settings, where only top-1 predicted labels are observable, pose a fundamentally constrained yet practically important feedback...
AttackMEDIUM
Building Production-Ready Probes For Gemini
János Kramár, Joshua Engels, Zheng Wang +4 more
Frontier language model capabilities are improving rapidly. We thus need stronger mitigations against bad actors misusing increasingly powerful...
BenchmarkLOW
LLM-Assisted Pseudo-Relevance Feedback
David Otero, Javier Parapar
Query expansion is a long-standing technique to mitigate vocabulary mismatch in ad hoc Information Retrieval. Pseudo-relevance feedback methods, such...
AttackMEDIUM
LoRA as Oracle
Marco Arazzi, Antonino Nocera
Backdoored and privacy-leaking deep neural networks pose a serious threat to the deployment of machine learning systems in security-critical...
AttackHIGH
SD-RAG: A Prompt-Injection-Resilient Framework for Selective Disclosure in Retrieval-Augmented Generation
Aiman Al Masoud, Marco Arazzi, Antonino Nocera
Retrieval-Augmented Generation (RAG) has attracted significant attention due to its ability to combine the generative capabilities of Large Language...
BenchmarkLOW
CoG: Controllable Graph Reasoning via Relational Blueprints and Failure-Aware Refinement over Knowledge Graphs
Yuanxiang Liu, Songze Li, Xiaoke Guo +4 more
Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities but often grapple with reliability challenges like hallucinations....
AttackHIGH
AJAR: Adaptive Jailbreak Architecture for Red-teaming
Yipu Dou, Wang Yang
Large language model (LLM) safety evaluation is moving from content moderation to action security as modern systems gain persistent state, tool...
ToolMEDIUM
Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents
Kaiyu Zhou, Yongsen Zheng, Yicheng He +5 more
The agent--tool interaction loop is a critical attack surface for modern Large Language Model (LLM) agents. Existing denial-of-service (DoS) attacks...
BenchmarkHIGH
Hidden-in-Plain-Text: A Benchmark for Social-Web Indirect Prompt Injection in RAG
Haoze Guo, Ziqi Wei
Retrieval-augmented generation (RAG) systems put more and more emphasis on grounding their responses in user-generated content found on the Web,...
Frequently asked questions
What is AI security research?
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
How many AI security papers does AI Threat Alert track?
AI Threat Alert indexes 4,319+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Where do the research papers come from?
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
What topics does the AI security research cover?
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
How is this different from a generic paper search?
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial