AI Threat Alert
Attack HIGH
Untargeted Jailbreak Attack
Xinzhe Huang, Wenjing Hu, Tianhang Zheng +5 more
Existing gradient-based jailbreak attacks on Large Language Models (LLMs) typically optimize adversarial suffixes to align the LLM output with...
AI Security Research
AI Threat Alert indexes 3,771+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
- Adversarial attacks
- Model defenses
- Red-teaming benchmarks
- Surveys
- Security tooling
- Total3,771
- Attack1,458
- Benchmark1,072
- Defense533
- Tool380
- Survey220
Showing 1201–1220 of 1,257 papers
Clear filters Attack HIGH
External Data Extraction Attacks against Retrieval-Augmented Large Language Models
Yu He, Yifei Chen, Yiming Li +5 more
In recent years, RAG has emerged as a key paradigm for enhancing large language models (LLMs). By integrating externally retrieved information, RAG...
Attack HIGH
Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs
Zhixin Xie, Xurui Song, Jun Luo
Despite substantial efforts in safety alignment, recent research indicates that Large Language Models (LLMs) remain highly susceptible to jailbreak...
Attack HIGH
A Statistical Method for Attack-Agnostic Adversarial Attack Detection with Compressive Sensing Comparison
Chinthana Wimalasuriya, Spyros Tragoudas
Adversarial attacks present a significant threat to modern machine learning systems. Yet, existing detection methods often lack the ability to detect...
Attack HIGH
ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks
Zhaorun Chen, Xun Liu, Mintong Kang +4 more
As vision-language models (VLMs) gain prominence, their multimodal interfaces also introduce new safety vulnerabilities, making the safety evaluation...
Benchmark HIGH
RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents
Chengquan Guo, Chulin Xie, Yu Yang +6 more
Code agents have gained widespread adoption due to their strong code generation capabilities and integration with code interpreters, enabling dynamic...
Tool HIGH
ToolTweak: An Attack on Tool Selection in LLM-based Agents
Jonathan Sneh, Ruomei Yan, Jialin Yu +6 more
As LLMs increasingly power agents that interact with external tools, tool use has become an essential mechanism for extending their capabilities....
Attack HIGH
Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks
Ruohao Guo, Afshin Oroojlooy, Roshan Sridhar +3 more
Despite recent rapid progress in AI safety, current large language models remain vulnerable to adversarial attacks in multi-turn interaction...
Attack HIGH
Dynamic Target Attack
Kedong Xiu, Churui Zeng, Tianhang Zheng +6 more
Existing gradient-based jailbreak attacks typically optimize an adversarial suffix to induce a fixed affirmative response, e.g., ``Sure, here...
Attack HIGH
Evaluating the Robustness of a Production Malware Detection System to Transferable Adversarial Attacks
Milad Nasr, Yanick Fratantonio, Luca Invernizzi +7 more
As deep learning models become widely deployed as components within larger production systems, their individual shortcomings can create system-level...
Attack HIGH
Machine Learning for Detection and Analysis of Novel LLM Jailbreaks
John Hawkins, Aditya Pramar, Rodney Beard +1 more
Large Language Models (LLMs) suffer from a range of vulnerabilities that allow malicious users to solicit undesirable responses through manipulation...
Attack HIGH
Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
Isha Gupta, Rylan Schaeffer, Joshua Kazdan +2 more
The field of adversarial robustness has long established that adversarial examples can successfully transfer between image classifiers and that text...
Tool HIGH
Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
Shoumik Saha, Jifan Chen, Sam Mayers +3 more
Code-capable large language model (LLM) agents are increasingly embedded into software engineering workflows where they can read, write, and execute...
Benchmark HIGH
WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents
Yinuo Liu, Ruohan Xu, Xilong Wang +2 more
Multiple prompt injection attacks have been proposed against web agents. At the same time, various methods have been developed to detect general...
Attack HIGH
Fine-Tuning Jailbreaks under Highly Constrained Black-Box Settings: A Three-Pronged Approach
Xiangfang Li, Yu Wang, Bo Li
With the rapid advancement of large language models (LLMs), ensuring their safe use becomes increasingly critical. Fine-tuning is a widely used...
Attack HIGH
Backdoor Attacks Against Speech Language Models
Alexandrine Fortier, Thomas Thebaud, Jesús Villalba +2 more
Large Language Models (LLMs) and their multimodal extensions are becoming increasingly popular. One common approach to enable multimodality is to...
Defense HIGH
Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability
Shojiro Yamabe, Jun Sakuma
Diffusion language models (DLMs) generate tokens in parallel through iterative denoising, which can reduce latency and enable bidirectional...
Attack HIGH
Attack logics, not outputs: Towards efficient robustification of deep neural networks by falsifying concept-based properties
Raik Dankworth, Gesina Schwalbe
Deep neural networks (NNs) for computer vision are vulnerable to adversarial attacks, i.e., miniscule malicious changes to inputs may induce...
Attack HIGH
SVDefense: Effective Defense against Gradient Inversion Attacks via Singular Value Decomposition
Chenxiang Luo, David K. Y. Yau, Qun Song
Federated learning (FL) enables collaborative model training without sharing raw data but is vulnerable to gradient inversion attacks (GIAs), where...
Benchmark HIGH
From Trace to Line: LLM Agent for Real-World OSS Vulnerability Localization
Haoran Xi, Minghao Shao, Brendan Dolan-Gavitt +2 more
Large language models show promise for vulnerability discovery, yet prevailing methods inspect code in isolation, struggle with long contexts, and...
Frequently asked questions
What is AI security research?
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
How many AI security papers does AI Threat Alert track?
AI Threat Alert indexes 3,771+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Where do the research papers come from?
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
What topics does the AI security research cover?
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
How is this different from a generic paper search?
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial