AI Threat Alert
Attack HIGH
Dynamic Target Attack
Kedong Xiu, Churui Zeng, Tianhang Zheng +6 more
Existing gradient-based jailbreak attacks typically optimize an adversarial suffix to induce a fixed affirmative response, e.g., ``Sure, here...
AI Security Research
AI Threat Alert indexes 3,371+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
- Adversarial attacks
- Model defenses
- Red-teaming benchmarks
- Surveys
- Security tooling
- Total3,371
- Attack1,303
- Benchmark970
- Defense471
- Tool340
- Survey195
Showing 1241–1260 of 1,303 papers
Clear filters Attack MEDIUM
Inverse Language Modeling towards Robust and Grounded LLMs
Davide Gabrielli, Simone Sestito, Iacopo Masi
The current landscape of defensive mechanisms for LLMs is fragmented and underdeveloped, unlike prior work on classifiers. To further promote...
Attack HIGH
Evaluating the Robustness of a Production Malware Detection System to Transferable Adversarial Attacks
Milad Nasr, Yanick Fratantonio, Luca Invernizzi +7 more
As deep learning models become widely deployed as components within larger production systems, their individual shortcomings can create system-level...
Attack HIGH
Machine Learning for Detection and Analysis of Novel LLM Jailbreaks
John Hawkins, Aditya Pramar, Rodney Beard +1 more
Large Language Models (LLMs) suffer from a range of vulnerabilities that allow malicious users to solicit undesirable responses through manipulation...
Attack MEDIUM
Securing generative artificial intelligence with parallel magnetic tunnel junction true randomness
Youwei Bao, Shuhan Yang, Hyunsoo Yang
Deterministic pseudo random number generators (PRNGs) used in generative artificial intelligence (GAI) models produce predictable patterns vulnerable...
Attack MEDIUM
AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning
Zhenyu Pan, Yiting Zhang, Zhuo Liu +13 more
LLM-based multi-agent systems excel at planning, tool use, and role coordination, but their openness and interaction complexity also expose them to...
Attack MEDIUM
Bypassing Prompt Guards in Production with Controlled-Release Prompting
Jaiden Fairoze, Sanjam Garg, Keewoo Lee +1 more
As large language models (LLMs) advance, ensuring AI safety and alignment is paramount. One popular approach is prompt guards, lightweight mechanisms...
Attack HIGH
Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
Isha Gupta, Rylan Schaeffer, Joshua Kazdan +2 more
The field of adversarial robustness has long established that adversarial examples can successfully transfer between image classifiers and that text...
Attack HIGH
Fine-Tuning Jailbreaks under Highly Constrained Black-Box Settings: A Three-Pronged Approach
Xiangfang Li, Yu Wang, Bo Li
With the rapid advancement of large language models (LLMs), ensuring their safe use becomes increasingly critical. Fine-tuning is a widely used...
Attack HIGH
Backdoor Attacks Against Speech Language Models
Alexandrine Fortier, Thomas Thebaud, Jesús Villalba +2 more
Large Language Models (LLMs) and their multimodal extensions are becoming increasingly popular. One common approach to enable multimodality is to...
Attack MEDIUM
Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors
Yen-Shan Chen, Sian-Yao Huang, Cheng-Lin Yang +1 more
Existing data poisoning attacks on retrieval-augmented generation (RAG) systems scale poorly because they require costly optimization of poisoned...
Attack HIGH
Attack logics, not outputs: Towards efficient robustification of deep neural networks by falsifying concept-based properties
Raik Dankworth, Gesina Schwalbe
Deep neural networks (NNs) for computer vision are vulnerable to adversarial attacks, i.e., miniscule malicious changes to inputs may induce...
Attack MEDIUM
Understanding Sensitivity of Differential Attention through the Lens of Adversarial Robustness
Tsubasa Takahashi, Shojiro Yamabe, Futa Waseda +1 more
Differential Attention (DA) has been proposed as a refinement to standard attention, suppressing redundant or noisy context through a subtractive...
Attack MEDIUM
Has the Two-Decade-Old Prophecy Come True? Artificial Bad Intelligence Triggered by Merely a Single-Bit Flip in Large Language Models
Yu Yan, Siqi Lu, Yang Gao +4 more
Recently, Bit-Flip Attack (BFA) has garnered widespread attention for its ability to compromise software system integrity remotely through hardware...
Attack HIGH
SVDefense: Effective Defense against Gradient Inversion Attacks via Singular Value Decomposition
Chenxiang Luo, David K. Y. Yau, Qun Song
Federated learning (FL) enables collaborative model training without sharing raw data but is vulnerable to gradient inversion attacks (GIAs), where...
Attack MEDIUM
A Call to Action for a Secure-by-Design Generative AI Paradigm
Dalal Alharthi, Ivan Roberto Kawaminami Garcia
Large language models have gained widespread prominence, yet their vulnerability to prompt injection and other adversarial attacks remains a critical...
Attack MEDIUM
MOLM: Mixture of LoRA Markers
Samar Fares, Nurbek Tastan, Noor Hussein +1 more
Generative models can generate photorealistic images at scale. This raises urgent concerns about the ability to detect synthetically generated images...
Attack MEDIUM
CHAI: Command Hijacking against embodied AI
Luis Burbano, Diego Ortiz, Qi Sun +5 more
Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning...
Attack MEDIUM
Are Robust LLM Fingerprints Adversarially Robust?
Anshul Nasery, Edoardo Contente, Alkin Kaz +2 more
Model fingerprinting has emerged as a promising paradigm for claiming model ownership. However, robustness evaluations of these schemes have mostly...
Attack MEDIUM
DeepProv: Behavioral Characterization and Repair of Neural Networks via Inference Provenance Graph Analysis
Firas Ben Hmida, Abderrahmen Amich, Ata Kaboudi +1 more
Deep neural networks (DNNs) are increasingly being deployed in high-stakes applications, from self-driving cars to biometric authentication. However,...
Frequently asked questions
What is AI security research?
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
How many AI security papers does AI Threat Alert track?
AI Threat Alert indexes 3,371+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Where do the research papers come from?
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
What topics does the AI security research cover?
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
How is this different from a generic paper search?
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial