AI Threat Alert
Benchmark MEDIUM
Breadcrumbing Search Agents
Xuebin Li, Hanqing Zhao, Siyuan Liang +4 more
LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security...
AI Security Research
AI Threat Alert indexes 3,371+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
- Adversarial attacks
- Model defenses
- Red-teaming benchmarks
- Surveys
- Security tooling
- Total3,371
- Attack1,303
- Benchmark970
- Defense471
- Tool340
- Survey195
Showing 41–60 of 237 papers
Clear filters Benchmark MEDIUM
Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness
Haoting Qian, Qingjie Zhang, Zhicong Huang +2 more
Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not....
Other MEDIUM
Emergence of Reputation-Based Cooperation in LLM Agents
Kazuya Horibe, Kenji Itao, Wataru Toyokawa
Can cooperation among large language model (LLM) agents be evolutionarily stable against free-rider invasion? We study an indirect reciprocity...
Benchmark MEDIUM
Combating Knowledge Corruption in Agent Systems: A Byzantine-Tolerant Secure Collaborative RAG Framework
Zhaoqi Wang, Daqing He, Zijian Zhang +13 more
While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new...
Defense MEDIUM
Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)
Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi +6 more
Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated...
Defense HIGH
LLM-based Vulnerability Discovery in Business Process Documentation
Ben Falchuk, Himanshu Garg, Euthimios Panagos +1 more
Just like software and hardware, business processes are susceptible to vulnerabilities that can lead to product quality issues, delays, and increased...
Attack MEDIUM
Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills
Peichun Hua, Haoxuan Xu, Mengyuan Li
Closed source agent skills may encode proprietary instructions, scripts, constants, and data. Providers may offer their capabilities as services...
Attack MEDIUM
Understanding Fault Tolerance of Adversarially Robust Pruned Models
Manali Dangarikar, Cory Merkel
Deep neural networks (DNNs) deployed on resource-constrained neuromorphic hardware face three concurrent challenges: the need for model compression...
Defense LOW
A game theory for foundation models shows new paths to rational cooperation through similarity inference
Alexander Meulemans, Maciej Wołczyk, Marissa A. Weis +11 more
As autonomous agents powered by foundation models are increasingly integrated into social and economic systems, understanding the principles...
Benchmark LOW
VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs
Andrei Chetvergov, Alexander Evseev, Timofei Sivoraksha +4 more
Large language models routinely describe socially salient targets, including political figures, countries, religions, organizations, historical...
Benchmark MEDIUM
An Inline Control Architecture for Language Models in Intelligent Transportation Systems
Narendra Kumar Dewangan, Mounira Msahli
Vehicle-to-everything (V2X) systems increasingly incorporate large language models (LLMs) for semantic tasks such as message summarization, operator...
Other MEDIUM
AutoSND: From Execution Evidence to Structural Policies for Automated Network Dismantling Heuristic Discovery
Zhijing Hu, Changjun Fan, Yufan Deng +1 more
Network dismantling is fundamental to analyzing the robustness and vulnerability of complex systems, yet practical heuristics must balance...
Benchmark HIGH
DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction
Xuyang Liu, Yibin Han, Zhenwei Zhang +8 more
Large Language Model (LLM) agents offer a promising approach to attack chain reconstruction by retrieving and interpreting heterogeneous telemetry to...
Defense MEDIUM
SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills
Nizhang Li, Zonghao Ying, Xiangfan Wu +7 more
External skills extend the capabilities of large language model agents, but also introduce an execution-time attack surface: a skill that appears...
Tool HIGH
AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection
Shihao Weng, Yang Feng, Xiaofei Xie +1 more
Prompt injection remains a critical threat to LLM agents, yet existing defenses treat each task as a self-contained problem, independent of previous...
Defense MEDIUM
When Modalities Fail to Tango: Conformal Backdoor Detection in Multimodal Contrastive Learning
Yiming Chen, Kemou Li, Haiwei Wu +1 more
Backdoor attacks in multimodal contrastive learning (MCL) have garnered growing attention in recent years, as many downstream tasks critically depend...
Defense MEDIUM
The Evolutionary Origin of Values: implications for AI alignment, sentience and existential risk
Francis Heylighen
AI systems based on Large Language Models (LLMs) have prompted fears that they may harbor hidden goals, seek to dominate or eliminate humanity, or...
Attack HIGH
Attacking and Defending Multi-Agent Collaborative Filtering Systems Through Connectivity
Anjun Hu, Hanting Xie, Saranya Govindan +2 more
Multi-agent collaborative filtering (CF) systems coordinate autonomous LLM-powered user and item agents through natural-language interaction to...
Attack MEDIUM
MalTotal: Cost-Effective and Language-Agnostic Malicious Code Poisoning Detection for Millions of Repositories
Jian Zhao, Shenao Wang, Qingyang Wu +3 more
The widespread adoption of open source software (OSS) has introduced significant security risks, with malicious code poisoning attacks increasingly...
Benchmark LOW
Internalizing Academic Writing Workflows for Introduction Generation via Struct-Aware Policy Learning
Meicong Zhang, Tiancheng Su, Jiahao Cheng +3 more
Generating a rigorous paper introduction with large language models (LLMs) remains challenging, since it requires coordinating background, gap...
Frequently asked questions
What is AI security research?
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
How many AI security papers does AI Threat Alert track?
AI Threat Alert indexes 3,371+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Where do the research papers come from?
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
What topics does the AI security research cover?
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
How is this different from a generic paper search?
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial