AI Threat AlertBenchmarkMEDIUM
Why Backdooring Neural Networks is so Easy?
Issam Seddik, Mohamed El Amine Seddik
Securing modern AI systems against backdoor attacks remains an open challenge and requires fundamentally principled estimates of the adversary's...
AI Security Research
AI Threat Alert indexes 4,319+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
- Adversarial attacks
- Model defenses
- Red-teaming benchmarks
- Surveys
- Security tooling
- Total4,319
- Attack1,676
- Benchmark1,229
- Defense610
- Tool432
- Survey252
Showing 521–540 of 1,741 papers
Clear filtersBenchmarkLOW
SAGE: A Statistical Acceptance Gate for Self-Evolving Agents
Yihao Wang, Linhan Xia, Rui Liu +6 more
Large Language Model (LLM)-based agents increasingly self-evolve by editing a persistent skill document that encodes their workflow, tool-use rules,...
SurveyMEDIUM
Tracekit: Tamper-Evident Intent-Reasoning-Action Auditing for Autonomous Coding Agents
Bravish Ghosh
Autonomous coding agents read untrusted files, run shell commands and spawn sub-agents with little supervision, yet their record is usually an...
AttackHIGH
Share-Borne AI Virus: Memory-Hopping Attacks Across LLM Agents
Sidharth Pulipaka, Ansh Sharma, Stanislau Hlebik +4 more
Large language models are increasingly deployed as stateful assistants that retain information across interactions and use tools to read, modify, and...
AttackHIGH
Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection
Yan Zhan, Yunze Song, Mengkai Hou +3 more
Prompt injection against LLM agents becomes much stronger when the injected instruction is wrapped in the model's own chat template. A forged...
DefenseMEDIUM
Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment
Shunchang Liu, Lukas Fluri, Xin Chen +1 more
Modern AI models are aligned through post-training to adapt them to downstream tasks. Recent work shows that fine-tuning language models on narrow...
DefenseMEDIUM
Physics-Attested Federated Learning: Securing Collaborative Anomaly Detection in Critical Water Infrastructure
Jeff Nijsse, Shu Su, Benjamin Oholeguy +1 more
Federated learning enables industrial operators to train shared intrusion detection models without disclosing proprietary operational telemetry....
BenchmarkMEDIUM
CoSec: Benchmarking Agent Security in Communities
Hao Chen, Wenhui Dong, Ye Chen +12 more
LLM agents operate in persistent collaborative environments involving multiple users, communities, memories, files, and tools. Community boundaries...
BenchmarkMEDIUM
CoSec: Benchmarking Agent Security in Communities
Hao Chen, Wenhui Dong, Ye Chen +12 more
LLM agents operate in persistent collaborative environments involving multiple users, communities, memories, files, and tools. Community boundaries...
ToolHIGH
Jailbreak Context Lingers: Divergent Safety Routing and Its Cross-Task Predictability in Tool Agents
Xi Wang, Songlei Jian, Yiming Zhang +5 more
As large language models increasingly operate as tool-using agents, post-jailbreak safety feedback is often assumed to serve as a reliable safeguard;...
AttackMEDIUM
Nudgeability: Reasoning Models Follow Confidence Signals Without Tracking Their Own Competence
Rohit Saxena, Utkarsh Upadhyay
Reasoning language models that can call tools must decide during inference whether to answer unaided or delegate. Any self-reflection mechanism for...
AttackHIGH
CoDeL: Co-Evolutionary Defense against Indirect Prompt Injection in LLM-based Agents
Xiao Yang, Yangchen Ou, Yuhan Gao +3 more
Large language model (LLM)-based agents increasingly rely on external tools and content, exposing them to indirect prompt injection (IPI). This...
BenchmarkMEDIUM
Breaking Windows Malware Detection: A Comprehensive Evaluation of Problem-Space Adversarial Robustness
Mashal Zainab, Salijona Dyrmishi, Hamid Bostani +2 more
Problem-space evasion attacks have exposed critical weaknesses in machine learning-based malware detectors; yet, their evaluation remains fragmented...
BenchmarkHIGH
ReproBench: Benchmarking LLM Agents on Reproducing Vulnerability From Scratch
Liang He, Sheng Wu, Haomiao Hao +3 more
Large language model (LLM) agents are increasingly evaluated on cybersecurity tasks such as vulnerability reproduction, exploitation, and patching....
OtherMEDIUM
Making LLMs Truly Forget: Deep Unlearning by Searching, Selecting, and Severing Knowledge Paths
Jialu Wang, Peizhi Niu, Haoteng Yin +3 more
While an unlearned language model may no longer recall a fact directly, the fact often remains recoverable through multi-hop reasoning over related...
BenchmarkLOW
AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering
Chanhee Park, Jeongho Yoon, Sungbin Han +2 more
Agentic tasks require a large language model to interact with the world, navigating information and gathering evidence across multiple steps with...
AttackMEDIUM
Certified Multi-Source Integrity for Structured Agent Actions
Anmol Pandey, Aditya Jain, Liang Chen +2 more
LLM agents increasingly take privileged, often irreversible structured actions, such as paying an invoice. They assemble each action from...
AttackLOW
The Privacy Fallacy of Crowdsourced Fine-Tuning: Extracting Proprietary Data via Topic-Based Poisoning
Sae Furukawa, Alina Oprea
Supervised fine-tuning (SFT) is widely used to adapt large language models to downstream tasks. Crowdsourcing user conversations is an established...
AttackMEDIUM
When Consent Outlives Context: Residual Authority Replay in Long-Lived Agents
Zhihao Zhang, Chao Wang, Rujia Li +3 more
LLM agents increasingly rely on user approval to authorize security-sensitive actions at runtime. Such approvals are granted within a specific task...
BenchmarkMEDIUM
Can Prompt Anonymity Protect Your Identity From LLM Providers?
Dzung Pham, Dillon Sheils, Naina Singh +1 more
User conversations with large language models (LLMs) often contain highly sensitive personal information that can be exploited by LLM providers to...
Frequently asked questions
What is AI security research?
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
How many AI security papers does AI Threat Alert track?
AI Threat Alert indexes 4,319+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Where do the research papers come from?
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
What topics does the AI security research cover?
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
How is this different from a generic paper search?
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial