AI Threat AlertAttackHIGH
Red-teaming Activation Probes using Prompted LLMs
Phil Blandfort, Robert Graham
Activation probes are attractive monitors for AI systems due to low cost and latency, but their real-world robustness remains underexplored. We ask:...
AI Security Research
AI Threat Alert indexes 4,319+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
- Adversarial attacks
- Model defenses
- Red-teaming benchmarks
- Surveys
- Security tooling
- Total4,319
- Attack1,676
- Benchmark1,229
- Defense610
- Tool432
- Survey252
Showing 3801–3820 of 4,319 papers
BenchmarkMEDIUM
Exploring and Mitigating Gender Bias in Encoder-Based Transformer Models
Ariyan Hossain, Khondokar Mohammad Ahanaf Hannan, Rakinul Haque +4 more
Gender bias in language models has gained increasing attention in the field of natural language processing. Encoder-based transformer models, which...
DefenseMEDIUM
Reimagining Safety Alignment with An Image
Yifan Xia, Guorui Chen, Wenqian Yu +3 more
Large language models (LLMs) excel in diverse applications but face dual challenges: generating harmful content under jailbreak attacks and...
DefenseMEDIUM
Proactive DDoS Detection and Mitigation in Decentralized Software-Defined Networking via Port-Level Monitoring and Zero-Training Large Language Models
Mohammed N. Swileh, Shengli Zhang
Centralized Software-Defined Networking (cSDN) offers flexible and programmable control of networks but suffers from scalability and reliability...
AttackHIGH
DRIP: Defending Prompt Injection via Token-wise Representation Editing and Residual Instruction Fusion
Ruofan Liu, Yun Lin, Zhiyong Huang +1 more
Large language models (LLMs) are increasingly integrated into IT infrastructures, where they process user data according to predefined instructions....
AttackHIGH
ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training
Xin Yao, Haiyang Zhao, Yimin Chen +3 more
The Contrastive Language-Image Pretraining (CLIP) model has significantly advanced vision-language modeling by aligning image-text pairs from...
AttackHIGH
Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks
Kayua Oleques Paim, Rodrigo Brandao Mansilha, Diego Kreutz +2 more
The rapid proliferation of Large Language Models (LLMs) has raised significant concerns about their security against adversarial attacks. In this...
AttackMEDIUM
Diffusion LLMs are Natural Adversaries for any LLM
David Lüdke, Tom Wollschläger, Paul Ungermann +2 more
We introduce a novel framework that transforms the resource-intensive (adversarial) prompt optimization problem into an \emph{efficient, amortized...
OtherLOW
Simulating Misinformation Vulnerabilities With Agent Personas
David Farr, Lynnette Hui Xian Ng, Stephen Prochaska +2 more
Disinformation campaigns can distort public perception and destabilize institutions. Understanding how different populations respond to information...
DefenseHIGH
On the Difficulty of Selecting Few-Shot Examples for Effective LLM-based Vulnerability Detection
Md Abdul Hannan, Ronghao Ni, Chi Zhang +3 more
Large language models (LLMs) have demonstrated impressive capabilities across a wide range of coding tasks, including summarization, translation,...
SurveyMEDIUM
Prevalence of Security and Privacy Risk-Inducing Usage of AI-based Conversational Agents
Kathrin Grosse, Nico Ebert
Recent improvement gains in large language models (LLMs) have lead to everyday usage of AI-based Conversational Agents (CAs). At the same time, LLMs...
AttackMEDIUM
Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels
Chenghao Du, Quanfeng Huang, Tingxuan Tang +3 more
Large Language Models (LLMs) have transformed software development, enabling AI-powered applications known as LLM-based agents that promise to...
BenchmarkMEDIUM
Self-HarmLLM: Can Large Language Model Harm Itself?
Heehwan Kim, Sungjune Park, Daeseon Choi
Large Language Models (LLMs) are generally equipped with guardrails to block the generation of harmful responses. However, existing defenses always...
BenchmarkMEDIUM
Adapting Large Language Models to Emerging Cybersecurity using Retrieval Augmented Generation
Arnabh Borah, Md Tanvirul Alam, Nidhi Rastogi
Security applications are increasingly relying on large language models (LLMs) for cyber threat detection; however, their opaque reasoning often...
AttackHIGH
Consistency Training Helps Stop Sycophancy and Jailbreaks
Alex Irpan, Alexander Matt Turner, Mark Kurzeja +2 more
An LLM's factuality and refusal training can be compromised by simple changes to a prompt. Models often adopt user beliefs (sycophancy) or satisfy...
BenchmarkMEDIUM
Reasoning Up the Instruction Ladder for Controllable Language Models
Zishuo Zheng, Vidhisha Balachandran, Chan Young Park +2 more
As large language model (LLM) based systems take on high-stakes roles in real-world decision-making, they must reconcile competing instructions from...
ToolHIGH
LLM-based Multi-class Attack Analysis and Mitigation Framework in IoT/IIoT Networks
Seif Ikbarieh, Maanak Gupta, Elmahedi Mahalal
The Internet of Things has expanded rapidly, transforming communication and operations across industries but also increasing the attack surface and...
DefenseLOW
The Oversight Game: Learning to Cooperatively Balance an AI Agent's Safety and Autonomy
William Overman, Mohsen Bayati
As increasingly capable agents are deployed, a central safety challenge is how to retain meaningful human control without modifying the underlying...
BenchmarkLOW
Using Copilot Agent Mode to Automate Library Migration: A Quantitative Assessment
Aylton Almeida, Laerte Xavier, Marco Tulio Valente
Keeping software systems up to date is essential to avoid technical debt, security vulnerabilities, and the rigidity typical of legacy systems....
BenchmarkMEDIUM
Broken-Token: Filtering Obfuscated Prompts by Counting Characters-Per-Token
Shaked Zychlinski, Yuval Kainan
Large Language Models (LLMs) are susceptible to jailbreak attacks where malicious prompts are disguised using ciphers and character-level encodings...
Frequently asked questions
What is AI security research?
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
How many AI security papers does AI Threat Alert track?
AI Threat Alert indexes 4,319+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Where do the research papers come from?
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
What topics does the AI security research cover?
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
How is this different from a generic paper search?
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial