AI Threat Alert
Attack HIGH
Prompt Injection as Role Confusion
Charles Ye, Jasmine Cui, Dylan Hadfield-Menell
Language models remain vulnerable to prompt injection attacks despite extensive safety training. We trace this failure to role confusion: models...
AI Security Research
2,077+ academic papers on AI security, attacks, and defenses
Total
2,077
Attack
809
Benchmark
603
Defense
272
Tool
226
Survey
113
Showing 81–100 of 522 papers
Clear filters Attack HIGH
Dark and Bright Side of Participatory Red-Teaming with Targets of Stereotyping for Eliciting Harmful Behaviors from Large Language Models
Sieun Kim, Yeeun Jo, Sungmin Na +5 more
Red-teaming, where adversarial prompts are crafted to expose harmful behaviors and assess risks, offers a dynamic approach to surfacing underlying...
Attack HIGH
When Backdoors Go Beyond Triggers: Semantic Drift in Diffusion Models Under Encoder Attacks
Shenyang Chen, Liuwan Zhu
Standard evaluations of backdoor attacks on text-to-image (T2I) models primarily measure trigger activation and visual fidelity. We challenge this...
Attack HIGH
When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models
Zafir Shamsi, Nikhil Chekuru, Zachary Guzman +1 more
Large Language Models (LLMs) are increasingly integrated into high-stakes applications, making robust safety guarantees a central practical and...
Attack HIGH
TFL: Targeted Bit-Flip Attack on Large Language Model
Jingkai Guo, Chaitali Chakrabarti, Deliang Fan
Large language models (LLMs) are increasingly deployed in safety and security critical applications, raising concerns about their robustness to model...
Attack HIGH
Trojan Horses in Recruiting: A Red-Teaming Case Study on Indirect Prompt Injection in Standard vs. Reasoning Models
Manuel Wirth
As Large Language Models (LLMs) are increasingly integrated into automated decision-making pipelines, specifically within Human Resources (HR), the...
Attack HIGH
Automating Agent Hijacking via Structural Template Injection
Xinhao Deng, Jiaqing Wu, Miao Chen +3 more
Agent hijacking, highlighted by OWASP as a critical threat to the Large Language Model (LLM) ecosystem, enables adversaries to manipulate execution...
Attack HIGH
Sequential Membership Inference Attacks
Thomas Michel, Debabrota Basu, Emilie Kaufmann
Modern AI models are not static. They go through multiple updates in their lifecycles. Thus, exploiting the model dynamics to create stronger...
Attack HIGH
SRFed: Mitigating Poisoning Attacks in Privacy-Preserving Federated Learning with Heterogeneous Data
Yiwen Lu
Federated Learning (FL) enables collaborative model training without exposing clients' private data, and has been widely adopted in privacy-sensitive...
Attack HIGH
The Vulnerability of LLM Rankers to Prompt Injection Attacks
Yu Yin, Shuai Wang, Bevan Koopman +1 more
Large Language Models (LLMs) have emerged as powerful re-rankers. Recent research has however showed that simple prompt injections embedded within a...
Attack HIGH
Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections
Xianglin Yang, Yufei He, Shuo Ji +2 more
Self-evolving LLM agents update their internal state across sessions, often by writing and reusing long-term memory. This design improves performance...
Attack HIGH
ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models
Mitchell Piehl, Zhaohan Xi, Zuobin Xiong +2 more
Large language models (LLMs) are increasingly augmented with long-term memory systems to overcome finite context windows and enable persistent...
Attack HIGH
Boundary Point Jailbreaking of Black-Box LLMs
Xander Davies, Giorgi Giglemiani, Edmund Lau +3 more
Frontier LLMs are safeguarded against attempts to extract harmful information via adversarial prompts known as "jailbreaks". Recently, defenders have...
Attack HIGH
Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks
Lukas Struppek, Adam Gleave, Kellin Pelrine
As the capabilities of large language models continue to advance, so does their potential for misuse. While closed-source models typically rely on...
Attack HIGH
Multi-Turn Adaptive Prompting Attack on Large Vision-Language Models
In Chong Choi, Jiacheng Zhang, Feng Liu +1 more
Multi-turn jailbreak attacks are effective against text-only large language models (LLMs) by gradually introducing malicious content across turns....
Attack HIGH
SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement
Xiaojun Jia, Jie Liao, Simeng Qin +5 more
Agent skills are becoming a core abstraction in coding agents, packaging long-form instructions and auxiliary scripts to extend tool-augmented...
Attack HIGH
AlignSentinel: Alignment-Aware Detection of Prompt Injection Attacks
Yuqi Jia, Ruiqi Wang, Xilong Wang +2 more
Prompt injection attacks insert malicious instructions into an LLM's input to steer it toward an attacker-chosen task instead of the intended one....
Attack HIGH
Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges
Ruomeng Ding, Yifei Pang, He Sun +3 more
Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-language...
Attack HIGH
AISA: Awakening Intrinsic Safety Awareness in Large Language Models against Jailbreak Attacks
Weiming Song, Xuan Xie, Ruiping Yin
Large language models (LLMs) remain vulnerable to jailbreak prompts that elicit harmful or policy-violating outputs, while many existing defenses...
Attack HIGH
Backdoor Attacks on Contrastive Continual Learning for IoT Systems
Alfous Tim, Kuniyilh Simi D
The Internet of Things (IoT) systems increasingly depend on continual learning to adapt to non-stationary environments. These environments can...
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial