When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shift
Max Fomin
Detecting prompt injection and jailbreak attacks is critical for deploying LLM-based agents safely. As agents increasingly process untrusted data...
AI Threat Alert indexes 3,023+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
Showing 381–400 of 562 papers
Clear filtersMax Fomin
Detecting prompt injection and jailbreak attacks is critical for deploying LLM-based agents safely. As agents increasingly process untrusted data...
Edibe Yilmaz, Kahraman Kostas
The integration of large language models (LLMs) into educational processes introduces significant constraints regarding data privacy and reliability,...
Haoyu Li, Xijia Che, Yanhao Wang +2 more
Proof-of-Vulnerability (PoV) generation is a critical task in software security, serving as a cornerstone for vulnerability validation, false...
Mohamed Shaaban, Mohamed Elmahallawy
Federated learning (FL) enables collaborative training across organizational silos without sharing raw data, making it attractive for...
Anudeep Das, Prach Chantasantitam, Gurjot Singh +3 more
Large language models (LLMs) are increasingly deployed in settings where inducing a bias toward a certain topic can have significant consequences,...
Xu Li, Simon Yu, Minzhou Pan +5 more
LLM-based agents are becoming increasingly capable, yet their safety lags behind. This creates a gap between what agents can do and should do. This...
Tailia Malloy, Tegawende F. Bissyande
Large Language Models are expanding beyond being a tool humans use and into independent agents that can observe an environment, reason about...
Nataša Krčo, Zexi Yao, Matthieu Meeus +1 more
Data containing personal information is increasingly used to train, fine-tune, or query Large Language Models (LLMs). Text is typically scrubbed of...
Rosie Zhao, Anshul Shah, Xiaoyu Zhu +5 more
Reinforcement learning (RL) fine-tuning has become a key technique for enhancing large language models (LLMs) on reasoning-intensive tasks,...
André Storhaug, Jiamou Sun, Jingyue Li
Identifying vulnerability-fixing commits corresponding to disclosed CVEs is essential for secure software maintenance but remains challenging at...
Faouzi El Yagoubi, Ranwa Al Mallah, Godwin Badu-Marfo
Multi-agent Large Language Model (LLM) systems create privacy risks that current benchmarks cannot measure. When agents coordinate on tasks,...
Aashish Kolluri, Rishi Sharma, Manuel Costa +5 more
Indirect prompt injection attacks threaten AI agents that execute consequential actions, motivating deterministic system-level defenses. Such...
Yang Liu, Armstrong Foundjem, Xingfang Wu +2 more
Context: In the fast-paced evolution of software development, Large Language Models (LLMs) have become indispensable tools for tasks such as code...
Arpit Singh Gautam, Kailash Talreja, Saurabh Jha
Large Language Models (LLMs) frequently hallucinate plausible but incorrect assertions, a vulnerability often missed by uncertainty metrics when...
Zhenhua Zou, Sheng Guo, Qiuyang Zhan +6 more
The evolution of Large Language Models (LLMs) has shifted mobile computing from App-centric interactions to system-level autonomous agents. Current...
Xinguo Feng, Zhongkui Ma, Zihan Wang +2 more
Training and fine-tuning large-scale language models largely benefit from collaborative learning, but the approach has been proven vulnerable to...
Matteo Migliarini, Berat Ercevik, Oluwagbemike Olowe +5 more
Large Language Models (LLMs) are increasingly deployed as active participants on public social media platforms, yet their behavior in these...
Yuxin Cao, Wei Song, Shangzhi Xu +2 more
Video Large Language Models (VideoLLMs) have recently achieved strong performance in video understanding tasks. However, we identify a previously...
Mohan Rajagopalan, Vinay Rao
Large Language Model (LLM) applications are vulnerable to prompt injection and context manipulation attacks that traditional security models cannot...
Yilin Yang, Zhenghui Guo, Yuke Wang +3 more
Large Vision-Language Models (VLMs) have achieved remarkable success across diverse multimodal tasks but remain vulnerable to hallucinations rooted...
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
AI Threat Alert indexes 3,023+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial