AI Threat AlertOtherLOW
Natural Language based Specification and Verification
Zhaorui Li, Chengyu Song
Recent frontier large language models (LLMs) have shown strong performance in identifying security vulnerabilities in large, mature open-source...
AI Security Research
AI Threat Alert indexes 4,319+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
- Adversarial attacks
- Model defenses
- Red-teaming benchmarks
- Surveys
- Security tooling
- Total4,319
- Attack1,676
- Benchmark1,229
- Defense610
- Tool432
- Survey252
Showing 1781–1800 of 4,319 papers
AttackHIGH
Context-Aware Spear Phishing: Generative AI-Enabled Attacks Against Individuals via Public Social Media Data
Elham Pourabbas Vafa, Sayak Saha Roy, Shirin Nilizadeh
We demonstrate how publicly available social-media data and generative AI (GenAI) can be misused to automate and scale highly personalized,...
DefenseMEDIUM
Leveraging RAG for Training-Free Alignment of LLMs
John T. Halloran
Large language model (LLM) alignment algorithms typically consist of post-training over preference pairs. While such algorithms are widely used to...
AttackHIGH
Adversarial SQL Injection Generation with LLM-Based Architectures
Ali Karakoc, H. Birkan Yilmaz
SQL injection (SQLi) attacks are still one of the serious attacks ranked in the Open Worldwide Application Security Project (OWASP) Top 10 threats....
BenchmarkLOW
Benchmarking LLM-Based Static Analysis for Secure Smart Contract Development: Reliability, Limitations, and Potential Hybrid Solutions
Stefan-Claudiu Susan, Andrei Arusoaie, Dorel Lucanu
The irreversible nature of blockchain transactions makes the identification of smart contract vulnerabilities an essential requirement for secure...
DefenseMEDIUM
Spurious Correlation Learning in Preference Optimization: Mechanisms, Consequences, and Mitigation via Tie Training
Christian Moya, Alex Semendinger, Guang Lin +1 more
Preference learning methods such as Direct Preference Optimization (DPO) are known to induce reliance on spurious correlations, leading to sycophancy...
DefenseMEDIUM
FedSurrogate: Backdoor Defense in Federated Learning via Layer Criticality and Surrogate Replacement
Fatima Z. Abacha, Sin G. Teo, Yuanxiang Wu +2 more
Federated Learning remains highly susceptible to backdoor attacks--malicious clients inject targeted behaviours into the global model. Existing...
AttackMEDIUM
Birds of a Feather Flock Together: Background-Invariant Representations via Linear Structure in VLMs
Youssef Zaazou, Mark Thomas
Vision-language models (VLMs), such as CLIP and SigLIP 2, are widely used for image classification, yet their vision encoders remain vulnerable to...
DefenseHIGH
Beyond Red-Teaming: Formal Guarantees of LLM Guardrail Classifiers
Nikita Kezins, Urbas Ekka, Pascal Berrang +1 more
Guardrail Classifiers defend production language models against harmful behavior, but although results seem promising in testing, they provide no...
ToolMEDIUM
RUBEN: Rule-Based Explanations for Retrieval-Augmented LLM Systems
Joel Rorseth, Parke Godfrey, Lukasz Golab +2 more
This paper demonstrates RUBEN, an interactive tool for discovering minimal rules to explain the outputs of retrieval-augmented large language models...
BenchmarkMEDIUM
From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World
Pedro Conde, Henrique Branquinho, Valerio Mazzone +3 more
AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will...
BenchmarkMEDIUM
Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights
Saba Pourhanifeh, AbdulAziz AbdulGhaffar, Ashraf Matrawy
Large Language Models(LLMs) are increasingly explored for cybersecurity applications such as vulnerability detection. In the domain of threat...
SurveyMEDIUM
LLMs for Secure Hardware Design and Related Problems: Opportunities and Challenges
Johann Knechtel, Ozgur Sinanoglu, Ramesh Karri
The integration of Large Language Models (LLMs) into Electronic Design Automation (EDA) and hardware security is rapidly reshaping the semiconductor...
BenchmarkHIGH
LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments
Chiyu Zhang, Huiqin Yang, Bendong Jiang +8 more
The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content...
AttackHIGH
Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximization
Mengqi He, Xinyu Tian, Xin Shen +6 more
Recent studies show that gradient-based universal image jailbreaks on vision-language models (VLMs) exhibit little or no cross-model transferability,...
ToolHIGH
MATRA: Modeling the Attack Surface of Agentic AI Systems -- OpenClaw Case Study
Tim Van hamme, Thomas Vissers, Javier Carnerero-Cano +4 more
LLMs are increasingly deployed as autonomous agents with access to tools, databases, and external services, yet practitioners (across different...
DefenseLOW
Conformity Generates Collective Misalignment in AI Agents Societies
Giordano De Marzo, Alessandro Bellina, Claudio Castellano +2 more
Artificial intelligence safety research focuses on aligning individual language models with human values, yet deployed AI systems increasingly...
BenchmarkLOW
The Bystander Effect in Multi-Agent Reasoning: Quantifying Cognitive Loafing in Collaborative Interactions
Dahlia Shehata, Ming Li
Multi-agent systems (MAS) assume that collaborating inherently improves Large Language Model (LLM) reasoning. We challenge this by demonstrating that...
ToolHIGH
MCPShield: Content-Aware Attack Detection for LLM Agent Tool-Call Traffic
Sultan Zavrak
The Model Context Protocol (MCP) has become a widely adopted interface for LLM agents to invoke external tools, yet learned monitoring of MCP...
DefenseMEDIUM
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs
Krishak Aneja, Manas Mittal, Anmol Goel +2 more
Fine-tuning Large Language Models (LLMs) on benign narrow data can sometimes induce broad harmful behaviors, a vulnerability termed emergent...
Frequently asked questions
What is AI security research?
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
How many AI security papers does AI Threat Alert track?
AI Threat Alert indexes 4,319+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Where do the research papers come from?
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
What topics does the AI security research cover?
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
How is this different from a generic paper search?
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Track AI security vulnerabilities in real time
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial