Red-teaming Activation Probes using Prompted LLMs
Phil Blandfort, Robert Graham
Activation probes are attractive monitors for AI systems due to low cost and latency, but their real-world robustness remains underexplored. We ask:...
AI Threat Alert indexes 3,397+ peer-reviewed and preprint papers on AI/ML security — covering adversarial attacks, model defenses, red-teaming benchmarks, surveys, and security tooling. Papers are sourced from arXiv, classified by type and by relevance to real-world threats, and cross-referenced with the CVEs and incidents they relate to.
Showing 1101–1120 of 1,314 papers
Clear filtersPhil Blandfort, Robert Graham
Activation probes are attractive monitors for AI systems due to low cost and latency, but their real-world robustness remains underexplored. We ask:...
Ruofan Liu, Yun Lin, Zhiyong Huang +1 more
Large language models (LLMs) are increasingly integrated into IT infrastructures, where they process user data according to predefined instructions....
Xin Yao, Haiyang Zhao, Yimin Chen +3 more
The Contrastive Language-Image Pretraining (CLIP) model has significantly advanced vision-language modeling by aligning image-text pairs from...
Kayua Oleques Paim, Rodrigo Brandao Mansilha, Diego Kreutz +2 more
The rapid proliferation of Large Language Models (LLMs) has raised significant concerns about their security against adversarial attacks. In this...
David Lüdke, Tom Wollschläger, Paul Ungermann +2 more
We introduce a novel framework that transforms the resource-intensive (adversarial) prompt optimization problem into an \emph{efficient, amortized...
Chenghao Du, Quanfeng Huang, Tingxuan Tang +3 more
Large Language Models (LLMs) have transformed software development, enabling AI-powered applications known as LLM-based agents that promise to...
Alex Irpan, Alexander Matt Turner, Mark Kurzeja +2 more
An LLM's factuality and refusal training can be compromised by simple changes to a prompt. Models often adopt user beliefs (sycophancy) or satisfy...
David Schmotz, Sahar Abdelnabi, Maksym Andriushchenko
Enabling continual learning in LLMs remains a key unresolved research challenge. In a recent announcement, a frontier LLM company made a step towards...
Haohua Duan, Liyao Xiang, Xin Zhang
Watermarking schemes for large language models (LLMs) have been proposed to identify the source of the generated text, mitigating the potential...
Lisha Shuai, Jiuling Dong, Nan Zhang +5 more
Local Differential Privacy (LDP) is a widely adopted privacy-protection model in the Internet of Things (IoT) due to its lightweight, decentralized,...
Guangzhi Su, Shuchang Huang, Yutong Ke +3 more
Multimodal large language models (MLLMs) have achieved impressive performance across diverse tasks by jointly reasoning over textual and visual...
Svetlana Churina, Niranjan Chebrolu, Kokil Jaidka
We show that continual pretraining on plausible misinformation can overwrite specific factual knowledge in large language models without degrading...
Zirui Cheng, Jikai Sun, Anjun Gao +4 more
Large language models (LLMs) have transformed natural language processing (NLP), enabling applications from content generation to decision support....
Elizabeth Lin, Jonah Ghebremichael, William Enck +5 more
Software supply chains, while providing immense economic and software development value, are only as strong as their weakest link. Over the past...
Sathwik Narkedimilli, N V Saran Kumar, Aswath Babu H +4 more
Current quantum machine learning approaches often face challenges balancing predictive accuracy, robustness, and interpretability. To address this,...
Viktoriia Zinkovich, Anton Antonov, Andrei Spiridonov +6 more
Multimodal large language models (MLLMs) have shown impressive capabilities in vision-language tasks such as reasoning segmentation, where models...
Ziyao Cui, Minxing Zhang, Jian Pei
Privacy concerns have become increasingly critical in modern AI and data science applications, where sensitive information is collected, analyzed,...
Yufan Liu, Wanqian Zhang, Huashan Chen +4 more
Despite rapid advancements in text-to-image (T2I) models, their safety mechanisms are vulnerable to adversarial prompts, which maliciously generate...
Yuchong Xie, Zesen Liu, Mingyu Luo +7 more
Modern coding agents integrated into IDEs orchestrate powerful tools and high-privilege system access, creating a high-stakes attack surface. Prior...
Myeongseob Ko, Nikhil Reddy Billa, Adam Nguyen +3 more
The memorization of training data in large language models (LLMs) poses significant privacy and copyright concerns. Existing data extraction methods,...
AI security research studies how AI and machine-learning systems can be attacked and defended — covering adversarial examples, prompt injection, model poisoning, training-data extraction, and the mitigations against them. AI Threat Alert curates this research from academic sources so security teams can track the threats behind emerging AI risks.
AI Threat Alert indexes 3,397+ papers on AI/ML security, classified across attack, defense, benchmark, survey, and tool categories and updated continuously.
Papers are sourced from arXiv, then classified by type and by relevance to real-world AI/ML threats, and cross-referenced with the CVEs and incidents they relate to.
Coverage spans adversarial attacks, model and system defenses, red-teaming benchmarks, literature surveys, and security tooling for LLMs, ML libraries, AI agents, and inference pipelines.
Every paper is filtered for AI security relevance and linked to the vulnerabilities, vendors, and incidents it relates to, so the research connects directly to operational threat intelligence.
Get breaking CVE alerts, compliance reports (ISO 42001, EU AI Act), and CISO risk assessments for your AI/ML stack.
Start 14-Day Free Trial