AI Threat AlertCVE-2026-85670: tokenizers: malformed vocab crashes process (DoS)
MEDIUMPoC AVAILABLECISA: TRACK*A malformed tokenizer.json file can crash any application built on Hugging Face's tokenizers library the moment it's loaded, before a single token is even encoded, because a scratch buffer sized to the longest vocabulary entry overflows when a merge rule produces a longer concatenated string. This matters because tokenizer configuration files are routinely pulled from Hugging Face Hub, third-party model repos, and other supply-chain sources with far less scrutiny than model weights, making the exposure effectively any LLM pipeline, inference server, or fine-tuning job that auto-downloads tokenizer assets. There is no public exploit or Nuclei scanner yet and it is not in CISA KEV, but exploitation requires only crafting a text file and getting it loaded — no encoding step, no privileges, and CVSS AC:L/PR:N keeps the bar low despite the UI:R requirement. A secondary usize underflow in the same code path can escalate from a clean panic to memory corruption in release builds when continuing_subword_prefix is configured, raising the ceiling above simple denial of service. Upgrade to the patched tokenizers release (fixed after 0.23.1, tracked in the linked GitHub advisory) on any host that loads tokenizer.json from external or user-supplied sources, and until patched, restrict tokenizer loading to vetted, pinned artifacts only.
What is the risk?
Medium severity (CVSS 6.5) but low barrier to trigger: the flaw fires at parse/load time via Tokenizer::from_file/from_str, requires no model inference and no special privileges, only that a victim application load an attacker-supplied tokenizer.json (UI:R). Impact is confined to availability (C:N/I:N/A:H) — the process aborts via Rust panic rather than being silently compromised, though the secondary usize underflow raises a theoretical memory-corruption path in release builds. No EPSS data, no CISA KEV listing, no public exploit or scanner template exist yet, so this is not under active exploitation; risk is driven by ubiquity of the affected library (tokenizers underpins most Hugging Face-based LLM stacks) rather than by attacker sophistication required.
How does the attack unfold?
What systems are affected?
| Package | Ecosystem | Vulnerable Range | Patched |
|---|---|---|---|
| Tokenizers | pip | — | No patch |
Do you use Tokenizers? You're affected.
How severe is it?
What is the attack surface?
What should I do?
1 stepUpgrade tokenizers past the affected 0.23.1 line to the patched release referenced in the GitHub advisory (huggingface/tokenizers, fix visible in the v0.23.2 branch of model.rs) as the primary fix. Until patched, avoid loading tokenizer.json files from untrusted sources — pin model/tokenizer versions from vetted Hugging Face Hub repos, validate or checksum tokenizer artifacts before deployment, and avoid dynamic tokenizer loading driven by user-controlled input. For defense in depth, run tokenizer-loading code paths (model servers, training workers) with process supervision/auto-restart so a panic-induced abort doesn't translate into extended downtime, and monitor for repeated process crashes/restarts correlated with new model or tokenizer deployments as a detection signal.
What does CISA's SSVC say?
Source: CISA Vulnrichment (SSVC v2.0). Decision based on the CISA Coordinator decision tree.
How is it classified?
Which compliance frameworks are affected?
This CVE is relevant to:
Frequently Asked Questions
What is CVE-2026-85670?
A malformed tokenizer.json file can crash any application built on Hugging Face's tokenizers library the moment it's loaded, before a single token is even encoded, because a scratch buffer sized to the longest vocabulary entry overflows when a merge rule produces a longer concatenated string. This matters because tokenizer configuration files are routinely pulled from Hugging Face Hub, third-party model repos, and other supply-chain sources with far less scrutiny than model weights, making the exposure effectively any LLM pipeline, inference server, or fine-tuning job that auto-downloads tokenizer assets. There is no public exploit or Nuclei scanner yet and it is not in CISA KEV, but exploitation requires only crafting a text file and getting it loaded — no encoding step, no privileges, and CVSS AC:L/PR:N keeps the bar low despite the UI:R requirement. A secondary usize underflow in the same code path can escalate from a clean panic to memory corruption in release builds when continuing_subword_prefix is configured, raising the ceiling above simple denial of service. Upgrade to the patched tokenizers release (fixed after 0.23.1, tracked in the linked GitHub advisory) on any host that loads tokenizer.json from external or user-supplied sources, and until patched, restrict tokenizer loading to vetted, pinned artifacts only.
Is CVE-2026-85670 actively exploited?
Proof-of-concept exploit code is publicly available for CVE-2026-85670, increasing the risk of exploitation.
How to fix CVE-2026-85670?
Upgrade tokenizers past the affected 0.23.1 line to the patched release referenced in the GitHub advisory (huggingface/tokenizers, fix visible in the v0.23.2 branch of model.rs) as the primary fix. Until patched, avoid loading tokenizer.json files from untrusted sources — pin model/tokenizer versions from vetted Hugging Face Hub repos, validate or checksum tokenizer artifacts before deployment, and avoid dynamic tokenizer loading driven by user-controlled input. For defense in depth, run tokenizer-loading code paths (model servers, training workers) with process supervision/auto-restart so a panic-induced abort doesn't translate into extended downtime, and monitor for repeated process crashes/restarts correlated with new model or tokenizer deployments as a detection signal.
What systems are affected by CVE-2026-85670?
This vulnerability affects the following AI/ML architecture patterns: model serving, training pipelines, agent frameworks, RAG pipelines.
What is the CVSS score for CVE-2026-85670?
CVE-2026-85670 has a CVSS v3.1 base score of 6.5 (MEDIUM). The EPSS exploitation probability is 0.47%.
What is the AI security impact?
Affected AI Architectures
MITRE ATLAS Techniques
AML.T0010.001AI SoftwareAML.T0011.000Unsafe AI ArtifactsAML.T0029Denial of AI ServiceCompliance Controls Affected
What are the technical details?
Original Advisory
tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.
Exploitation Scenario
An adversary publishes a poisoned model or standalone tokenizer artifact to a public repository (or compromises an existing one) with a tokenizer.json containing a merge rule whose concatenated token is longer than any entry in the vocabulary. A victim's inference service, fine-tuning pipeline, or developer tooling pulls this artifact — via an automated model-refresh job, a dependency on a community tokenizer, or a user pointing the pipeline at an external repo — and calls Tokenizer::from_file/from_str to load it. BpeBuilder::build sizes its scratch buffer to the longest known vocabulary key, then overruns it while writing the oversized merge, triggering a Rust panic that aborts the host process (or, if continuing_subword_prefix is set and a merge token is shorter than the prefix, a usize underflow that could corrupt memory in release builds). The result is an immediate crash of the serving process or training job, repeatable on demand simply by redistributing the same file, producing a low-effort, high-repeatability denial-of-service against any pipeline that trusts external tokenizer configs.
Weaknesses (CWE)
CWE-787 — Out-of-bounds Write: The product writes data past the end, or before the beginning, of the intended buffer.
- [Requirements] Use a language that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid. For example, many languages that perform their own memory management, such as Java and Perl, are not subject to buffer overflows. Other languages, such as Ada and C#, typically provide overflow protection, but the protection can be disabled by the programmer. Be wary that a language's interface to native code may still be subject to overflows, even if the language itself is theoretically safe.
- [Architecture and Design] Use a vetted library or framework that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid. Examples include the Safe C String Library (SafeStr) by Messier and Viega [REF-57], and the Strsafe.h library from Microsoft [REF-56]. These libraries provide safer versions of overflow-prone string-handling functions.
Source: MITRE CWE corpus.
CVSS Vector
CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:HReferences
Timeline
Related Vulnerabilities
CVE-2026-416807.5marked: infinite recursion DoS crashes Node.js via OOM
Same package: tokenizersCVE-2026-122595.3NLTK: unverified downloads allow malicious data swap
Same package: tokenizersGHSA-j95f-988m-3j2ftiptap: ReDoS in Markdown attribute parsers
Same package: tokenizersCVE-2025-5952810.0Flowise: Unauthenticated RCE via MCP config injection
Same attack type: Supply ChainCVE-2025-512010.0smolagents: sandbox escape enables unauthenticated RCE
Same attack type: Supply Chain