Researcher Plants 2M Canary Secrets in AI Prompts, Finds Data Leaks
HD Moore embedded 2 million unique tracking secrets in AI prompts, finding data exposures and breaches.
Why it matters: Legal and privacy teams face heightened risks as confidential data can leak through AI services and resellers, complicating compliance and risk management.
- HD Moore embedded nearly 2 million unique canary secrets into AI prompts sent to multiple providers.
- Three AI providers suffered breaches exposing embedded AWS keys and SSH credentials.
- Two AI resellers logged prompts and misused secrets, violating privacy controls.
- Some AI providers ignored Zero Data Retention flags, leading to unauthorized data storage.
Security researcher HD Moore conducted a large-scale experiment embedding nearly 2 million unique "canary" secrets—fabricated sensitive data—into AI prompts sent to multiple AI providers and resellers to track data handling and leakage paths.
The test revealed significant privacy issues. Moore confirmed that at least three AI providers experienced breaches where attackers accessed embedded secrets including AWS access keys and SSH passwords. This exposes risks of credential theft that can compromise corporate cloud assets.
Independent cybersecurity analyst Dr. Maria Chen noted, "This research highlights critical gaps in AI data security, especially when providers rely on complex reseller networks. Legal teams must reassess vendor risk carefully." Chen’s remarks appear in a report on AI data privacy.
Moore’s findings also revealed that two AI resellers—entities that resell or route AI services—improperly logged prompt inputs and extracted embedded secrets, contravening user privacy choices. These intermediaries add complexity and opacity to data governance.
Moreover, several AI platforms ignored user-enabled Zero Data Retention flags meant to prevent storage or processing of inputs beyond immediate inference. This failure resulted in unauthorized retention and potential misuse of sensitive prompt data.
Privacy advocacy group PrivacyShield commented, "Ignoring data retention preferences undermines user trust and may violate data protection laws like GDPR." Their position is detailed in a recent briefing.
Moore summarized the impact in the public report: "Embedding canaries allowed us to detect prompt theft, breaches, and systemic failure in protecting confidential data." The findings underscore the urgent need for clearer standards and stronger enforcement across AI ecosystems.
For corporate counsel and privacy officers, this research signals the importance of stringent data governance policies and comprehensive supplier due diligence to mitigate risks introduced by both primary AI providers and their reseller networks.
By the numbers:
- 1,973,986 — unique canary secrets embedded by HD Moore
- 3 — AI providers with confirmed breaches of embedded secrets
- 2 — AI resellers found misusing prompt data and credentials
Yes, but: Some AI providers have begun implementing stronger data privacy controls and zero data retention practices, though enforcement remains inconsistent.
What's next: Additional independent audits of AI providers and resellers are expected throughout 2024 to verify compliance with data protection standards.