Secrets in the Machine: Preventing Sensitive Data Leaks Through LLM APIs

In this webinar, we break down a simple but increasingly common problem: secrets leak wherever text flows, and modern LLM apps and agentic workflows are built to move text fast.
We walk through concrete demos showing how API keys and passwords can surface through RAG-based assistants when secrets accidentally live in knowledge bases (tickets, docs, internal wikis). We also show why “just harden the system prompt” isn’t a reliable fix, and how output-only redaction can be bypassed (for example by simple formatting/encoding tricks).
Most importantly, we explore real-world agent architectures: chat + tools + logging/tracing + external services. Even if your chatbot output is guarded, secrets can still be exfiltrated out-of-band through connected tools (e.g., creating a GitHub issue) or silently propagated into traces, datasets, and third-party platforms.
The takeaway is practical: to reduce risk, aim to make your agent a “no-secret zone” by filtering/redacting secrets before they ever reach the model, and combine that with a shift-left mindset. Scan and clean the sources (docs, tickets, repos) where secrets shouldn’t exist in the first place.