18 regex patterns and 22 active probes: how we find leaked secrets

In January 2025, a researcher found valid AWS keys in the client-side JavaScript bundle of a publicly traded company's web application. The keys had been there for fourteen months. They granted full S3 access to a bucket containing 2.3 million customer records. The company had passed two penetration tests during that period.

This is the problem with secrets: they don't look like vulnerabilities. There's no broken authentication flow, no injection point, no malformed input. There's just a string that shouldn't be there, sitting in plain text, waiting for someone to notice. BrokenApp's exposure scanner is designed to notice.

Why secrets leak

Secrets end up in production through three primary vectors, and understanding them matters because each requires a different detection strategy.

Passive scanning: the 18 regex patterns

The first phase of the exposure scanner is passive. BrokenApp downloads every JavaScript bundle, CSS file, HTML page, and JSON response the application serves, then runs 18 regex patterns against the content. Each pattern targets a specific secret format with high specificity — we match the known structure of each provider's key format, not just "long random string."

The first 17 patterns are provider-specific and have near-zero false positive rates because cloud providers use deterministic key prefixes. Pattern 18 — the generic high-entropy matcher — is intentionally broader and carries a higher false-positive risk. It exists to catch secrets from providers we don't have a specific pattern for. Findings from pattern 18 are flagged as "needs review" rather than confirmed.

Active scanning: the 22 path probes

Passive scanning only finds secrets that appear in content the application already serves. But many secrets hide behind known paths that aren't linked from the application itself. BrokenApp's active scanner makes 22 targeted HTTP requests to paths where secrets commonly live.

Each probe is scored by both the HTTP status code and a content-type analysis. A 200 response to /.env that returns text/plain with lines matching KEY=VALUE format is a confirmed exposure. A 200 that returns an HTML error page is a false positive — many servers return 200 with a custom "not found" page rather than a proper 404.

How secrets are masked in reports

Here's a design decision we made early: BrokenApp never stores the full secret. When a pattern match fires, the scanner captures enough to confirm the finding — the prefix, the length, and a 4-character suffix — then discards the rest. The scan report shows:

This matters for two reasons. First, if the scan report itself leaks (shared in a Slack channel, committed to a repo, attached to a Jira ticket), the full secret isn't compromised. Second, it keeps BrokenApp out of the chain of custody for secrets — we never have them, so there's no risk of us being a secondary exposure vector.

Real-world examples

In our first three months of beta testing, the exposure scanner found secrets in 23% of applications scanned. The most common findings, in order:

• Stripe publishable keys in client bundles (41% of findings) — technically not secret, but indicate the developer may also be leaking secret keys through the same pattern
• Firebase server keys in client JavaScript (18%) — these grant push notification access and should never be client-side
• AWS access keys compiled into static assets (14%) — typically from NEXT_PUBLIC_ or REACT_APP_ env vars that should have been server-only
• Exposed .env files via path probe (11%) — usually on staging environments with less restrictive server configurations
• Spring Boot Actuator /actuator/env returning full environment (8%) — the default configuration exposes everything unless explicitly locked down
• Database connection strings in debug responses (8%) — Laravel Telescope and Django debug toolbar were the most common sources

The median time between secret deployment and discovery was 47 days. The longest-lived secret we found had been exposed for 11 months. In every case, the development team was unaware of the exposure until the scan flagged it.

False positive handling

False positives destroy trust in any scanning tool. If developers learn to ignore findings, real secrets get ignored too. BrokenApp uses three layers to minimize false positives:

Across our beta dataset, the provider-specific patterns (1-17) had a combined false positive rate of 2.1%. The generic pattern had a false positive rate of 18%, which is why it's reported separately as "needs review." We're actively tuning the entropy threshold and context heuristics to bring this down, but we'd rather show a few false positives than miss a real secret.

The allowlist file lets you permanently suppress specific findings — for example, if your application intentionally exposes a Stripe publishable key (which is designed to be public). Each allowlist entry requires a reason field, so there's an audit trail for why a finding was dismissed.