Implementation guide

PII Redaction

PII redaction is how you stop personally identifiable information from reaching models, logs, vector stores, and citations. This guide explains what it is, where it belongs in AI pipelines, and how to implement it locally in Node.js before prompts ever leave your process.

Getting started docs•Redact before OpenAI•Try the playground

What is PII redaction?

PII redaction is the practice of detecting and masking information that can directly or indirectly identify an individual before that data is stored, processed, or exposed downstream. In AI systems, that includes prompts, model outputs, logs, traces, embeddings, analytics, and any citation that points back to user data.

Unlike a one-off filter, effective PII redaction is a control plane: the same entity taxonomy and masking policy applied at every trust boundary so a single prompt cannot fan out into vendor APIs, RAG indexes, and observability tools unchanged.

PII broadly covers two categories:

  • Direct identifiers: email addresses, phone numbers, names, national IDs, account numbers
  • Indirect or quasi-identifiers: dates of birth, postcodes, job titles, and combinations of attributes that re-identify a person when linked

What matters most is not only what counts as PII, but where it can appear. For detection primitives and pattern coverage, see the PII detection overview.

Why PII redaction matters

AI systems amplify traditional privacy risks because they copy, transform, and log data across multiple layers. Without redaction, a single prompt can end up in model vendor logs, vector stores, observability platforms, and audit trails.

Common injection points

  • Inbound streams: user prompts, uploads, and pasted exports (CSV, DOCX, CRM snapshots)
  • Processing layers: system logs, traces, APM instrumentation, and replay tools
  • Storage: vector databases, embeddings, RAG indexes, and training sets
  • Outbound channels: model responses that echo prompts, retrieved snippets, or internal context
  • Regulatory compliance: GDPR, CCPA, HIPAA, and sector rules expect proactive controls on personal data (see GDPR redaction and HIPAA redaction)
  • Security posture: masking high-value identifiers reduces the blast radius of any breach
  • Trust and adoption: users and customers adopt AI features faster when privacy controls are concrete and inspectable

Core PII entities and redaction targets

Start with clear entity definitions and a consistent masking strategy per entity. In AI pipelines, the same entities show up in prompts, tool outputs, RAG documents, citations, logs, traces, and embeddings.

Contact information

Emails, phone numbers, postal addresses, postcodes

Identity numbers

National IDs, SSNs, passport numbers, driver’s licenses, NHS/NINO-style identifiers

Financial data

Credit cards, IBANs, account numbers, routing / sort codes

Health identifiers

Patient IDs, clinical record IDs, insurance member IDs (HIPAA-bound systems)

Online / device identifiers

IP addresses, device IDs, cookies where regulations treat them as personal data

Free-text personal details

Names, employers, locations, and attribute combinations that can re-identify someone

Ask for your use case: which of these entity types is most likely to appear in logs or citations? Start there, then expand coverage once recall and false-positive rates are measured.

Redaction vs anonymization vs tokenization

Different privacy techniques serve different goals. Mixing them up leads to brittle designs.

TechniqueWhat it doesBest for
RedactionRemove or mask the sensitive value (e.g. [EMAIL_9619])Prompts, logs, model traffic when structure/context remains useful
AnonymizationTransform so individuals are no longer identifiable, even when linkedLong-term analytics, public datasets, aggregated reporting
TokenizationReplace values with reversible tokens plus a secure mapInternal workflows that still need authorised re-identification

For AI citations, redaction and tokenization matter most: citations should not leak real identifiers, but privileged services may still need to resolve a placeholder back to a record.

Pattern-first vs machine learning detection

There are two dominant paradigms for detecting sensitive text: pattern-first (regex-based) and ML/NLP-based. Production systems usually combine both—starting with the layer you can deploy everywhere without shipping raw text to another processor.

Pattern-first (regex / rule-based)

Regex-driven detectors catch structured identifiers—emails, phone numbers, credit cards, postal codes, and national IDs— with deterministic precision. They are fast, local, and auditable, which makes them the right first hop before content reaches an LLM API.

ML / named entity recognition (NER)

NER expands coverage to unstructured narrative—names, organisations, and contextual references that rigid patterns miss. Trade-offs are latency, cost, and data residency if inference runs outside your boundary.

Optimal architecture: run high-precision pattern redaction locally, optionally apply NER inside a private VPC, then merge spans in a single pass. OpenRedaction focuses on the deterministic layer you can ship in every Node process; optional local NER sits behind the same boundary when you need free-text name coverage.

Defense-in-depth: a multi-layer strategy

Modern AI systems need PII controls at multiple layers, not a single filter at the edge.

  • Input layer: redact or tokenise before data reaches the model or vector store
  • Logging layer: scrub structured logs, traces, and metrics (including OpenTelemetry attributes)
  • Storage and analytics: anonymize or aggregate before long-term retention
  • Output and citations: ensure responses, explanations, and provenance never re-emit raw identifiers

Keep a shared entity taxonomy and redaction policy across layers so one hop does not undo another.

How to implement PII redaction with OpenRedaction

OpenRedaction is a regex-first, open-source library that detects and redacts PII locally in Node.js. Use it as the deterministic control plane before LLM calls, RAG indexing, and log export. The steps below mirror the pattern used in production AI agent stacks: install a specialized redactor, wrap sensitive payloads, then keep observability tools from storing the originals.

Step 1: Install the library

npm install openredaction

Step 2: Detect and redact a string

detect() is async and returns the redacted text plus a redactionMap for authorised restore flows.

import { OpenRedaction } from "openredaction";

const redactor = new OpenRedaction({
  preset: "gdpr",
  redactionMode: "placeholder",
  deterministic: true,
});

const input =
  "Hi, I'm Jane Smith. Email jane@acme.com or call 07700900123.";

const result = await redactor.detect(input);

console.log(result.redacted);
// e.g. "Hi, I'm [NAME_...]. Email [EMAIL_...] or call [PHONE_UK_MOBILE_...]."

console.log(result.detections.map((d) => d.type));
// ["NAME", "EMAIL", "PHONE_UK_MOBILE"]

Step 3: Redact before an LLM API call

Never pass raw user text to a third-party model when PII is possible. Full walkthrough: redact PII before OpenAI.

import OpenAI from "openai";
import { OpenRedaction } from "openredaction";

const client = new OpenAI();
const redactor = new OpenRedaction({ redactionMode: "placeholder" });

async function safeCompletion(userPrompt: string) {
  const { redacted } = await redactor.detect(userPrompt);

  return client.chat.completions.create({
    model: "gpt-4.1-mini",
    messages: [{ role: "user", content: redacted }],
  });
}

Step 4: Scrub request bodies with Express middleware

Put redaction at the gateway so every route inherits the same policy. See also the Node.js redaction guide.

import express from "express";
import { openredactionMiddleware } from "@openredaction/express";

const app = express();
app.use(express.json());

app.use(
  openredactionMiddleware({
    autoRedact: true,
    fields: ["prompt", "message", "content"],
  }),
);

app.post("/chat", (req, res) => {
  // req.body fields are already redacted
  res.json({ ok: true, body: req.body, pii: req.pii });
});

Step 5: Keep session-scoped maps for citations (optional)

Stable placeholders keep RAG citations readable. Store redactionMap only in an authorised, short-lived context—never in the vector store or public citation payload.

const redactor = new OpenRedaction({
  redactionMode: "placeholder",
  deterministic: true,
});

async function prepareChunkForIndex(chunk: string) {
  const { redacted, redactionMap } = await redactor.detect(chunk);

  // Persist only redacted text in your vector DB / citation index
  await indexDocument({ text: redacted });

  // Keep the map in a privileged session store if restore is required
  await sessionStore.set(sessionId, redactionMap);

  return redacted;
}

// Later, privileged internal tooling only:
const map = await sessionStore.get(sessionId);
const restored = redactor.restore(redactedCitation, map);

Before / after

Before
Contact jane@acme.com about order #4412, SSN 078-05-1120
After
Contact [EMAIL_…] about order #4412, SSN [SSN_…]

Where to redact in AI pipelines

Placement is as important as detection quality. Treat every hop where text leaves a trust boundary as a redaction checkpoint.

  • Before inference (gateway): scrub inputs and tool results before third-party LLMs
  • Application / RAG layer: redact early in chunking so embeddings never store raw identifiers
  • Observability layer: use collectors and span processors to scrub attributes before export
  • Response path: scan generated replies before storage or display—models can echo user inputs or retrieved snippets
  • Storage and analytics: tokenize or aggregate before dashboards and training sets

In practice, many teams combine a model gateway with collector-level redaction for logs and traces.

Redaction styles and consistency

How you represent scrubbed values matters as much as catching them. Pick a style, document it, and apply it globally—auditors prefer a stable schema over clever one-offs.

  • Full placeholders (e.g. [EMAIL_9619]) for external model traffic and vendor APIs
  • Partial masking (e.g. jo***@domain.com) only for internal dashboards or controlled analytics
  • Token replace when a privileged service must restore the original later

OpenRedaction supports placeholder, mask-middle, mask-all, format-preserving, and token-replace modes so you can use irreversible placeholders for vendors and softer masking for internal tools.

Redaction in logs, traces, and telemetry

Most compliance findings involving AI stacks come from logs and observability data—not just primary databases.

  • Prefer field-level redaction in structured logs before serialisation
  • Scrub span attributes and events that may contain emails, tokens, or prompt bodies
  • Ensure debug logs and stack traces never dump raw prompts
  • Separate operational logs from compliance audit logs—neither should retain raw identifiers by default

You can also apply collector-level masking (useful when many services need a central policy). Example OpenTelemetry Collector snippet:

processors:
  attributes/pii:
    actions:
      - key: user.email
        action: delete
      - key: http.url
        regex: '(\?|&)(token|password)=([^&]+)'
        action: update
        value: '[REDACTED]'

service:
  pipelines:
    traces:
      processors: [attributes/pii]

Collector rules catch known attribute keys. Application-layer OpenRedaction still matters for free-text prompts and message bodies those keys will not cover.

Designing redaction for AI citations

AI citations refer model outputs back to source documents, prompts, or intermediate artifacts. If those sources contain PII, citations become a leak point even when the answer looks safe.

  • Stable placeholders: replace PII with consistent tokens ([PERSON_1], [EMAIL_1]) so citations remain meaningful
  • Session-scoped redaction maps: only privileged services can de-redact
  • Redacted metadata: titles, IDs, and document labels must not encode PII
  • Constraint-based outputs: prevent the model from reconstructing raw identifiers from context

Policies, classification, and governance

Effective PII redaction is driven by policy, not only tooling.

  • Classify data tiers (public, internal, sensitive, regulated)
  • Map entity types to regulations (for example, SSN → GDPR / HIPAA expectations)
  • Define who can see de-tokenised data versus fully masked outputs
  • Treat redaction configuration as code—versioned, reviewed, and approved

OpenRedaction presets (gdpr, hipaa, ccpa, finance, education, healthcare) give teams a concrete starting policy they can inspect and override.

Evaluation: precision, recall, and quality

Privacy assurance is not theoretical—it needs continuous, automated proof. Treat redaction checks as part of CI/CD, not a post-incident cleanup.

  • Maintain fixture sets with known PII and assert nothing sensitive reaches models or logs
  • Inject canary values (e.g. a unique test email) into prompts and verify they never appear in logs, embeddings, or LLM responses
  • Prioritise recall for high-risk entities even when that means some over-redaction
  • Track latency budgets (for example under 50ms per prompt)—if redaction slows the path, teams will bypass it under pressure
  • Periodically scan vector stores and citation indexes for regressions

No automated system catches everything. Keep review queues for low-confidence matches, especially in healthcare, legal, and HR workflows—see also PII in support tickets.

Where OpenRedaction fits in

OpenRedaction provides an open-source foundation for detecting and redacting a broad set of PII entities using extensive regex libraries, validators, and configurable masking modes. Pair it with optional NER, structured logging, and clear policies for end-to-end coverage.

  • Gateways in front of LLMs
  • Middleware scrubbing for APIs and support tools
  • Pre-processing documents and prompts before RAG or search indexes
  • Local, auditable redaction for compliance reviews

Compare approaches on open source AI redaction tools, or read how the library was built in Building OpenRedaction.

FAQ

Related guides

Try the playgroundInstall with npm