Files
presidio/docs/recipes/german-language-support/README.md
Michael van den Berg 98f79b9c64 feat(analyzer): add German PII recognizers (DE_*) (#1909)
* feat(analyzer): add German PII recognizers (DE_*)

Adds 9 new predefined recognizers for German personally identifiable
information, all disabled by default (enabled: false) and scoped to
supported_language: de.

Recognizers with checksum validation:
- DE_TAX_ID: Steueridentifikationsnummer (§§ 139a-e AO)
  ISO 7064 Mod 11,10 checksum (Bundeszentralamt für Steuern)
- DE_SOCIAL_SECURITY: Rentenversicherungsnummer / RVNR (§ 147 SGB VI)
  Deutsche Rentenversicherung Bund checksum algorithm
- DE_HEALTH_INSURANCE: Krankenversicherungsnummer KVNR (§ 290 SGB V)
  GKV-Spitzenverband checksum; DSGVO Art. 9 (health data)

Pattern-based recognizers:
- DE_TAX_NUMBER: Steuernummer (§ 139a AO) – ELSTER 13-digit and
  state-specific slash-separated formats
- DE_PASSPORT: Reisepassnummer (PassG § 4, ICAO Doc 9303)
- DE_ID_CARD: Personalausweisnummer (PAuswG) – nPA and legacy format
- DE_KFZ: KFZ-Kennzeichen (FZV § 8, ECJ C-582/14)
- DE_HANDELSREGISTER: Handelsregisternummer HRA/HRB (§§ 9, 14 HGB)
- DE_PLZ: Postleitzahl (DSGVO Art. 4 Nr. 1) – very low base confidence
  (0.05), context words required for actionable results

Also updates:
- predefined_recognizers/__init__.py: imports and __all__
- conf/default_recognizers.yaml: all 9 recognizers registered
- docs/supported_entities.md: new Germany section
- CHANGELOG.md: entry under [unreleased]

133 new tests pass (9 test files, one per recognizer).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: enforce LF line endings for shell scripts in containers

Added .gitattributes to force LF for *.sh files.
CRLF endings in entrypoint.sh caused 'no such file or directory'
when running the presidio-analyzer Docker container on Linux.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(config): enable German recognizers and add 'de' language support

Enabled all 9 DE_* recognizers in default_recognizers.yaml and added
'de' to supported_languages so they are loaded and visible to clients
(e.g. LiteLLM) without requiring ad-hoc configuration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(config): add 'de' to analyzer engine supported_languages

The registry and engine supported_languages must match.
Missing 'de' in default_analyzer.yaml caused a startup crash.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* revert(config): restore default enabled=false and single-language defaults

Reverts the deployment-specific changes made for local testing:
- German recognizers back to enabled=false (upstream contribution convention)
- supported_languages back to [en] only in both config files

Users who want German PII detection should enable the DE_* recognizers
explicitly in their own deployment configuration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: fix ruff linting issues in German recognizers

- D205: add blank line between summary and description (de_health_insurance_recognizer)
- E501: shorten long lines in de_health_insurance_recognizer, de_kfz_recognizer,
        de_social_security_recognizer, de_tax_id_recognizer
- D214/D406: fix Examples section formatting in de_kfz_recognizer (auto-fixed)

All 133 tests still pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(config): enable German language support and fix DE recognizer patterns

Enable German (de) as a supported language alongside English:
- default_analyzer.yaml: add 'de' to supported_languages
- default_recognizers.yaml: add 'de' to supported_languages, enable all
  nine German PII recognizers (DeTaxIdRecognizer, DeTaxNumberRecognizer,
  DePassportRecognizer, DeIdCardRecognizer, DeSocialSecurityRecognizer,
  DeHealthInsuranceRecognizer, DeKfzRecognizer, DeHandelsregisterRecognizer,
  DePlzRecognizer)
- spacy_en_de.yaml: add dedicated en+de NLP config (en_core_web_lg +
  de_core_news_md) for deployments that only need English and German

Fix DeKfzRecognizer:
- Replace \b word-boundary anchors with (?<![\w-]) / (?!\w) lookarounds
  to prevent false matches starting mid-plate after a hyphen separator
  (e.g. 'M-AB 123' was matching as 'AB 123' instead of 'M-AB 123')
- Add two missing patterns for the common Bindestrich+Leerzeichen format
  '[District]-[Letters] [Digits]' (e.g. M-AB 123, HH-XY 999)

Fix DeTaxNumberRecognizer:
- Replace \b anchors with (?<!\w) / (?!\w) lookarounds on slash-format
  patterns to prevent boundary bleed into surrounding words
- Raise Bayern/BW (3/3/5) pattern confidence from 0.3 to 0.4

NOTE: After this PR is merged a corresponding update to the LiteLLM
Presidio guardrail configuration is needed to map the newly active
German entity types (DE_KFZ, DE_TAX_ID, DE_TAX_NUMBER, DE_ID_CARD,
DE_PASSPORT, DE_SOCIAL_SECURITY, DE_HEALTH_INSURANCE,
DE_HANDELSREGISTER, DE_PLZ) to the desired masking actions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(config): restore English-only defaults and fix import ordering

- Remove 'de' from supported_languages in default_analyzer.yaml and
  default_recognizers.yaml so defaults remain ["en"] as tests expect
- Add enabled: false to all German recognizers in default_recognizers.yaml
  (opt-in instead of opt-out, consistent with other disabled recognizers)
- Move Germany imports to correct alphabetical position in __init__.py
  (after Finland, before India) to fix ruff I001 import-ordering error

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(de-recognizers): address Copilot review feedback on German recognizers

- de_id_card_recognizer: fix docstring example T220001292 (10 chars) -> T22000129 (9 chars)
- de_plz_recognizer: tighten regex to exclude boundary values 01000 and 99999
  via negative lookahead; update docstring accordingly
- test_de_plz_recognizer: add negative test cases for 01000 and 99999
- de_social_security_recognizer: fix day regex [4-7]\d (40-79) -> 5[1-9]|[67]\d
  so supplemental range is correctly 51-81 per spec, not 41-81
- de_tax_number_recognizer: remove undocumented 10-digit plain format from
  docstring (no corresponding pattern exists)
- de_passport_recognizer: remove unused _ICAO_CHARS constant; fix docstring
  example F204004812 (10 chars) -> F20400481 (9 chars)
- de_health_insurance_recognizer: fix docstring example A123456780 (invalid
  checksum) -> A123456787 (valid checksum)
- de_tax_id_recognizer: remove invalid example 02476291358 (leading zero);
  remove unimplemented digit-frequency constraint from docstring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(de-recognizers): add DE_LANR, DE_BSNR, DE_VAT_ID, DE_FUEHRERSCHEIN recognizers

Adds four new German PII recognizers with tests:

- DE_LANR (Lebenslange Arztnummer): 9-digit physician number with KBV
  check digit validation (weights [4,9,2,10,5,3], cross-sum, mod 10).
  Legal basis: § 75 Abs. 7 SGB V.

- DE_BSNR (Betriebsstättennummer): 9-digit practice/site-of-care number,
  no public checksum, relies on context words for high-confidence matches.
  Legal basis: § 75 Abs. 7 SGB V.

- DE_VAT_ID (Umsatzsteuer-Identifikationsnummer): fixed-prefix pattern
  "DE" + 9 digits per § 27a UStG / BZSt format.

- DE_FUEHRERSCHEIN (Führerscheinnummer): post-2013 EU-harmonized format
  [A-Z]{2}\d{8}[A-Z0-9] (11 chars) per FeV Anlage 8 / EU Directive
  2006/126/EC. No checksum (KBA algorithm not published). Pre-2013
  card formats are explicitly out of scope.

Also moves spacy_en_de.yaml from presidio_analyzer/conf/ to
docs/recipes/german-language-support/ per reviewer feedback, and
reverts the docker-compose.yml NLP_CONF_FILE build arg.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(recipes): align german-language-support README with recipe template

Rewrites the README to follow the template.md structure: Overview,
Quick Start, Approach (with entity/checksum table), Results (TBD
pending formal evaluation), Key Findings, and Tips sections.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
2026-03-19 20:25:22 +02:00

5.2 KiB
Raw Blame History

German Language Support

Domain: Healthcare, Legal, Finance, General Data Type: German-language free text (medical documents, contracts, invoices, ID documents) Goal: Detect German PII and sensitive identifiers using Presidio's built-in German recognizers with a bilingual spaCy NLP engine

Overview

Domain: Healthcare / Legal / Finance Data Type: German-language documents Goal: Detect German PII — including healthcare identifiers, tax numbers, official document numbers, and vehicle plates — using the German spaCy model alongside the English model so that bilingual (EN + DE) documents are handled correctly.

This recipe provides:

  • spacy_en_de.yaml — a ready-to-use NLP engine configuration that loads both en_core_web_lg and de_core_news_md
  • An overview of all German-specific recognizers available in Presidio

Quick Start

Prerequisites

pip install presidio-analyzer presidio-anonymizer
python -m spacy download en_core_web_lg
python -m spacy download de_core_news_md

Sample Data

sample_text = """
Sehr geehrter Herr Müller,

Ihre Krankenversicherungsnummer (KVNR): A123456787
Steuer-IdNr.: 86095742719
Arztnummer (LANR): 123456901
Betriebsstättennummer (BSNR): 021234568
USt-IdNr.: DE123456789
Führerscheinnummer: BO12345678A
Reisepassnummer: C01X00T47
"""

Basic Configuration

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.nlp_engine import NlpEngineProvider
from presidio_anonymizer import AnonymizerEngine

# Load the bilingual EN + DE spaCy configuration
provider = NlpEngineProvider(conf_file="spacy_en_de.yaml")
nlp_engine = provider.create_engine()

analyzer = AnalyzerEngine(
    nlp_engine=nlp_engine,
    supported_languages=["en", "de"],
)
anonymizer = AnonymizerEngine()

# Analyze German text
results = analyzer.analyze(text=sample_text, language="de")
anonymized = anonymizer.anonymize(text=sample_text, analyzer_results=results)

print(anonymized.text)

Approach

Presidio ships pattern-based recognizers for 13 German entity types (see table below). Each recognizer targets a single entity, uses \b-anchored regex patterns with base confidence between 0.2 and 0.5, and relies on:

  1. Context words (German terminology near the match) to boost confidence
  2. Check digit validation where the official specification defines one (KVNR, Rentenversicherungsnummer, Steuer-IdNr., LANR)

The spaCy de_core_news_md model adds named-entity recognition for PERSON, LOCATION, and ORGANIZATION on top of the pattern recognizers.

Supported German Entities

Entity Name Check digit
DE_TAX_ID Steueridentifikationsnummer ISO 7064 Mod 11,10
DE_TAX_NUMBER Steuernummer (Länder format)
DE_SOCIAL_SECURITY Rentenversicherungsnummer DRV algorithm
DE_HEALTH_INSURANCE Krankenversicherungsnummer (KVNR) GKV-Spitzenverband
DE_PASSPORT Reisepassnummer
DE_ID_CARD Personalausweisnummer
DE_KFZ Kfz-Kennzeichen
DE_PLZ Postleitzahl
DE_HANDELSREGISTER Handelsregisternummer
DE_LANR Lebenslange Arztnummer KBV weights algorithm
DE_BSNR Betriebsstättennummer
DE_VAT_ID Umsatzsteuer-Identifikationsnummer
DE_FUEHRERSCHEIN Führerscheinnummer (post-2013)

Results

Formal evaluation against a labelled German dataset has not yet been performed. To benchmark this recipe follow the Presidio Research evaluation workflow:

  1. Generate synthetic German text with the data generator
  2. Configure the analyzer with spacy_en_de.yaml
  3. Run the evaluator and report precision / recall / F₂ / latency

Precision: TBD Recall: TBD F₂ Score: TBD Latency: TBD

Key Findings

  • Recognizers with check digit validation (KVNR, RVNR, Steuer-IdNr., LANR) achieve very low false-positive rates even on ambiguous digit strings.
  • Recognizers without a checksum (BSNR, PLZ, KFZ) rely heavily on context words; setting score_threshold=0.5 when no context is present is recommended.
  • The DE_PLZ (postal code) and DE_KFZ (vehicle plate) recognizers overlap with generic patterns; use the entities parameter to restrict detection when only specific entity types are needed.

Tips for Others

  • Set score_threshold to 0.40.5 for production use to filter out low-confidence pattern-only matches from context-free digit strings.
  • Use the entities parameter to limit detection to the entity types relevant to your domain (e.g. only healthcare identifiers in clinical notes).
  • Pre-2013 Führerschein numbers use locally defined, non-standardized formats that are not covered by DE_FUEHRERSCHEIN; handle them with a custom recognizer if needed.
  • DE_TELEMATIK_ID was evaluated and rejected as a generic recognizer: the format (\d{1,2}-<up to 128 chars>) is too ambiguous for reliable free-text detection.

Author: MvdB Date: 2026-03-19