Files
presidio/presidio-analyzer/tests/test_de_health_insurance_recognizer.py
Michael van den Berg 98f79b9c64 feat(analyzer): add German PII recognizers (DE_*) (#1909)
* feat(analyzer): add German PII recognizers (DE_*)

Adds 9 new predefined recognizers for German personally identifiable
information, all disabled by default (enabled: false) and scoped to
supported_language: de.

Recognizers with checksum validation:
- DE_TAX_ID: Steueridentifikationsnummer (§§ 139a-e AO)
  ISO 7064 Mod 11,10 checksum (Bundeszentralamt für Steuern)
- DE_SOCIAL_SECURITY: Rentenversicherungsnummer / RVNR (§ 147 SGB VI)
  Deutsche Rentenversicherung Bund checksum algorithm
- DE_HEALTH_INSURANCE: Krankenversicherungsnummer KVNR (§ 290 SGB V)
  GKV-Spitzenverband checksum; DSGVO Art. 9 (health data)

Pattern-based recognizers:
- DE_TAX_NUMBER: Steuernummer (§ 139a AO) – ELSTER 13-digit and
  state-specific slash-separated formats
- DE_PASSPORT: Reisepassnummer (PassG § 4, ICAO Doc 9303)
- DE_ID_CARD: Personalausweisnummer (PAuswG) – nPA and legacy format
- DE_KFZ: KFZ-Kennzeichen (FZV § 8, ECJ C-582/14)
- DE_HANDELSREGISTER: Handelsregisternummer HRA/HRB (§§ 9, 14 HGB)
- DE_PLZ: Postleitzahl (DSGVO Art. 4 Nr. 1) – very low base confidence
  (0.05), context words required for actionable results

Also updates:
- predefined_recognizers/__init__.py: imports and __all__
- conf/default_recognizers.yaml: all 9 recognizers registered
- docs/supported_entities.md: new Germany section
- CHANGELOG.md: entry under [unreleased]

133 new tests pass (9 test files, one per recognizer).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: enforce LF line endings for shell scripts in containers

Added .gitattributes to force LF for *.sh files.
CRLF endings in entrypoint.sh caused 'no such file or directory'
when running the presidio-analyzer Docker container on Linux.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(config): enable German recognizers and add 'de' language support

Enabled all 9 DE_* recognizers in default_recognizers.yaml and added
'de' to supported_languages so they are loaded and visible to clients
(e.g. LiteLLM) without requiring ad-hoc configuration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(config): add 'de' to analyzer engine supported_languages

The registry and engine supported_languages must match.
Missing 'de' in default_analyzer.yaml caused a startup crash.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* revert(config): restore default enabled=false and single-language defaults

Reverts the deployment-specific changes made for local testing:
- German recognizers back to enabled=false (upstream contribution convention)
- supported_languages back to [en] only in both config files

Users who want German PII detection should enable the DE_* recognizers
explicitly in their own deployment configuration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: fix ruff linting issues in German recognizers

- D205: add blank line between summary and description (de_health_insurance_recognizer)
- E501: shorten long lines in de_health_insurance_recognizer, de_kfz_recognizer,
        de_social_security_recognizer, de_tax_id_recognizer
- D214/D406: fix Examples section formatting in de_kfz_recognizer (auto-fixed)

All 133 tests still pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(config): enable German language support and fix DE recognizer patterns

Enable German (de) as a supported language alongside English:
- default_analyzer.yaml: add 'de' to supported_languages
- default_recognizers.yaml: add 'de' to supported_languages, enable all
  nine German PII recognizers (DeTaxIdRecognizer, DeTaxNumberRecognizer,
  DePassportRecognizer, DeIdCardRecognizer, DeSocialSecurityRecognizer,
  DeHealthInsuranceRecognizer, DeKfzRecognizer, DeHandelsregisterRecognizer,
  DePlzRecognizer)
- spacy_en_de.yaml: add dedicated en+de NLP config (en_core_web_lg +
  de_core_news_md) for deployments that only need English and German

Fix DeKfzRecognizer:
- Replace \b word-boundary anchors with (?<![\w-]) / (?!\w) lookarounds
  to prevent false matches starting mid-plate after a hyphen separator
  (e.g. 'M-AB 123' was matching as 'AB 123' instead of 'M-AB 123')
- Add two missing patterns for the common Bindestrich+Leerzeichen format
  '[District]-[Letters] [Digits]' (e.g. M-AB 123, HH-XY 999)

Fix DeTaxNumberRecognizer:
- Replace \b anchors with (?<!\w) / (?!\w) lookarounds on slash-format
  patterns to prevent boundary bleed into surrounding words
- Raise Bayern/BW (3/3/5) pattern confidence from 0.3 to 0.4

NOTE: After this PR is merged a corresponding update to the LiteLLM
Presidio guardrail configuration is needed to map the newly active
German entity types (DE_KFZ, DE_TAX_ID, DE_TAX_NUMBER, DE_ID_CARD,
DE_PASSPORT, DE_SOCIAL_SECURITY, DE_HEALTH_INSURANCE,
DE_HANDELSREGISTER, DE_PLZ) to the desired masking actions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(config): restore English-only defaults and fix import ordering

- Remove 'de' from supported_languages in default_analyzer.yaml and
  default_recognizers.yaml so defaults remain ["en"] as tests expect
- Add enabled: false to all German recognizers in default_recognizers.yaml
  (opt-in instead of opt-out, consistent with other disabled recognizers)
- Move Germany imports to correct alphabetical position in __init__.py
  (after Finland, before India) to fix ruff I001 import-ordering error

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(de-recognizers): address Copilot review feedback on German recognizers

- de_id_card_recognizer: fix docstring example T220001292 (10 chars) -> T22000129 (9 chars)
- de_plz_recognizer: tighten regex to exclude boundary values 01000 and 99999
  via negative lookahead; update docstring accordingly
- test_de_plz_recognizer: add negative test cases for 01000 and 99999
- de_social_security_recognizer: fix day regex [4-7]\d (40-79) -> 5[1-9]|[67]\d
  so supplemental range is correctly 51-81 per spec, not 41-81
- de_tax_number_recognizer: remove undocumented 10-digit plain format from
  docstring (no corresponding pattern exists)
- de_passport_recognizer: remove unused _ICAO_CHARS constant; fix docstring
  example F204004812 (10 chars) -> F20400481 (9 chars)
- de_health_insurance_recognizer: fix docstring example A123456780 (invalid
  checksum) -> A123456787 (valid checksum)
- de_tax_id_recognizer: remove invalid example 02476291358 (leading zero);
  remove unimplemented digit-frequency constraint from docstring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(de-recognizers): add DE_LANR, DE_BSNR, DE_VAT_ID, DE_FUEHRERSCHEIN recognizers

Adds four new German PII recognizers with tests:

- DE_LANR (Lebenslange Arztnummer): 9-digit physician number with KBV
  check digit validation (weights [4,9,2,10,5,3], cross-sum, mod 10).
  Legal basis: § 75 Abs. 7 SGB V.

- DE_BSNR (Betriebsstättennummer): 9-digit practice/site-of-care number,
  no public checksum, relies on context words for high-confidence matches.
  Legal basis: § 75 Abs. 7 SGB V.

- DE_VAT_ID (Umsatzsteuer-Identifikationsnummer): fixed-prefix pattern
  "DE" + 9 digits per § 27a UStG / BZSt format.

- DE_FUEHRERSCHEIN (Führerscheinnummer): post-2013 EU-harmonized format
  [A-Z]{2}\d{8}[A-Z0-9] (11 chars) per FeV Anlage 8 / EU Directive
  2006/126/EC. No checksum (KBA algorithm not published). Pre-2013
  card formats are explicitly out of scope.

Also moves spacy_en_de.yaml from presidio_analyzer/conf/ to
docs/recipes/german-language-support/ per reviewer feedback, and
reverts the docker-compose.yml NLP_CONF_FILE build arg.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(recipes): align german-language-support README with recipe template

Rewrites the README to follow the template.md structure: Overview,
Quick Start, Approach (with entity/checksum table), Results (TBD
pending formal evaluation), Key Findings, and Tips sections.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
2026-03-19 20:25:22 +02:00

91 lines
3.0 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""
Tests for DeHealthInsuranceRecognizer (Krankenversicherungsnummer / KVNR).
Format: 10 characters 1 uppercase letter (birth surname initial) +
8 digits (birth date + serial) + 1 check digit.
Valid numbers are generated with the GKV-Spitzenverband checksum algorithm
(letter expanded to 2-digit ordinal, weights [2,9,8,7,6,5,4,3,2,1],
products digit-summed if ≥ 10, sum mod 10).
Legal basis: § 290 SGB V. DSGVO Art. 9 (Gesundheitsdaten).
Pre-calculated valid examples (fictitious):
A123456787 letter A (=01), data 12345678, check = 7
M123456789 letter M (=13), data 12345678, check = 9
B123456787 letter B (=02), data 12345678, check = 7
"""
import pytest
from tests import assert_result
from presidio_analyzer.predefined_recognizers import DeHealthInsuranceRecognizer
@pytest.fixture(scope="module")
def recognizer():
return DeHealthInsuranceRecognizer()
@pytest.fixture(scope="module")
def entities():
return ["DE_HEALTH_INSURANCE"]
@pytest.mark.parametrize(
"text, expected_len, expected_positions",
[
# fmt: off
# Valid KVNR checksum passes → result at MAX_SCORE
("A123456787", 1, ((0, 10),)),
("M123456789", 1, ((0, 10),)),
("B123456787", 1, ((0, 10),)),
("Krankenkasse KVNR: A123456787", 1, ((19, 29),)),
("eGK-Nummer M123456789 bitte angeben.", 1, ((11, 21),)),
# Invalid: wrong check digit
("A123456780", 0, ()),
("M123456781", 0, ()),
# Invalid: starts with digit instead of letter
("1123456787", 0, ()),
# Lowercase: Presidio uses global IGNORECASE; validate_result calls .upper()
# so a valid lowercase KVNR is matched and validated correctly
("a123456787", 1, ((0, 10),)),
# Too short (9 chars)
("A12345678", 0, ()),
# Too long (11 chars)
("A1234567890", 0, ()),
# fmt: on
],
)
def test_when_all_de_health_insurance_numbers_then_succeed(
text, expected_len, expected_positions, recognizer, entities, max_score
):
results = recognizer.analyze(text, entities)
assert len(results) == expected_len
for res, (st_pos, fn_pos) in zip(results, expected_positions):
assert_result(res, entities[0], st_pos, fn_pos, max_score)
@pytest.mark.parametrize(
"number, expected",
[
# Valid
("A123456787", True),
("M123456789", True),
("B123456787", True),
# Wrong check digit
("A123456780", False),
("M123456781", False),
# Starts with digit (after .upper() it's still a digit → re.match fails)
("1123456787", False),
# Wrong length
("A12345678", False),
("A1234567890", False),
# Lowercase: .upper() converts to valid → True (IGNORECASE is global)
("a123456787", True),
],
)
def test_when_de_health_insurance_validated_then_checksum_result_is_correct(
number, expected, recognizer
):
assert recognizer.validate_result(number) == expected