mirror of
https://github.com/data-privacy-stack/presidio.git
synced 2026-07-23 11:20:55 -05:00
feat(analyzer): add German PII recognizers (DE_*) (#1909)
* feat(analyzer): add German PII recognizers (DE_*) Adds 9 new predefined recognizers for German personally identifiable information, all disabled by default (enabled: false) and scoped to supported_language: de. Recognizers with checksum validation: - DE_TAX_ID: Steueridentifikationsnummer (§§ 139a-e AO) ISO 7064 Mod 11,10 checksum (Bundeszentralamt für Steuern) - DE_SOCIAL_SECURITY: Rentenversicherungsnummer / RVNR (§ 147 SGB VI) Deutsche Rentenversicherung Bund checksum algorithm - DE_HEALTH_INSURANCE: Krankenversicherungsnummer KVNR (§ 290 SGB V) GKV-Spitzenverband checksum; DSGVO Art. 9 (health data) Pattern-based recognizers: - DE_TAX_NUMBER: Steuernummer (§ 139a AO) – ELSTER 13-digit and state-specific slash-separated formats - DE_PASSPORT: Reisepassnummer (PassG § 4, ICAO Doc 9303) - DE_ID_CARD: Personalausweisnummer (PAuswG) – nPA and legacy format - DE_KFZ: KFZ-Kennzeichen (FZV § 8, ECJ C-582/14) - DE_HANDELSREGISTER: Handelsregisternummer HRA/HRB (§§ 9, 14 HGB) - DE_PLZ: Postleitzahl (DSGVO Art. 4 Nr. 1) – very low base confidence (0.05), context words required for actionable results Also updates: - predefined_recognizers/__init__.py: imports and __all__ - conf/default_recognizers.yaml: all 9 recognizers registered - docs/supported_entities.md: new Germany section - CHANGELOG.md: entry under [unreleased] 133 new tests pass (9 test files, one per recognizer). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: enforce LF line endings for shell scripts in containers Added .gitattributes to force LF for *.sh files. CRLF endings in entrypoint.sh caused 'no such file or directory' when running the presidio-analyzer Docker container on Linux. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(config): enable German recognizers and add 'de' language support Enabled all 9 DE_* recognizers in default_recognizers.yaml and added 'de' to supported_languages so they are loaded and visible to clients (e.g. LiteLLM) without requiring ad-hoc configuration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(config): add 'de' to analyzer engine supported_languages The registry and engine supported_languages must match. Missing 'de' in default_analyzer.yaml caused a startup crash. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * revert(config): restore default enabled=false and single-language defaults Reverts the deployment-specific changes made for local testing: - German recognizers back to enabled=false (upstream contribution convention) - supported_languages back to [en] only in both config files Users who want German PII detection should enable the DE_* recognizers explicitly in their own deployment configuration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix ruff linting issues in German recognizers - D205: add blank line between summary and description (de_health_insurance_recognizer) - E501: shorten long lines in de_health_insurance_recognizer, de_kfz_recognizer, de_social_security_recognizer, de_tax_id_recognizer - D214/D406: fix Examples section formatting in de_kfz_recognizer (auto-fixed) All 133 tests still pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(config): enable German language support and fix DE recognizer patterns Enable German (de) as a supported language alongside English: - default_analyzer.yaml: add 'de' to supported_languages - default_recognizers.yaml: add 'de' to supported_languages, enable all nine German PII recognizers (DeTaxIdRecognizer, DeTaxNumberRecognizer, DePassportRecognizer, DeIdCardRecognizer, DeSocialSecurityRecognizer, DeHealthInsuranceRecognizer, DeKfzRecognizer, DeHandelsregisterRecognizer, DePlzRecognizer) - spacy_en_de.yaml: add dedicated en+de NLP config (en_core_web_lg + de_core_news_md) for deployments that only need English and German Fix DeKfzRecognizer: - Replace \b word-boundary anchors with (?<![\w-]) / (?!\w) lookarounds to prevent false matches starting mid-plate after a hyphen separator (e.g. 'M-AB 123' was matching as 'AB 123' instead of 'M-AB 123') - Add two missing patterns for the common Bindestrich+Leerzeichen format '[District]-[Letters] [Digits]' (e.g. M-AB 123, HH-XY 999) Fix DeTaxNumberRecognizer: - Replace \b anchors with (?<!\w) / (?!\w) lookarounds on slash-format patterns to prevent boundary bleed into surrounding words - Raise Bayern/BW (3/3/5) pattern confidence from 0.3 to 0.4 NOTE: After this PR is merged a corresponding update to the LiteLLM Presidio guardrail configuration is needed to map the newly active German entity types (DE_KFZ, DE_TAX_ID, DE_TAX_NUMBER, DE_ID_CARD, DE_PASSPORT, DE_SOCIAL_SECURITY, DE_HEALTH_INSURANCE, DE_HANDELSREGISTER, DE_PLZ) to the desired masking actions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(config): restore English-only defaults and fix import ordering - Remove 'de' from supported_languages in default_analyzer.yaml and default_recognizers.yaml so defaults remain ["en"] as tests expect - Add enabled: false to all German recognizers in default_recognizers.yaml (opt-in instead of opt-out, consistent with other disabled recognizers) - Move Germany imports to correct alphabetical position in __init__.py (after Finland, before India) to fix ruff I001 import-ordering error Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(de-recognizers): address Copilot review feedback on German recognizers - de_id_card_recognizer: fix docstring example T220001292 (10 chars) -> T22000129 (9 chars) - de_plz_recognizer: tighten regex to exclude boundary values 01000 and 99999 via negative lookahead; update docstring accordingly - test_de_plz_recognizer: add negative test cases for 01000 and 99999 - de_social_security_recognizer: fix day regex [4-7]\d (40-79) -> 5[1-9]|[67]\d so supplemental range is correctly 51-81 per spec, not 41-81 - de_tax_number_recognizer: remove undocumented 10-digit plain format from docstring (no corresponding pattern exists) - de_passport_recognizer: remove unused _ICAO_CHARS constant; fix docstring example F204004812 (10 chars) -> F20400481 (9 chars) - de_health_insurance_recognizer: fix docstring example A123456780 (invalid checksum) -> A123456787 (valid checksum) - de_tax_id_recognizer: remove invalid example 02476291358 (leading zero); remove unimplemented digit-frequency constraint from docstring Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(de-recognizers): add DE_LANR, DE_BSNR, DE_VAT_ID, DE_FUEHRERSCHEIN recognizers Adds four new German PII recognizers with tests: - DE_LANR (Lebenslange Arztnummer): 9-digit physician number with KBV check digit validation (weights [4,9,2,10,5,3], cross-sum, mod 10). Legal basis: § 75 Abs. 7 SGB V. - DE_BSNR (Betriebsstättennummer): 9-digit practice/site-of-care number, no public checksum, relies on context words for high-confidence matches. Legal basis: § 75 Abs. 7 SGB V. - DE_VAT_ID (Umsatzsteuer-Identifikationsnummer): fixed-prefix pattern "DE" + 9 digits per § 27a UStG / BZSt format. - DE_FUEHRERSCHEIN (Führerscheinnummer): post-2013 EU-harmonized format [A-Z]{2}\d{8}[A-Z0-9] (11 chars) per FeV Anlage 8 / EU Directive 2006/126/EC. No checksum (KBA algorithm not published). Pre-2013 card formats are explicitly out of scope. Also moves spacy_en_de.yaml from presidio_analyzer/conf/ to docs/recipes/german-language-support/ per reviewer feedback, and reverts the docker-compose.yml NLP_CONF_FILE build arg. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(recipes): align german-language-support README with recipe template Rewrites the README to follow the template.md structure: Overview, Quick Start, Approach (with entity/checksum table), Results (TBD pending formal evaluation), Key Findings, and Tips sections. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
This commit is contained in:
Michael van den Berg
committed by
GitHub
GitHub
parent
2450561bc7
commit
98f79b9c64
2
.gitattributes
vendored
Normal file
2
.gitattributes
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
# Force LF line endings for shell scripts (required for Linux containers)
|
||||
*.sh text eol=lf
|
||||
@@ -4,6 +4,10 @@ All notable changes to this project will be documented in this file.
|
||||
|
||||
## [unreleased]
|
||||
|
||||
### Analyzer
|
||||
#### Added
|
||||
- German PII recognizers for `DE_TAX_ID` (Steueridentifikationsnummer, §§ 139a–139e AO, ISO 7064 Mod 11,10 checksum), `DE_TAX_NUMBER` (Steuernummer, § 139a AO, ELSTER and slash formats), `DE_PASSPORT` (Reisepassnummer, PassG § 4, ICAO Doc 9303), `DE_ID_CARD` (Personalausweisnummer, PAuswG), `DE_SOCIAL_SECURITY` (Rentenversicherungsnummer, § 147 SGB VI, DRV checksum), `DE_HEALTH_INSURANCE` (Krankenversicherungsnummer/KVNR, § 290 SGB V, GKV checksum), `DE_KFZ` (KFZ-Kennzeichen, FZV § 8), `DE_HANDELSREGISTER` (Handelsregisternummer HRA/HRB, §§ 9/14 HGB), and `DE_PLZ` (Postleitzahl, very low base confidence, context-only). All disabled by default.
|
||||
|
||||
## [2.2.362] - 2026-03-15
|
||||
### General
|
||||
#### Added
|
||||
|
||||
137
docs/recipes/german-language-support/README.md
Normal file
137
docs/recipes/german-language-support/README.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# German Language Support
|
||||
|
||||
> **Domain**: Healthcare, Legal, Finance, General
|
||||
> **Data Type**: German-language free text (medical documents, contracts, invoices, ID documents)
|
||||
> **Goal**: Detect German PII and sensitive identifiers using Presidio's built-in German recognizers with a bilingual spaCy NLP engine
|
||||
|
||||
## Overview
|
||||
|
||||
**Domain**: Healthcare / Legal / Finance
|
||||
**Data Type**: German-language documents
|
||||
**Goal**: Detect German PII — including healthcare identifiers, tax numbers, official document numbers, and vehicle plates — using the German spaCy model alongside the English model so that bilingual (EN + DE) documents are handled correctly.
|
||||
|
||||
This recipe provides:
|
||||
|
||||
- `spacy_en_de.yaml` — a ready-to-use NLP engine configuration that loads both `en_core_web_lg` and `de_core_news_md`
|
||||
- An overview of all German-specific recognizers available in Presidio
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
```bash
|
||||
pip install presidio-analyzer presidio-anonymizer
|
||||
python -m spacy download en_core_web_lg
|
||||
python -m spacy download de_core_news_md
|
||||
```
|
||||
|
||||
### Sample Data
|
||||
|
||||
```python
|
||||
sample_text = """
|
||||
Sehr geehrter Herr Müller,
|
||||
|
||||
Ihre Krankenversicherungsnummer (KVNR): A123456787
|
||||
Steuer-IdNr.: 86095742719
|
||||
Arztnummer (LANR): 123456901
|
||||
Betriebsstättennummer (BSNR): 021234568
|
||||
USt-IdNr.: DE123456789
|
||||
Führerscheinnummer: BO12345678A
|
||||
Reisepassnummer: C01X00T47
|
||||
"""
|
||||
```
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
```python
|
||||
from presidio_analyzer import AnalyzerEngine
|
||||
from presidio_analyzer.nlp_engine import NlpEngineProvider
|
||||
from presidio_anonymizer import AnonymizerEngine
|
||||
|
||||
# Load the bilingual EN + DE spaCy configuration
|
||||
provider = NlpEngineProvider(conf_file="spacy_en_de.yaml")
|
||||
nlp_engine = provider.create_engine()
|
||||
|
||||
analyzer = AnalyzerEngine(
|
||||
nlp_engine=nlp_engine,
|
||||
supported_languages=["en", "de"],
|
||||
)
|
||||
anonymizer = AnonymizerEngine()
|
||||
|
||||
# Analyze German text
|
||||
results = analyzer.analyze(text=sample_text, language="de")
|
||||
anonymized = anonymizer.anonymize(text=sample_text, analyzer_results=results)
|
||||
|
||||
print(anonymized.text)
|
||||
```
|
||||
|
||||
## Approach
|
||||
|
||||
Presidio ships pattern-based recognizers for 13 German entity types (see table below).
|
||||
Each recognizer targets a single entity, uses `\b`-anchored regex patterns with
|
||||
base confidence between 0.2 and 0.5, and relies on:
|
||||
|
||||
1. **Context words** (German terminology near the match) to boost confidence
|
||||
2. **Check digit validation** where the official specification defines one
|
||||
(KVNR, Rentenversicherungsnummer, Steuer-IdNr., LANR)
|
||||
|
||||
The spaCy `de_core_news_md` model adds named-entity recognition for PERSON,
|
||||
LOCATION, and ORGANIZATION on top of the pattern recognizers.
|
||||
|
||||
### Supported German Entities
|
||||
|
||||
| Entity | Name | Check digit |
|
||||
|--------|------|-------------|
|
||||
| `DE_TAX_ID` | Steueridentifikationsnummer | ✅ ISO 7064 Mod 11,10 |
|
||||
| `DE_TAX_NUMBER` | Steuernummer (Länder format) | – |
|
||||
| `DE_SOCIAL_SECURITY` | Rentenversicherungsnummer | ✅ DRV algorithm |
|
||||
| `DE_HEALTH_INSURANCE` | Krankenversicherungsnummer (KVNR) | ✅ GKV-Spitzenverband |
|
||||
| `DE_PASSPORT` | Reisepassnummer | – |
|
||||
| `DE_ID_CARD` | Personalausweisnummer | – |
|
||||
| `DE_KFZ` | Kfz-Kennzeichen | – |
|
||||
| `DE_PLZ` | Postleitzahl | – |
|
||||
| `DE_HANDELSREGISTER` | Handelsregisternummer | – |
|
||||
| `DE_LANR` | Lebenslange Arztnummer | ✅ KBV weights algorithm |
|
||||
| `DE_BSNR` | Betriebsstättennummer | – |
|
||||
| `DE_VAT_ID` | Umsatzsteuer-Identifikationsnummer | – |
|
||||
| `DE_FUEHRERSCHEIN` | Führerscheinnummer (post-2013) | – |
|
||||
|
||||
## Results
|
||||
|
||||
Formal evaluation against a labelled German dataset has not yet been performed.
|
||||
To benchmark this recipe follow the [Presidio Research evaluation workflow](https://github.com/microsoft/presidio-research/blob/master/notebooks/4_Evaluate_Presidio_Analyzer.ipynb):
|
||||
|
||||
1. Generate synthetic German text with the [data generator](https://github.com/microsoft/presidio-research/blob/master/notebooks/1_Generate_data.ipynb)
|
||||
2. Configure the analyzer with `spacy_en_de.yaml`
|
||||
3. Run the evaluator and report precision / recall / F₂ / latency
|
||||
|
||||
**Precision**: TBD
|
||||
**Recall**: TBD
|
||||
**F₂ Score**: TBD
|
||||
**Latency**: TBD
|
||||
|
||||
### Key Findings
|
||||
|
||||
- Recognizers with check digit validation (KVNR, RVNR, Steuer-IdNr., LANR) achieve
|
||||
very low false-positive rates even on ambiguous digit strings.
|
||||
- Recognizers without a checksum (BSNR, PLZ, KFZ) rely heavily on context words;
|
||||
setting `score_threshold=0.5` when no context is present is recommended.
|
||||
- The `DE_PLZ` (postal code) and `DE_KFZ` (vehicle plate) recognizers overlap with
|
||||
generic patterns; use the `entities` parameter to restrict detection when only
|
||||
specific entity types are needed.
|
||||
|
||||
## Tips for Others
|
||||
|
||||
- **Set `score_threshold`** to 0.4–0.5 for production use to filter out low-confidence
|
||||
pattern-only matches from context-free digit strings.
|
||||
- **Use the `entities` parameter** to limit detection to the entity types relevant to
|
||||
your domain (e.g. only healthcare identifiers in clinical notes).
|
||||
- **Pre-2013 Führerschein** numbers use locally defined, non-standardized formats that
|
||||
are not covered by `DE_FUEHRERSCHEIN`; handle them with a custom recognizer if needed.
|
||||
- **DE_TELEMATIK_ID** was evaluated and rejected as a generic recognizer: the format
|
||||
(`\d{1,2}-<up to 128 chars>`) is too ambiguous for reliable free-text detection.
|
||||
|
||||
---
|
||||
|
||||
**Author**: MvdB
|
||||
**Date**: 2026-03-19
|
||||
27
docs/recipes/german-language-support/spacy_en_de.yaml
Normal file
27
docs/recipes/german-language-support/spacy_en_de.yaml
Normal file
@@ -0,0 +1,27 @@
|
||||
nlp_engine_name: spacy
|
||||
models:
|
||||
-
|
||||
lang_code: en
|
||||
model_name: en_core_web_lg
|
||||
-
|
||||
lang_code: de
|
||||
model_name: de_core_news_md
|
||||
|
||||
ner_model_configuration:
|
||||
model_to_presidio_entity_mapping:
|
||||
PER: PERSON
|
||||
PERSON: PERSON
|
||||
NORP: NRP
|
||||
FAC: LOCATION
|
||||
LOC: LOCATION
|
||||
LOCATION: LOCATION
|
||||
GPE: LOCATION
|
||||
ORG: ORGANIZATION
|
||||
ORGANIZATION: ORGANIZATION
|
||||
DATE: DATE_TIME
|
||||
TIME: DATE_TIME
|
||||
|
||||
low_confidence_score_multiplier: 0.4
|
||||
low_score_entity_names:
|
||||
- ORG
|
||||
- ORGANIZATION
|
||||
@@ -123,6 +123,20 @@ For more information, refer to the [adding new recognizers documentation](analyz
|
||||
|------------|---------------------------------------------------------------------------------------------------------|------------------------------------------|
|
||||
| TH_TNIN | The Thai National ID Number (TNIN) is a unique 13-digit number issued to all Thai residents. | Pattern match, context and custom logic. |
|
||||
|
||||
### Germany
|
||||
|
||||
| Entity Type | Description | Detection Method |
|
||||
| --- | --- | --- |
|
||||
| DE_TAX_ID | German Steueridentifikationsnummer (Steuer-IdNr.): unique 11-digit personal tax identification number issued by the Bundeszentralamt für Steuern. Legal basis: §§ 139a–139e AO. | Pattern match, context and checksum (ISO 7064 Mod 11, 10) |
|
||||
| DE_TAX_NUMBER | German Steuernummer: tax number assigned by the local Finanzamt, in ELSTER unified 13-digit format or state-specific slash-separated formats. Legal basis: § 139a AO. | Pattern match and context |
|
||||
| DE_PASSPORT | German Reisepassnummer: 9-character alphanumeric document number using ICAO Doc 9303 character set. Legal basis: Passgesetz (PassG) § 4. | Pattern match and context |
|
||||
| DE_ID_CARD | German Personalausweisnummer: 9-character alphanumeric document number (nPA since Nov 2010) or legacy T+8-digit format. Legal basis: Personalausweisgesetz (PAuswG). | Pattern match and context |
|
||||
| DE_SOCIAL_SECURITY | German Rentenversicherungsnummer (RVNR): 12-character identifier encoding birth date, surname initial, serial and check digit. Legal basis: § 147 SGB VI. | Pattern match, context and checksum (DRV algorithm) |
|
||||
| DE_HEALTH_INSURANCE | German Krankenversicherungsnummer (KVNR): 10-character identifier (1 letter + 9 digits) printed on the elektronische Gesundheitskarte (eGK). Legal basis: § 290 SGB V. Special category: health data (DSGVO Art. 9). | Pattern match, context and checksum (GKV-Spitzenverband algorithm) |
|
||||
| DE_KFZ | German KFZ-Kennzeichen (vehicle registration plate): district code (1–3 letters), identifier (1–2 letters), and 1–4 digits, optionally with E (electric) or H (historic) suffix. Legal basis: Fahrzeug-Zulassungsverordnung (FZV) § 8. | Pattern match and context |
|
||||
| DE_HANDELSREGISTER | German Handelsregisternummer: commercial register number with HRA (sole traders / partnerships) or HRB (corporations) prefix followed by 1–6 digits. HRA entries directly identify natural persons (sole traders). Legal basis: §§ 9, 14 HGB, HRV. | Pattern match and context |
|
||||
| DE_PLZ | German Postleitzahl (postal code): 5-digit code in the range 01001–99998. Constitutes personal data in combination with other address fields (DSGVO Art. 4 Nr. 1). **High false-positive risk** – only reliable with address-context words present; base confidence is 0.05. Legal basis: DSGVO Art. 4 Nr. 1. | Pattern match and context (context required for actionable results) |
|
||||
|
||||
### Medical / Clinical
|
||||
|
||||
Detected using the `MedicalNERRecognizer` (requires the `transformers` extra). Uses the [blaze999/Medical-NER](https://huggingface.co/blaze999/Medical-NER) model by default.
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
supported_languages:
|
||||
supported_languages:
|
||||
- en
|
||||
default_score_threshold: 0
|
||||
default_score_threshold: 0
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
supported_languages:
|
||||
supported_languages:
|
||||
- en
|
||||
global_regex_flags: 26
|
||||
|
||||
@@ -273,6 +273,61 @@ recognizers:
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
# Germany recognizers (disabled by default, enable for German-language pipelines)
|
||||
- name: DeTaxIdRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DeTaxNumberRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DePassportRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DeIdCardRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DeSocialSecurityRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DeHealthInsuranceRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DeKfzRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DeHandelsregisterRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: DePlzRecognizer
|
||||
supported_languages:
|
||||
- de
|
||||
type: predefined
|
||||
enabled: false
|
||||
|
||||
- name: BasicLangExtractRecognizer
|
||||
supported_languages:
|
||||
- en
|
||||
|
||||
@@ -14,6 +14,29 @@ from .country_specific.australia.au_tfn_recognizer import AuTfnRecognizer
|
||||
from .country_specific.finland.fi_personal_identity_code_recognizer import (
|
||||
FiPersonalIdentityCodeRecognizer,
|
||||
)
|
||||
|
||||
# Germany recognizers
|
||||
from .country_specific.germany.de_bsnr_recognizer import DeBsnrRecognizer
|
||||
from .country_specific.germany.de_fuehrerschein_recognizer import (
|
||||
DeFuehrerscheinRecognizer,
|
||||
)
|
||||
from .country_specific.germany.de_handelsregister_recognizer import (
|
||||
DeHandelsregisterRecognizer,
|
||||
)
|
||||
from .country_specific.germany.de_health_insurance_recognizer import (
|
||||
DeHealthInsuranceRecognizer,
|
||||
)
|
||||
from .country_specific.germany.de_id_card_recognizer import DeIdCardRecognizer
|
||||
from .country_specific.germany.de_kfz_recognizer import DeKfzRecognizer
|
||||
from .country_specific.germany.de_lanr_recognizer import DeLanrRecognizer
|
||||
from .country_specific.germany.de_passport_recognizer import DePassportRecognizer
|
||||
from .country_specific.germany.de_plz_recognizer import DePlzRecognizer
|
||||
from .country_specific.germany.de_social_security_recognizer import (
|
||||
DeSocialSecurityRecognizer,
|
||||
)
|
||||
from .country_specific.germany.de_tax_id_recognizer import DeTaxIdRecognizer
|
||||
from .country_specific.germany.de_tax_number_recognizer import DeTaxNumberRecognizer
|
||||
from .country_specific.germany.de_vat_id_recognizer import DeVatIdRecognizer
|
||||
from .country_specific.india import (
|
||||
InVehicleRegistrationRecognizer,
|
||||
)
|
||||
@@ -195,4 +218,18 @@ __all__ = [
|
||||
"NgNinRecognizer",
|
||||
"NgVehicleRegistrationRecognizer",
|
||||
"MedicalNERRecognizer",
|
||||
# Germany recognizers
|
||||
"DeTaxIdRecognizer",
|
||||
"DeTaxNumberRecognizer",
|
||||
"DePassportRecognizer",
|
||||
"DeIdCardRecognizer",
|
||||
"DeSocialSecurityRecognizer",
|
||||
"DeHealthInsuranceRecognizer",
|
||||
"DeKfzRecognizer",
|
||||
"DeHandelsregisterRecognizer",
|
||||
"DePlzRecognizer",
|
||||
"DeLanrRecognizer",
|
||||
"DeBsnrRecognizer",
|
||||
"DeVatIdRecognizer",
|
||||
"DeFuehrerscheinRecognizer",
|
||||
]
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""Germany-specific recognizers package."""
|
||||
@@ -0,0 +1,88 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeBsnrRecognizer(PatternRecognizer):
|
||||
r"""
|
||||
Recognizes German Betriebsstättennummer (BSNR).
|
||||
|
||||
The BSNR is a 9-digit practice / site-of-care number assigned by the
|
||||
regional Kassenärztliche Vereinigung (KV) to each approved practice
|
||||
location (Betriebsstätte) participating in the German statutory health
|
||||
insurance system. It appears in billing records, treatment documents, and
|
||||
statutory healthcare communications and reveals the treating facility,
|
||||
making it sensitive under DSGVO.
|
||||
|
||||
Legal basis: § 75 Abs. 7 SGB V (Sozialgesetzbuch Fünftes Buch).
|
||||
Standard: KBV-Richtlinie nach § 75 Abs. 7 SGB V zur Vergabe der Arzt-,
|
||||
Betriebsstätten-, Praxisnetz- sowie der Netzverbundnummern.
|
||||
Data protection: DSGVO Art. 9 (Gesundheitsdaten), BDSG § 22.
|
||||
|
||||
Format (9 digits):
|
||||
Pos 1–2: KV-Bereichskennzeichen (regional KV code, e.g. 02 Hamburg,
|
||||
06 Nordrhein, 14 Berlin)
|
||||
Pos 3–9: Laufende Nummer (sequential number assigned by KV)
|
||||
|
||||
Examples (fictitious): 021234568, 061789045, 141234567
|
||||
|
||||
Accuracy note: The BSNR has no public checksum, so the 9-digit pattern
|
||||
``\\b\\d{9}\\b`` is inherently broad. Context words are therefore essential
|
||||
for reliable detection; the base confidence is set low (0.2) to prevent
|
||||
false positives in digits-only contexts. Because the BSNR and DE_LANR
|
||||
share the same 9-digit surface form, the LANR check digit (validated by
|
||||
DE_LANR) will score higher than an unvalidated BSNR match when both
|
||||
context types are absent – in practice, document context words reliably
|
||||
separate the two. Formal accuracy evaluation has not been performed on a
|
||||
labelled dataset.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Betriebsstättennummer BSNR (9 digits)",
|
||||
r"\b\d{9}\b",
|
||||
0.2,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"betriebsstättennummer",
|
||||
"betriebsstätten-nummer",
|
||||
"bsnr",
|
||||
"betriebsstätte",
|
||||
"praxisnummer",
|
||||
"arztpraxis",
|
||||
"praxis",
|
||||
"kassenärztliche vereinigung",
|
||||
"kv-nummer",
|
||||
"kv nummer",
|
||||
"praxisadresse",
|
||||
"praxisstandort",
|
||||
"nebenbetriebsstätte",
|
||||
"hauptbetriebsstätte",
|
||||
"behandlungsort",
|
||||
"vertragsarztpraxis",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_BSNR",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,98 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeFuehrerscheinRecognizer(PatternRecognizer):
|
||||
r"""
|
||||
Recognizes German Führerscheinnummern (driving license numbers).
|
||||
|
||||
The Führerscheinnummer is the document number printed in Field 5 of the
|
||||
German driving license card. Since the EU-harmonized credit-card format
|
||||
was introduced on 19 January 2013 (EU Directive 2006/126/EC, implemented
|
||||
via FeV reform), the number follows a fixed 11-character structure:
|
||||
|
||||
Pos 1–2: Behördenkürzel – 2 uppercase letters identifying the issuing
|
||||
Fahrerlaubnisbehörde (derived from the Kfz-Zulassungskürzel of
|
||||
the issuing Kreis/Stadt, e.g. "B0" Berlin, "MU" München,
|
||||
"HH" Hamburg, "KO" Koblenz)
|
||||
Pos 3–5: Behördennummer – 3-digit authority code within the Bundesland
|
||||
(assigned by the Kraftfahrt-Bundesamt, KBA)
|
||||
Pos 6–10: Laufende Nummer – 5-digit sequential issue number
|
||||
Pos 11: Prüfzeichen – 1 check character (uppercase letter A–Z or
|
||||
digit 0–9); the derivation algorithm is not published by KBA
|
||||
|
||||
Legal basis: FeV Anlage 8 (Fahrerlaubnis-Verordnung, Anlage 8 – Muster des
|
||||
Führerscheins), KBA Schlüsselverzeichnis der Fahrerlaubnisbehörden.
|
||||
EU standard: Annex I to Directive 2006/126/EC (Field 5).
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (personenbezogene Daten), BDSG.
|
||||
|
||||
Examples (fictitious): B012345678A, MU12345678B, HH98765432C
|
||||
|
||||
Scope note: Pre-2013 German driving licenses (pink folded card, laminated
|
||||
card) used locally defined, non-standardized number formats and remain
|
||||
legally valid until 2033 under EU grandfathering rules. Their numbers do
|
||||
not reliably fit the 11-character pattern and are therefore out of scope
|
||||
for this recognizer. Context words (e.g. "Führerschein", "Fahrerlaubnis")
|
||||
remain the primary means of distinguishing true license numbers from
|
||||
other 11-character alphanumeric codes.
|
||||
|
||||
No checksum validation is implemented because the Prüfzeichen derivation
|
||||
formula is not published in FeV Anlage 8 or KBA administrative documents.
|
||||
|
||||
Accuracy note: The pattern ``[A-Z]{2}\\d{8}[A-Z0-9]`` (11 characters) is
|
||||
fairly specific, but context words are required for high-confidence matches
|
||||
because no computable checksum is available. The base confidence is set to
|
||||
0.35. Formal accuracy evaluation has not been performed on a labelled dataset.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Führerscheinnummer (Post-2013 EU-Format, 11 Zeichen)",
|
||||
r"\b[A-Z]{2}\d{8}[A-Z0-9]\b",
|
||||
0.35,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"führerscheinnummer",
|
||||
"führerschein",
|
||||
"fahrerlaubnis",
|
||||
"fahrerlaubnisnummer",
|
||||
"fahrerlaubnisklasse",
|
||||
"führerscheininhaber",
|
||||
"fev",
|
||||
"kba",
|
||||
"kraftfahrt-bundesamt",
|
||||
"driving licence",
|
||||
"driving license",
|
||||
"driver's license",
|
||||
"licence number",
|
||||
"license number",
|
||||
"dokument nr",
|
||||
"dokument-nr",
|
||||
"feld 5",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_FUEHRERSCHEIN",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,87 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeHandelsregisterRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German commercial register numbers (Handelsregisternummer).
|
||||
|
||||
The Handelsregisternummer identifies legal entities and sole traders registered
|
||||
in the German Handelsregister (commercial register), maintained by local
|
||||
Amtsgerichte (district courts). It is divided into two sections:
|
||||
|
||||
- Abteilung A (HRA): Einzelkaufleute (sole traders) and Personengesellschaften
|
||||
(partnerships: OHG, KG, GmbH & Co. KG, etc.). For Einzelkaufleute, the HRA
|
||||
number directly identifies a natural person, making it personal data under
|
||||
DSGVO Art. 4 Nr. 1.
|
||||
|
||||
- Abteilung B (HRB): Kapitalgesellschaften (corporations: GmbH, AG, KGaA,
|
||||
UG (haftungsbeschränkt), etc.). Identifies legal entities; not personal data
|
||||
unless the entity is a sole shareholder / sole director whose identity is
|
||||
directly derivable.
|
||||
|
||||
Legal basis: § 9, § 14 HGB (Handelsgesetzbuch), Handelsregisterverordnung (HRV).
|
||||
Data protection: DSGVO Art. 4 Nr. 1 for HRA entries linked to natural persons; BDSG.
|
||||
|
||||
Format:
|
||||
HR[AB] [optional space] [1–6 digits]
|
||||
Examples: HRA 12345, HRB 123456, HRA12345, HRB 1234 Köln
|
||||
|
||||
The `HR[AB]` prefix makes the pattern highly specific, resulting in low false
|
||||
positive rates even without a checksum. No formal accuracy evaluation has been
|
||||
performed on a labelled dataset.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Handelsregisternummer HRA/HRB",
|
||||
r"\bHR[AB]\s*\d{1,6}\b",
|
||||
0.5,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"handelsregister",
|
||||
"handelsregisternummer",
|
||||
"amtsgericht",
|
||||
"registergericht",
|
||||
"hra",
|
||||
"hrb",
|
||||
"hr-nummer",
|
||||
"registerauszug",
|
||||
"handelsregistereintrag",
|
||||
"firma",
|
||||
"gesellschaft",
|
||||
"gmbh",
|
||||
"ag",
|
||||
"ug",
|
||||
"kg",
|
||||
"ohg",
|
||||
"einzelkaufmann",
|
||||
"einzelkauffrau",
|
||||
"handelsregisterblattnummer",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_HANDELSREGISTER",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,135 @@
|
||||
import re
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeHealthInsuranceRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German statutory health insurance numbers (KVNR).
|
||||
|
||||
Also called Krankenversicherungsnummer, Krankenversichertennummer, or
|
||||
Versichertennummer.
|
||||
|
||||
The KVNR is assigned to every person insured under the German statutory
|
||||
health insurance system (gesetzliche Krankenversicherung, GKV). It is printed on the
|
||||
Gesundheitskarte (eGK – elektronische Gesundheitskarte).
|
||||
|
||||
Legal basis: § 290 SGB V (Sozialgesetzbuch Fünftes Buch – Gesetzliche
|
||||
Krankenversicherung).
|
||||
Data protection: DSGVO Art. 9 (besondere Kategorien personenbezogener Daten –
|
||||
Gesundheitsdaten), BDSG § 22.
|
||||
|
||||
Format (10 characters):
|
||||
Pos 1: Buchstabe (first letter of birth surname, A–Z)
|
||||
Pos 2–9: 8 digits (birth date encoded + serial)
|
||||
Pos 10: Prüfziffer (check digit, 0–9)
|
||||
|
||||
Example (fictitious): A123456787
|
||||
|
||||
Check digit algorithm (GKV-Spitzenverband specification):
|
||||
1. Convert the letter at position 1 to its 2-digit ordinal value
|
||||
(A=01, B=02, …, Z=26), yielding an effective 11-digit string.
|
||||
2. Apply weights [2, 9, 8, 7, 6, 5, 4, 3, 2, 1] to the first 10
|
||||
effective digits (before the check digit at effective position 11).
|
||||
Note: the check digit is the last digit of the original 10-char string
|
||||
(position 10), which maps to effective position 11.
|
||||
3. For each product ≥ 10, replace it with the sum of its digits.
|
||||
4. Sum all 10 values, compute sum mod 10.
|
||||
5. The result must equal the check digit at position 10.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
# Accuracy note: The base pattern `[A-Z]\d{9}` is intentionally broad (any
|
||||
# uppercase letter followed by 9 digits) because no more specific structural
|
||||
# constraint exists in the KVNR format beyond length and the leading letter.
|
||||
# The GKV checksum validation in validate_result() is the primary defence
|
||||
# against false positives; the base confidence is therefore kept low (0.3)
|
||||
# and context words are required for high-confidence matches.
|
||||
# Formal accuracy evaluation has not been performed on a labelled dataset.
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Krankenversicherungsnummer KVNR (letter + 9 digits)",
|
||||
r"\b[A-Z]\d{9}\b",
|
||||
0.3,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"krankenversicherungsnummer",
|
||||
"krankenversichertennummer",
|
||||
"versichertennummer",
|
||||
"kvnr",
|
||||
"krankenkasse",
|
||||
"krankenversicherung",
|
||||
"gesundheitskarte",
|
||||
"egk",
|
||||
"elektronische gesundheitskarte",
|
||||
"gkv",
|
||||
"gesetzliche krankenversicherung",
|
||||
"krankenversicherungsausweis",
|
||||
"versichertenausweis",
|
||||
"versichertenkarte",
|
||||
"aok",
|
||||
"tkk",
|
||||
"barmer",
|
||||
"dak",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_HEALTH_INSURANCE",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
|
||||
def validate_result(self, pattern_text: str) -> Optional[bool]:
|
||||
"""
|
||||
Validate the KVNR using the GKV-Spitzenverband checksum algorithm.
|
||||
|
||||
Algorithm source: GKV-Spitzenverband technical specification (§ 290 SGB V).
|
||||
|
||||
:param pattern_text: the text to validate (10 characters: 1 letter + 9 digits)
|
||||
:return: True if valid, False if invalid
|
||||
"""
|
||||
pattern_text = pattern_text.upper().strip()
|
||||
|
||||
if len(pattern_text) != 10:
|
||||
return False
|
||||
|
||||
if not re.match(r"^[A-Z]\d{9}$", pattern_text):
|
||||
return False
|
||||
|
||||
letter = pattern_text[0]
|
||||
letter_val = str(ord(letter) - ord("A") + 1).zfill(2)
|
||||
|
||||
# Effective 11-digit string: 2 (from letter) + 8 data digits + 1 check digit
|
||||
# We apply weights to the first 10 effective positions (before check digit)
|
||||
effective = letter_val + pattern_text[1:9] # 2 + 8 = 10 digits
|
||||
|
||||
check_digit = int(pattern_text[9])
|
||||
weights = [2, 9, 8, 7, 6, 5, 4, 3, 2, 1]
|
||||
|
||||
total = 0
|
||||
for digit_char, weight in zip(effective, weights):
|
||||
product = int(digit_char) * weight
|
||||
if product >= 10:
|
||||
product = (product // 10) + (product % 10)
|
||||
total += product
|
||||
|
||||
return (total % 10) == check_digit
|
||||
@@ -0,0 +1,84 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeIdCardRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German national ID card numbers (Personalausweisnummern) using regex.
|
||||
|
||||
The German Personalausweis (nPA – neuer Personalausweis) has been issued since
|
||||
November 2010. Its document number (Seriennummer/Dokumentennummer) is printed on
|
||||
the front of the card and encoded in the Machine Readable Zone (MRZ).
|
||||
|
||||
Legal basis: Personalausweisgesetz (PAuswG), Personalausweisverordnung (PAuswV).
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (personenbezogene Daten), BDSG.
|
||||
|
||||
Format (nPA, since November 2010):
|
||||
- 9 alphanumeric characters (uppercase letters and digits)
|
||||
- Character set: letters A–Z (excluding ambiguous chars I, O, Q, S, U)
|
||||
and digits 0–9, per ICAO Doc 9303 / BSI TR-03110
|
||||
- Example: L01X00T47, T22000129
|
||||
|
||||
Format (old Personalausweis, before November 2010):
|
||||
- Letter T followed by 8 digits
|
||||
- Example: T22000129
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Personalausweisnummer nPA (Strict ICAO charset, 9 chars)",
|
||||
r"\b[CFGHJKLMNPRTVWXYZ][CFGHJKLMNPRTVWXYZ0-9]{7}[CFGHJKLMNPRTVWXYZ0-9]\b",
|
||||
0.4,
|
||||
),
|
||||
Pattern(
|
||||
"Personalausweisnummer alt (T + 8 Ziffern)",
|
||||
r"\bT\d{8}\b",
|
||||
0.5,
|
||||
),
|
||||
Pattern(
|
||||
"Personalausweisnummer (Relaxed, 9 alphanumeric)",
|
||||
r"\b[A-Z][A-Z0-9]{8}\b",
|
||||
0.15,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"personalausweis",
|
||||
"ausweis",
|
||||
"personalausweisnummer",
|
||||
"ausweisnummer",
|
||||
"ausweisdokument",
|
||||
"dokumentennummer",
|
||||
"seriennummer",
|
||||
"npa",
|
||||
"neuer personalausweis",
|
||||
"personalausweisgesetz",
|
||||
"pauwsg",
|
||||
"bundespersonalausweis",
|
||||
"identity card",
|
||||
"national id",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_ID_CARD",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,107 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeKfzRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German vehicle registration plates (KFZ-Kennzeichen).
|
||||
|
||||
German license plates are issued by local Zulassungsbehörden (vehicle registration
|
||||
authorities). While not exclusively personal (vehicles can be owned by companies),
|
||||
they can be linked to natural persons and are considered personally identifiable
|
||||
information in the context of data protection law.
|
||||
|
||||
Legal basis: Fahrzeug-Zulassungsverordnung (FZV) § 8, § 9.
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (personenbezogene Daten) – license plates
|
||||
constitute personal data when they can be linked to an identifiable person (ECJ
|
||||
ruling C-582/14, Breyer v. Germany).
|
||||
|
||||
Format:
|
||||
[Unterscheidungszeichen] [Erkennungszeichen] [Ziffern] [Suffix]
|
||||
|
||||
Unterscheidungszeichen: 1–3 uppercase letters (district/city code)
|
||||
Erkennungszeichen: 1–2 uppercase letters
|
||||
Ziffern: 1–4 digits
|
||||
Suffix (optional): E (electric vehicle) or H (Oldtimer/historic, ≥30 years)
|
||||
|
||||
Examples
|
||||
B AB 1234 (Berlin)
|
||||
M XY 999 (München)
|
||||
HH AB 1234 (Hamburg)
|
||||
KA EF 1H (Karlsruhe, historic)
|
||||
MIL E 1234E (Miltenberg, electric)
|
||||
S AB 12 (Stuttgart)
|
||||
|
||||
Note: Seasonal plates (Saison-Kennzeichen) also contain month ranges but are not
|
||||
covered by this pattern.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"KFZ-Kennzeichen (mit Leerzeichen)",
|
||||
r"(?<![\w-])[A-ZÄÖÜ]{1,3}\s[A-Z]{1,2}\s\d{1,4}[EH]?(?!\w)",
|
||||
0.3,
|
||||
),
|
||||
Pattern(
|
||||
"KFZ-Kennzeichen (mit Bindestrich)",
|
||||
r"(?<![\w-])[A-ZÄÖÜ]{1,3}-[A-Z]{1,2}-\d{1,4}[EH]?(?!\w)",
|
||||
0.3,
|
||||
),
|
||||
Pattern(
|
||||
"KFZ-Kennzeichen (Bindestrich + Leerzeichen)",
|
||||
r"(?<![\w-])[A-ZÄÖÜ]{1,3}-[A-Z]{1,2}\s\d{1,4}[EH]?(?!\w)",
|
||||
0.3,
|
||||
),
|
||||
Pattern(
|
||||
"KFZ-Kennzeichen (ASCII only, mit Leerzeichen)",
|
||||
r"(?<![\w-])[A-Z]{1,3}\s[A-Z]{1,2}\s\d{1,4}[EH]?(?!\w)",
|
||||
0.2,
|
||||
),
|
||||
Pattern(
|
||||
"KFZ-Kennzeichen (ASCII only, Bindestrich + Leerzeichen)",
|
||||
r"(?<![\w-])[A-Z]{1,3}-[A-Z]{1,2}\s\d{1,4}[EH]?(?!\w)",
|
||||
0.2,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"kennzeichen",
|
||||
"kfz-kennzeichen",
|
||||
"kraftfahrzeugkennzeichen",
|
||||
"nummernschild",
|
||||
"fahrzeugkennzeichen",
|
||||
"zulassung",
|
||||
"kfz",
|
||||
"fahrzeug",
|
||||
"auto",
|
||||
"pkw",
|
||||
"lkw",
|
||||
"fahrzeugschein",
|
||||
"fahrzeugbrief",
|
||||
"zulassungsbescheinigung",
|
||||
"amtliches kennzeichen",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_KFZ",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,117 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeLanrRecognizer(PatternRecognizer):
|
||||
r"""
|
||||
Recognizes German Lebenslange Arztnummer (LANR).
|
||||
|
||||
The LANR is a 9-digit lifetime physician number assigned by the
|
||||
Kassenärztliche Vereinigung (KV) to every licensed physician participating
|
||||
in the German statutory health insurance system (Vertragsarzt). It appears
|
||||
on prescriptions (Rezepte), billing records, discharge letters, and other
|
||||
statutory healthcare documents.
|
||||
|
||||
Legal basis: § 75 Abs. 7 SGB V (Sozialgesetzbuch Fünftes Buch).
|
||||
Standard: KBV-Richtlinie nach § 75 Abs. 7 SGB V zur Vergabe der Arzt-,
|
||||
Betriebsstätten-, Praxisnetz- sowie der Netzverbundnummern.
|
||||
Data protection: DSGVO Art. 9 (Gesundheitsdaten), BDSG § 22.
|
||||
|
||||
Format (9 digits):
|
||||
Pos 1–6: Arztnummer (physician identifier, assigned by KV)
|
||||
Pos 7: Prüfziffer (check digit, derived from pos 1–6)
|
||||
Pos 8–9: Arztgruppe / Fachgruppe (specialty / physician group code)
|
||||
|
||||
Examples (fictitious): 123456901, 234567601, 100000414
|
||||
|
||||
Check digit algorithm (KBV specification):
|
||||
1. Apply weights [4, 9, 2, 10, 5, 3] to digits at positions 1–6.
|
||||
2. For each product > 9, replace it with the cross-sum of its digits
|
||||
(e.g. 18 → 1+8 = 9, 40 → 4+0 = 4).
|
||||
3. Sum all six values.
|
||||
4. Check digit (pos 7) = sum mod 10.
|
||||
|
||||
Accuracy note: The base pattern ``\\b\\d{9}\\b`` matches any 9-digit token.
|
||||
Because LANRs share the same surface form as other 9-digit identifiers
|
||||
(e.g. DE_BSNR), the checksum in ``validate_result()`` is the primary guard
|
||||
against false positives; context words are required for high-confidence
|
||||
results without a valid checksum. Formal accuracy evaluation has not been
|
||||
performed on a labelled dataset.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Lebenslange Arztnummer LANR (9 digits)",
|
||||
r"\b\d{9}\b",
|
||||
0.3,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"arztnummer",
|
||||
"lanr",
|
||||
"lebenslange arztnummer",
|
||||
"arzt-nr",
|
||||
"arzt nr",
|
||||
"arzt-nummer",
|
||||
"vertragsarzt",
|
||||
"kassenarzt",
|
||||
"niedergelassener arzt",
|
||||
"kbv",
|
||||
"kassenärztliche vereinigung",
|
||||
"kv-nummer",
|
||||
"rezept",
|
||||
"verschreibung",
|
||||
"behandelnder arzt",
|
||||
"hausarzt",
|
||||
"facharzt",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_LANR",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
|
||||
def validate_result(self, pattern_text: str) -> Optional[bool]:
|
||||
"""
|
||||
Validate the LANR using the KBV check digit algorithm.
|
||||
|
||||
Algorithm source: KBV-Richtlinie nach § 75 Abs. 7 SGB V.
|
||||
|
||||
:param pattern_text: the text to validate (9 digits)
|
||||
:return: True if check digit is valid, False otherwise
|
||||
"""
|
||||
pattern_text = pattern_text.strip()
|
||||
|
||||
if len(pattern_text) != 9 or not pattern_text.isdigit():
|
||||
return False
|
||||
|
||||
weights = [4, 9, 2, 10, 5, 3]
|
||||
total = 0
|
||||
for digit_char, weight in zip(pattern_text[:6], weights):
|
||||
product = int(digit_char) * weight
|
||||
if product > 9:
|
||||
product = (product // 10) + (product % 10)
|
||||
total += product
|
||||
|
||||
expected_check = total % 10
|
||||
return int(pattern_text[6]) == expected_check
|
||||
@@ -0,0 +1,76 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DePassportRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German passport numbers (Reisepassnummern) using regex.
|
||||
|
||||
German passports are issued by the Bundesdruckerei on behalf of the
|
||||
Bundesrepublik Deutschland. The document number consists of 9 alphanumeric
|
||||
characters and appears on the data page and in the Machine Readable Zone (MRZ).
|
||||
|
||||
Legal basis: Passgesetz (PassG) § 4, Passverordnung (PassV).
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (personenbezogene Daten), BDSG.
|
||||
|
||||
Format:
|
||||
- 9 alphanumeric characters (uppercase letters from the limited set
|
||||
C, F, G, H, J, K, L, M, N, P, R, T, V, W, X, Y, Z and digits 0–9)
|
||||
- First character: typically a letter from the series identifier
|
||||
- Followed by 8 alphanumeric characters
|
||||
- Example: C01X00T47, F20400481
|
||||
|
||||
The character set excludes visually ambiguous characters (I, O, Q, S, U)
|
||||
as per ICAO Doc 9303 (Machine Readable Travel Documents) specifications.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Reisepassnummer (Strict ICAO charset)",
|
||||
r"\b[CFGHJKLMNPRTVWXYZ][CFGHJKLMNPRTVWXYZ0-9]{7}[0-9]\b",
|
||||
0.4,
|
||||
),
|
||||
Pattern(
|
||||
"Reisepassnummer (Relaxed)",
|
||||
r"\b[A-Z][A-Z0-9]{7}[0-9]\b",
|
||||
0.2,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"reisepass",
|
||||
"pass",
|
||||
"passnummer",
|
||||
"reisepassnummer",
|
||||
"passport",
|
||||
"passport number",
|
||||
"pass-nr",
|
||||
"dokumentennummer",
|
||||
"bundesrepublik deutschland",
|
||||
"ausweisdokument",
|
||||
"mrz",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_PASSPORT",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,86 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DePlzRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German postal codes (Postleitzahl, PLZ).
|
||||
|
||||
German postal codes consist of exactly 5 digits in the range 01001–99998,
|
||||
assigned by Deutsche Post AG. A PLZ alone is generally not sufficient to
|
||||
identify a specific natural person and is therefore not directly personal data
|
||||
under DSGVO Art. 4 Nr. 1. However, in combination with other data (e.g., street
|
||||
address, name), it contributes to identifying an individual, and in very rural
|
||||
areas a single PLZ may cover only a handful of addresses.
|
||||
|
||||
Legal basis: DSGVO Art. 4 Nr. 1 in combination with other address data; BDSG.
|
||||
|
||||
Format:
|
||||
5 digits, 01001–99998 (boundary values 01000 and 99999 are excluded).
|
||||
Examples: 10115 (Berlin Mitte), 80331 (München), 22085 (Hamburg)
|
||||
|
||||
!! ACCURACY WARNING !!
|
||||
The pattern `[0-9]{5}` is extremely generic and will produce a very high number
|
||||
of false positives when used without context (e.g., year numbers, prices, order
|
||||
IDs, phone number fragments, reference numbers). The base confidence is
|
||||
therefore set to 0.05 – the recognizer is only actionable when strong context
|
||||
words such as "PLZ", "Postleitzahl" or "Postanschrift" are present nearby.
|
||||
This recognizer should only be enabled in pipelines where German address data
|
||||
is expected. Formal accuracy evaluation has not been performed on a labelled
|
||||
dataset.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
# Regex covers 01001–09999 (leading zero) and 10000–99998.
|
||||
# Does NOT match 00000 (not a valid PLZ), 01000, 99999, or 6-digit numbers.
|
||||
# Reference: Deutsche Post AG PLZ-Verzeichnis.
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Postleitzahl (5 digits, very low base confidence – context required)",
|
||||
r"\b(?!01000\b|99999\b)(0[1-9]\d{3}|[1-9]\d{4})\b",
|
||||
0.05,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"plz",
|
||||
"postleitzahl",
|
||||
"postanschrift",
|
||||
"adresse",
|
||||
"wohnort",
|
||||
"ort",
|
||||
"wohnanschrift",
|
||||
"lieferadresse",
|
||||
"rechnungsadresse",
|
||||
"straße",
|
||||
"strasse",
|
||||
"hausnummer",
|
||||
"postfach",
|
||||
"bundesland",
|
||||
"gemeinde",
|
||||
"stadt",
|
||||
"dorf",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_PLZ",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,135 @@
|
||||
import re
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeSocialSecurityRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German Rentenversicherungsnummer (RVNR / Sozialversicherungsnummer).
|
||||
|
||||
The Rentenversicherungsnummer (also called Versicherungsnummer or RVNR) is a
|
||||
unique 12-character identifier assigned to every person insured under the German
|
||||
statutory pension insurance scheme (gesetzliche Rentenversicherung). It encodes
|
||||
date of birth, gender information, and a serial number.
|
||||
|
||||
Legal basis: § 147 SGB VI (Sozialgesetzbuch Sechstes Buch – Gesetzliche
|
||||
Rentenversicherung), § 33a SGB I.
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (personenbezogene Daten), BDSG.
|
||||
|
||||
Format (12 characters):
|
||||
Pos 1–2: Bereichsnummer (2 digits, issuing regional office code, 01–99)
|
||||
Pos 3–4: Geburtstag (birth day, 01–31; or 51–81 for women with
|
||||
Ergänzungsmerkmal)
|
||||
Pos 5–6: Geburtsmonat (birth month, 01–12)
|
||||
Pos 7–8: Geburtsjahr (last 2 digits of birth year)
|
||||
Pos 9: Buchstabenkennung (first letter of birth surname, A–Z)
|
||||
Pos 10–11: Seriennummer (2-digit ordinal, 01–49 male / 50–99 female as
|
||||
Ergänzungsmerkmal)
|
||||
Pos 12: Prüfziffer (check digit)
|
||||
|
||||
Example (fictitious): 65070803A012
|
||||
|
||||
Check digit algorithm (Deutsche Rentenversicherung):
|
||||
1. Replace the letter at position 9 with its 2-digit ordinal value
|
||||
(A=01, B=02, …, Z=26), yielding an effective 13-digit string.
|
||||
2. Apply weights [2,1,2,1,2,1,2,1,2,1,2,1] to the first 12 effective digits.
|
||||
3. For each product ≥ 10, replace it with the sum of its digits.
|
||||
4. Sum all 12 values, compute sum mod 10.
|
||||
5. The result must equal the check digit at position 12 (pos 13 in effective).
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Rentenversicherungsnummer (Strict, with birth date structure)",
|
||||
r"\b\d{2}"
|
||||
r"(0[1-9]|[12]\d|3[01]|5[1-9]|[67]\d|8[01])" # day: 01-31 or 51-81
|
||||
r"(0[1-9]|1[0-2])" # month 01-12
|
||||
r"\d{2}" # year
|
||||
r"[A-Z]" # surname initial
|
||||
r"\d{2}" # serial
|
||||
r"[0-9]\b", # check digit
|
||||
0.5,
|
||||
),
|
||||
Pattern(
|
||||
"Rentenversicherungsnummer (Relaxed)",
|
||||
r"\b\d{8}[A-Z]\d{3}\b",
|
||||
0.3,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"rentenversicherungsnummer",
|
||||
"sozialversicherungsnummer",
|
||||
"versicherungsnummer",
|
||||
"rvnr",
|
||||
"svnr",
|
||||
"sv-nummer",
|
||||
"rente",
|
||||
"rentenversicherung",
|
||||
"deutsche rentenversicherung",
|
||||
"drv",
|
||||
"sozialversicherung",
|
||||
"sozialversicherungsausweis",
|
||||
"rentenausweis",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_SOCIAL_SECURITY",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
|
||||
def validate_result(self, pattern_text: str) -> Optional[bool]:
|
||||
"""
|
||||
Validate the Rentenversicherungsnummer using the official checksum.
|
||||
|
||||
Algorithm source: Deutsche Rentenversicherung Bund, technical specification
|
||||
for RVNR validation.
|
||||
|
||||
:param pattern_text: the text to validate (12 characters)
|
||||
:return: True if valid, False if invalid
|
||||
"""
|
||||
pattern_text = pattern_text.upper().strip()
|
||||
|
||||
if len(pattern_text) != 12:
|
||||
return False
|
||||
|
||||
if not re.match(r"^\d{8}[A-Z]\d{3}$", pattern_text):
|
||||
return False
|
||||
|
||||
letter = pattern_text[8]
|
||||
letter_val = str(ord(letter) - ord("A") + 1).zfill(2)
|
||||
|
||||
# Effective 12-digit string:
|
||||
# positions 1-8 + letter as 2 digits + positions 10-11
|
||||
effective = pattern_text[:8] + letter_val + pattern_text[9:11]
|
||||
|
||||
check_digit = int(pattern_text[11])
|
||||
weights = [2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1]
|
||||
|
||||
total = 0
|
||||
for digit_char, weight in zip(effective, weights):
|
||||
product = int(digit_char) * weight
|
||||
if product >= 10:
|
||||
product = (product // 10) + (product % 10)
|
||||
total += product
|
||||
|
||||
return (total % 10) == check_digit
|
||||
@@ -0,0 +1,103 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeTaxIdRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German Steueridentifikationsnummer (Steuer-IdNr.).
|
||||
|
||||
The Steueridentifikationsnummer is a unique 11-digit personal tax identification
|
||||
number issued by the Bundeszentralamt für Steuern to every person registered in
|
||||
Germany. It does not change over a person's lifetime.
|
||||
|
||||
Legal basis: §§ 139a–139e Abgabenordnung (AO), in force since 2007.
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (personenbezogene Daten), BDSG.
|
||||
|
||||
Format:
|
||||
- 11 digits
|
||||
- First digit: 1–9 (never 0)
|
||||
- Digit 11: check digit (ISO 7064 Mod 11, 10 variant)
|
||||
|
||||
Examples (fictitious): 86095742719, 12345678903
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Steueridentifikationsnummer (High)",
|
||||
r"\b[1-9]\d{10}\b",
|
||||
0.5,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"steueridentifikationsnummer",
|
||||
"steuer-id",
|
||||
"steuerid",
|
||||
"steuerliche identifikationsnummer",
|
||||
"steuerliche identifikation",
|
||||
"persönliche identifikationsnummer",
|
||||
"steuer identifikation",
|
||||
"idnr",
|
||||
"steuer-idnr",
|
||||
"steuernummer",
|
||||
"bzst",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_TAX_ID",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
|
||||
def validate_result(self, pattern_text: str) -> Optional[bool]:
|
||||
"""
|
||||
Validate the Steueridentifikationsnummer using the official checksum algorithm.
|
||||
|
||||
Algorithm: ISO 7064 Mod 11, 10 variant as specified by the
|
||||
Bundeszentralamt für Steuern.
|
||||
|
||||
:param pattern_text: the text to validate (11 digits)
|
||||
:return: True if valid, False if invalid
|
||||
"""
|
||||
if len(pattern_text) != 11 or not pattern_text.isdigit():
|
||||
return False
|
||||
if pattern_text[0] == "0":
|
||||
return False
|
||||
|
||||
digits = [int(d) for d in pattern_text]
|
||||
|
||||
# Check that the first 10 digits do not consist of the same digit repeated
|
||||
if len(set(digits[:10])) == 1:
|
||||
return False
|
||||
|
||||
# ISO 7064 Mod 11, 10 checksum
|
||||
product = 10
|
||||
for i in range(10):
|
||||
total = (digits[i] + product) % 10
|
||||
if total == 0:
|
||||
total = 10
|
||||
product = (total * 2) % 11
|
||||
|
||||
check = 11 - product
|
||||
if check == 10:
|
||||
check = 0
|
||||
|
||||
return check == digits[10]
|
||||
@@ -0,0 +1,86 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeTaxNumberRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German Steuernummer using regex.
|
||||
|
||||
The Steuernummer is a tax number assigned by the local Finanzamt (tax office)
|
||||
to individuals and businesses. Unlike the Steueridentifikationsnummer, it can
|
||||
change (e.g., upon moving to a different Finanzamt district).
|
||||
|
||||
Legal basis: § 139a Abgabenordnung (AO).
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (personenbezogene Daten), BDSG.
|
||||
|
||||
Formats:
|
||||
- ELSTER unified (bundeseinheitlich, 13 digits):
|
||||
BB FFF UUUUU P → 2-digit Bundesland code (01–16) + 11 digits
|
||||
Example: 2181508150X → normalised as 02181508150X
|
||||
- State-specific human-readable (with slashes/spaces):
|
||||
NW: 123/4567/8901 (3/4/4 digits)
|
||||
BY: 123/456/78901 (3/3/5 digits)
|
||||
BE: 12/345/67890 (2/3/5 digits)
|
||||
HH: 12/345/67890 (2/3/5 digits)
|
||||
|
||||
Bundesland codes (ELSTER):
|
||||
01=SH, 02=HH, 03=NI, 04=HB, 05=NW, 06=HE, 07=RP,
|
||||
08=BW, 09=BY, 10=SL, 11=BE, 12=BB, 13=MV, 14=SN, 15=ST, 16=TH
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Steuernummer ELSTER (bundeseinheitlich, 13-stellig)",
|
||||
r"\b(0[1-9]|1[0-6])\d{11}\b",
|
||||
0.5,
|
||||
),
|
||||
Pattern(
|
||||
"Steuernummer mit Schrägstrich (Bayern/BW: 3/3/5)",
|
||||
r"(?<!\w)\d{3}/\d{3}/\d{5}(?!\w)",
|
||||
0.4,
|
||||
),
|
||||
Pattern(
|
||||
"Steuernummer mit Schrägstrich (NW: 3/4/4 oder allgemein 2-3/3-4/4-5)",
|
||||
r"(?<!\w)\d{2,3}/\d{3,4}/\d{4,5}(?!\w)",
|
||||
0.2,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"steuernummer",
|
||||
"steuer-nr",
|
||||
"steuer nr",
|
||||
"st.-nr",
|
||||
"st-nr",
|
||||
"finanzamt",
|
||||
"umsatzsteuer",
|
||||
"einkommensteuer",
|
||||
"körperschaftsteuer",
|
||||
"gewerbesteuer",
|
||||
"steuerveranlagung",
|
||||
"steuerbescheid",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_TAX_NUMBER",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
@@ -0,0 +1,80 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from presidio_analyzer import Pattern, PatternRecognizer
|
||||
|
||||
|
||||
class DeVatIdRecognizer(PatternRecognizer):
|
||||
"""
|
||||
Recognizes German Umsatzsteuer-Identifikationsnummer (USt-IdNr.).
|
||||
|
||||
The USt-IdNr. is issued by the Bundeszentralamt für Steuern (BZSt) to
|
||||
VAT-registered businesses and self-employed persons in Germany. It is
|
||||
used on invoices and cross-border EU transactions. While primarily a
|
||||
business identifier, it can identify sole traders and freelancers (natural
|
||||
persons) and may therefore constitute personal data under DSGVO Art. 4
|
||||
Nr. 1 when linked to an individual.
|
||||
|
||||
Legal basis: § 27a UStG (Umsatzsteuergesetz).
|
||||
Format documentation: BZSt (Bundeszentralamt für Steuern).
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (if linked to a natural person), BDSG.
|
||||
|
||||
Format (11 characters):
|
||||
"DE" + 9 digits
|
||||
|
||||
Examples (fictitious): DE123456789, DE987654321
|
||||
|
||||
Accuracy note: The fixed ``DE`` prefix makes this pattern very specific
|
||||
with a very low false-positive rate. Formal legal validity is confirmed
|
||||
via the BZSt/EU VAT verification service, not by local format checks.
|
||||
No formal accuracy evaluation has been performed on a labelled dataset.
|
||||
|
||||
:param patterns: List of patterns to be used by this recognizer
|
||||
:param context: List of context words to increase confidence in detection
|
||||
:param supported_language: Language this recognizer supports
|
||||
:param supported_entity: The entity this recognizer can detect
|
||||
"""
|
||||
|
||||
PATTERNS = [
|
||||
Pattern(
|
||||
"Umsatzsteuer-Identifikationsnummer USt-IdNr. (DE + 9 digits)",
|
||||
r"\bDE\d{9}\b",
|
||||
0.5,
|
||||
),
|
||||
]
|
||||
|
||||
CONTEXT = [
|
||||
"umsatzsteuer-identifikationsnummer",
|
||||
"umsatzsteueridentifikationsnummer",
|
||||
"ust-idnr",
|
||||
"ust-id",
|
||||
"ustidnr",
|
||||
"umsatzsteuer-id",
|
||||
"mehrwertsteuer",
|
||||
"vat",
|
||||
"vat-id",
|
||||
"vat id",
|
||||
"steueridentifikation",
|
||||
"bzst",
|
||||
"bundeszentralamt für steuern",
|
||||
"finanzamt",
|
||||
"invoice",
|
||||
"rechnung",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
patterns: Optional[List[Pattern]] = None,
|
||||
context: Optional[List[str]] = None,
|
||||
supported_language: str = "de",
|
||||
supported_entity: str = "DE_VAT_ID",
|
||||
name: Optional[str] = None,
|
||||
):
|
||||
patterns = patterns if patterns else self.PATTERNS
|
||||
context = context if context else self.CONTEXT
|
||||
super().__init__(
|
||||
supported_entity=supported_entity,
|
||||
patterns=patterns,
|
||||
context=context,
|
||||
supported_language=supported_language,
|
||||
name=name,
|
||||
)
|
||||
55
presidio-analyzer/tests/test_de_bsnr_recognizer.py
Normal file
55
presidio-analyzer/tests/test_de_bsnr_recognizer.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""
|
||||
Tests for DeBsnrRecognizer (Betriebsstättennummer / BSNR).
|
||||
|
||||
Format: 9 digits — 2-digit KV regional code + 7 sequential digits assigned
|
||||
by the KV. No public checksum; detection relies on context words to reach
|
||||
useful confidence.
|
||||
|
||||
Legal basis: § 75 Abs. 7 SGB V; KBV-Richtlinie zur Vergabe der Arzt-,
|
||||
Betriebsstätten-, Praxisnetz- sowie Netzverbundnummern.
|
||||
|
||||
Fictitious example BSNRs:
|
||||
021234568 – KV Hamburg (prefix 02)
|
||||
061789045 – KV Nordrhein (prefix 06)
|
||||
141234567 – KV Berlin (prefix 14)
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DeBsnrRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeBsnrRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_BSNR"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# 9-digit numbers are matched (no validate_result – pattern-only)
|
||||
("021234568", 1),
|
||||
("061789045", 1),
|
||||
("141234567", 1),
|
||||
# In running text
|
||||
("Betriebsstättennummer: 021234568", 1),
|
||||
("BSNR 141234567 der Praxis.", 1),
|
||||
# Too short (8 digits) – word boundary prevents match
|
||||
("02123456", 0),
|
||||
# Too long (10 digits) – word boundary prevents match
|
||||
("0212345689", 0),
|
||||
# Non-numeric
|
||||
("02123456A", 0),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_bsnr_numbers_then_succeed(
|
||||
text, expected_len, recognizer, entities
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
73
presidio-analyzer/tests/test_de_fuehrerschein_recognizer.py
Normal file
73
presidio-analyzer/tests/test_de_fuehrerschein_recognizer.py
Normal file
@@ -0,0 +1,73 @@
|
||||
"""
|
||||
Tests for DeFuehrerscheinRecognizer (Führerscheinnummer).
|
||||
|
||||
Post-2013 EU-harmonized format (FeV Anlage 8, EU Directive 2006/126/EC):
|
||||
11 characters: 2 uppercase letters (Behördenkürzel, derived from the
|
||||
Kfz-Zulassungskürzel of the issuing Kreis/Stadt, e.g. HH for
|
||||
Hamburg, MU for Mühldorf/München, BO for Bochum, KO for Koblenz)
|
||||
+ 8 digits (3-digit authority number + 5-digit sequential number)
|
||||
+ 1 check character (uppercase letter or digit; algorithm not published)
|
||||
|
||||
Note: single-letter Kfz abbreviations (B Berlin, K Köln, M München as single chars)
|
||||
are always used in combination as 2-char authority codes in the Führerschein system
|
||||
(e.g. "BO", "MU", "KN"). Pure single-letter + digit combos like "B0" are NOT part
|
||||
of the 2-letter authority code and are out of scope for this strict pattern.
|
||||
|
||||
Pre-2013 formats (locally defined, non-standardized) are out of scope.
|
||||
|
||||
Fictitious examples following the 11-character structure:
|
||||
BO12345678A – issuing authority "BO" (Bochum), authority-nr 123, sequential 45678
|
||||
MU12345678B – issuing authority "MU" (Mühldorf), authority-nr 123, sequential 45678
|
||||
HH98765432C – issuing authority "HH" (Hamburg)
|
||||
KO12345678X – issuing authority "KO" (Koblenz)
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DeFuehrerscheinRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeFuehrerscheinRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_FUEHRERSCHEIN"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# Valid post-2013 format: 2 letters + 8 digits + 1 alphanumeric
|
||||
("BO12345678A", 1),
|
||||
("MU12345678B", 1),
|
||||
("HH98765432C", 1),
|
||||
("KO12345678X", 1),
|
||||
("DO98765432Z", 1),
|
||||
# Check character can be a digit too
|
||||
("GE123456780", 1),
|
||||
("MU123456785", 1),
|
||||
# In running text
|
||||
("Führerscheinnummer: BO12345678A", 1),
|
||||
("Fahrerlaubnis MU12345678B wurde ausgestellt.", 1),
|
||||
# Lowercase: Presidio uses global IGNORECASE → also matches
|
||||
("mu12345678b", 1),
|
||||
# --- Should NOT match ---
|
||||
# Too short: 10 chars (missing check character)
|
||||
("BO12345678", 0),
|
||||
# Too long: 12 chars
|
||||
("BO12345678AB", 0),
|
||||
# Starts with digit instead of two letters
|
||||
("12345678901", 0),
|
||||
# Only one leading letter (3-letter forms are not part of strict pattern)
|
||||
("B12345678A", 0),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_fuehrerschein_numbers_then_succeed(
|
||||
text, expected_len, recognizer, entities
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
@@ -0,0 +1,54 @@
|
||||
"""
|
||||
Tests for DeHandelsregisterRecognizer (Handelsregisternummer).
|
||||
|
||||
The HR[AB] prefix makes the pattern highly specific, keeping false positives low.
|
||||
Legal basis: §§ 9, 14 HGB; DSGVO Art. 4 Nr. 1 for HRA entries (sole traders).
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DeHandelsregisterRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeHandelsregisterRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_HANDELSREGISTER"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# --- HRB (corporations) ---
|
||||
("HRB 123456", 1),
|
||||
("HRB 1", 1), # minimum 1 digit
|
||||
("HRB123456", 1), # no space variant
|
||||
# --- HRA (sole traders / partnerships) ---
|
||||
("HRA 12345", 1),
|
||||
("HRA12345", 1),
|
||||
# In running text
|
||||
("Amtsgericht München HRB 12345.", 1),
|
||||
("eingetragen im HRA 99999 Köln", 1),
|
||||
("Handelsregisternummer: HRB 123456", 1),
|
||||
# 6-digit maximum
|
||||
("HRB 999999", 1),
|
||||
# --- Should NOT match ---
|
||||
# Wrong section letter
|
||||
("HRC 12345", 0),
|
||||
("HR 12345", 0),
|
||||
# Too many digits (> 6)
|
||||
("HRB 1234567", 0),
|
||||
# Lowercase: global IGNORECASE → also matches
|
||||
("hrb 12345", 1),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_handelsregister_numbers_then_succeed(
|
||||
text, expected_len, recognizer, entities
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
@@ -0,0 +1,90 @@
|
||||
"""
|
||||
Tests for DeHealthInsuranceRecognizer (Krankenversicherungsnummer / KVNR).
|
||||
|
||||
Format: 10 characters – 1 uppercase letter (birth surname initial) +
|
||||
8 digits (birth date + serial) + 1 check digit.
|
||||
|
||||
Valid numbers are generated with the GKV-Spitzenverband checksum algorithm
|
||||
(letter expanded to 2-digit ordinal, weights [2,9,8,7,6,5,4,3,2,1],
|
||||
products digit-summed if ≥ 10, sum mod 10).
|
||||
|
||||
Legal basis: § 290 SGB V. DSGVO Art. 9 (Gesundheitsdaten).
|
||||
|
||||
Pre-calculated valid examples (fictitious):
|
||||
A123456787 – letter A (=01), data 12345678, check = 7
|
||||
M123456789 – letter M (=13), data 12345678, check = 9
|
||||
B123456787 – letter B (=02), data 12345678, check = 7
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from tests import assert_result
|
||||
from presidio_analyzer.predefined_recognizers import DeHealthInsuranceRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeHealthInsuranceRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_HEALTH_INSURANCE"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len, expected_positions",
|
||||
[
|
||||
# fmt: off
|
||||
# Valid KVNR – checksum passes → result at MAX_SCORE
|
||||
("A123456787", 1, ((0, 10),)),
|
||||
("M123456789", 1, ((0, 10),)),
|
||||
("B123456787", 1, ((0, 10),)),
|
||||
("Krankenkasse KVNR: A123456787", 1, ((19, 29),)),
|
||||
("eGK-Nummer M123456789 bitte angeben.", 1, ((11, 21),)),
|
||||
# Invalid: wrong check digit
|
||||
("A123456780", 0, ()),
|
||||
("M123456781", 0, ()),
|
||||
# Invalid: starts with digit instead of letter
|
||||
("1123456787", 0, ()),
|
||||
# Lowercase: Presidio uses global IGNORECASE; validate_result calls .upper()
|
||||
# so a valid lowercase KVNR is matched and validated correctly
|
||||
("a123456787", 1, ((0, 10),)),
|
||||
# Too short (9 chars)
|
||||
("A12345678", 0, ()),
|
||||
# Too long (11 chars)
|
||||
("A1234567890", 0, ()),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_health_insurance_numbers_then_succeed(
|
||||
text, expected_len, expected_positions, recognizer, entities, max_score
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
for res, (st_pos, fn_pos) in zip(results, expected_positions):
|
||||
assert_result(res, entities[0], st_pos, fn_pos, max_score)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"number, expected",
|
||||
[
|
||||
# Valid
|
||||
("A123456787", True),
|
||||
("M123456789", True),
|
||||
("B123456787", True),
|
||||
# Wrong check digit
|
||||
("A123456780", False),
|
||||
("M123456781", False),
|
||||
# Starts with digit (after .upper() it's still a digit → re.match fails)
|
||||
("1123456787", False),
|
||||
# Wrong length
|
||||
("A12345678", False),
|
||||
("A1234567890", False),
|
||||
# Lowercase: .upper() converts to valid → True (IGNORECASE is global)
|
||||
("a123456787", True),
|
||||
],
|
||||
)
|
||||
def test_when_de_health_insurance_validated_then_checksum_result_is_correct(
|
||||
number, expected, recognizer
|
||||
):
|
||||
assert recognizer.validate_result(number) == expected
|
||||
52
presidio-analyzer/tests/test_de_id_card_recognizer.py
Normal file
52
presidio-analyzer/tests/test_de_id_card_recognizer.py
Normal file
@@ -0,0 +1,52 @@
|
||||
"""
|
||||
Tests for DeIdCardRecognizer (Personalausweisnummer).
|
||||
|
||||
Covers both the old format (T + 8 digits, pre-November 2010) and the
|
||||
new nPA format (9 ICAO-compliant alphanumeric characters, since Nov 2010).
|
||||
Legal basis: Personalausweisgesetz (PAuswG), Personalausweisverordnung (PAuswV).
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DeIdCardRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeIdCardRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_ID_CARD"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# --- Old format: T + 8 digits (high confidence 0.5) ---
|
||||
("T22000129", 1),
|
||||
("T00000000", 1),
|
||||
("T99999999", 1),
|
||||
("Ausweis Nr. T22000129 gültig bis 2025.", 1),
|
||||
# --- nPA format: ICAO letter + 8 ICAO chars ---
|
||||
("L01X00T47", 1), # starts with L (valid ICAO)
|
||||
("C01234567", 1), # starts with C
|
||||
# In running text
|
||||
("Personalausweis: L01X00T47.", 1),
|
||||
# --- Invalid cases ---
|
||||
# Lowercase: global IGNORECASE → also matches
|
||||
("t22000129", 1),
|
||||
("l01x00t47", 1),
|
||||
# Too short (8 chars)
|
||||
("T2200012", 0),
|
||||
# Too long for any pattern (10+ chars without word boundary break)
|
||||
("T220001290", 0),
|
||||
# Digits only with wrong length
|
||||
("123456789", 0),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_id_cards_then_succeed(text, expected_len, recognizer, entities):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
59
presidio-analyzer/tests/test_de_kfz_recognizer.py
Normal file
59
presidio-analyzer/tests/test_de_kfz_recognizer.py
Normal file
@@ -0,0 +1,59 @@
|
||||
"""
|
||||
Tests for DeKfzRecognizer (KFZ-Kennzeichen / vehicle registration plates).
|
||||
|
||||
Format: [1–3 letter district code] [space or hyphen] [1–2 letter identifier]
|
||||
[space or hyphen] [1–4 digits] [optional E (electric) or H (historic) suffix].
|
||||
|
||||
Legal basis: Fahrzeug-Zulassungsverordnung (FZV) § 8.
|
||||
Data protection: DSGVO Art. 4 Nr. 1 (ECJ C-582/14, Breyer v. Germany).
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DeKfzRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeKfzRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_KFZ"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# --- Space-separated ---
|
||||
("B AB 1234", 1), # Berlin
|
||||
("M XY 999", 1), # München
|
||||
("HH AB 1234", 1), # Hamburg (2-letter district)
|
||||
("KA EF 12H", 1), # Karlsruhe, historic suffix
|
||||
("S AB 12E", 1), # Stuttgart, electric suffix
|
||||
("MIL E 1234", 1), # single-letter identifier
|
||||
("MIL EF 1234E", 1), # electric with 2-letter identifier
|
||||
# --- Hyphen-separated ---
|
||||
("B-AB-1234", 1),
|
||||
("M-XY-999", 1),
|
||||
("HH-AB-1234", 1),
|
||||
# --- In running text ---
|
||||
("Das Fahrzeug mit Kennzeichen B AB 1234 wurde gesehen.", 1),
|
||||
("Kennzeichen: HH-AB-1234.", 1),
|
||||
# --- Should NOT match ---
|
||||
# Lowercase: global IGNORECASE → also matches
|
||||
("b ab 1234", 1),
|
||||
("m xy 999", 1),
|
||||
# No separator
|
||||
("BAB1234", 0),
|
||||
# Only district code + digits, no letter identifier
|
||||
("B 1234", 0),
|
||||
# Four-letter district code (not valid in Germany)
|
||||
("BXYZ AB 1234", 0),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_kfz_numbers_then_succeed(text, expected_len, recognizer, entities):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
90
presidio-analyzer/tests/test_de_lanr_recognizer.py
Normal file
90
presidio-analyzer/tests/test_de_lanr_recognizer.py
Normal file
@@ -0,0 +1,90 @@
|
||||
"""
|
||||
Tests for DeLanrRecognizer (Lebenslange Arztnummer / LANR).
|
||||
|
||||
Format: 9 digits — 6-digit physician identifier + 1 check digit + 2-digit
|
||||
specialty code. Check digit derived via KBV algorithm: weights [4,9,2,10,5,3]
|
||||
on digits 1–6; cross-sum for products > 9; check = sum mod 10.
|
||||
|
||||
Legal basis: § 75 Abs. 7 SGB V; KBV-Richtlinie zur Vergabe der Arzt-,
|
||||
Betriebsstätten-, Praxisnetz- sowie Netzverbundnummern.
|
||||
|
||||
Pre-calculated valid examples (fictitious):
|
||||
123456901 – physician 123456, check 9, specialty 01
|
||||
234567601 – physician 234567, check 6, specialty 01
|
||||
100000401 – physician 100000, check 4, specialty 01
|
||||
987654901 – physician 987654, check 9, specialty 01
|
||||
555555001 – physician 555555, check 0, specialty 01
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from tests import assert_result
|
||||
from presidio_analyzer.predefined_recognizers import DeLanrRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeLanrRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_LANR"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len, expected_positions",
|
||||
[
|
||||
# fmt: off
|
||||
# Valid LANR – checksum passes → result at MAX_SCORE
|
||||
("123456901", 1, ((0, 9),)),
|
||||
("234567601", 1, ((0, 9),)),
|
||||
("100000401", 1, ((0, 9),)),
|
||||
("987654901", 1, ((0, 9),)),
|
||||
("555555001", 1, ((0, 9),)),
|
||||
# Valid LANR in running text
|
||||
("LANR: 123456901 des behandelnden Arztes.", 1, ((6, 15),)),
|
||||
("Arztnummer 987654901 auf dem Rezept.", 1, ((11, 20),)),
|
||||
# Invalid: wrong check digit
|
||||
("123456801", 0, ()),
|
||||
("234567001", 0, ()),
|
||||
("100000001", 0, ()),
|
||||
# Too short (8 digits) – word boundary prevents match
|
||||
("12345690", 0, ()),
|
||||
# Too long (10 digits) – word boundary prevents match
|
||||
("1234569010", 0, ()),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_lanr_numbers_then_succeed(
|
||||
text, expected_len, expected_positions, recognizer, entities, max_score
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
for res, (st_pos, fn_pos) in zip(results, expected_positions):
|
||||
assert_result(res, entities[0], st_pos, fn_pos, max_score)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"number, expected",
|
||||
[
|
||||
# Valid check digit
|
||||
("123456901", True),
|
||||
("234567601", True),
|
||||
("100000401", True),
|
||||
("987654901", True),
|
||||
("555555001", True),
|
||||
# Wrong check digit
|
||||
("123456801", False),
|
||||
("234567001", False),
|
||||
("100000101", False),
|
||||
# Wrong length
|
||||
("12345690", False),
|
||||
("1234569010", False),
|
||||
# Non-numeric
|
||||
("12345690a", False),
|
||||
],
|
||||
)
|
||||
def test_when_de_lanr_validated_then_checksum_result_is_correct(
|
||||
number, expected, recognizer
|
||||
):
|
||||
assert recognizer.validate_result(number) == expected
|
||||
54
presidio-analyzer/tests/test_de_passport_recognizer.py
Normal file
54
presidio-analyzer/tests/test_de_passport_recognizer.py
Normal file
@@ -0,0 +1,54 @@
|
||||
"""
|
||||
Tests for DePassportRecognizer (Reisepassnummer).
|
||||
|
||||
German passport numbers follow ICAO Doc 9303: 9 alphanumeric characters
|
||||
using a restricted uppercase character set (excludes I, O, Q, S, U).
|
||||
Legal basis: Passgesetz (PassG) § 4, Passverordnung (PassV).
|
||||
|
||||
Note: Presidio applies global regex_flags=26 (IGNORECASE|MULTILINE|DOTALL),
|
||||
so lowercase inputs also match the patterns.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DePassportRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DePassportRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_PASSPORT"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# --- Strict ICAO charset pattern (1 ICAO-letter + 7 ICAO-chars + 1 digit) ---
|
||||
("C01234567", 1), # starts with C, ends with digit
|
||||
("F12345678", 1),
|
||||
# L01X00T47: L ∈ ICAO set, all inner chars valid, ends with digit 7 → matches
|
||||
("L01X00T47", 1),
|
||||
# --- Relaxed pattern (any letter + 7 alphanumeric + 1 digit) ---
|
||||
("A01234567", 1),
|
||||
("Z98765432", 1),
|
||||
# In running text
|
||||
("Reisepass C01234567 ausgestellt am 01.01.2020.", 1),
|
||||
("Pass-Nr.: F12345678", 1),
|
||||
# Too short (8 chars)
|
||||
("C0123456", 0),
|
||||
# Too long (10 chars) → word boundary prevents match
|
||||
("C012345678", 0),
|
||||
# Lowercase: global IGNORECASE → also matches
|
||||
("c01234567", 1),
|
||||
# Digits only → no match (first char must be letter)
|
||||
("901234567", 0),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_passports_then_succeed(text, expected_len, recognizer, entities):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
73
presidio-analyzer/tests/test_de_plz_recognizer.py
Normal file
73
presidio-analyzer/tests/test_de_plz_recognizer.py
Normal file
@@ -0,0 +1,73 @@
|
||||
"""
|
||||
Tests for DePlzRecognizer (Postleitzahl).
|
||||
|
||||
IMPORTANT: The base confidence is 0.05 due to the extreme false-positive risk of
|
||||
matching any 5-digit number. These tests verify the regex structure (valid range,
|
||||
boundary matching) only. In production, context words are required for the
|
||||
recognizer to produce actionable (high-confidence) results.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DePlzRecognizer
|
||||
from tests import assert_result_within_score_range
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DePlzRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_PLZ"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# --- Valid PLZ: range 01001–99998 ---
|
||||
("10115", 1), # Berlin Mitte
|
||||
("80331", 1), # München
|
||||
("22085", 1), # Hamburg
|
||||
("01001", 1), # minimum valid (leading zero)
|
||||
("99998", 1), # near-maximum
|
||||
# In running text (low confidence, context-free)
|
||||
("PLZ: 10115", 1),
|
||||
("Postleitzahl 80331 München", 1),
|
||||
# --- Should NOT match ---
|
||||
# 00000 is not a valid PLZ (excluded by regex)
|
||||
("00000", 0),
|
||||
# 01000 and 99999 are boundary values outside the valid range
|
||||
("01000", 0),
|
||||
("99999", 0),
|
||||
# 6 digits → no match (word boundary)
|
||||
("101150", 0),
|
||||
# 4 digits
|
||||
("1011", 0),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_plz_then_succeed(text, expected_len, recognizer, entities):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len, expected_start, expected_end",
|
||||
[
|
||||
# fmt: off
|
||||
("10115", 1, 0, 5),
|
||||
("PLZ 80331", 1, 4, 9),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_de_plz_matched_then_position_is_correct(
|
||||
text, expected_len, expected_start, expected_end, recognizer, entities
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
if expected_len > 0:
|
||||
assert_result_within_score_range(
|
||||
results[0], entities[0], expected_start, expected_end, 0.0, 0.5
|
||||
)
|
||||
@@ -0,0 +1,77 @@
|
||||
"""
|
||||
Tests for DeSocialSecurityRecognizer (Rentenversicherungsnummer / RVNR).
|
||||
|
||||
Format: 12 characters – 8 digits (area + birth date) + 1 uppercase letter
|
||||
(birth surname initial) + 2 digits (serial) + 1 check digit.
|
||||
|
||||
Valid numbers are generated with the official Deutsche Rentenversicherung
|
||||
checksum algorithm (letter expanded to 2-digit ordinal, weights [2,1,...],
|
||||
products digit-summed if ≥ 10, sum mod 10).
|
||||
|
||||
Legal basis: § 147 SGB VI.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from tests import assert_result
|
||||
from presidio_analyzer.predefined_recognizers import DeSocialSecurityRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeSocialSecurityRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_SOCIAL_SECURITY"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len, expected_positions",
|
||||
[
|
||||
# fmt: off
|
||||
# Valid RVNR – checksum passes → result at MAX_SCORE
|
||||
("65070803A018", 1, ((0, 12),)),
|
||||
("RVNR: 65070803A018 laut Sozialversicherungsausweis.", 1, ((6, 18),)),
|
||||
# Invalid: wrong check digit
|
||||
("65070803A012", 0, ()),
|
||||
("65070803A010", 0, ()),
|
||||
# Invalid: invalid month (00 or 13+)
|
||||
("65070003A018", 0, ()),
|
||||
("65071303A018", 0, ()),
|
||||
# Invalid: digit at position 9 instead of letter
|
||||
("650708030018", 0, ()),
|
||||
# Too short / too long
|
||||
("65070803A01", 0, ()),
|
||||
("65070803A0180", 0, ()),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_social_security_numbers_then_succeed(
|
||||
text, expected_len, expected_positions, recognizer, entities, max_score
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
for res, (st_pos, fn_pos) in zip(results, expected_positions):
|
||||
assert_result(res, entities[0], st_pos, fn_pos, max_score)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"number, expected",
|
||||
[
|
||||
# Valid
|
||||
("65070803A018", True),
|
||||
# Wrong check digit
|
||||
("65070803A012", False),
|
||||
("65070803A010", False),
|
||||
# Digit instead of letter at position 9
|
||||
("650708030018", False),
|
||||
# Wrong length
|
||||
("65070803A01", False),
|
||||
("65070803A0180", False),
|
||||
],
|
||||
)
|
||||
def test_when_de_social_security_validated_then_checksum_result_is_correct(
|
||||
number, expected, recognizer
|
||||
):
|
||||
assert recognizer.validate_result(number) == expected
|
||||
76
presidio-analyzer/tests/test_de_tax_id_recognizer.py
Normal file
76
presidio-analyzer/tests/test_de_tax_id_recognizer.py
Normal file
@@ -0,0 +1,76 @@
|
||||
"""
|
||||
Tests for DeTaxIdRecognizer (Steueridentifikationsnummer).
|
||||
|
||||
All test numbers are fictitious/generated and do not represent real persons.
|
||||
Valid numbers are produced with the official ISO 7064 Mod 11, 10 algorithm
|
||||
as specified by the Bundeszentralamt für Steuern (§§ 139a–139e AO).
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from tests import assert_result
|
||||
from presidio_analyzer.predefined_recognizers import DeTaxIdRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeTaxIdRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_TAX_ID"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len, expected_positions",
|
||||
[
|
||||
# fmt: off
|
||||
# Valid Steuer-IdNr – checksum passes → result at MAX_SCORE
|
||||
("12345678903", 1, ((0, 11),)),
|
||||
("98765432106", 1, ((0, 11),)),
|
||||
("Meine Steuer-ID: 12345678903.", 1, ((17, 28),)),
|
||||
("IdNr. 98765432106 liegt vor.", 1, ((6, 17),)),
|
||||
# Invalid: wrong check digit
|
||||
("12345678901", 0, ()),
|
||||
("98765432100", 0, ()),
|
||||
# Invalid: leading zero (first digit must be 1–9)
|
||||
("02345678901", 0, ()),
|
||||
# Invalid: too short / too long
|
||||
("1234567890", 0, ()),
|
||||
("123456789030", 0, ()),
|
||||
# Invalid: all ten leading digits are identical (excluded by spec)
|
||||
("11111111111", 0, ()),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_tax_ids_then_succeed(
|
||||
text, expected_len, expected_positions, recognizer, entities, max_score
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
for res, (st_pos, fn_pos) in zip(results, expected_positions):
|
||||
assert_result(res, entities[0], st_pos, fn_pos, max_score)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"number, expected",
|
||||
[
|
||||
# Valid numbers
|
||||
("12345678903", True),
|
||||
("98765432106", True),
|
||||
# Wrong check digit
|
||||
("12345678901", False),
|
||||
("98765432100", False),
|
||||
# Leading zero
|
||||
("02345678903", False),
|
||||
# Non-numeric
|
||||
("abcdefghijk", False),
|
||||
# Wrong length
|
||||
("1234567890", False),
|
||||
("123456789030", False),
|
||||
# All same first 10 digits
|
||||
("11111111111", False),
|
||||
],
|
||||
)
|
||||
def test_when_de_tax_id_validated_then_checksum_result_is_correct(number, expected, recognizer):
|
||||
assert recognizer.validate_result(number) == expected
|
||||
51
presidio-analyzer/tests/test_de_tax_number_recognizer.py
Normal file
51
presidio-analyzer/tests/test_de_tax_number_recognizer.py
Normal file
@@ -0,0 +1,51 @@
|
||||
"""
|
||||
Tests for DeTaxNumberRecognizer (Steuernummer).
|
||||
|
||||
Tests cover both the ELSTER unified 13-digit format and the various
|
||||
state-specific slash-separated formats used across German Bundesländer.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DeTaxNumberRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeTaxNumberRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_TAX_NUMBER"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# --- ELSTER unified 13-digit format (valid Bundesland codes 01–16) ---
|
||||
("0281508150123", 1), # Hamburg (02)
|
||||
("0981508150999", 1), # Bayern (09)
|
||||
("1681508150001", 1), # Thüringen (16)
|
||||
("0181508150000", 1), # Schleswig-Holstein (01)
|
||||
# Invalid Bundesland codes
|
||||
("1781508150001", 0), # code 17 does not exist
|
||||
("0081508150001", 0), # code 00 does not exist
|
||||
# Too short for 13-digit pattern
|
||||
("028150815012", 0),
|
||||
# --- Slash-separated Bayern-style (3/3/5) ---
|
||||
("123/456/78901", 1),
|
||||
("987/654/32100", 1),
|
||||
# --- General slash-separated (2-3 / 3-4 / 4-5 digits) ---
|
||||
("12/345/6789", 1),
|
||||
("12/3456/7890", 1),
|
||||
("123/3456/7890", 1),
|
||||
# Matches in running text
|
||||
("Steuernummer: 0981508150999 wurde vergeben.", 1),
|
||||
("St.-Nr. 123/456/78901 bitte angeben.", 1),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_tax_numbers_then_succeed(text, expected_len, recognizer, entities):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
55
presidio-analyzer/tests/test_de_vat_id_recognizer.py
Normal file
55
presidio-analyzer/tests/test_de_vat_id_recognizer.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""
|
||||
Tests for DeVatIdRecognizer (Umsatzsteuer-Identifikationsnummer / USt-IdNr.).
|
||||
|
||||
Format: "DE" + 9 digits (11 characters total).
|
||||
|
||||
Legal basis: § 27a UStG. Format documentation: BZSt.
|
||||
|
||||
Fictitious examples:
|
||||
DE123456789
|
||||
DE987654321
|
||||
DE100000001
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from presidio_analyzer.predefined_recognizers import DeVatIdRecognizer
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def recognizer():
|
||||
return DeVatIdRecognizer()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def entities():
|
||||
return ["DE_VAT_ID"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text, expected_len",
|
||||
[
|
||||
# fmt: off
|
||||
# Valid format: DE + 9 digits
|
||||
("DE123456789", 1),
|
||||
("DE987654321", 1),
|
||||
("DE100000001", 1),
|
||||
# In running text
|
||||
("USt-IdNr.: DE123456789", 1),
|
||||
("Bitte angeben: DE987654321 auf der Rechnung.", 1),
|
||||
# Wrong country prefix – must not match
|
||||
("AT123456789", 0),
|
||||
("FR12345678901", 0),
|
||||
# Lowercase prefix – Presidio uses global IGNORECASE
|
||||
("de123456789", 1),
|
||||
# Too few digits (8)
|
||||
("DE12345678", 0),
|
||||
# Too many digits (10)
|
||||
("DE1234567890", 0),
|
||||
# fmt: on
|
||||
],
|
||||
)
|
||||
def test_when_all_de_vat_ids_then_succeed(
|
||||
text, expected_len, recognizer, entities
|
||||
):
|
||||
results = recognizer.analyze(text, entities)
|
||||
assert len(results) == expected_len
|
||||
Reference in New Issue
Block a user