mirror of
https://github.com/data-privacy-stack/presidio.git
synced 2026-09-21 13:38:05 -05:00
채희원andOmri Mendels
07ba0344fd
* fix(analyzer): remove a non-language code and a duplicate entry from default_recognizers.yaml The five Korean entries listed `kr` next to `ko`. `ko` is the ISO 639-1 language code; `kr` is the ISO 3166-1 country code, which the entries already carry in `country_code`. The loader builds one recognizer per listed language, so `kr` either logged a "language is not supported" warning on every load, or - for a registry configured with it - registered a second copy of all five Korean recognizers that no NLP engine can serve. UkPostcodeRecognizer was listed twice: #1858 added it, and #1857 added it again and merged later. The registry built two identical instances. Generated-by: Claude Opus 5 * test(analyzer): guard default_recognizers.yaml against duplicates and non-language codes Both tests fail on main and pass with the config fix. The language guard keeps an explicit set of ISO 639-1 codes so that adding a language is a deliberate edit rather than a side effect of a new entry. Generated-by: Claude Opus 5 * test(analyzer): stop asserting kr as a servable language for KrPassport #2170 parametrised this over ["ko", "kr"] to match the four sibling Kr* entries. `kr` is the ISO 3166-1 country code the entry already carries in `country_code`, and no NLP engine can serve it, so the `kr` case asserted the defect rather than the contract. Generated-by: Claude Opus 5 * test(analyzer): read YAML entries the way the loader does Both guards indexed every entry as a dict. `RecognizerListLoader` also accepts bare-string entries such as `- SpacyRecognizer`, so one of those in the shipped file would have crashed the guards with a `TypeError` instead of failing on the assertion they exist to make. Names now go through `_entry_name`, which prefers the instance `name` and falls back to `RecognizerListLoader.get_recognizer_name` for a bare string or a class-only entry. The duplicate check stays keyed on the instance name rather than the class: two entries may share a class as long as `class_name` carries it and `name` tells the instances apart. Languages go through `_declared_languages`, which covers all three shapes `RecognizerListLoader._get_recognizer_languages` reads -- absent or `None`, a list of codes, and a list of `{language, context}` mappings -- rather than only the one the shipped file happens to use today. Generated-by: Claude Opus 5 * fix(analyzer): keep the kr alias on KrRrnRecognizer and KrPassportRecognizer Per review, `kr` goes only from the three entries that can have no legacy users. `KrDriverLicenseRecognizer`, `KrBrnRecognizer` and `KrFrnRecognizer` have only ever defaulted to `supported_language="ko"`, so no configuration can depend on `kr` for them. `KrRrnRecognizer` and `KrPassportRecognizer` are a deprecation question instead: both defaulted to `kr` until it was moved to `ko` -- #1742 (2025-10-08) and #2170 (2026-08-05) respectively -- so a registry configured against the old default would go quiet if the alias were dropped now. They keep `kr`, and the removal is deferred to a later release. The language guard gains a carve-out naming those two entries, why the alias is there, and the release it goes away in. That also restores the `kr` case of `test_loads_from_default_recognizers_yaml`, which now asserts a documented alias rather than the defect it was reverted for. Generated-by: Claude Opus 5 * test(analyzer): assert the deprecated kr alias still loads its recognizers The carve-out in the language guard records why `kr` survives on two entries but does not prove the reason is real. This builds a registry through `RecognizerRegistryProvider` with `supported_languages: ["kr"]` and the Korean entries enabled, and asserts that exactly `KrRrnRecognizer` and `KrPassportRecognizer` load, both under `kr`. That is the configuration the alias exists for. The loader builds one instance per declared language and `RecognizerListLoader._is_language_supported_globally` then drops every instance whose language the registry does not list, so the entries that declare only `ko` are built and discarded. Remove the alias and this registry loads nothing, which is the breakage the deprecation is staged to avoid. Generated-by: Claude Opus 5 * docs(changelog): note that the kr language code is deprecated Records the user-visible half of the change: `kr` is deprecated in favour of the ISO 639-1 code `ko`, the alias survives on `KrRrnRecognizer` and `KrPassportRecognizer` for registries configured against their original class defaults, and it is gone from the other three entries as of this release. The repository has no migration or upgrade guide -- `docs/project_transition.md` covers the move to Data Privacy Stack and `docs/build_release.md` the release machinery -- so this goes under a new Analyzer/Deprecated heading, which is where the changelog already records this kind of change. Generated-by: Claude Opus 5 * docs(changelog): move the kr deprecation note under Analyzer/Changed Per review, the changelog should not grow a new `Deprecated` heading for this. The note moves verbatim into the Analyzer component's `Changed` section, which did not exist under `[unreleased]` yet and is created here in the Added/Changed/Fixed order the 2.2.363 release already uses. Text is unchanged, `<release TBD>` placeholder included. Generated-by: Claude Opus 5 --------- Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
Presidio - Data Protection and De-identification SDK
📣 Presidio is moving to a new home! Read more here 📣
Context aware, pluggable and customizable PII de-identification service for text and images.
| Component | Downloads | Coverage |
|---|---|---|
| Presidio Analyzer |  |
 |
| Presidio Anonymizer |  |
 |
| Presidio Image-Redactor |  |
 |
| Presidio Structured |  |
 |
What is Presidio
Presidio (Origin from Latin praesidium ‘protection, garrison’) helps to ensure sensitive data is properly managed and governed. It provides fast identification and anonymization modules for private entities in text such as credit card numbers, names, locations, social security numbers, bitcoin wallets, US phone numbers, financial data and more.
📘 Full documentation
📣 Project transition update
❓ Frequently Asked Questions
💭 Demo
🛫 Examples
Goals
- Allow organizations to preserve privacy in a simpler way by democratizing de-identification technologies and introducing transparency in decisions.
- Embrace extensibility and customizability to a specific business need.
- Facilitate both fully automated and semi-automated PII de-identification flows on multiple platforms.
Main features
- Predefined or custom PII recognizers leveraging Named Entity Recognition, regular expressions, rule based logic and checksum with relevant context in multiple languages.
- Options for connecting to external PII detection models.
- Multiple usage options, from Python or PySpark workloads through Docker to Kubernetes.
- Customizability in PII identification and de-identification.
- Module for redacting PII text in images (standard image types and DICOM medical images).
⚠️ Presidio can help identify sensitive/PII data in un/structured text. However, because it is using automated detection mechanisms, there is no guarantee that Presidio will find all sensitive information. Consequently, additional systems and protections should be employed.
Installing Presidio
Running Presidio
- Getting started
- Setting up a development environment
- PII de-identification in text
- PII de-identification in images
- Usage samples and example deployments
Support
- Before you submit an issue, please go over the documentation.
- For general discussions, please use the GitHub repo's discussion board.
- If you have a usage question, found a bug or have a suggestion for improvement, please file a GitHub issue.
- For other matters, please email presidio@dataprivacystack.org.
Contributing
For details on contributing to this repository, see the contributing guide.
This project has adopted the Contributor Covenant Code of Conduct.