mirror of
https://github.com/data-privacy-stack/presidio.git
synced 2026-09-21 05:27:53 -05:00
채희원andOmri Mendels
07ba0344fd
* fix(analyzer): remove a non-language code and a duplicate entry from default_recognizers.yaml The five Korean entries listed `kr` next to `ko`. `ko` is the ISO 639-1 language code; `kr` is the ISO 3166-1 country code, which the entries already carry in `country_code`. The loader builds one recognizer per listed language, so `kr` either logged a "language is not supported" warning on every load, or - for a registry configured with it - registered a second copy of all five Korean recognizers that no NLP engine can serve. UkPostcodeRecognizer was listed twice: #1858 added it, and #1857 added it again and merged later. The registry built two identical instances. Generated-by: Claude Opus 5 * test(analyzer): guard default_recognizers.yaml against duplicates and non-language codes Both tests fail on main and pass with the config fix. The language guard keeps an explicit set of ISO 639-1 codes so that adding a language is a deliberate edit rather than a side effect of a new entry. Generated-by: Claude Opus 5 * test(analyzer): stop asserting kr as a servable language for KrPassport #2170 parametrised this over ["ko", "kr"] to match the four sibling Kr* entries. `kr` is the ISO 3166-1 country code the entry already carries in `country_code`, and no NLP engine can serve it, so the `kr` case asserted the defect rather than the contract. Generated-by: Claude Opus 5 * test(analyzer): read YAML entries the way the loader does Both guards indexed every entry as a dict. `RecognizerListLoader` also accepts bare-string entries such as `- SpacyRecognizer`, so one of those in the shipped file would have crashed the guards with a `TypeError` instead of failing on the assertion they exist to make. Names now go through `_entry_name`, which prefers the instance `name` and falls back to `RecognizerListLoader.get_recognizer_name` for a bare string or a class-only entry. The duplicate check stays keyed on the instance name rather than the class: two entries may share a class as long as `class_name` carries it and `name` tells the instances apart. Languages go through `_declared_languages`, which covers all three shapes `RecognizerListLoader._get_recognizer_languages` reads -- absent or `None`, a list of codes, and a list of `{language, context}` mappings -- rather than only the one the shipped file happens to use today. Generated-by: Claude Opus 5 * fix(analyzer): keep the kr alias on KrRrnRecognizer and KrPassportRecognizer Per review, `kr` goes only from the three entries that can have no legacy users. `KrDriverLicenseRecognizer`, `KrBrnRecognizer` and `KrFrnRecognizer` have only ever defaulted to `supported_language="ko"`, so no configuration can depend on `kr` for them. `KrRrnRecognizer` and `KrPassportRecognizer` are a deprecation question instead: both defaulted to `kr` until it was moved to `ko` -- #1742 (2025-10-08) and #2170 (2026-08-05) respectively -- so a registry configured against the old default would go quiet if the alias were dropped now. They keep `kr`, and the removal is deferred to a later release. The language guard gains a carve-out naming those two entries, why the alias is there, and the release it goes away in. That also restores the `kr` case of `test_loads_from_default_recognizers_yaml`, which now asserts a documented alias rather than the defect it was reverted for. Generated-by: Claude Opus 5 * test(analyzer): assert the deprecated kr alias still loads its recognizers The carve-out in the language guard records why `kr` survives on two entries but does not prove the reason is real. This builds a registry through `RecognizerRegistryProvider` with `supported_languages: ["kr"]` and the Korean entries enabled, and asserts that exactly `KrRrnRecognizer` and `KrPassportRecognizer` load, both under `kr`. That is the configuration the alias exists for. The loader builds one instance per declared language and `RecognizerListLoader._is_language_supported_globally` then drops every instance whose language the registry does not list, so the entries that declare only `ko` are built and discarded. Remove the alias and this registry loads nothing, which is the breakage the deprecation is staged to avoid. Generated-by: Claude Opus 5 * docs(changelog): note that the kr language code is deprecated Records the user-visible half of the change: `kr` is deprecated in favour of the ISO 639-1 code `ko`, the alias survives on `KrRrnRecognizer` and `KrPassportRecognizer` for registries configured against their original class defaults, and it is gone from the other three entries as of this release. The repository has no migration or upgrade guide -- `docs/project_transition.md` covers the move to Data Privacy Stack and `docs/build_release.md` the release machinery -- so this goes under a new Analyzer/Deprecated heading, which is where the changelog already records this kind of change. Generated-by: Claude Opus 5 * docs(changelog): move the kr deprecation note under Analyzer/Changed Per review, the changelog should not grow a new `Deprecated` heading for this. The note moves verbatim into the Analyzer component's `Changed` section, which did not exist under `[unreleased]` yet and is created here in the Added/Changed/Fixed order the 2.2.363 release already uses. Text is unchanged, `<release TBD>` placeholder included. Generated-by: Claude Opus 5 --------- Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>