Files
채희원andOmri Mendels 07ba0344fd fix(analyzer): remove a non-language code and a duplicate entry from default_recognizers.yaml (#2236)
* fix(analyzer): remove a non-language code and a duplicate entry from default_recognizers.yaml

The five Korean entries listed `kr` next to `ko`. `ko` is the ISO 639-1 language code; `kr` is the ISO 3166-1 country code, which the entries already carry in `country_code`. The loader builds one recognizer per listed language, so `kr` either logged a "language is not supported" warning on every load, or - for a registry configured with it - registered a second copy of all five Korean recognizers that no NLP engine can serve.

UkPostcodeRecognizer was listed twice: #1858 added it, and #1857 added it again and merged later. The registry built two identical instances.

Generated-by: Claude Opus 5

* test(analyzer): guard default_recognizers.yaml against duplicates and non-language codes

Both tests fail on main and pass with the config fix. The language guard keeps an explicit set of ISO 639-1 codes so that adding a language is a deliberate edit rather than a side effect of a new entry.

Generated-by: Claude Opus 5

* test(analyzer): stop asserting kr as a servable language for KrPassport

#2170 parametrised this over ["ko", "kr"] to match the four sibling Kr* entries. `kr` is the ISO 3166-1 country code the entry already carries in `country_code`, and no NLP engine can serve it, so the `kr` case asserted the defect rather than the contract.

Generated-by: Claude Opus 5

* test(analyzer): read YAML entries the way the loader does

Both guards indexed every entry as a dict. `RecognizerListLoader` also
accepts bare-string entries such as `- SpacyRecognizer`, so one of those in
the shipped file would have crashed the guards with a `TypeError` instead of
failing on the assertion they exist to make.

Names now go through `_entry_name`, which prefers the instance `name` and
falls back to `RecognizerListLoader.get_recognizer_name` for a bare string or
a class-only entry. The duplicate check stays keyed on the instance name
rather than the class: two entries may share a class as long as `class_name`
carries it and `name` tells the instances apart.

Languages go through `_declared_languages`, which covers all three shapes
`RecognizerListLoader._get_recognizer_languages` reads -- absent or `None`,
a list of codes, and a list of `{language, context}` mappings -- rather than
only the one the shipped file happens to use today.

Generated-by: Claude Opus 5

* fix(analyzer): keep the kr alias on KrRrnRecognizer and KrPassportRecognizer

Per review, `kr` goes only from the three entries that can have no legacy
users. `KrDriverLicenseRecognizer`, `KrBrnRecognizer` and `KrFrnRecognizer`
have only ever defaulted to `supported_language="ko"`, so no configuration
can depend on `kr` for them.

`KrRrnRecognizer` and `KrPassportRecognizer` are a deprecation question
instead: both defaulted to `kr` until it was moved to `ko` -- #1742
(2025-10-08) and #2170 (2026-08-05) respectively -- so a registry configured
against the old default would go quiet if the alias were dropped now. They
keep `kr`, and the removal is deferred to a later release.

The language guard gains a carve-out naming those two entries, why the alias
is there, and the release it goes away in. That also restores the `kr` case
of `test_loads_from_default_recognizers_yaml`, which now asserts a documented
alias rather than the defect it was reverted for.

Generated-by: Claude Opus 5

* test(analyzer): assert the deprecated kr alias still loads its recognizers

The carve-out in the language guard records why `kr` survives on two entries
but does not prove the reason is real. This builds a registry through
`RecognizerRegistryProvider` with `supported_languages: ["kr"]` and the
Korean entries enabled, and asserts that exactly `KrRrnRecognizer` and
`KrPassportRecognizer` load, both under `kr`.

That is the configuration the alias exists for. The loader builds one
instance per declared language and
`RecognizerListLoader._is_language_supported_globally` then drops every
instance whose language the registry does not list, so the entries that
declare only `ko` are built and discarded. Remove the alias and this registry
loads nothing, which is the breakage the deprecation is staged to avoid.

Generated-by: Claude Opus 5

* docs(changelog): note that the kr language code is deprecated

Records the user-visible half of the change: `kr` is deprecated in favour of
the ISO 639-1 code `ko`, the alias survives on `KrRrnRecognizer` and
`KrPassportRecognizer` for registries configured against their original class
defaults, and it is gone from the other three entries as of this release.

The repository has no migration or upgrade guide -- `docs/project_transition.md`
covers the move to Data Privacy Stack and `docs/build_release.md` the release
machinery -- so this goes under a new Analyzer/Deprecated heading, which is
where the changelog already records this kind of change.

Generated-by: Claude Opus 5

* docs(changelog): move the kr deprecation note under Analyzer/Changed

Per review, the changelog should not grow a new `Deprecated` heading for
this. The note moves verbatim into the Analyzer component's `Changed`
section, which did not exist under `[unreleased]` yet and is created here
in the Added/Changed/Fixed order the 2.2.363 release already uses.

Text is unchanged, `<release TBD>` placeholder included.

Generated-by: Claude Opus 5

---------

Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
2026-09-15 08:59:58 +00:00
..
2026-03-22 14:12:21 +02:00
2021-02-08 14:34:25 +02:00
2026-07-22 10:43:33 +03:00

Presidio analyzer

Description

The Presidio analyzer is a Python based service for detecting PII entities in text.

During analysis, it runs a set of different PII Recognizers, each one in charge of detecting one or more PII entities using different mechanisms.

Presidio analyzer comes with a set of predefined recognizers, but can easily be extended with other types of custom recognizers. Predefined and custom recognizers leverage regex, Named Entity Recognition and other types of logic to detect PII in unstructured text.

Language Model-based PII/PHI Detection

Presidio analyzer supports language model-based PII/PHI detection (LLMs, SLMs) for flexible entity recognition. The current implementation uses LangExtract with support for multiple providers:

  • Ollama - Local model deployment for privacy-sensitive environments
  • Azure OpenAI - Cloud-based deployment with enterprise features
pip install "presidio-analyzer[langextract]"

Quick Usage

Ollama (local models):

from presidio_analyzer.predefined_recognizers import BasicLangExtractRecognizer
recognizer = BasicLangExtractRecognizer()  # Uses default config

Azure OpenAI (cloud models):

from presidio_analyzer.predefined_recognizers import AzureOpenAILangExtractRecognizer

# Simple usage - pass everything as parameters
recognizer = AzureOpenAILangExtractRecognizer(
    model_id="gpt-4",  # Your Azure deployment name
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_key="your-api-key"
)

# Or use environment variables (AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY):
recognizer = AzureOpenAILangExtractRecognizer(
    model_id="gpt-4"  # Your Azure deployment name
)

# Advanced: Customize entities/prompts with config file
recognizer = AzureOpenAILangExtractRecognizer(
    model_id="gpt-4",
    config_path="./custom_config.yaml",  # Optional: for custom entities/prompts
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_key="your-api-key"
)

Note: LangExtract recognizers do not validate connectivity during initialization. Connection errors or missing models will be reported when analyze() is first called.

See the Language Model-based PII/PHI Detection guide for complete setup and usage instructions.

Deploy Presidio analyzer to Azure

Use the following button to deploy presidio analyzer to your Azure subscription.

Deploy to Azure

Simple usage example

from presidio_analyzer import AnalyzerEngine

# Set up the engine, loads the NLP module (spaCy model by default) and other PII recognizers
analyzer = AnalyzerEngine()

# Call analyzer to get results
results = analyzer.analyze(text="My phone number is 212-555-5555",
                           entities=["PHONE_NUMBER"],
                           language='en')
print(results)

GPU Acceleration

For GPU acceleration, install the appropriate dependencies for your hardware:

  • Linux with NVIDIA GPU: cupy-cuda12x (or the version matching your CUDA installation)
  • macOS with Apple Silicon: MPS (Metal Performance Shaders) is currently not supported. The analyzer will use CPU for PyTorch operations.

Documentation

Additional documentation on installation, usage and extending the Analyzer can be found under the Analyzer section of Presidio Documentation