* Add LangExtract recognizer for PII extraction
- Introduced LangExtract recognizer to enhance PII detection capabilities.
- Added configuration files for LangExtract prompts and examples.
- Implemented LangExtractRecognizer class to handle PII extraction using LangExtract.
- Created tests for LangExtract recognizer to ensure functionality and reliability.
- Added a simple standalone test script for quick validation of LangExtract setup.
- Updated pyproject.toml to include langextract as a dependency.
* refine the docs
* narrow support for oollama only
* Refactor LangExtract tests to use Ollama; remove API key dependency
* adding first draft of docker compose
* Update model_id in tests to use 'gemma2:2b' instead of 'gemini-2.5-flash'
* Refactor LangExtract documentation to focus on Ollama support; remove references to other LLM providers
* Update README to remove Ollama setup instructions and clarify integration guide reference
* Enhance Ollama installation script with progress messages and error handling; update model download method for better user feedback
* auto ruff fixes
* Enhance LangExtractRecognizer tests with real Ollama integration
- Updated `langextract_recognizer_class` fixture to create a test-specific configuration for LangExtractRecognizer, enabling it for testing.
- Refactored tests in `test_langextract_recognizer.py` to utilize the new configuration and validate the recognizer's behavior with real Ollama.
- Removed mock-based tests for LangExtract and replaced them with integration tests that check the recognizer's functionality against a running Ollama instance.
- Added tests to verify the recognizer's initialization, entity detection, and error handling when the Ollama server is unreachable.
- Ensured that only requested entities are returned and that results include analysis explanations.
* Remove unnecessary line breaks in LLM-based PII detection section of README
* Add LangExtract LLM-based PII detection test and configuration
* Improve Ollama availability check with setup attempt message
* Increase wait time for services and update healthcheck parameters for Ollama service
* Add Ollama setup for Analyzer tests and improve availability check
* Set timeout for Ollama setup in Analyzer tests to 8 minutes
* Enhance Ollama setup for Analyzer tests with improved installation and readiness checks
* Update Ollama model references from gemma2:2b to llama3.2:1b across configuration and test files
* Update model references from llama3.2:1b to gemma2:2b across configuration, scripts, and tests
* Remove 'enabled' configuration from LangExtract settings in YAML files and update tests accordingly
* Update Ollama service configuration: change port mapping and modify healthcheck command
* Refactor logging in LangExtractRecognizer: reduce verbosity and improve clarity of extraction results
* Update Ollama service configuration: modify port mapping and healthcheck command
* Update CI workflow and tests: reduce sleep duration and add environment variable for LangExtract recognizer
* Reduce sleep duration in CI workflow from 150 to 60 seconds
* Update LangExtract model references from gemma2:2b to gemma3:1b and remove obsolete installation script
* docs and prompt fixes
* finalizing the pr
* Update Ollama image to latest version and add LangExtract PII/PHI extraction examples and prompts
* fix bad example
* fix unit-tests
* refactor: clean up .env file and simplify skip_engine logic in tests
* chore: add a new line to .env file for better readability
* chore: remove unnecessary blank line from .env file
* revert .env
* intial commit
* Remove skip marker for spacy_nlp_engine fixture
* Remove skip markers for stanza and transformers NLP engine fixtures
* Remove Ollama recognizer test and update default recognizers configuration
* Remove unused Ollama recognizer configuration and update prompt file references
* Add end-to-end tests for API anonymization and redaction features
- Implemented tests for the anonymization API in `test_api_anonymizer.py`, covering various scenarios including valid requests, empty inputs, malformed requests, and custom anonymizers.
- Created integration tests in `test_api_e2e_integration_flows.py` to validate the analyze and anonymize workflow with PII detection.
- Added tests for image redaction functionality in `test_api_image_redactor.py`, ensuring proper handling of image data and error responses.
- Developed package-level tests in `test_package_e2e_integration_flows.py` to verify the functionality of the analyzer and anonymizer engines, including support for third-party recognizers.
* Remove unused Ollama recognizer imports and related tests
* Update requirements and improve Ollama recognizer availability checks in e2e tests
* Fix formatting in requirements.txt for analyzer and anonymizer dependencies
* Update Ollama model ID from gemma3:1b to gemma2:2b in configuration and tests
* gemma2:2b
* finalizing the pr
* Remove unused ABC import from lm_recognizer.py
* Fix indentation in docker-compose.yml for volumes section
* Fix line break for clarity in adding_recognizers.md
* Add timeout settings for Ollama recognizer and test cases
* Refactor timeout comment for clarity in OllamaLangExtractRecognizer
* Update Ollama model version and add configuration for LangExtract recognizer tests
* Update examples_file path in configuration for Ollama recognizer
* Remove timeout decorator from Ollama recognizer
* Add rerun settings to unit and E2E tests for improved stability
* Remove rerun settings from unit and E2E test commands for simplification
* Set max-parallel to 2 for local build and E2E tests
* Remove max-parallel setting from local build and E2E tests
* move poetry cache dir
* pr changes
* code review changes
* ruff check
* remove unused json import in test_ollama_recognizer.py
* refactor test names for clarity and consistency in test_ollama_recognizer.py
* finalizing the PR
* self code review fixes
* Refactor LangExtractRecognizer to use yaml for configuration loading
* ruff fixes
* Update error messages in OllamaLangExtractRecognizer tests for clarity
* CR comment addressed
* Remove unused variables from Jinja2 prompt rendering in LangExtractRecognizer
* exporting functionality to helpers enlarging composition
* composition
* Refactor entity mapper and langextract utilities for improved clarity and consistency
* Refactor tests to use get_langextract_module for mocking LangExtract availability
* Refactor langextract utilities to improve clarity and error handling; remove deprecated functions and update tests accordingly
* Refactor LLM utilities by simplifying docstrings and consolidating imports for improved readability and maintainability
* Update error message for missing Jinja2 installation to include poetry installation instructions
* Add Ollama recognizer configuration and tests for YAML integration
- Introduced `test_ollama_enabled_recognizers.yaml` to define recognizers including OllamaLangExtractRecognizer.
- Enhanced `test_package_e2e_integration_flows.py` with a test to validate loading of Ollama recognizer from YAML configuration.
- Updated `OllamaLangExtractRecognizer` to support configuration path and language parameters.
- Improved handling of relative paths for configuration files.
* Refactor docstrings in OllamaLangExtractRecognizer for improved clarity and formatting
* Enhance OllamaLangExtractRecognizer initialization docstring to clarify kwargs usage
* pr comments
* Refactor OllamaLangExtractRecognizer to streamline config path handling and remove redundant comments
* Refactor Ollama recognizer test to improve clarity and enhance entity detection validation
* Refactor tests for Ollama recognizer and LMRecognizer to improve exception handling and configuration validation
* Update config path for Ollama recognizer in test configuration
* Update config paths for Ollama recognizer and add test configuration for LangExtract
* Remove test configuration for Ollama LangExtract
* Remove test configuration for Ollama LangExtract recognizer
* Fix formatting in resolve_config_path function for improved readability
* Enable UsLangExtractRecognizer and update its config path
* change all configs to use gemma3:1b
* Disable Ollama LangExtract recognizer and update its configuration path
* Update langextract configuration paths to use absolute paths for prompt and examples files
* Remove test script for Ollama recognizer configuration loading
* Refactor config loading in examples and prompt loaders to use resolve_config_path; update logging level in LMRecognizer; add langextract availability check in OllamaLangExtractRecognizer.
* Refactor parameter description in load_yaml_examples and clean up imports in prompt_loader
* Update langextract paths to use repo-root-relative paths in tests and prompt loader
* Enhance documentation for Ollama setup and improve __init__.py imports for clarity and maintainability
* code review changes
* pr comments & align to main
---------
Co-authored-by: Tamir Kamara <26870601+tamirkamara@users.noreply.github.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
