* fix: add non-root user to all Dockerfiles for security
- Create dedicated user 'presidio' with UID 1001 in all Linux containers
- Add USER directive to ensure containers don't run as root
- Set proper file ownership with chown for production images
- Apply same security fix to Windows containers using net user
- Update sample Dockerfiles to follow security best practices
This addresses the container security requirement that containers
should not run with root privileges.
* fix: configure Poetry to not use virtualenvs in containers
When running as non-root user, Poetry was trying to create virtualenvs
in the user's home directory, causing 'Command not found: gunicorn' errors.
Setting POETRY_VIRTUALENVS_CREATE=false ensures Poetry installs packages
system-wide (already isolated by the container), making them accessible
to the non-root user at runtime.
Tested locally with analyzer container - health and analyze endpoints
working correctly.
* fix: pre-install spaCy model in image-redactor for non-root user
The image-redactor was attempting to install en_core_web_lg at runtime,
which failed when running as non-root user due to write permission issues
on system site-packages.
Installing the model during Docker build (as root) ensures it's available
system-wide for the non-root user to read and use at runtime.
Using en_core_web_lg as the default model, which matches the presidio-analyzer
default configuration.
* fix: add POETRY_VIRTUALENVS_CREATE=false to Windows Dockerfiles
Windows containers also need Poetry configured to install packages
system-wide rather than in virtualenvs to ensure the non-root user
can access the installed dependencies.
* Added a new section to the deployments docs, best practices to deanonymize data when using OpenAI
* dded a line to mkdocs.yaml under samples
* fixed link title
* PR fixes
---------
Co-authored-by: itye-msft <ityer@microsoft.com>
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* presidio-structured
changelog
Static analysis
docstrings, types
preliminary tests engine
static analysis
isort
Minor refactorings
Update README.md
Fix late binding issues and example
removal of old samples
Refactoring, adding example
pre-clean-break-commit
broken commit, fixing TabularConfigBuilder
Rename TabularConfig
pre-breaking replace commit
removal of some old experimental files
rename tabular to structured
restructuring presidio tabular - pre del commit
Add project TODOs
testing dump presidio tabular
* Add unit tests
* rename engine, add buildfile
* Update setup.py
* lint-build-test
* Update lint-build-test.yml
* Add packages to setup.py
* Update presidio-structured to alpha version
* Update Presidio structured README.md
* Add logging configuration to presidio-structured
module
* Refactor AnalysisBuilder constructor to accept an
optional AnalyzerEngine parameter
* Fix entity mapping in JsonAnalysisBuilder
* Drop type in docstring in analysis builder classes
* Refactor TabularAnalysisBuilder to use
BatchAnalyzerEngine for all columns
* Update data_reader.py with type hints for file
paths
* Update data_reader.py to include additional
keyword arguments in read() method
* Update Transformer to Processor term in
StructuredEngine
* Add PandasDataProcessor as default to StructuredEngine
init
* Move structured sample files to the docs
* Add Presidio Structured Notebook to samples index
* Remove unnecessary imports in structured sample
* Update to processors in structured __init__ files
* Add explanation for structured table sample
* Delete unnecessary __init__s in structured test
* Fix bug in JsonAnalysisBuilder entity mapping
* pr comments, nits, minor tests
* README
* Add TabularAnalysisBuilder
* Some basic logging
* linting
* Fix typo in logger variable name
* Refactor analysis builder to include score
threshold
* Linting, continued
* Update Pipfile
* Refactor JsonAnalysisBuilder to support language
parameter
* Fix not camel case in TabularAnalysisBuilder
* Add score_threshold parameter to AnalysisBuilder
* Refactor JSON analysis builder to gain consistency
* Remove low score results in JsonAnalysisBuilder
* Add tests to json analysis with score threshold
* Fix bug in JSON analysis to update map with
nested_mappings
* Fix bug in JSON analysis to take only entity types
* Fix typos in test anl json names and assert values
* Update build-structured.yml
* Create __init__.py
* Type hint fix python <3.10, loggger typo
* Update setup.py
* PR comments variety
* further pr comments
* readme, refactor score, refactor tabular analysis
* Update test_analysis_builder.py
* lint
---------
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
Co-authored-by: enrique.botia <enrique.botia@netzima.com>
* Adding methods to enable plotting of custom bboxes
* Adding test for get_pii_bboxes
* Linting fixes and addition of test for add_custom_bboxes
* Adding use of custom bbox into DICOM verification engine
* Fixing tests for the DICOM verification engine
* Adding in ocr kwargs compatibility and updating tests
* Adding example notebook
* Linting fixes
* Remaining linting fix
---------
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Enabling allow list approach
* Adding in empty list for allow_list in existing unit tests
* Added unit tests for newly introduced methods
* Adding unit test for allow list functionality
* Linting fixes
* Removing spaces in empty lines
* Fix integration test not accounting for empty space removal
* Updating notebook with more examples and adding ad_hoc_recognizers approach to standard image redactor engine as well
* Linting fixes
* Removing incomplete example code
* Fixing section header numbers
* Removing duplicate comment
---------
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Initial commit with old paths (will need to fix)
* Fixing paths
* Adding in notebook and updating documentation
* Adding in WIP notebook and docs
* Adding example notebook
* Clearing some output to reduce file size
* Linting fix v1
* Linting v2
* Add rounding to integration test to account for minor differences in real testing
* Fixing typo with wrong variable name
* Removing check for exact or rounded conf
* Updating logic for comparing ocr_results
* Minor fixes on DICOM verify code based on PR comments
* Changing results and gt format from dictionary to list of dictionaries
* Updating notebook
* Updating docstring format
* Linting fix
* Changing fixture scope for mock_engine
* Update docs/image-redactor/evaluating_dicom_redaction.md
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Docstring updates
* Improving efficiency of _remove_duplicate_entities
* Add handling for divide by zero in recall and precision calculations
* Making precision and recall calculation functions public
* Moving common fixtures to conftest
* PR comments
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
