* presidio-structured
changelog
Static analysis
docstrings, types
preliminary tests engine
static analysis
isort
Minor refactorings
Update README.md
Fix late binding issues and example
removal of old samples
Refactoring, adding example
pre-clean-break-commit
broken commit, fixing TabularConfigBuilder
Rename TabularConfig
pre-breaking replace commit
removal of some old experimental files
rename tabular to structured
restructuring presidio tabular - pre del commit
Add project TODOs
testing dump presidio tabular
* Add unit tests
* rename engine, add buildfile
* Update setup.py
* lint-build-test
* Update lint-build-test.yml
* Add packages to setup.py
* Update presidio-structured to alpha version
* Update Presidio structured README.md
* Add logging configuration to presidio-structured
module
* Refactor AnalysisBuilder constructor to accept an
optional AnalyzerEngine parameter
* Fix entity mapping in JsonAnalysisBuilder
* Drop type in docstring in analysis builder classes
* Refactor TabularAnalysisBuilder to use
BatchAnalyzerEngine for all columns
* Update data_reader.py with type hints for file
paths
* Update data_reader.py to include additional
keyword arguments in read() method
* Update Transformer to Processor term in
StructuredEngine
* Add PandasDataProcessor as default to StructuredEngine
init
* Move structured sample files to the docs
* Add Presidio Structured Notebook to samples index
* Remove unnecessary imports in structured sample
* Update to processors in structured __init__ files
* Add explanation for structured table sample
* Delete unnecessary __init__s in structured test
* Fix bug in JsonAnalysisBuilder entity mapping
* pr comments, nits, minor tests
* README
* Add TabularAnalysisBuilder
* Some basic logging
* linting
* Fix typo in logger variable name
* Refactor analysis builder to include score
threshold
* Linting, continued
* Update Pipfile
* Refactor JsonAnalysisBuilder to support language
parameter
* Fix not camel case in TabularAnalysisBuilder
* Add score_threshold parameter to AnalysisBuilder
* Refactor JSON analysis builder to gain consistency
* Remove low score results in JsonAnalysisBuilder
* Add tests to json analysis with score threshold
* Fix bug in JSON analysis to update map with
nested_mappings
* Fix bug in JSON analysis to take only entity types
* Fix typos in test anl json names and assert values
* Update build-structured.yml
* Create __init__.py
* Type hint fix python <3.10, loggger typo
* Update setup.py
* PR comments variety
* further pr comments
* readme, refactor score, refactor tabular analysis
* Update test_analysis_builder.py
* lint
---------
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
Co-authored-by: enrique.botia <enrique.botia@netzima.com>
* Adding methods to enable plotting of custom bboxes
* Adding test for get_pii_bboxes
* Linting fixes and addition of test for add_custom_bboxes
* Adding use of custom bbox into DICOM verification engine
* Fixing tests for the DICOM verification engine
* Adding in ocr kwargs compatibility and updating tests
* Adding example notebook
* Linting fixes
* Remaining linting fix
---------
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Enabling allow list approach
* Adding in empty list for allow_list in existing unit tests
* Added unit tests for newly introduced methods
* Adding unit test for allow list functionality
* Linting fixes
* Removing spaces in empty lines
* Fix integration test not accounting for empty space removal
* Updating notebook with more examples and adding ad_hoc_recognizers approach to standard image redactor engine as well
* Linting fixes
* Removing incomplete example code
* Fixing section header numbers
* Removing duplicate comment
---------
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Initial commit with old paths (will need to fix)
* Fixing paths
* Adding in notebook and updating documentation
* Adding in WIP notebook and docs
* Adding example notebook
* Clearing some output to reduce file size
* Linting fix v1
* Linting v2
* Add rounding to integration test to account for minor differences in real testing
* Fixing typo with wrong variable name
* Removing check for exact or rounded conf
* Updating logic for comparing ocr_results
* Minor fixes on DICOM verify code based on PR comments
* Changing results and gt format from dictionary to list of dictionaries
* Updating notebook
* Updating docstring format
* Linting fix
* Changing fixture scope for mock_engine
* Update docs/image-redactor/evaluating_dicom_redaction.md
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Docstring updates
* Improving efficiency of _remove_duplicate_entities
* Add handling for divide by zero in recall and precision calculations
* Making precision and recall calculation functions public
* Moving common fixtures to conftest
* PR comments
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Adding DicomImageRedactorEngine class without test data for now
* Updating and putting placeholder in package doc
* Updating docs
* Moving presidio-dicom-image-redactor module into presidio-image-redactor module as an extension
* Updating docs to reflect refactor from presidio-dicom-image-redactor module to extension of presidio-image-redactor module
* Docs fixes continued
* Reflecting move from new module to updating existing module
* Updating requirements
* Linting fixes
* Linting fixes. Function and variable name casing
* Fixing getting started code blocks
* Updating image redactor version
* Fix minor typo: phi -> PHI in comment
* Update presidio-image-redactor/presidio_image_redactor/utils/dicom_image_redact_utils.py
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Update presidio-image-redactor/presidio_image_redactor/utils/dicom_image_redact_utils.py
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Fixing new typing adding added from PR comment commit
* Moving all utils into the DicomImageRedactorEngine class
* Fixing integration test now that utils have been moved
* Moving to ad-hoc recognizer approach
* Updating redact to have similar format as in ImageRedactorEngine. Moved existing logic for redact into redact_from_file
* Updating requirements
* Updating docs with new getting started code
* Versioning and changelog updates
* Adding in data with citations
* Linting fixes
* Minor change: from multiple list append to list extend
* Adding pytest-mock to requirements
* Fixing paths to test data
* Making file extension search more flexible and work with both Windows and Unix systems
* Changing docstring style to match rest of repo
* Making statement about Tesseract version more general in one other doc (was caught in another earlier)
* Changing box_color_setting to fill for consistency
* Updating changelog based on PR comments
* Update docs/image-redactor/index.md
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
* Updating docs
* Adding **kwargs support and addressing minor comments
* Adjusting with clause duration
* Adding note about custom ImageAnalyzerEngine support
* Changing minor comment in docstring
* Splitting redact_from_file into redact_from_file and redact_from_directory
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
