3 Commits

Author SHA1 Message Date
Sharon Hart
ce63783fcc Unlock numpy after dropping 3.8 (#1480) 2024-11-06 15:21:45 +02:00
Jakob Serlier
966d17a5a5 Feature/presidio-structured (#1192)
* presidio-structured

changelog

Static analysis

docstrings, types

preliminary tests engine

static analysis

isort

Minor refactorings

Update README.md

Fix late binding issues and example

removal of old samples

Refactoring, adding example

pre-clean-break-commit

broken commit, fixing TabularConfigBuilder

Rename TabularConfig

pre-breaking replace commit

removal of some old experimental files

rename tabular to structured

restructuring presidio tabular - pre del commit

Add project TODOs

testing dump presidio tabular

* Add unit tests

* rename engine, add buildfile

* Update setup.py

* lint-build-test

* Update lint-build-test.yml

* Add packages to setup.py

* Update presidio-structured to alpha version

* Update Presidio structured README.md

* Add logging configuration to presidio-structured
module

* Refactor AnalysisBuilder constructor to accept an
optional AnalyzerEngine parameter

* Fix entity mapping in JsonAnalysisBuilder

* Drop type in docstring in analysis builder classes

* Refactor TabularAnalysisBuilder to use
BatchAnalyzerEngine for all columns

* Update data_reader.py with type hints for file
paths

* Update data_reader.py to include additional
keyword arguments in read() method

* Update Transformer to Processor term in
StructuredEngine

* Add PandasDataProcessor as default to StructuredEngine
init

* Move structured sample files to the docs

* Add Presidio Structured  Notebook to samples index

* Remove unnecessary imports in structured sample

* Update to processors in structured __init__ files

* Add explanation for structured table sample

* Delete unnecessary __init__s in structured test

* Fix bug in JsonAnalysisBuilder entity mapping

* pr comments, nits, minor tests

* README

* Add TabularAnalysisBuilder

* Some basic logging

* linting

* Fix typo in logger variable name

* Refactor analysis builder to include score
threshold

* Linting, continued

* Update Pipfile

* Refactor JsonAnalysisBuilder to support language
parameter

* Fix not camel case in TabularAnalysisBuilder

* Add score_threshold parameter to AnalysisBuilder

* Refactor JSON analysis builder to gain consistency

* Remove low score results in JsonAnalysisBuilder

* Add tests to json analysis  with score threshold

* Fix bug in JSON analysis to update map with
nested_mappings

* Fix bug in JSON analysis to take only entity types

* Fix typos in test anl json names and assert values

* Update build-structured.yml

* Create __init__.py

* Type hint fix python <3.10, loggger typo

* Update setup.py

* PR comments variety

* further pr comments

* readme, refactor score, refactor tabular analysis

* Update test_analysis_builder.py

* lint

---------

Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
Co-authored-by: enrique.botia <enrique.botia@netzima.com>
2024-01-14 11:22:44 +02:00
Nava Vaisman Levy
aab4d20ee7 Sample - analyze and anonymize csv file (#990) 2022-12-29 14:59:06 +02:00