* Initial plan
* feat: add Python 3.14 compatibility to all remaining Presidio packages
- Update requires-python to <3.15 in anonymizer, image-redactor, cli,
structured, and umbrella packages
- Add Python 3.14 PyPI classifier to all affected packages
- Split spacy dependency markers in image-redactor to exclude 3.8.14
on Python 3.14 (no compatible wheel), matching analyzer pattern
- Extend CI matrix to test all components on Python 3.14
- Document Python 3.14 as a supported version in installation docs
- Add CHANGELOG entry under [unreleased]
Closes#2096
* ci: move Python 3.14 into main python-version matrix
All packages now support 3.14, so add it directly to the matrix array
instead of duplicating each component in `include` entries.
* build: regenerate uv.lock files to match updated requires-python bounds
* fix: merge duplicate Changed headings in CHANGELOG.md Unreleased section
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
* ci: use uv for test dependency installation + add timeout safety net
The analyzer test job runs `poetry install --all-extras` with no committed
lock, so Poetry performs a universal resolve of the full optional-dependencies
graph on every run. That resolve backtracks for many minutes — effectively
hanging — driven by the langextract extra's deep, loosely-bounded transitive
tree. Reproduced locally: Poetry >6 min (never completed); uv ~2s for the same
158-package resolution.
Switch the test jobs to uv:
- Install uv via astral-sh/setup-uv.
- Create a per-component venv with `uv venv --seed` (seed keeps pip for the
wheel-build step) and expose it via $GITHUB_PATH / $VIRTUAL_ENV.
- `uv pip install -e '.<extras>'` plus the test tooling that previously came
from the Poetry dev group (uv does not read that group).
- Add a job-level `timeout-minutes: 60` (and 30 on the install step) so a bad
resolve fails fast instead of hanging.
No package metadata changed — pyproject.toml is untouched; this only swaps the
CI install tooling.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: run tests via uv run inside the uv-managed venv
Use "uv run --no-sync" for spaCy model downloads, pytest, and diff-cover
instead of exposing the venv on $GITHUB_PATH. "uv sync" builds the
project's .venv (resolving the analyzer --all-extras graph in seconds
where Poetry's lockless universal resolve hangs), and the subsequent
steps execute inside it idiomatically. The --all-extras selection stays
in the job matrix, unchanged from main. Wheel-build keeps using the
system interpreter, matching the previous Poetry behaviour.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: install uv via pip instead of the setup-uv action
The astral-sh/setup-uv action is not on the org's allowed-actions list,
so adding it made the CI workflow fail at startup (startup_failure)
before any job ran. Install a pinned uv from PyPI with the already
present setup-python instead; this needs no new third-party action and
keeps the resolver benefits that motivated the Poetry -> uv switch.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: add pip to the uv venv for spaCy model downloads
spaCy's `download` command shells out to `python -m pip install`, but
uv-created virtualenvs ship without pip, so runtime/first-use model
downloads (e.g. the presidio-cli conftest fetching en_core_web_lg)
failed with SystemExit: 1. Install pip alongside the test tooling.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: pre-install tomli for the wheel build on Python < 3.11
`python -m build` imports `tomli` on Python < 3.11 to read pyproject.toml,
but it is not in the hashed requirements-build.txt (pip-compiled on 3.11+
where tomllib is stdlib). Poetry previously pulled tomli onto the runner
as one of its own dependencies, so `pip install --require-hashes` found it
already satisfied. uv has no Python dependencies, so the 3.10 wheel-build
jobs began failing with "requirements must be pinned with ==". Pre-install
a marker-guarded tomli to restore that condition.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: build service Docker images with uv to fix the E2E resolve hang
The local-build-and-E2E job builds the analyzer/anonymizer/image-redactor
images via `docker compose build`. Each Dockerfile ran
`poetry install --no-root --only=main -E server`, which triggered the same
lockless Poetry backtracking that hangs the test jobs: the analyzer image
sat on "Resolving dependencies" for ~40 minutes until the job was
cancelled. Switch the dependency install to `uv pip install --system
-r pyproject.toml --extra server`, which resolves the same graph in
seconds (verified locally: the analyzer install layer completes in ~9s and
imports spacy/flask/gunicorn while correctly omitting the project itself,
matching --no-root). Drop the now-unused POETRY_VIRTUALENVS_CREATE env and
switch `poetry run python install_nlp_models.py` to a direct call.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: run gunicorn directly in container entrypoints (drop poetry run)
The Docker images no longer install Poetry (deps come from uv), so the
`exec poetry run gunicorn ...` entrypoints crashed at startup with
"exec: poetry: not found". The analyzer/anonymizer/image-redactor
containers never came up, so every E2E test failed with connection
refused on ports 5002/5003. uv installs gunicorn system-wide, so invoke
it directly. Verified locally: the anonymizer container now starts
healthy and /health returns 200.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: keep uv's wheel cache on the /mnt ephemeral disk
Restore the intent of the old POETRY_CACHE_DIR=/mnt/poetry_cache setup for
uv: point UV_CACHE_DIR at the runner's large ephemeral disk (/mnt, ~65 GB)
so the heavy analyzer --all-extras wheel cache does not fill the smaller
root volume. /mnt is root-owned, so a prep step creates the directory and
hands it to the runner user. UV_LINK_MODE=copy silences the cross-filesystem
hardlink warning, since the project venv lives on the root volume under the
checkout while the cache is on /mnt.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: broaden CHANGELOG/CI comment to cover the Docker image changes
Address review feedback that the PR scope grew beyond the test jobs: the
CHANGELOG now notes the analyzer/anonymizer/image-redactor image builds
and entrypoints also moved from Poetry to uv, and clarifies pyproject.toml
metadata is unchanged. Also clarify the ci.yml install-step comment that
the wheel-build step deliberately uses the system interpreter (not uv run).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* build: adopt PEP 735 dependency groups and commit uv.lock files
Migrate each CI-tested package's development tooling from Poetry's
[tool.poetry.group.dev.dependencies] to the standard PEP 735
[dependency-groups] table, and commit a uv.lock per package. This makes
pyproject.toml the single source of truth for dev tooling and lets CI and
Docker install a pinned, reproducible dependency graph instead of resolving
on every run. The [project] metadata and the poetry-core build backend are
unchanged, so wheels/sdists are byte-for-byte identical.
Covers presidio-analyzer, presidio-anonymizer, presidio-cli,
presidio-image-redactor, presidio-structured.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: install from the committed uv.lock in CI and Docker builds
CI now runs `uv sync --locked --group dev`, which installs exactly the
locked graph and fails if pyproject.toml and uv.lock disagree rather than
silently resolving new versions. The dev group provides the test tooling
(and pip, which spaCy's model download shells out to), so the manual
`uv pip install pytest ...` line is gone. The uv cache is persisted across
runs via actions/cache keyed on uv.lock and trimmed with `uv cache prune
--ci`.
The analyzer/anonymizer/image-redactor images consume the same lockfile
with `uv sync --locked --no-default-groups --extra server
--no-install-project` into UV_PROJECT_ENVIRONMENT=/usr/local, guaranteeing
images use the same dependency graph as CI. Splitting the dependency layer
from the source COPY keeps it cached until pyproject/uv.lock change.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: allow pre-existing transformers advisories in dependency review
The newly committed uv.lock files pin exact versions, so dependency-review
now sees two transformers advisories that already exist on main (capped at
transformers <5 by spacy-huggingface-pipelines and tracked as Dependabot
alerts). They are unrelated to the Poetry->uv migration and will be
addressed in separate security PRs, so allow-list the two GHSAs here to
keep this PR's dependency review unblocked.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: require uv.lock review and commits alongside pyproject changes
Address review feedback from @SharonHart:
- CODEOWNERS: require presidio-administrators approval for **/uv.lock, the
same as **/pyproject.toml, so lockfile changes get dependency review.
- copilot-instructions: switch the local dev commands to uv and add an
explicit rule that editing pyproject.toml dependencies requires
regenerating and committing that package's uv.lock in the same change,
since CI installs with `uv sync --locked` and fails on drift.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: reference UV_CACHE_DIR in cache path; align dev-docs to uv
Address review feedback:
- ci.yml: use ${{ env.UV_CACHE_DIR }} for the cache action path instead of
hardcoding /mnt/uv-cache, so the cache and uv stay in sync if the dir
changes.
- copilot-instructions: update the "Technology Stack" bullet to name uv as
the dependency/install tool (poetry-core retained only as build backend),
matching the uv-based dev commands already documented.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* ci: shorten workflow comments; move transformers allow-ghsas out
Trim the verbose explanatory comments in the test job to 1-3 lines each,
and drop the dependency-review `allow-ghsas` entry for the transformers
advisories. That suppression is a security concern rather than part of the
Poetry->uv migration, so it moves to the stacked security PR alongside the
.trivyignore.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: drop stale pip cache; trim CHANGELOG entry
Remove `cache: 'pip'` (keyed on pyproject.toml) from the test job's
setup-python: it is leftover Poetry-era config. Project dependencies now use
the uv cache, and the only remaining pip installs (uv, tomli, build tools)
are tiny and unrelated to pyproject.toml. Also condense the unreleased
CHANGELOG entry to a single concise line; the implementation detail lives in
the PR description.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* small commit to trigger the pipeline again.
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Fix CI and release container publishing
Build analyzer release images on native platform runners and create manifests afterward. Stop CI from publishing ephemeral gha images by building and testing Docker images locally in the same job.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revise container workflow handoff
Run release container builds through a shared image/platform matrix on native runners before creating manifests. Keep CI image builds as ephemeral artifacts that E2E jobs download and load instead of publishing gha tags to GHCR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Share release workflow platform list
* Reuse shared release platform list
* Share release workflow image list
* Restore GHCR login for CI cache access
* Simplify release container platform handling
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* docs: migrate documentation site from MkDocs to Zensical
Replace the Material for MkDocs toolchain with Zensical (its successor),
leaving no parallel MkDocs build behind.
- release-docs.yml builds with scripts/zensical_build.py and publishes the
generated ./site to GitHub Pages (was: pip install requirements-docs.txt
+ mkdocs gh-deploy).
- requirements-docs.txt pins zensical + nbconvert + mkdocstrings-python in
place of mkdocs / mkdocs-material / mkdocs-jupyter.
- scripts/zensical_build.py pre-converts the notebook samples with nbconvert
(Zensical has no MkDocs plugin lifecycle, so mkdocs-jupyter never runs) and
emits a generated zensical.yml from mkdocs.yml, which Zensical reads natively.
- mkdocs.yml: drop the now-unused mkdocs-jupyter plugin (the build script
handles notebooks); the Material-compatible theme, custom_dir overrides
(Microsoft Clarity + cookie consent) and mkdocstrings API reference are all
honored by Zensical.
- docs/stylesheets/extra.css: style the nbconvert-rendered notebook output.
- NOTICE: attribute Zensical (MIT) and nbconvert (BSD-3 Clause); drop the
MkDocs and mkdocs-jupyter notices, keep mkdocstrings and pymdown-extensions.
Verified locally: 104 pages build, all 18 sample notebooks render, the API
reference and brand styling are intact.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: improve Zensical mobile homepage
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Reverts a725ecfa ("updates to docs and markdown") and bee25c35 ("docs and md renames"), which were pushed to main by mistake. The migration work is preserved on a separate branch and will be reintroduced via a proper branch/PR.
* Initial plan
* Fix numpy IndentationError in Test Analyzer (Python 3.12) CI job
Root cause: GitHub Actions reuses the same runner VM on re-run attempts,
leaving the /mnt/poetry_cache virtualenv from the previous run intact.
When poetry installs a new numpy version (2.4.x) on top of the cached
older version, it creates a corrupted/mixed _type_aliases.py that causes
IndentationError: unexpected indent at line 134.
Fix 1 (ci.yml): Add 'rm -rf $POETRY_CACHE_DIR/virtualenvs' to the
'Setup Poetry Cache Directory' step. This ensures each run starts with
a clean virtualenv, preventing partial numpy upgrades from corrupting
the installation.
Fix 2 (pyproject.toml): Update numpy upper bound from '<2.5.0' to
'<3.0.0'. numpy 2.5.0 is now the current stable release; the old bound
was preventing its use and restricting the project to numpy 2.4.x which
is more susceptible to the cache corruption issue.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
