presidio/docs/samples/python/presidio_notebook.ipynb

{
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.7-final"
  },
  "orig_nbformat": 2,
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2,
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from presidio_analyzer import AnalyzerEngine, PatternRecognizer\n",
    "from presidio_anonymizer import AnonymizerEngine\n",
    "from presidio_anonymizer.entities import AnonymizerRequest"
   ]
  },
  {
   "source": [
    "# Analyze Text for PII Entities\n",
    "\n",
    "<br>Using Presidio Analyzer, analyze a text to identify PII entities. \n",
    "<br>The Presidio analyzer is using pre-defined entity recognizers, and offers the option to create custom recognizers.\n",
    "\n",
    "<br>The following code sample will:\n",
    "<ol>\n",
    "<li>Set up the Analyzer engine - load the NLP module (spaCy model by default) and other PII recognizers</li>\n",
    "<li> Call analyzer to get analyzed results for \"PHONE_NUMBER\" entity type</li>\n",
    "</ol>"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_to_anonymize = \"His name is Mr. Jones and his phone number is 212-555-5555\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "analyzer = AnalyzerEngine()\n",
    "analyzer_results = analyzer.analyze(text=text_to_anonymize, entities=[\"PHONE_NUMBER\"], language='en')\n",
    "\n",
    "print(analyzer_results)"
   ]
  },
  {
   "source": [
    "# Create Custom PII Entity Recognizers\n",
    "\n",
    "<br>Presidio Analyzer comes with a pre-defined set of entity recognizers. It also allows adding new recognizers without changing the analyzer base code,\n",
    "<b>by creating custom recognizers. \n",
    "<br>In the following example, we will create two new recognizers of type `PatternRecognizer` to identify titles and pronouns in the analyzed text.\n",
    "<br>A `PatternRecognizer` is a PII entity recognizer which uses regular expressions or deny-lists.\n",
    "\n",
    "<br>The following code sample will:\n",
    "<ol>\n",
    "<li>Create custom recognizers</li>\n",
    "<li>Add the new custom recognizers to the analyzer</li>\n",
    "<li>Call analyzer to get results from the new recognizers</li>\n",
    "</ol>\n"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titles_recognizer = PatternRecognizer(supported_entity=\"TITLE\",\n",
    "                                      deny_list=[\"Mr.\",\"Mrs.\",\"Miss\"])\n",
    "\n",
    "pronoun_recognizer = PatternRecognizer(supported_entity=\"PRONOUN\",\n",
    "                                      deny_list=[\"he\", \"He\", \"his\", \"His\", \"she\", \"She\", \"hers\" \"Hers\"])\n",
    "\n",
    "analyzer.registry.add_recognizer(titles_recognizer)\n",
    "analyzer.registry.add_recognizer(pronoun_recognizer)\n",
    "\n",
    "analyzer_results = analyzer.analyze(text=text_to_anonymize,\n",
    "                            entities=[\"TITLE\", \"PRONOUN\"],\n",
    "                            language=\"en\")\n",
    "print(analyzer_results)\n"
   ]
  },
  {
   "source": [
    "Call Presidio Analyzer and get analyzed results with all the configured recognizers - default and new custom recognizers"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "analyzer_results = analyzer.analyze(text=text_to_anonymize, language='en')\n",
    "\n",
    "print(analyzer_results)"
   ]
  },
  {
   "source": [
    "# Anonymize Text with Identified PII Entities\n",
    "\n",
    "<br>Presidio Anonymizer iterates over the Presidio Analyzer result, and provides anonymization capabilities for the identified text.\n",
    "<br>The anonymizer provides 4 types of anonymizers - replace, redact, mask and hash. The default is **replace**\n",
    "\n",
    "<br>The following code sample will:\n",
    "<ol>\n",
    "<li>Convert analyzer results to anonymizer input - a list of dict</li>\n",
    "<li>Setup the anonymizer engine </li>\n",
    "<li>Create an anonymizer request - text to anonymize, list of anonymizers to apply and the results from the analyzer request</li>\n",
    "<li>Anonymize the text</li>\n",
    "</ol>"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "converted_analyzer_results = [result.to_dict() for result in analyzer_results]\n",
    "\n",
    "print(converted_analyzer_results)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "anonymizer = AnonymizerEngine()\n",
    "\n",
    "request = {\n",
    "        \"text\": text_to_anonymize,\n",
    "        \"anonymizers\": {\n",
    "            \"DEFAULT\": {\"type\": \"replace\", \"new_value\": \"<ANONYMIZED>\"},\n",
    "             \"PHONE_NUMBER\": {\n",
    "                \"type\": \"mask\",\n",
    "                \"masking_char\": \"*\",\n",
    "                \"chars_to_mask\": 12,\n",
    "                \"from_end\": True,\n",
    "            },\n",
    "            \"TITLE\": {\n",
    "                \"type\": \"redact\"\n",
    "            }\n",
    "        },\n",
    "        \"analyzer_results\": converted_analyzer_results\n",
    "    }\n",
    "\n",
    "data = AnonymizerRequest(request, AnonymizerEngine().builtin_anonymizers)\n",
    "\n",
    "anonymized_results = anonymizer.anonymize(data)\n",
    "\n",
    "print(anonymized_results)"
   ]
  }
 ]
}