upstream-archive/presidio
1
0
Fork 0
mirror of https://github.com/data-privacy-stack/presidio.git synced 2026-08-05 01:30:43 -05:00
Code Issues Packages Projects Releases Wiki Activity
Files
dda4444fdab0dd9396f2cae4380ce99e93285afd
presidio/docs/samples/python/GPT3_synth_data.ipynb
Omri Mendels 60911edf16 GPT-3 based synthetic data generation (#1051)
2023-04-16 12:41:36 +03:00

1208 lines
72 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "markdown",
"id": "21410c42",
"metadata": {},
"source": [
"# Use Presidio + GPT-3 to turn real text into fake text\n",
"\n",
"This notebook uses Presidio to turn text with PII into text where PII entities are replaced with placeholders, e.g. \"`My name is David`\" turns into \"`My name is {{PERSON}}`\". Then, it calls the OpenAI GPT-3 API to create a fake record which is based on the original one.\n",
"\n",
"\n",
"Flow:\n",
"1. `My friend David lives in Paris. He likes it.`\n",
"1. `My friend {{PERSON}} lives in {{CITY}}. He likes it.`\n",
"1. `My friend Lucy lives in Beirut. She likes it.`\n",
" \n",
"Note that OpenAI completion models could possibly detect PII values and replace them in one call, but it is suggested to validate that all PII entities are indeed detected.\n",
"\n",
"## Imports and set up OpenAI Key"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "dc5146e0",
"metadata": {},
"outputs": [],
"source": [
"#!pip install openai\n",
"import pprint\n",
"from dotenv import load_dotenv\n",
"import os\n",
"import pandas as pd\n",
"import openai\n",
"\n",
"load_dotenv()\n",
"\n",
"openai.api_key = os.getenv(\"OPENAI_KEY\") #Or put explicitly in notebook. Find out more here: https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key"
]
},
{
"cell_type": "markdown",
"id": "7c793237",
"metadata": {},
"source": [
"## Define request for the OpenAI Completion service"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "a15b64db",
"metadata": {},
"outputs": [],
"source": [
"def call_completion_model(prompt:str, model:str=\"text-davinci-003\", max_tokens:int=512) ->str:\n",
" \"\"\"Creates a request for the OpenAI Completion service and returns the response.\n",
" \n",
" :param prompt: The prompt for the completion model\n",
" :param model: OpenAI model name\n",
" :param max_tokens: Model's max tokens parameter\n",
" \"\"\"\n",
"\n",
" response = openai.Completion.create(\n",
" model=model,\n",
" prompt= prompt,\n",
" max_tokens=max_tokens\n",
" )\n",
"\n",
" return response['choices'][0].text"
]
},
{
"cell_type": "markdown",
"id": "b0f30aed",
"metadata": {},
"source": [
"## De-identify data using Presidio Analyzer and Anonymizer"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "d0dd7820",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Hello, my name is <PERSON> and I live in <LOCATION>.\n",
"My credit card number is <CREDIT_CARD> and my crypto wallet id is <CRYPTO>.\n",
"\n",
"On <DATE_TIME> I visited <URL> and sent an email to <EMAIL_ADDRESS>, from the IP <IP_ADDRESS>.\n",
"\n",
"My passport: <US_PASSPORT> and my phone number: <PHONE_NUMBER>.\n",
"\n",
"This is a valid International Bank Account Number: <IBAN_CODE> . Can you please check the status on bank account <US_BANK_NUMBER>?\n",
"\n",
"<PERSON>'s social security number is <US_SSN>. Her driver license? it is <US_DRIVER_LICENSE>.\n",
"\n"
]
}
],
"source": [
"from presidio_analyzer import AnalyzerEngine\n",
"from presidio_anonymizer import AnonymizerEngine\n",
"\n",
"analyzer = AnalyzerEngine()\n",
"anonymizer = AnonymizerEngine()\n",
"\n",
"sample = \"\"\"\n",
"Hello, my name is David Johnson and I live in Maine.\n",
"My credit card number is 4095-2609-9393-4932 and my crypto wallet id is 16Yeky6GMjeNkAiNcBY7ZhrLoMSgg1BoyZ.\n",
"\n",
"On September 18 I visited microsoft.com and sent an email to test@presidio.site, from the IP 192.168.0.1.\n",
"\n",
"My passport: 191280342 and my phone number: (212) 555-1234.\n",
"\n",
"This is a valid International Bank Account Number: IL150120690000003111111 . Can you please check the status on bank account 954567876544?\n",
"\n",
"Kate's social security number is 078-05-1126. Her driver license? it is 1234567A.\n",
"\"\"\"\n",
"\n",
"results = analyzer.analyze(sample, language=\"en\")\n",
"anonymized = anonymizer.anonymize(text=sample, analyzer_results=results)\n",
"anonymized_text = anonymized.text\n",
"print(anonymized_text)\n"
]
},
{
"cell_type": "markdown",
"id": "f77b0c0f",
"metadata": {},
"source": [
"## Create prompt (instructions + text to manipulate)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "d2a63fda",
"metadata": {},
"outputs": [],
"source": [
"def create_prompt(anonymized_text: str) -> str:\n",
" \"\"\"\n",
" Create the prompt with instructions to GPT-3.\n",
" \n",
" :param anonymized_text: Text with placeholders instead of PII values, e.g. My name is <PERSON>.\n",
" \"\"\"\n",
"\n",
" prompt = f\"\"\"\n",
" Your role is to create synthetic text based on de-identified text with placeholders instead of personally identifiable information.\n",
" Replace the placeholders (e.g. , , {{DATE}}, {{ip_address}}) with fake values.\n",
"\n",
" Instructions:\n",
"\n",
" Use completely random numbers, so every digit is drawn between 0 and 9.\n",
" Use realistic names that come from diverse genders, ethnicities and countries.\n",
" If there are no placeholders, return the text as is and provide an answer.\n",
" input: How do I change the limit on my credit card {{credit_card_number}}?\n",
" output: How do I change the limit on my credit card 2539 3519 2345 1555?\n",
" input: {anonymized_text}\n",
" output:\n",
" \"\"\"\n",
" return prompt"
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "1c02d0df",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"This is the prompt with de-identified values:\n",
"\n",
" Your role is to create synthetic text based on de-identified text with placeholders instead of personally identifiable information.\n",
" Replace the placeholders (e.g. , , {DATE}, {ip_address}) with fake values.\n",
"\n",
" Instructions:\n",
"\n",
" Use completely random numbers, so every digit is drawn between 0 and 9.\n",
" Use realistic names that come from diverse genders, ethnicities and countries.\n",
" If there are no placeholders, return the text as is and provide an answer.\n",
" If there is any additional PII/PHI in the sentence, replace it too.\n",
" \n",
" input: How do I change the limit on my credit card {credit_card_number}?\n",
" output: How do I change the limit on my credit card 2539 3519 2345 1555?\n",
" input: \n",
"Hello, my name is <PERSON> and I live in <LOCATION>.\n",
"My credit card number is <CREDIT_CARD> and my crypto wallet id is <CRYPTO>.\n",
"\n",
"On <DATE_TIME> I visited <URL> and sent an email to <EMAIL_ADDRESS>, from the IP <IP_ADDRESS>.\n",
"\n",
"My passport: <US_PASSPORT> and my phone number: <PHONE_NUMBER>.\n",
"\n",
"This is a valid International Bank Account Number: <IBAN_CODE> . Can you please check the status on bank account <US_BANK_NUMBER>?\n",
"\n",
"<PERSON>'s social security number is <US_SSN>. Her driver license? it is <US_DRIVER_LICENSE>.\n",
"\n",
" output:\n",
" \n"
]
}
],
"source": [
"print(\"This is the prompt with de-identified values:\")\n",
"print(create_prompt(anonymized_text))"
]
},
{
"cell_type": "markdown",
"id": "49aabd81",
"metadata": {},
"source": [
"## Call GPT-3"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "23653b0b",
"metadata": {},
"outputs": [],
"source": [
"gpt_res = call_completion_model(create_prompt(anonymized_text))"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "561f0565",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Hello, my name is Kalyn Carranza and I live in Kenya.\n",
"My credit card number is 9871 8047 5709 9872 and my crypto wallet id is FV7POW12.\n",
"\n",
"On 12th August 2021, 12:31pm I visited http://www.example.com and sent an email to vrystrkbnb@example.com, from the IP 109.98.184.102.\n",
"\n",
"My passport: 975-56-3481 and my phone number: 501-678-2198.\n",
"\n",
"This is a valid International Bank Account Number: GB89 CPFX 4256 1763 8810 31 . Can you please check the status on bank account 855-31-6381?\n",
"\n",
"Kalyn Carranza's social security number is 270-10-8543. Her driver license? it is 1183 2066 152 66.\n"
]
}
],
"source": [
"print(gpt_res)"
]
},
{
"cell_type": "markdown",
"id": "cdf4dbe1",
"metadata": {},
"source": [
"### Alternatively, run on a list of template sentences:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "234b6204",
"metadata": {},
"outputs": [],
"source": [
"import urllib\n",
"\n",
"templates = []\n",
"\n",
"url = \"https://raw.githubusercontent.com/microsoft/presidio-research/master/presidio_evaluator/data_generator/raw_data/templates.txt\"\n",
"for line in urllib.request.urlopen(url):\n",
" templates.append(line.decode('utf-8')) "
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "da1773fb",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Example templates:\n"
]
},
{
"data": {
"text/plain": [
"['I want to increase limit on my card # {{credit_card_number}} for certain duration of time. is it possible?\\n',\n",
" 'My credit card {{credit_card_number}} has been lost, Can I request you to block it.\\n',\n",
" 'Need to change billing date of my card {{credit_card_number}}\\n',\n",
" 'I want to update my primary and secondary address to the same: {{address}}\\n',\n",
" \"In case of my child's account, we need to add {{person}} as guardian\\n\"]"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"print(\"Example templates:\")\n",
"templates[:5]"
]
},
{
"cell_type": "code",
"execution_count": 42,
"id": "3757461b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{ 'original': 'I want to increase limit on my card # {{credit_card_number}} for certain duration of time. is '\n",
" 'it possible?\\n',\n",
" 'synthetic': 'I want to increase limit on my card # 2539 3519 2345 1555 for certain duration of time. Is '\n",
" 'it possible?'}\n",
"--------------\n",
"{ 'original': 'My credit card {{credit_card_number}} has been lost, Can I request you to block it.\\n',\n",
" 'synthetic': 'My credit card 2539 3519 2345 1555 has been lost, Can I request you to block it.'}\n",
"--------------\n",
"{ 'original': 'Need to change billing date of my card {{credit_card_number}}\\n',\n",
" 'synthetic': 'Need to change billing date of my card 2539 3519 2345 1555?'}\n",
"--------------\n",
"{ 'original': 'I want to update my primary and secondary address to the same: {{address}}\\n',\n",
" 'synthetic': 'I want to update my primary and secondary address to the same: 3241 Tulip Lane, London, NW9 '\n",
" '5RX.'}\n",
"--------------\n",
"{ 'original': \"In case of my child's account, we need to add {{person}} as guardian\\n\",\n",
" 'synthetic': \"In case of my child's account, we need to add Abigail Jones as guardian.\"}\n",
"--------------\n",
"{ 'original': 'Are there any charges applied for money transfer from {{iban}} to other bank accounts\\n',\n",
" 'synthetic': 'Are there any charges applied for money transfer from GB55TRRX92363841637179 to other bank '\n",
" 'accounts?'}\n",
"--------------\n",
"{ 'original': 'Are there any charges applied to withdraw money from ATM with the card '\n",
" '{{credit_card_number}}\\n',\n",
" 'synthetic': 'Are there any charges applied to withdraw money from ATM with the card 2539 3519 2345 1555?'}\n",
"--------------\n",
"{ 'original': 'Not getting bank documents to my address. Can you please validate the following? '\n",
" '{{address}}\\n',\n",
" 'synthetic': 'Not getting bank documents to my address. Can you please validate the following? 943 Elm '\n",
" 'Street, Fort Wayne, IN 46825.'}\n",
"--------------\n",
"{ 'original': 'Please update the billing address with {{address}} for this card: {{credit_card_number}}\\n',\n",
" 'synthetic': 'Please update the billing address with 3989 Elm Street, San Francisco, CA 94103 for this '\n",
" 'card: 2539 3519 2345 1555.'}\n",
"--------------\n",
"{ 'original': 'Need to see last 10 transaction of card {{credit_card_number}}\\n',\n",
" 'synthetic': 'Need to see last 10 transactions of card 2539 3519 2345 1555'}\n",
"--------------\n",
"{ 'original': 'I have lost my card {{credit_card_number}}. Could you please block my credit card ASAP ? My '\n",
" 'name is {{person}}.\\n',\n",
" 'synthetic': 'I have lost my card 2539 3519 2345 1555. Could you please block my credit card ASAP ? My '\n",
" 'name is John Smith.'}\n",
"--------------\n",
"{ 'original': \"My card {{credit_card_number}} is expiring this month. Please let me know process to it's \"\n",
" 'extend validity.\\n',\n",
" 'synthetic': \"My card 2539 3519 2345 1555 is expiring this month. Please let me know process to it's \"\n",
" 'extend validity.'}\n",
"--------------\n",
"{ 'original': \"I have done an online order but didn't get any message on my registered {{phone_number}}. \"\n",
" 'Could you please look into it ?\\n',\n",
" 'synthetic': \"I have done an online order but didn't get any message on my registered (502) 183-3752. \"\n",
" 'Could you please look into it?'}\n",
"--------------\n",
"{ 'original': 'What is procedure to redeem points won on credit card {{credit_card_number}} transactions ?\\n',\n",
" 'synthetic': 'What is procedure to redeem points won on credit card 2539 3519 2345 1555 transactions?'}\n",
"--------------\n",
"{ 'original': 'My card {{credit_card_number}} expires soon <20> when will I get a new one?\\n',\n",
" 'synthetic': 'My card 2539 3519 2345 1555 expires soon <20> when will I get a new one?'}\n",
"--------------\n",
"{ 'original': 'How do I check my balance on my credit card?\\n',\n",
" 'synthetic': 'How do I check my balance on my credit card?'}\n",
"--------------\n",
"{ 'original': 'Could I change the payment due date of my credit card?\\n',\n",
" 'synthetic': 'Could I change the payment due date of my credit card?'}\n",
"--------------\n",
"{ 'original': 'How can I request a new credit card pin ?\\n',\n",
" 'synthetic': 'How can I request a new credit card pin?'}\n",
"--------------\n",
"{ 'original': 'Can I withdraw cash using my card {{credit_card_number}} at aTM center ?\\n',\n",
" 'synthetic': 'Can I withdraw cash using my card 2539 3519 2345 1555 at ATM center?'}\n",
"--------------\n",
"{ 'original': 'How do I change the address linked to my credit card to {{address}}?\\n',\n",
" 'synthetic': 'How do I change the address linked to my credit card to 1325 Valley View Rd, Yelm, WA 98597?'}\n",
"--------------\n",
"{ 'original': 'How do I open my credit card statement?\\n',\n",
" 'synthetic': 'How do I open my credit card statement?'}\n",
"--------------\n",
"{'original': \"I'm originally from {{country}}\\n\", 'synthetic': \"I'm originally from Canada.\"}\n",
"--------------\n",
"{ 'original': 'I will be travelling to {{country}} next week, so I need my passport to be ready by then\\n',\n",
" 'synthetic': 'I will be travelling to Mexico next week, so I need my passport to be ready by then.'}\n",
"--------------\n",
"{'original': \"Who's coming to {{country}} with me?\\n\", 'synthetic': \"Who's coming to Mexico with me?\"}\n",
"--------------\n",
"{'original': '{{country}} was super fun to visit!\\n', 'synthetic': 'France was super fun to visit!'}\n",
"--------------\n",
"{ 'original': 'Could you please email me the statement for last month , my credit card number is '\n",
" '{{credit_card_number}}?\\n',\n",
" 'synthetic': 'Could you please email me the statement for last month, my credit card number is 2539 3519 '\n",
" '2345 1555?'}\n",
"--------------\n",
"{ 'original': 'Could you please send me the last billed amount for cc {{credit_card_number}} on my e-mail '\n",
" '{{email}}?\\n',\n",
" 'synthetic': 'Could you please send me the last billed amount for cc 2539 3519 2345 1555 on my email '\n",
" 'sue.perkins@example.com?'}\n",
"--------------\n",
"{ 'original': 'How do I change my address to {{address}} for post mail?\\n',\n",
" 'synthetic': 'How do I change my address to 2514 Golden Street, San Diego, CA 90203 for post mail?'}\n",
"--------------\n",
"{ 'original': 'My name appears incorrectly on credit card statement could you please correct it to '\n",
" '{{prefix_male}} {{name_male}}?\\n',\n",
" 'synthetic': 'My name appears incorrectly on credit card statement could you please correct it to Mr. '\n",
" 'Gregor?'}\n",
"--------------\n",
"{ 'original': 'My name appears incorrectly on credit card statement could you please correct it to '\n",
" '{{prefix_female}} {{name_female}}?\\n',\n",
" 'synthetic': 'My name appears incorrectly on credit card statement could you please correct it to Ms. '\n",
" 'Emma?'}\n",
"--------------\n",
"{ 'original': 'card number {{credit_card_number}} is lost, can you please send a new one to {{address}}? I '\n",
" 'am in {{city}} for a business trip\\n',\n",
" 'synthetic': 'card number 2539 3519 2345 1555 is lost, can you please send a new one to 16 Meadow Lane, '\n",
" 'Hicksville, NY 11801? I am in New York City for a business trip.'}\n",
"--------------\n",
"{ 'original': \"Please transfer all funds from my account to this hackers' {{email}}\\n\",\n",
" 'synthetic': \"Please transfer all funds from my account to this hackers' mitchellejess@gmail.com\"}\n",
"--------------\n",
"{ 'original': \"I can't browse to your site, keep getting address {{ip_address}} blocked error\\n\",\n",
" 'synthetic': \"I can't browse to your site, keep getting address 135.238.125.181 blocked error.\"}\n",
"--------------\n",
"{ 'original': 'My religion does not allow speaking to bots, they are evil and hacked by the Devil\\n',\n",
" 'synthetic': 'My religion does not allow speaking to bots, they are evil and hacked by the Devil'}\n",
"--------------\n",
"{ 'original': \"Excuse me, Sir bot, but I really don't like this tone\\n\",\n",
" 'synthetic': \"Excuse me, Sir bot, but I really don't like this tone\"}\n",
"--------------\n",
"{ 'original': \"WHAT ??? I DON'T KNOW WHAT TO PRESS NEXT!!! ? !! ?!\\n\",\n",
" 'synthetic': \"WHAT ??? I DON'T KNOW WHAT TO PRESS NEXT!!! ? !! ?!\"}\n",
"--------------\n",
"{ 'original': \"Please have the manager call me at {{phone_number}} I'd like to join accounts with ms. \"\n",
" '{{first_name}}\\n',\n",
" 'synthetic': \"Please have the manager call me at 443-214-5568 I'd like to join accounts with ms. Tanya.\"}\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------\n",
"{ 'original': 'Inject SELECT * FROM Users WHERE client_ip = ?%//!%20\\\\|{{ip_address}}|%20/\\n',\n",
" 'synthetic': 'Inject SELECT * FROM Users WHERE client_ip = 192.162.94.21'}\n",
"--------------\n",
"{ 'original': '{{first_name}}, can I please speak to your boss?\\n',\n",
" 'synthetic': 'Abigail, can I please speak to your boss?'}\n",
"--------------\n",
"{ 'original': 'May I request to have the statement sent to {{address}}?\\n',\n",
" 'synthetic': 'May I request to have the statement sent to 14 Maple Street, Chicago, IL 60647?'}\n",
"--------------\n",
"{ 'original': \"Will my account stay active? It's under my partner's name {{person}}\\n\",\n",
" 'synthetic': \"Will my account stay active? It's under my partner's name John Smith.\"}\n",
"--------------\n",
"{'original': 'What are my options?\\n', 'synthetic': 'What are my options?'}\n",
"--------------\n",
"{ 'original': 'Bot: Where would you like this to be sent to? User: {{address}}\\n',\n",
" 'synthetic': 'Bot: Where would you like this to be sent to? User: 369 Center Street, New York, NY 10018'}\n",
"--------------\n",
"{ 'original': \"Bot: What's the name on the account? User: {{person}}\\n\",\n",
" 'synthetic': \"Bot: What's the name on the account? User: John Smith\"}\n",
"--------------\n",
"{ 'original': 'I would like to stop receiving messages to {{phone_number}}\\n',\n",
" 'synthetic': 'I would like to stop receiving messages to 714-572-8764.'}\n",
"--------------\n",
"{'original': 'CAN I SPEAK TO A REAL PERSON?!?!\\n', 'synthetic': 'CAN I SPEAK TO A REAL PERSON?!?!'}\n",
"--------------\n",
"{ 'original': \"I'll meet you at {{address}} after the concert.\\n\",\n",
" 'synthetic': \"I'll meet you at 4465 West Street after the concert.\"}\n",
"--------------\n",
"{ 'original': 'I would like to remove my kid {{first_name}} from the will. How do I do that?\\n',\n",
" 'synthetic': 'I would like to remove my kid Sarah from the will. How do I do that?'}\n",
"--------------\n",
"{ 'original': 'The name in the account is not correct, please change it to {{person}}\\n',\n",
" 'synthetic': 'The name in the account is not correct, please change it to Mary Smith.'}\n",
"--------------\n",
"{ 'original': 'Hello I moved, please update my new address is {{address}}\\n',\n",
" 'synthetic': 'Hello I moved, please update my new address is 14 Benson Street, Dolton, Illinois, 60419.'}\n",
"--------------\n",
"{ 'original': 'I need to add my addresses, here they are: {{address}}, and {{address}}\\n',\n",
" 'synthetic': 'I need to add my addresses, here they are: 4284 Taloncourt Road, Lake Travis, TX 76884, and '\n",
" '12 Needle Street, Lawrenceburg, IN 47025.'}\n",
"--------------\n",
"{ 'original': 'Please send my portfolio to this email {{email}}\\n',\n",
" 'synthetic': 'Please send my portfolio to this email johnsmith@example.com'}\n",
"--------------\n",
"{ 'original': 'Hello, this is {{prefix_male}} {{name_male}}. Who are you?\\n',\n",
" 'synthetic': 'Hello, this is Mr. Jackson. Who are you?'}\n",
"--------------\n",
"{ 'original': 'I want to add {{person}} as a beneficiary to my account\\n',\n",
" 'synthetic': 'I want to add Mary Smith as a beneficiary to my account.'}\n",
"--------------\n",
"{ 'original': 'I want to cancel my card {{credit_card_number}} because I lost it\\n',\n",
" 'synthetic': 'I want to cancel my card 2539 3519 2345 1555 because I lost it'}\n",
"--------------\n",
"{ 'original': 'Please block card no {{credit_card_number}}\\n',\n",
" 'synthetic': 'Please block card no 2539 3519 2345 1555'}\n",
"--------------\n",
"{ 'original': 'What is the limit for card {{credit_card_number}}?\\n',\n",
" 'synthetic': 'What is the limit for card 2539 3519 2345 1555?'}\n",
"--------------\n",
"{ 'original': 'Can someone call me on {{phone_number}}? I have some questions about opening an account.\\n',\n",
" 'synthetic': 'Can someone call me on 639-243-7534? I have some questions about opening an account.'}\n",
"--------------\n",
"{'original': 'My name is {{first_name}}\\n', 'synthetic': 'My name is Olivia.'}\n",
"--------------\n",
"{ 'original': \"I'm moving out of the country, so please cancel my subscription\\n\",\n",
" 'synthetic': \"I'm moving out of the country, so please cancel my subscription.\"}\n",
"--------------\n",
"{ 'original': 'My name is {{person}} but everyone calls me {{first_name}}\\n',\n",
" 'synthetic': 'My name is Zara Smith but everyone calls me Zara.'}\n",
"--------------\n",
"{ 'original': \"Please tell me your date of birth. It's {{date_of_birth}}\\n\",\n",
" 'synthetic': \"Please tell me your date of birth. It's May 18th 1996.\"}\n",
"--------------\n",
"{ 'original': 'You said your email is {{email}}. Is that correct?\\n',\n",
" 'synthetic': 'You said your email is jane.doe99@example.com. Is that correct?'}\n",
"--------------\n",
"{ 'original': 'I once lived in {{address}}. I now live in {{address}}\\n',\n",
" 'synthetic': 'I once lived in 189 Maple Street. I now live in 45 McLeod Avenue.'}\n",
"--------------\n",
"{ 'original': \"I'd like to call a taxi to {{address}}. Please call me when you're here.\\n\",\n",
" 'synthetic': \"I'd like to call a taxi to 125 Washington Street. Please call me when you're here.\"}\n",
"--------------\n",
"{ 'original': 'Please charge my credit card. Number is {{credit_card_number}}\\n',\n",
" 'synthetic': 'Please charge my credit card. Number is 9147 3623 4882 7568.'}\n",
"--------------\n",
"{'original': \"What's your email? {{email}}\\n\", 'synthetic': \"What's your email? janet.smith@example.com\"}\n",
"--------------\n",
"{ 'original': \"What's your credit card? {{credit_card_number}}\\n\",\n",
" 'synthetic': \"What's your credit card? 6527 4235 1093 1522?\"}\n",
"--------------\n",
"{'original': \"What's your name? {{person}}\\n\", 'synthetic': \"What's your name? Ibrahim Karim\"}\n",
"--------------\n",
"{'original': \"What's your last name? {{last_name}}\\n\", 'synthetic': \"What's your last name? Lopez\"}\n",
"--------------\n",
"{ 'original': 'How can we reach you? You can call {{phone_number}}\\n',\n",
" 'synthetic': 'How can we reach you? You can call 029 7783 8252.'}\n",
"--------------\n",
"{ 'original': \"I'd like it to be sent to {{address}}\\n\",\n",
" 'synthetic': \"I'd like it to be sent to 1234 Smith Street, Chicago, IL 60616.\"}\n",
"--------------\n",
"{'original': 'Meet me at {{address}}\\n', 'synthetic': 'Meet me at 328 Anderson Street, Boston, MA 02109.'}\n",
"--------------\n",
"{ 'original': 'The restaurant is located at {{address}}. It serves great {{nationality}} food.\\n',\n",
" 'synthetic': 'The restaurant is located at 1214 W. Main Ave. It serves great Italian food.'}\n",
"--------------\n",
"{ 'original': \"So where are we meeting? There's this nice new {{nationality}} place downtown. Cool, what's \"\n",
" \"the address? Oh do they serve vegan stuff? It's in {{address}}\\n\",\n",
" 'synthetic': \"So where are we meeting? There's this nice new Mexican place downtown. Cool, what's the \"\n",
" \"address? Oh do they serve vegan stuff? It's in 9083 Holcomb Street.\"}\n",
"--------------\n",
"{ 'original': \"Hi {{first_name}}, I'm contacting you about a problem I have with sending a wire transfer \"\n",
" 'using this IBAN {{iban}}\\n',\n",
" 'synthetic': \"Hi Kim, I'm contacting you about a problem I have with sending a wire transfer using this \"\n",
" 'IBAN DE56 8123 6888 0139 8956 61.'}\n",
"--------------\n",
"{ 'original': 'She was born on {{date_of_birth}}. Her maiden name is {{last_name}}\\n',\n",
" 'synthetic': 'She was born on 02/15/1965. Her maiden name is Garcia.'}\n",
"--------------\n",
"{'original': 'Sometimes people call me {{first_name}}\\n', 'synthetic': 'Sometimes people call me Evan'}\n",
"--------------\n",
"{'original': \"Maybe it's under {{person}}\\n\", 'synthetic': \"Maybe it's under Malik Robinson\"}\n",
"--------------\n",
"{'original': \"It's like that since {{date_of_birth}}\\n\", 'synthetic': \"It's like that since 06/14/1988\"}\n",
"--------------\n",
"{ 'original': 'Just posted a photo {{url}}\\n',\n",
" 'synthetic': 'Just posted a photo http://www.example.com/q3jx1zgca2.'}\n",
"--------------\n",
"{'original': 'My website is {{url}}\\n', 'synthetic': 'My website is www.synthetictestwebsite.com'}\n",
"--------------\n",
"{'original': 'My IBAN is {{iban}}\\n', 'synthetic': 'My IBAN is FR14 2004 1010 0505 0001 3M02 606.'}\n",
"--------------\n",
"{ 'original': \"I've shared files with you {{url}}\\n\",\n",
" 'synthetic': \"I've shared files with you https://www.myfilesharing.com/agfsosid90\"}\n",
"--------------\n",
"{'original': 'I work for {{organization}}\\n', 'synthetic': 'I work for CorporationX.'}\n",
"--------------\n",
"{ 'original': '{{person}} from {{organization}} is the keynote speaker\\n',\n",
" 'synthetic': 'John Smith from Harvard University is the keynote speaker.'}\n",
"--------------\n",
"{'original': '{{first_name}} is from {{organization}}\\n', 'synthetic': 'Maxwell is from Acme Corporation.'}\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------\n",
"{ 'original': 'The address of {{organization}} is {{address}}\\n',\n",
" 'synthetic': 'The address of Acme Corporation is 1020 Grand Avenue, Los Angeles, CA 90015.'}\n",
"--------------\n",
"{ 'original': 'His social security number is {{ssn}}\\n',\n",
" 'synthetic': 'His social security number is 271-97-6469.'}\n",
"--------------\n",
"{'original': \"Here's my SSN: {{ssn}}\\n\", 'synthetic': \"Here's my SSN: 697-18-3566\"}\n",
"--------------\n",
"{ 'original': '{{first_name_nonbinary}} is a very sympathetic person. They are also good listeners.\\n',\n",
" 'synthetic': 'Piper is a very sympathetic person. They are also good listeners.'}\n",
"--------------\n",
"{ 'original': '{{first_name}} is very reliable. You can always depend on him.\\n',\n",
" 'synthetic': 'Fred is very reliable. You can always depend on him.'}\n",
"--------------\n",
"{'original': 'Why is {{first_name}} so impulsive?\\n', 'synthetic': 'Why is Daniella so impulsive?'}\n",
"--------------\n",
"{ 'original': '{{person}} will be talking in the conference\\n',\n",
" 'synthetic': 'Sally Smith will be talking in the conference.'}\n",
"--------------\n",
"{'original': 'have you heard {{person}} speak yet?\\n', 'synthetic': 'Have you heard Keira Emerson speak yet?'}\n",
"--------------\n",
"{ 'original': 'Have you been to a {{person}} concert before?\\n',\n",
" 'synthetic': 'Have you been to a John Smith concert before?'}\n",
"--------------\n",
"{ 'original': \"I'm so jealous! said {{first_name}} to {{first_name}}\\n\",\n",
" 'synthetic': \"I'm so jealous! said Veronika to Trevor.\"}\n",
"--------------\n",
"{ 'original': 'The true gender of {{first_name}} has been under debate for years, but the riff and building '\n",
" 'energy is a rock masterpiece regardless.\\n',\n",
" 'synthetic': 'The true gender of Miguel has been under debate for years, but the riff and building energy '\n",
" 'is a rock masterpiece regardless.'}\n",
"--------------\n",
"{ 'original': 'For my take on {{prefix_female}} {{last_name_female}}, see Guilty Pleasures: 5 Musicians Of '\n",
" \"The 70s You're Supposed To Hate (But Secretly Love)\\n\",\n",
" 'synthetic': \"For my take on Mrs. Miller, see Guilty Pleasures: 5 Musicians Of The 70s You're Supposed To \"\n",
" 'Hate (But Secretly Love)'}\n",
"--------------\n",
"{ 'original': \"Unlike the {{last_name}} novel, it's not about necrophilia. What it is about, I suppose is \"\n",
" \"anyone's guess. A brilliant piece of baroque pop.\\n\",\n",
" 'synthetic': \"Unlike the Smith novel, it's not about necrophilia. What it is about, I suppose is anyone's \"\n",
" 'guess. A brilliant piece of baroque pop.'}\n",
"--------------\n",
"{ 'original': \"One of the most depressing songs on the list. He's injured from the waist down from \"\n",
" \"{{country}}, but {{first_name}} just has to get laid. Don't go to town, {{first_name}}!\\n\",\n",
" 'synthetic': \"One of the most depressing songs on the list. He's injured from the waist down from Uganda, \"\n",
" \"but Stacey just has to get laid. Don't go to town, Stacey!\"}\n",
"--------------\n",
"{ 'original': 'Is there a better crafted pop song on this list? {{last_name}} and {{last_name}} were '\n",
" 'precision engineers.\\n',\n",
" 'synthetic': 'Is there a better crafted pop song on this list? Smith and Jones were precision engineers.'}\n",
"--------------\n",
"{ 'original': 'C\\'mon, sing it with me: \"You picked a fine time to leave me {{first_name}}, four hungry '\n",
" 'children and a crop in the field...\"\\n',\n",
" 'synthetic': 'C\\'mon, sing it with me: \"You picked a fine time to leave me Lucas, four hungry children and '\n",
" 'a crop in the field...\"'}\n",
"--------------\n",
"{ 'original': \"A tribute to {{person}} – sadly, she wasn't impressed.\\n\",\n",
" 'synthetic': \"A tribute to Paula Harris – sadly, she wasn't impressed.\"}\n",
"--------------\n",
"{ 'original': \"When they weren't singing about Hobbits, satanic felines and interstellar journeys, they were \"\n",
" \"singing about the verses from {{person}}'s Cautionary Tales. Is there a better example of \"\n",
" 'unbridled creativity than early {{last_name}}?\\n',\n",
" 'synthetic': \"When they weren't singing about Hobbits, satanic felines and interstellar journeys, they \"\n",
" \"were singing about the verses from Sally Smith's Cautionary Tales. Is there a better example \"\n",
" \"of unbridled creativity than early Jones'?\"}\n",
"--------------\n",
"{ 'original': 'A great song made even greater by a mandolin coda (not by {{person}}).\\n',\n",
" 'synthetic': 'A great song made even greater by a mandolin coda (not by Justin Bieber).'}\n",
"--------------\n",
"{ 'original': '{{name_male}} listed his top 20 songs for Entertainment Weekly and had the balls to list this '\n",
" 'song at #15. (What did he put at #1 you ask? Answer:\"Tube Snake Boogie\" by {{person}} – go '\n",
" 'figure)\\n',\n",
" 'synthetic': 'John listed his top 20 songs for Entertainment Weekly and had the balls to list this song at '\n",
" '#15. (What did he put at #1 you ask? Answer:\"Tube Snake Boogie\" by ZZ Top – go figure)'}\n",
"--------------\n",
"{ 'original': \"From the film {{nationality}} graffiti (also features {{person}}. What's not to love?\\n\",\n",
" 'synthetic': \"From the film Canadian graffiti (also features Zakari Rigori. What's not to love?\"}\n",
"--------------\n",
"{ 'original': 'You can tell {{first_name}} was a huge {{person}} fan. Written when he was {{age}}.\\n',\n",
" 'synthetic': 'You can tell Isabella was a huge basketball fan. Written when she was 19.'}\n",
"--------------\n",
"{ 'original': \"This song by ex-Zombie {{last_name}} is a perfect example of why you shouldn't concentrate on \"\n",
" 'the order of this list. An argument could be made that this should be at number one, and I '\n",
" \"wouldn't argue with it.\\n\",\n",
" 'synthetic': \"This song by ex-Zombie Williams is a perfect example of why you shouldn't concentrate on the \"\n",
" 'order of this list. An argument could be made that this should be at number one, and I '\n",
" \"wouldn't argue with it.\"}\n",
"--------------\n",
"{ 'original': 'The title refers to {{street_name}} street in {{city}}. It was on this street that many of '\n",
" 'the clubs where Metallica first played were situated. \"Battery is found in me\" shows that '\n",
" 'these early shows on {{street_name}} Street were important to them. Battery is where \"lunacy '\n",
" 'finds you\" and you \"smash through the boundaries.\"\\n',\n",
" 'synthetic': 'The title refers to Washington Street in Chicago. It was on this street that many of the '\n",
" 'clubs where Metallica first played were situated. \"Battery is found in me\" shows that these '\n",
" 'early shows on Washington Street were important to them. Battery is where \"lunacy finds you\" '\n",
" 'and you \"smash through the boundaries.\"'}\n",
"--------------\n",
"{ 'original': 'Blink-182 pay tribute here to the {{country}}. Producer {{person}} explained to Fuse TV: \"We '\n",
" \"all liked the idea of writing a song about our state, where we live and love. To me it's the \"\n",
" 'most beautiful place in the world, this song was us giving credit to how lucky we are to have '\n",
" 'lived here and grown up here, raising families here, the whole thing.\"\\n',\n",
" 'synthetic': 'Blink-182 pay tribute here to the United States. Producer Kim Johnson explained to Fuse TV: '\n",
" '\"We all liked the idea of writing a song about our state, where we live and love. To me '\n",
" \"it's the most beautiful place in the world, this song was us giving credit to how lucky we \"\n",
" 'are to have lived here and grown up here, raising families here, the whole thing.\"'}\n",
"--------------\n",
"{ 'original': 'It may be too that {{last_name}} was influenced by an earlier song, \"Carry Me Back To '\n",
" '{{country}},\" which was arranged and sung by {{person}} in {{year}} (though {{last_name}}\\'s '\n",
" 'song was actually about a boat!).\\n',\n",
" 'synthetic': 'It may be too that Jenkins was influenced by an earlier song, \"Carry Me Back To Trinidad,\" '\n",
" \"which was arranged and sung by Mark Twain in 1901 (though Jenkins's song was actually about \"\n",
" 'a boat!).'}\n",
"--------------\n",
"{ 'original': 'The {{person}} version recorded for {{organization}} became the first celebrity recording by '\n",
" 'a classical musician to sell one million copies. The song was awarded the seventh gold disc '\n",
" 'ever granted.\\n',\n",
" 'synthetic': 'The Helen Smith version recorded for Universal Music Group became the first celebrity '\n",
" 'recording by a classical musician to sell one million copies. The song was awarded the '\n",
" 'seventh gold disc ever granted.'}\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------\n",
"{ 'original': 'In {{country}} they have company songs, musical expressions of employee loyalty sung by '\n",
" 'salarymen. Unfortunately, as regular RR commenter {{person}} points out, \"most are '\n",
" 'horrible\".\\n',\n",
" 'synthetic': 'In Japan they have company songs, musical expressions of employee loyalty sung by salarymen. '\n",
" 'Unfortunately, as regular RR commenter Hannah Jackson points out, \"most are horrible\".'}\n",
"--------------\n",
"{ 'original': '\"The big three\" of The Big Three Killed My Baby are the car manufacturers that dominate the '\n",
" \"economy of the White Stripes' home city {{city}}: {{organization}}, {{organization}} and \"\n",
" '{{organization}}. \"Don\\'t feed me planned obsolescence,\" says {{person}} in an '\n",
" 'uncharacteristically political song, lamenting the demise of the unions in the 60s.\\n',\n",
" 'synthetic': '\"The big three\" of The Big Three Killed My Baby are the car manufacturers that dominate the '\n",
" 'economy of the White Stripes\\' home city Detroit: Ford, General Motors and Chrysler. \"Don\\'t '\n",
" 'feed me planned obsolescence,\" says Thomas Johnson in an uncharacteristically political '\n",
" 'song, lamenting the demise of the unions in the 60s.'}\n",
"--------------\n",
"{ 'original': '{{organization}} songwriter {{name_female}} employs corporate lingo in the first verse of her '\n",
" '{{organization}} resignation Letter\\n',\n",
" 'synthetic': 'Acme Corporation songwriter Hannah Smith employs corporate lingo in the first verse of her '\n",
" 'Acme Corporation resignation Letter.'}\n",
"--------------\n",
"{ 'original': 'Mission Statement: This non-profit founded by radio executives \"serves as an advocate for the '\n",
" 'value of music\" and \"supports its songwriters, composers and publishers by taking care of an '\n",
" 'important aspect of their careers – getting paid,\" according to the {{organization}} website. '\n",
" 'They offer blanket music licenses to businesses and organizations that allow them to play '\n",
" 'nearly 13 million musical works.\\n',\n",
" 'synthetic': 'Mission Statement: This non-profit founded by radio executives \"serves as an advocate for '\n",
" 'the value of music\" and \"supports its songwriters, composers and publishers by taking care '\n",
" 'of an important aspect of their careers – getting paid,\" according to the Music Alliance '\n",
" 'website. They offer blanket music licenses to businesses and organizations that allow them '\n",
" 'to play nearly 13 million musical works.'}\n",
"--------------\n",
"{ 'original': 'The {{organization}} Orchestra was founded in {{year}}. Since then, it has grown from a '\n",
" 'volunteer community orchestra to a fully professional orchestra serving Southern '\n",
" '{{country}}\\n',\n",
" 'synthetic': 'The Chaz Symphony Orchestra was founded in 2015. Since then, it has grown from a volunteer '\n",
" 'community orchestra to a fully professional orchestra serving Southern France.'}\n",
"--------------\n",
"{ 'original': 'Celebrating its 10th year in {{city}}, {{organization}} is a 501(c)3 that invites songwriters '\n",
" 'from around the world to {{city}} to share the universal language of music in collaborations '\n",
" 'designed to bridge cultures, build friendships and cultivate peace.\\n',\n",
" 'synthetic': 'Celebrating its 10th year in Honolulu, Artslana is a 501(c)3 that invites songwriters from '\n",
" 'around the world to Honolulu to share the universal language of music in collaborations '\n",
" 'designed to bridge cultures, build friendships and cultivate peace.'}\n",
"--------------\n",
"{ 'original': '{{organization}} is the brainchild of our 3 founders: {{last_name}}, {{last_name}} and '\n",
" '{{last_name}}. The idea was born (on the beach) while they were constructing a website to be '\n",
" 'the basis of another start-up idea.\\n',\n",
" 'synthetic': 'Acme Co. is the brainchild of our 3 founders: Smith, Nguyen and Takahashi. The idea was born '\n",
" '(on the beach) while they were constructing a website to be the basis of another start-up '\n",
" 'idea.'}\n",
"--------------\n",
"{ 'original': '{{organization}} is an {{nationality}} multinational investment bank and financial services '\n",
" 'company\\n',\n",
" 'synthetic': 'Goldman Sachs is an American multinational investment bank and financial services company.'}\n",
"--------------\n",
"{ 'original': 'Zoolander is a {{year}} {{nationality}} action-comedy film directed by {{person}} and '\n",
" 'starring {{last_name}}\\n',\n",
" 'synthetic': 'Zoolander is a 2001 American action-comedy film directed by Ben Stiller and starring '\n",
" 'Stiller.'}\n",
"--------------\n",
"{ 'original': 'During {{year}}, {{organization}} invested heavily in new microprocessor designs fostering '\n",
" 'the rapid growth of the computer industry.\\n',\n",
" 'synthetic': 'During 2019, Acme Technologies invested heavily in new microprocessor designs fostering the '\n",
" 'rapid growth of the computer industry.'}\n",
"--------------\n",
"{ 'original': 'On {{date_time}}, the {{nationality}} government formally began the process of withdrawal by '\n",
" 'invoking Article 50 of the Treaty on European Union\\n',\n",
" 'synthetic': 'On June 23rd, 2020, the British government formally began the process of withdrawal by '\n",
" 'invoking Article 50 of the Treaty on European Union.'}\n",
"--------------\n",
"{ 'original': '{{first_name}} shouted at {{first_name}}: \"What are you doing here?\"\\n',\n",
" 'synthetic': 'Shadeed shouted at Latonya: \"What are you doing here?\"'}\n",
"--------------\n",
"{ 'original': '{{first_name}} spent a year at {{organization}} as the assistant to {{person}}, and the '\n",
" 'following year at {{organization}} in {{city}}, which later became {{organization}} in '\n",
" '{{year}}.\\n',\n",
" 'synthetic': 'Nina spent a year at Kingsley Academy as the assistant to Dean Roberts, and the following '\n",
" 'year at Brighty Design in San Francisco, which later became Thompkins Designs in 2005.'}\n",
"--------------\n",
"{ 'original': '{{last_name}} began writing as a teenager, publishing her first story, \"The Dimensions of a '\n",
" 'Shadow\", in {{year}} while studying English and journalism at the University of {{city}}.\\n',\n",
" 'synthetic': 'Smith began writing as a teenager, publishing her first story, \"The Dimensions of a Shadow\", '\n",
" 'in 1994 while studying English and journalism at the University of Chicago.'}\n",
"--------------\n",
"{ 'original': \"My driver's license number is {{us_driver_license}}\\n\",\n",
" 'synthetic': \"My driver's license number is VFJ600DN5051EA1N\"}\n",
"--------------\n",
"{ 'original': '{{person}}\\\\n\\\\n{{building_number}} {{street_name}}\\\\n {{secondary_address}}\\\\n {{city}}\\\\n '\n",
" '{{country}} {{postcode}}\\n',\n",
" 'synthetic': 'John Smith \\n123 Main Street \\nApartment 6 \\nNew York City \\nUSA 12254'}\n",
"--------------\n",
"{ 'original': '{{person}}\\\\n\\\\n{{building_number}} {{street_name}}\\\\n {{secondary_address}}\\\\n '\n",
" '{{city}}\\\\n\\\\n {{country}} {{postcode}}\\n',\n",
" 'synthetic': 'John Smith\\n 843 Easton Street\\n Suite 3\\n Hamden\\n \\nUnited States 12345'}\n",
"--------------\n",
"{ 'original': '{{prefix_female}} {{name_female}} {{secondary_address}} {{building_number}} '\n",
" '{{street_name}}\\\\n{{city}} {{state_abbr}} {{zipcode}}\\n',\n",
" 'synthetic': 'Ms. Allison Grobman Apt 12 975 Hamilton Street\\\\\\\\New York NY 11111'}\n",
"--------------\n",
"{ 'original': '{{building_number}} {{street_name}}\\\\n {{secondary_address}}\\\\n {{city}}\\\\n {{country}} '\n",
" '{{postcode}}\\n',\n",
" 'synthetic': '456 Maple Street\\\\n Apt 5D\\\\n San Francisco \\\\n USA 94110'}\n",
"--------------\n",
"{ 'original': 'The corner of {street_name} and {street_name}\\n',\n",
" 'synthetic': 'The corner of Johnson Street and Monroe Avenue'}\n",
"--------------\n",
"{ 'original': 'The restaurant is at {{building_number}} {{street_name}}\\n',\n",
" 'synthetic': 'The restaurant is at 1243 Willow Street.'}\n",
"--------------\n",
"{'original': 'My friend lives in {{city}}\\n', 'synthetic': 'My friend lives in Cairo.'}\n",
"--------------\n",
"{ 'original': '{{first_name}} lives on {{street_name}} street.\\n',\n",
" 'synthetic': 'Maria lives on Liberty Street.'}\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------\n",
"{ 'original': '{{name}} lives at {{building_number}} {{street_name}}, {{city}}\\n',\n",
" 'synthetic': 'John Smith lives at 635 Poplar Street, Houston'}\n",
"--------------\n",
"{ 'original': '{{first_name_male}} had given {{first_name}} his address: {{building_number}} '\n",
" '{{street_name}}\\n',\n",
" 'synthetic': 'Adam had given Sarah his address: 44 Apple Street'}\n",
"--------------\n",
"{ 'original': '{{first_name_male}} had given {{first_name}} his address: {{building_number}} '\n",
" '{{street_name}}, {{city}}\\n',\n",
" 'synthetic': 'David had given Emma his address: 515 Elm Street, Camden.'}\n",
"--------------\n",
"{ 'original': 'What is your address? it is {{address}}\\n',\n",
" 'synthetic': 'What is your address? it is 3498 Allensby Street, Los Angeles, CA 90011.'}\n",
"--------------\n",
"{'original': 'We moved here from {{city}}\\n', 'synthetic': 'We moved here from Paris.'}\n",
"--------------\n",
"{'original': 'We moved here from {{country}}\\n', 'synthetic': 'We moved here from Venezuela.'}\n",
"--------------\n",
"{ 'original': '{{person}}\\\\n\\\\n{{building_number}} {{street_name}}\\\\n {{secondary_address}}\\\\n {{city}}\\\\n '\n",
" '{{country}} {{postcode}}\\\\n{{phone_number}}-Office\\\\,{{phone_number}}-Fax\\n',\n",
" 'synthetic': 'Jessica Thompson\\n'\n",
" ' 8745 West Drive\\n'\n",
" ' Suite 1402\\n'\n",
" ' Brooklyn, NY USA 12009\\n'\n",
" ' 789-534-9921-Office, 567-945-0023-Fax'}\n",
"--------------\n",
"{ 'original': '{{person}}\\\\n{{job}}\\\\n{{organization}}\\\\n{{address}}\\n',\n",
" 'synthetic': 'John Smith\\\\nAccountant\\\\nGlobalTech Solutions\\\\n25 Speedwell Street, Richmond, VA 23223'}\n",
"--------------\n",
"{ 'original': 'Our offices are located at {{address}}\\n',\n",
" 'synthetic': 'Our offices are located at 1234 Main St, Los Angeles, CA 91234.'}\n",
"--------------\n",
"{ 'original': 'Please return to {{address}} in case of an issue.\\n',\n",
" 'synthetic': 'Please return to 123 Cherry Street, El Monte, CA 91731 in case of an issue.'}\n",
"--------------\n",
"{ 'original': '{{organization}}\\\\n\\\\n{{address}}\\n',\n",
" 'synthetic': 'ABC Inc.\\n 1234 Main Street, Anytown, ST 12345'}\n",
"--------------\n",
"{ 'original': 'The {{organization}} office is at {{address}}\\n',\n",
" 'synthetic': 'The ABC Corporation office is at 123 Redwood Street, Anytown, USA.'}\n",
"--------------\n",
"{ 'original': '{{name}}\\\\n{{organization}}\\\\n{{address}}\\\\n{{phone_number}} office\\\\n{{phone_number}} '\n",
" 'fax\\\\n{{phone_number}} mobile\\\\n\\n',\n",
" 'synthetic': 'Larry Fernandez\\\\nStar Enterprises\\\\n421 5th Avenue, Los Angeles, CA 90012\\\\n123-456-7890 '\n",
" 'office\\\\n456-789-0123 fax\\\\n256-454-2397 mobile\\\\n'}\n",
"--------------\n",
"{ 'original': '{{name}}\\\\n{{organization}}\\\\n{{address}}\\\\nMobile: {{phone_number}}\\\\nDesk: '\n",
" '{{phone_number}}\\\\nFax: {{phone_number}}\\\\n\\n',\n",
" 'synthetic': 'John Brown\\n'\n",
" ' ABC Consulting\\n'\n",
" ' 5th St., Suite 116, LA CA 90004\\n'\n",
" ' Mobile: 213-294-4497\\n'\n",
" ' Desk: 424-348-1275\\n'\n",
" ' Fax: 323-456-2545'}\n",
"--------------\n",
"{ 'original': 'Billing address: {{name}}\\\\n {{building_number}} {{street_name}} '\n",
" '{{secondary_address}}\\\\n {{city}}\\\\n {{state_abbr}}\\\\n {{zipcode}}\\\\n\\n',\n",
" 'synthetic': 'Billing address: Mariam Rajput\\n'\n",
" ' 576 Broadway Street Apartment A8\\n'\n",
" ' Sacramento\\n'\n",
" ' CA\\n'\n",
" ' 95349'}\n",
"--------------\n",
"{ 'original': \"As promised, here's {{first_name}}'s address:\\\\n\\\\n{{address}}\\n\",\n",
" 'synthetic': \"As promised, here's Aamir's address:\\\\n\\\\n2166 Sesame Street, Fortaleza, Ceará, Brazil.\"}\n",
"--------------\n",
"{ 'original': '>{{name}}\\\\n>{{organization}}\\\\n>{{person}}\\\\n>{{building_number}} '\n",
" '{{street_name}}\\\\n>{{secondary_address}}\\\\n>{{city}}\\\\n>{{country}} {{postcode}}\\n',\n",
" 'synthetic': '>Freda Chen\\n'\n",
" '>Example Inc.\\n'\n",
" '>John Doe\\n'\n",
" '>50 Main Street\\n'\n",
" '>Apt. 2\\n'\n",
" '>New York\\n'\n",
" '>United States 10005'}\n",
"--------------\n",
"{ 'original': '??? {{name}}\\\\n??? {{organization}}\\\\n??? {{building_number}} {{street_name}}\\\\n??? '\n",
" '{{secondary_address}}\\\\n??? {{city}}\\\\n??? {{country}} {{postcode}}\\n',\n",
" 'synthetic': 'John Smith\\n'\n",
" ' ABC Corporation\\n'\n",
" ' 192 Main Street\\n'\n",
" ' Suite 100\\n'\n",
" ' Austin\\n'\n",
" ' United States 78701'}\n",
"--------------\n",
"{ 'original': '> \\\\n> {{name}}\\\\n> {{organization}}\\\\n> {{person}}\\\\n> {{building_number}} '\n",
" '{{street_name}}\\\\n> {{secondary_address}}\\\\n> {{city}}\\\\n> {{country}} {{postcode}}\\n',\n",
" 'synthetic': '> John Doe\\n'\n",
" ' > ABC Corp\\n'\n",
" ' > Jane Smith\\n'\n",
" ' > 123 Main Street\\n'\n",
" ' > APT 8\\n'\n",
" ' > San Francisco\\n'\n",
" ' > United States 12345'}\n",
"--------------\n",
"{ 'original': 'Pedestrians must enter on {{street_name}} St. the first three months\\n',\n",
" 'synthetic': 'Pedestrians must enter on Jericho Avenue St. the first three months'}\n",
"--------------\n",
"{ 'original': 'When: {{date_time}}\\\\nWhere: {{city}} Country Club.\\n',\n",
" 'synthetic': 'When: 05/01/2020 10:00am\\\\nWhere: Richmond Country Club.'}\n",
"--------------\n",
"{ 'original': \"We'll meet {{day_of_week}} at {{organization}}, {{building_number}} {{street_name}}, \"\n",
" '{{city}}\\n',\n",
" 'synthetic': \"We'll meet Monday at Smartdel Solutions, 145 King Street, San Diego.\"}\n",
"--------------\n",
"{ 'original': 'They had 6: {{first_name}}, {{first_name}}, {{first_name}}, {{first_name}}, {{first_name}} '\n",
" 'and {{first_name}}.\\n',\n",
" 'synthetic': 'They had 6: Sarah, Micheal, Kanak, Hana, Mei and Dan.'}\n",
"--------------\n",
"{'original': 'She moved here from {{country}}\\n', 'synthetic': 'She moved here from Mexico.'}\n",
"--------------\n",
"{'original': 'My zip code is {{zipcode}}\\n', 'synthetic': 'My zip code is 47713.'}\n",
"--------------\n",
"{'original': 'ZIP: {{zipcode}}\\n', 'synthetic': 'ZIP: 08547'}\n",
"--------------\n",
"{'original': 'The bus station is on {{street_name}}\\n', 'synthetic': 'The bus station is on Wilson Avenue.'}\n",
"--------------\n",
"{ 'original': \"They're not answering at {{phone_number}}\\n\",\n",
" 'synthetic': \"They're not answering at 654-339-1013.\"}\n",
"--------------\n",
"{ 'original': 'God gave rock and roll to you, gave rock and roll to you, put it in the soul of everyone.\\n',\n",
" 'synthetic': 'God gave rock and roll to you, gave rock and roll to you, put it in the soul of everyone.'}\n",
"--------------\n",
"{'original': '3... 2... 1... liftoff!\\n', 'synthetic': '3... 2... 1... liftoff!'}\n",
"--------------\n",
"{ 'original': 'My great great grandfather was called {{name_male}}, and my great great grandmother was '\n",
" 'called {{name_female}}\\n',\n",
" 'synthetic': 'My great great grandfather was called Michael, and my great great grandmother was called '\n",
" 'Emma.'}\n",
"--------------\n",
"{'original': 'She named him {{first_name_male}}\\n', 'synthetic': 'She named him Juan.'}\n",
"--------------\n",
"{ 'original': 'Name: {{name}}\\\\nAddress: {{address}}\\n',\n",
" 'synthetic': 'Name: Amari Walters\\\\nAddress: 32 Webster Street, Salem, MA 01819'}\n",
"--------------\n",
"{ 'original': 'Follow up with {{name}} in a couple of months.\\n',\n",
" 'synthetic': 'Follow up with Beatriz Lawrence in a couple of months.'}\n",
"--------------\n",
"{ 'original': '{{prefix_male}} {{last_name_male}} is a {{age}} year old man who grew up in {{city}}.\\n',\n",
" 'synthetic': 'Mr.Williams is a 28 year old man who grew up in Dallas.'}\n",
"--------------\n",
"{ 'original': 'Date: {{date_time}}\\\\nName: {{name}}\\\\nPhone: {{phone_number}}\\n',\n",
" 'synthetic': 'Date: 01/03/2021 13:45 \\\\nName: Pratima Joshi \\\\nPhone: 467-562-8954'}\n",
"--------------\n",
"{ 'original': '{{first_name}}: \"Who are you?\"\\\\n{{first_name_female}}:\"I\\'m {{first_name}}\\'s daughter\".\\n',\n",
" 'synthetic': 'Bob: \"Who are you?\"\\\\nMaria:\"I\\'m Bob\\'s daughter\".'}\n",
"--------------\n",
"{ 'original': 'At my suggestion, one morning over breakfast, she agreed, and on the last Sunday before Labor '\n",
" 'Day we returned to {{city}} by helicopter.\\n',\n",
" 'synthetic': 'At my suggestion, one morning over breakfast, she agreed, and on the last Sunday before '\n",
" 'Labor Day we returned to Paris by helicopter.'}\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------\n",
"{ 'original': \"It was a done thing between him and {{first_name}}'s kid; and everybody thought so.\\n\",\n",
" 'synthetic': \"It was a done thing between him and Jeffery's kid; and everybody thought so.\"}\n",
"--------------\n",
"{ 'original': 'Capitalized words like Wisdom and Discipline are often mistaken with names.\\n',\n",
" 'synthetic': 'Capitalized words like Wisdom and Discipline are often mistaken with names.'}\n",
"--------------\n",
"{ 'original': 'The letter arrived at {{address}} last night.\\n',\n",
" 'synthetic': 'The letter arrived at 1143 Orange Street last night.'}\n",
"--------------\n",
"{ 'original': 'The Princess Royal arrived at {{city}} this morning from {{country}}.\\n',\n",
" 'synthetic': 'The Princess Royal arrived at London this morning from France.'}\n",
"--------------\n",
"{'original': \"I'm in {{city}}, at the conference\\n\", 'synthetic': \"I'm in Toronto, at the conference.\"}\n",
"--------------\n",
"{ 'original': '{{name}}, the {{job}}, said: \"I\\'m glad to hear that this has been withdrawn – quite why they '\n",
" 'thought this would go down well is beyond me.\"\\n',\n",
" 'synthetic': 'Gloria Green, the Nurse Practitioner, said: \"I\\'m glad to hear that this has been withdrawn '\n",
" '– quite why they thought this would go down well is beyond me.\"'}\n",
"--------------\n",
"{ 'original': '\"I\\'m glad to hear that {{country}} is moving in that direction,\" says {{last_name}}.\\n',\n",
" 'synthetic': '\"I\\'m glad to hear that Canada is moving in that direction,\" says Smith.'}\n",
"--------------\n",
"{ 'original': 'I am {{nation_woman}} but I live in {{country}}.\\n',\n",
" 'synthetic': 'I am Marianna Montenegro but I live in Ukraine.'}\n",
"--------------\n",
"{'original': 'We are proud {{nation_plural}}\\n', 'synthetic': 'We are proud Americans.'}\n",
"--------------\n",
"{ 'original': \"{{person}}'s killers sentenced to life in prison\\n\",\n",
" 'synthetic': \"John Smith's killers sentenced to life in prison\"}\n",
"--------------\n",
"{ 'original': \"{{country}} leader gives 'kill without warning' order\\n\",\n",
" 'synthetic': \"Brazilian leader gives 'kill without warning' order\"}\n",
"--------------\n",
"{ 'original': 'The {{nationality}} Border Force have detained top-flight tennis player {{name_female}} over '\n",
" 'visa disputes.\\n',\n",
" 'synthetic': 'The British Border Force have detained top-flight tennis player Maria Rodriguez over visa '\n",
" 'disputes.'}\n",
"--------------\n",
"{ 'original': 'You will be responsible for the husbandry and care of a large variety of species including '\n",
" 'lemurs, antelope, camels, and more\\n',\n",
" 'synthetic': 'You will be responsible for the husbandry and care of a large variety of species including '\n",
" 'lemurs, antelope, camels, and more.'}\n",
"--------------\n",
"{ 'original': '{{name}}\\\\n\\\\n{{job}}\\\\n\\\\nPersonal '\n",
" 'Info:\\\\nPhone:\\\\n{{phone_number}}\\\\n\\\\nE-mail:\\\\n{{email}}\\\\n\\\\nWebsite:\\\\n{{url}}\\\\n\\\\nAddress:\\\\n{{address}}.\\n',\n",
" 'synthetic': 'Robert James\\\\n\\\\nSoftware Engineer\\\\n\\\\nPersonal '\n",
" 'Info:\\\\nPhone:\\\\n555-847-8915\\\\n\\\\nE-mail:\\\\nrobertjames@example.com\\\\n\\\\nWebsite:\\\\nwww.example.com\\\\n\\\\nAddress:\\\\n277 '\n",
" 'Park Ave North, Denver, CO 80100.'}\n",
"--------------\n",
"{ 'original': '{{name}}\\\\n\\\\n{{city}}\\\\n{{country}}\\n',\n",
" 'synthetic': 'John Smith\\n Los Angeles\\n United States'}\n",
"--------------\n",
"{ 'original': 'Title VII of the Civil Rights Act of {{year}} protects individuals against employment '\n",
" 'discrimination on the basis of race and color as well as national origin, sex, or religion.\\n',\n",
" 'synthetic': 'Title VII of the Civil Rights Act of 1964 protects individuals against employment '\n",
" 'discrimination on the basis of race and color as well as national origin, sex, or religion.'}\n",
"--------------\n",
"{ 'original': 'Energetic and driven salesperson with 8+ years of professional experience in inbound and '\n",
" 'outbound sales. Awarded Salesperson of the Month three times. Helped increase inbound sales '\n",
" 'by 16% within the first year of employment. Looking to support {{organization}} in {{city}} '\n",
" '{{zipcode}} in its mission to become a market-leading solution.\\n',\n",
" 'synthetic': 'Energetic and driven salesperson with 8+ years of professional experience in inbound and '\n",
" 'outbound sales. Awarded Salesperson of the Month three times. Helped increase inbound sales '\n",
" 'by 16% within the first year of employment. Looking to support Acme Corporation in Los '\n",
" 'Angeles 90018 in its mission to become a market-leading solution.'}\n",
"--------------\n",
"{ 'original': 'The bus drops you off at {{building_number}} {{street_name}} St.\\n',\n",
" 'synthetic': 'The bus drops you off at 2774 Chestnut St.'}\n",
"--------------\n",
"{ 'original': 'Ask the driver to stop at the corner of {{street_name}} St. and {{street_name}} St.\\n',\n",
" 'synthetic': 'Ask the driver to stop at the corner of Maple St. and Sycamore St.'}\n",
"--------------\n",
"{ 'original': 'He lives on the north side of {{street_name}}.\\n',\n",
" 'synthetic': 'He lives on the north side of Abbey Road.'}\n",
"--------------\n",
"{ 'original': \"I used to work for {{organization}} as {{job}}, but quit a few months ago. Now I'm \"\n",
" 'unemployed.\\n',\n",
" 'synthetic': \"I used to work for ABC Corporation as Software Engineer, but quit a few months ago. Now I'm \"\n",
" 'unemployed.'}\n",
"--------------\n",
"{'original': '{{city}} bridge is falling down.\\n', 'synthetic': 'Berlin bridge is falling down.'}\n",
"--------------\n",
"{ 'original': '{{name}} of {{organization}} is the CEO of the year. ABC Business considered several other '\n",
" \"influential CEOs for this year's honor, including {{name}} of {{organization}}, \"\n",
" \"{{organization}}'s {{name}}, {{name}} of {{organization}}'s, {{name}} of {{organization}}, \"\n",
" \"and {{organization}}'s {{name}}.\\n\",\n",
" 'synthetic': 'Franklin Smith of Technology Solutions International is the CEO of the year. ABC Business '\n",
" \"considered several other influential CEOs for this year's honor, including Madison Chang of \"\n",
" \"Radiance Digital, Radiance Digital's Ashleigh Jones, Cash Huang of Clark & Partner's, Sierra \"\n",
" \"Urbina of Intelicity, and Clark & Partners's Asher Kenney.\"}\n",
"--------------\n",
"{ 'original': '{{organization}} is a design agency based in {{city}}.\\n',\n",
" 'synthetic': 'Maestro Design Inc. is a design agency based in Amsterdam.'}\n",
"--------------\n",
"{ 'original': 'Action & Adventure, Animation, Comedy, Kids & Family, Mystery & Suspense\\\\nDirected By: '\n",
" '{{name}}\\n',\n",
" 'synthetic': 'Action & Adventure, Animation, Comedy, Kids & Family, Mystery & Suspense\\n'\n",
" 'Directed By: Esther Jones'}\n",
"--------------\n",
"{ 'original': '{{first_name}}: What a wife.\\\\n{{first_name}}: Remember me, {{first_name}}? When I killed '\n",
" 'your brother, I talked just like this!\\\\n{{first_name}}: You saved my life! How can I ever '\n",
" 'repay you?\\n',\n",
" 'synthetic': 'Emma: What a wife.\\n'\n",
" ' Emma: Remember me, Emma? When I killed your brother, I talked just like this!\\n'\n",
" ' Emma: You saved my life! How can I ever repay you?'}\n",
"--------------\n",
"{'original': 'He just turned {{age}} years old\\n', 'synthetic': 'He just turned 7 years old'}\n",
"--------------\n",
"{ 'original': \"I'm {{name}}, originally from {{city}}, and i'm {{age}} y/o.\\n\",\n",
" 'synthetic': \"I'm Emily Evanston, originally from London, and I'm 24 y/o.\"}\n",
"--------------\n",
"{ 'original': 'Patient is a {{age}}-year-old male with a history of headaches\\n',\n",
" 'synthetic': 'Patient is a 35-year-old male with a history of headaches'}\n",
"--------------\n",
"{'original': 'I just turned {{age}}\\n', 'synthetic': 'I just turned 24.'}\n",
"--------------\n",
"{'original': 'My father retired at the age of {{age}}\\n', 'synthetic': 'My father retired at the age of 60.'}\n",
"--------------\n",
"{ 'original': 'This {{age}} year old female complaining of stomach pain.\\n',\n",
" 'synthetic': 'This 28 year old female complaining of stomach pain.'}\n",
"--------------\n",
"{ 'original': \"My birthday is on the weekend. I'll turn {{age}}.\\n\",\n",
" 'synthetic': \"My birthday is on the weekend. I'll turn 20.\"}\n",
"--------------\n",
"{'original': 'My brother just turned {{age}}\\n', 'synthetic': 'My brother just turned 18.'}\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------\n",
"{ 'original': '{{prefix}} {{last_name}} flew to {{city}} on {{day_of_week}} morning.',\n",
" 'synthetic': 'Dr. Nguyen flew to Los Angeles on Tuesday morning.'}\n",
"--------------\n"
]
}
],
"source": [
"import time\n",
"pp = pprint.PrettyPrinter(indent=2, width=110)\n",
"sentences = []\n",
"for template in templates:\n",
" synth_sentence = call_completion_model(create_prompt(template))\n",
" sentence_dict = {\"original\": template, \"synthetic\":synth_sentence.strip()}\n",
" sentences.append(sentence_dict)\n",
" pp.pprint(sentence_dict)\n",
" time.sleep(3) # wait to not get blocked by service (only applicable for the free tier)\n",
" print(\"--------------\")\n"
]
},
{
"cell_type": "markdown",
"id": "3981c913",
"metadata": {},
"source": [
"This notebook demonstrates how to leverage OpenAI models for fake/surrogate data generation. It uses Presidio to first de-identify data (as de-identification might be required prior to passing the model to OpenAI), and then uses OpenAI completion models to create synthetic/fake/surrogate data based on real data. OpenAI models would also potentially remove additional PII entities, if those are not detected by Presidio.\n",
"\n",
"Some impressions:\n",
"1. GPT-3 sometimes gives additonal output, especially if the text is a question or concerning a human/bot interaction. Engineering the prompt can mitigate some of these issues. Potential post-processing might be required.\n",
"2. GPT-3 sometimes creates fake values even in the absence of placeholders.\n",
"3. GPT-3 re-uses context from other sentences, which could cause phone numbers are sometimes generated using a credit card pattern or other similar mistakes.\n",
"4. Co-references are sometimes missed (i.e. two name placeholders that should be filled with the same name, or referencing he/she to a male/female name)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "presidio",
"language": "python",
"name": "presidio"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in New Issue View Git Blame Copy Permalink