PII-Detection-Bot

Steps to build a PII Detection Bot using Microsoft Bot Framework SDK and Azure Cognitive Service for Language.

Introduction

Personally Identifiable Information (PII) detection is one of the features offered by Azure Cognitive Service for Language which can identify, categorize, and even redact sensitive information in natural unstructured text. For example: phone numbers, email addresses, and forms of identification.

Redaction example:

Categorization example:

Here, we will be building this bot using:

• Azure Cognitive Service for Language which is a cloud-based service for Natural Language Processing(NLP) helping in understanding and analyzing text.
• Azure Bot Service which is a cloud platform to host bots and make them available to channels like Facebook, MS Teams etc.
• Microsoft Bot Framework SDK which is used to develop conversational applications in Python, Java, JavaScript and C#, that can be hosted on Azure Bot Service.

Pre-requisites

• Python 3.X version:
  • Installation using Anaconda distribution
  • Python Website
• Microsoft Azure Account
• Editor of choice:
  • VSCode
• Terminal of choice with Azure CLI installed.

Steps to create the PII-Detection-Bot

1. Sign-in to your Microsoft Azure account.

2. Create a Resource Group (in a region closest to your location) to logically contain your Azure Resources.

   

3. Create an Azure Cognitive Service for Language inside the Resource Group created above.

   

   • After the Language Service is successfully created, navigate to the language resource to make a note of the Keys and Endpoint which will be used for accessing this resource.

     

4. Open a Python project(like the one explained in echo-bot) in editor of choice(like VSCode).

5. Install the required dependencies as shown in the requirements.txt file using the command:

   pip install -r requirements.txt
   

   This will install the dependencies like botbuilder-core, asyncio, aiohttp, cookiecutter

   

6. Login to Microsoft Azure Account and set your subscription:

   az login
   
   az account set --subscription "<< subscription-id >>"
   

   

7. Create an App Registration in the Azure portal and make a note of its Application (client) ID and appSecret.

   

   

8. Update the config.py file in VSCode with the copied appId and appSecret in the MicrosoftAppId and MicrosoftAppPassword respectively.

   

9. Update bot.py with the Keys and Endpoint of the Azure Language Service which are used to authenticate and connect to the text-analytics-client. The text-analytics-client is used to recognize PII entities in the input user text. The bot contains mainly two functions:

   • on_members_added_activity: this function gets called whenever a user accesses the bot and greets the user with "Hello and welcome to the PII detection bot".

   • on_message_activity: this function gets called whenever a user interacts with the bot. This function takes the user input text and calls the text-analytics-client for PII recognition. This function prints the redacted text and the PII categories of the redacted text.

     

10. Create an App Service and App Service Plan with Azure Resource Manager(ARM) template:

    • Modify the parameters-for-template-BotApp-with-rg.json with values for appServiceName, newAppServicePlanName, newAppServicePlanLocation, appId, appSecret

      

    • Change directory to deploymentTemplates\deployUseExistResourceGroup\

      cd .\deploymentTemplates\deployUseExistResourceGroup\
      
      

    • Create a deployment for App Service and App Service Plan with ARM template-file template-BotApp-with-rg.json:

       az deployment group create --resource-group "<<Resource-Group-Name>>" --template-file "<<template-file-json>>" --parameters "@<<parameters-file-json>>"
      
      

      

11. Create an Azure Bot Service with Azure Resource Manager(ARM) template:

    • Modify the parameters-for-template-AzureBot-with-rg.json with values for azureBotId, botEndpoint, appId.

      

    • Create a deployment for Azure Bot Service with ARM template-file template-AzureBot-with-rg.json:

       az deployment group create --resource-group "<<Resource-Group-Name>>" --template-file "<<template-file-json>>" --parameters "@<<parameters-file-json>>"
      
      

      

12. The creation of Azure Bot Service, App Service, App Service Plan can be verified by navigating to the Azure portal and the respective Resource Group.

    

13. Package the Microsoft Bot Framework SDK(python code) into a .zip file.

14. Deploy the packaged SDK as a web-app

    • Change the working directory to the root of the project.

    • Run the command:

      az webapp deployment source config-zip --resource-group "<<resource-group-name>>" --name "<<app-servicename>>" --src "<<packaged-zip-code>>"
      
      

    • If the command runs successfully, it will deploy the Bot framework SDK as a web-application and the deployment endpoint responds with an HTTP status code 202.

      

15. The deployed Azure Bot PII service can be tested in the Azure portal with any example as shown below:

    • Example 1: My name is John Doe and my phone number is 1234567890. I can be contacted over john.doe@gmail.com. I live in Seattle.

      

    • Example 2: Barack Obama is an American politician who served as the 44th President of the United States from 2009 to 2017. He is the first African American to have served as president, as well as the first born outside the contiguous United States.

      

    • Example 3: You can even pre-order from their online menu at www.contososteakhouse.com, call 312-555-0176 or send email to order@contososteakhouse.com!