Microsoft Azure Form Recognizer Client Library for Python
Project description
Azure Form Recognizer client library for Python
Azure Cognitive Services Form Recognizer is a cloud service that uses machine learning to recognize text and table data from form documents. It includes the following main functionalities:
- Custom models - Recognize field values and table data from forms. These models are trained with your own data, so they're tailored to your forms. You can then take these custom models and recognize forms. You can also manage the custom models you've created and see how close you are to the limit of custom models your account can hold.
- Content API - Recognize text and table structures, along with their bounding box coordinates, from documents. Corresponds to the REST service's Layout API.
- Prebuilt receipt model - Recognize data from USA sales receipts using a prebuilt model.
Source code | Package (PyPI) | API reference documentation| Product documentation | Samples
Getting started
Prerequisites
- Python 2.7, or 3.5 or later is required to use this package.
- You must have an Azure subscription and a Cognitive Services or Form Recognizer resource to use this package.
Install the package
Install the Azure Form Recognizer client library for Python with pip:
pip install azure-ai-formrecognizer
Note: This version of the client library supports the v2.0-preview version of the Form Recognizer service
Create a Form Recognizer resource
Form Recognizer supports both multi-service and single-service access. Create a Cognitive Services resource if you plan to access multiple cognitive services under a single endpoint/key. For Form Recognizer access only, create a Form Recognizer resource.
You can create the resource using
Option 1: Azure Portal
Option 2: Azure CLI. Below is an example of how you can create a Form Recognizer resource using the CLI:
# Create a new resource group to hold the form recognizer resource -
# if using an existing resource group, skip this step
az group create --name my-resource-group --location westus2
# Create form recognizer
az cognitiveservices account create \
--name form-recognizer-resource \
--resource-group my-resource-group \
--kind FormRecognizer \
--sku F0 \
--location westus2 \
--yes
Authenticate the client
Looking up the endpoint
You can find the endpoint for your form recognizer resource using the Azure Portal or Azure CLI:
# Get the endpoint for the form recognizer resource
az cognitiveservices account show --name "resource-name" --resource-group "resource-group-name" --query "endpoint"
Types of credentials
The credential
parameter may be provided as a AzureKeyCredential from azure.core.
See the full details regarding authentication of cognitive services.
To use an API key,
pass the key as a string into an instance of AzureKeyCredential("<api_key>")
.
The API key can be found in the Azure Portal or by running the following Azure CLI command:
az cognitiveservices account keys list --name "resource-name" --resource-group "resource-group-name"
Use the key as the credential parameter to authenticate the client:
from azure.ai.formrecognizer import FormRecognizerClient
from azure.core.credentials import AzureKeyCredential
endpoint = "https://<region>.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("<api_key>")
form_recognizer_client = FormRecognizerClient(endpoint, credential)
Key concepts
FormRecognizerClient
FormRecognizerClient
provides operations for:
- Recognizing form fields and content using custom models trained to recognize your custom forms. These values are returned in a collection of
RecognizedForm
objects. - Recognizing form content, including tables, lines and words, without the need to train a model. Form content is returned in a collection of
FormPage
objects. - Recognizing common fields from US receipts, using a pre-trained receipt model on the Form Recognizer service. These fields and meta-data are returned in a collection of
USReceipt
objects.
FormTrainingClient
FormTrainingClient
provides operations for:
- Training custom models to recognize all fields and values found in your custom forms. A
CustomFormModel
is returned indicating the form types the model will recognize, and the fields it will extract for each form type. See the service's documents for a more detailed explanation. - Training custom models to recognize specific fields and values you specify by labeling your custom forms. A
CustomFormModel
is returned indicating the fields the model will extract, as well as the estimated accuracy for each field. See the service's documents for a more detailed explanation. - Managing models created in your account.
Please note that models can also be trained using a graphical user interface such as the Form Recognizer Labeling Tool.
Long-Running Operations
Long-running operations are operations which consist of an initial request sent to the service to start an operation, followed by polling the service at intervals to determine whether the operation has completed or failed, and if it has succeeded, to get the result.
Methods that train models or recognize values from forms are modeled as long-running operations. The client exposes
a begin_<method-name>
method that returns an LROPoller
. Callers should wait for the operation to complete by
calling result()
on the operation returned from the begin_<method-name>
method. Sample code snippets are provided
to illustrate using long-running operations below.
Examples
The following section provides several code snippets covering some of the most common Form Recognizer tasks, including:
- Recognize Forms Using a Custom Model
- Recognize Content
- Recognize Receipts
- Train a Model
- Manage Your Models
Recognize Forms Using a Custom Model
Recognize name/value pairs and table data from forms. These models are trained with your own data, so they're tailored to your forms. You should only recognize forms of the same form type that the custom model was trained on.
from azure.ai.formrecognizer import FormRecognizerClient
from azure.core.credentials import AzureKeyCredential
endpoint = "https://<region>.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("<api_key>")
form_recognizer_client = FormRecognizerClient(endpoint, credential)
model_id = "<your custom model id>"
# Make sure the form type is one of the types of forms your custom model can recognize
with open("<path to your form>", "rb") as fd:
form = fd.read()
poller = form_recognizer_client.begin_recognize_custom_forms(model_id=model_id, stream=form)
result = poller.result()
for recognized_form in result:
print("Form type ID: {}".format(recognized_form.form_type))
for label, field in recognized_form.fields.items():
print("Field '{}' has value '{}' with a confidence score of {}".format(
label, field.value, field.confidence
))
Recognize Content
Recognize text and table structures, along with their bounding box coordinates, from documents.
from azure.ai.formrecognizer import FormRecognizerClient
from azure.core.credentials import AzureKeyCredential
endpoint = "https://<region>.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("<api_key>")
form_recognizer_client = FormRecognizerClient(endpoint, credential)
with open("<path to your form>", "rb") as fd:
form = fd.read()
poller = form_recognizer_client.begin_recognize_content(form)
page = poller.result()
table = page[0].tables[0] # page 1, table 1
for cell in table.cells:
print(cell.text)
print(cell.bounding_box)
print(cell.confidence)
Recognize Receipts
Recognize data from USA sales receipts using a prebuilt model.
from azure.ai.formrecognizer import FormRecognizerClient
from azure.core.credentials import AzureKeyCredential
endpoint = "https://<region>.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("<api_key>")
form_recognizer_client = FormRecognizerClient(endpoint, credential)
with open("<path to your receipt>", "rb") as fd:
receipt = fd.read()
poller = form_recognizer_client.begin_recognize_receipts(receipt)
result = poller.result()
r = result[0]
print("Receipt contained the following values with confidences: ")
print("Receipt Type: {} has confidence: {}".format(r.receipt_type.type, r.receipt_type.confidence))
print("Merchant Name: {} has confidence: {}".format(r.merchant_name.value, r.merchant_name.confidence))
print("Transaction Date: {} has confidence: {}".format(r.transaction_date.value, r.transaction_date.confidence))
print("Receipt items:")
for item in r.receipt_items:
print("...Item Name: {} has confidence: {}".format(item.name.value, item.name.confidence))
print("...Item Quantity: {} has confidence: {}".format(item.quantity.value, item.quantity.confidence))
print("...Individual Item Price: {} has confidence: {}".format(item.price.value, item.price.confidence))
print("...Total Item Price: {} has confidence: {}".format(item.total_price.value, item.total_price.confidence))
print("Subtotal: {} has confidence: {}".format(r.subtotal.value, r.subtotal.confidence))
print("Tax: {} has confidence: {}".format(r.tax.value, r.tax.confidence))
print("Tip: {} has confidence: {}".format(r.tip.value, r.tip.confidence))
print("Total: {} has confidence: {}".format(r.total.value, r.total.confidence))
Train a model
Train a machine-learned model on your own form type. The resulting model will be able to recognize values from the types of forms it was trained on. Provide a container SAS url to your Azure Storage Blob container where you're storing the training documents. See details on setting this up in the service quickstart documentation.
from azure.ai.formrecognizer import FormTrainingClient
from azure.core.credentials import AzureKeyCredential
endpoint = "https://<region>.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("<api_key>")
form_training_client = FormTrainingClient(endpoint, credential)
container_sas_url = "xxx" # training documents uploaded to blob storage
poller = form_training_client.begin_train_model(container_sas_url)
model = poller.result()
# Custom model information
print("Model ID: {}".format(model.model_id))
print("Status: {}".format(model.status))
print("Created on: {}".format(model.created_on))
print("Last modified: {}".format(model.last_modified))
print("Recognized fields:")
# looping through the submodels, which contains the fields they were trained on
for submodel in model.models:
print("The submodel with form type '{}' has recognized the following fields: {}".format(
submodel.form_type,
", ".join([label for label in submodel.fields])
))
# Training result information
for doc in model.training_documents:
print("Document name: {}".format(doc.document_name))
print("Document status: {}".format(doc.status))
print("Document page count: {}".format(doc.page_count))
print("Document errors: {}".format(doc.errors))
Manage Your Models
Manage the custom models attached to your account.
from azure.ai.formrecognizer import FormTrainingClient
from azure.core.credentials import AzureKeyCredential
from azure.core.exceptions import ResourceNotFoundError
endpoint = "https://<region>.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("<api_key>")
form_training_client = FormTrainingClient(endpoint, credential)
account_properties = form_training_client.get_account_properties()
print("Our account has {} custom models, and we can have at most {} custom models".format(
account_properties.custom_model_count, account_properties.custom_model_limit
))
# Here we get a paged list of all of our custom models
custom_models = form_training_client.list_model_infos()
print("We have models with the following ids: {}".format(
", ".join([m.model_id for m in custom_models])
))
# Now we get the custom model from the "Train a model" sample
model_id = "<model id from the Train a Model sample>"
custom_model = form_training_client.get_custom_model(model_id=model_id)
print("Model ID: {}".format(custom_model.model_id))
print("Status: {}".format(custom_model.status))
print("Created on: {}".format(custom_model.created_on))
print("Last modified: {}".format(custom_model.last_modified))
# Finally, we will delete this model by ID
form_training_client.delete_model(model_id=custom_model.model_id)
try:
form_training_client.get_custom_model(model_id=custom_model.model_id)
except ResourceNotFoundError:
print("Successfully deleted model with id {}".format(custom_model.model_id))
Optional Configuration
Optional keyword arguments can be passed in at the client and per-operation level. The azure-core reference documentation describes available configurations for retries, logging, transport protocols, and more.
Troubleshooting
General
Form Recognizer client library will raise exceptions defined in Azure Core.
Logging
This library uses the standard logging library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.
Detailed DEBUG level logging, including request/response bodies and unredacted
headers, can be enabled on a client with the logging_enable
keyword argument:
import sys
import logging
from azure.ai.formrecognizer import FormRecognizerClient
from azure.core.credentials import AzureKeyCredential
# Create a logger for the 'azure' SDK
logger = logging.getLogger('azure')
logger.setLevel(logging.DEBUG)
# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)
endpoint = "https://<my-custom-subdomain>.cognitiveservices.azure.com/"
credential = AzureKeyCredential("<api_key>")
# This client will log detailed information about its HTTP sessions, at DEBUG level
form_recognizer_client = FormRecognizerClient(endpoint, credential, logging_enable=True)
Similarly, logging_enable
can enable detailed logging for a single operation,
even when it isn't enabled for the client:
poller = form_recognizer_client.begin_recognize_receipts(receipt, logging_enable=True)
Next steps
The following section provides several code snippets illustrating common patterns used in the Form Recognizer Python API.
More sample code
These code samples show common scenario operations with the Azure Form Recognizer client library.
The async versions of the samples (the python sample files appended with _async
) show asynchronous operations
with Form Recognizer and require Python 3.5 or later.
- Recognize receipts: sample_recognize_receipts.py (async version)
- Recognize receipts from a URL: sample_recognize_receipts_from_url.py (async version)
- Recognize content: sample_recognize_content.py (async version)
- Recognize custom forms: sample_recognize_custom_forms.py (async version)
- Train a model without labels: sample_train_model_without_labels.py (async version)
- Train a model with labels: sample_train_model_with_labels.py (async version)
- Manage custom models: sample_manage_custom_models.py (async_version)
Additional documentation
For more extensive documentation on Azure Cognitive Services Form Recognizer, see the Form Recognizer documentation on docs.microsoft.com.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
Change Log azure-ai-formrecognizer
1.0.0b2 (2020-05-06)
Fixes and improvements
- Bug fixed where
confidence
==0.0
was erroneously getting set to1.0
__repr__
has been added to all of the models
1.0.0b1 (2020-04-23)
Version (1.0.0b1) is the first preview of our efforts to create a user-friendly and Pythonic client library for Azure Form Recognizer. This library replaces the package found here: https://pypi.org/project/azure-cognitiveservices-formrecognizer/
For more information about this, and preview releases of other Azure SDK libraries, please visit https://azure.github.io/azure-sdk/releases/latest/python.html.
Breaking changes: New API design
- New namespace/package name:
- The namespace/package name for the Form Recognizer client library has changed from
azure.cognitiveservices.formrecognizer
toazure.ai.formrecognizer
- The namespace/package name for the Form Recognizer client library has changed from
- Two client design:
- FormRecognizerClient to analyze fields/values on custom forms, receipts, and form content/layout
- FormTrainingClient to train custom models (with/without labels), and manage the custom models on your account
- Different analyze methods based on input type: file stream or URL.
- URL input should use the method with suffix
from_url
- Stream methods will automatically detect content-type of the input file
- URL input should use the method with suffix
- Asynchronous APIs added under
azure.ai.formrecognizer.aio
namespace - Authentication with API key supported using
AzureKeyCredential("<api_key>")
fromazure.core.credentials
- New underlying REST pipeline implementation based on the azure-core library
- Client and pipeline configuration is now available via keyword arguments at both the client level, and per-operation. See README for a link to optional configuration arguments
- New error hierarchy:
- All service errors will now use the base type:
azure.core.exceptions.HttpResponseError
- All service errors will now use the base type:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for azure-ai-formrecognizer-1.0.0b2.zip
Algorithm | Hash digest | |
---|---|---|
SHA256 | 68e10a17890a5e686069bb1395cb94cb456ede17b50253fcd6f316796bcd859a |
|
MD5 | 6b302613fbcbab108c91ea4daa7f1798 |
|
BLAKE2b-256 | de4ae5691d2a681637fcfcc89d13368290485f6fb3d85db6c7d21ed6fb9494b1 |
Hashes for azure_ai_formrecognizer-1.0.0b2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f13257c7ee74e62ba6aa95a03f5fa3ed9266e04bcb6e938ffdd963ef25b600f8 |
|
MD5 | 29b77b4d14ee37378d696602e825858a |
|
BLAKE2b-256 | eedaec45ed0fdb7c13725f0d4ca60e53a3e0bb61383db225480d7dff399b9547 |