AI Builder Form Processing Ideas

We’re in the Digital Transformation age and everyone are using mobile phones, computers, and tablet to solve business problems. We now automate business processes to work productively and efficiently. One of the main process that business users wanted to automate is the paper approval. Business users want to automate this because of the issue where you need to look for the approver in his/her desk to approve your form. If the approver is out of reach, your paper will just be in your desk waiting until the Approver arrives.

With all the digital transformation and automation, Power Platform released General Availability of AI Builder last October 2019 but some of its features are still in preview. And one of the features that I’m playing around is the Form-Processing (Preview).


While I’m doing some proposal for work, I did this scene where the user will create a list in SharePoint, attached an invoice form to the item and then submit. Once submitted, the Power Automate will trigger to get the attached file from the newly created item. It will then read the invoice form using AI Model Form-Processing. Once it is extracted, the data from the invoice form using the AI Model will then update the list item in SharePoint. The below image is the Power Automate that I’ve been working on. I will show you the detailed setup that I have for my Power Automate in my next post.

For now, below is the image for the flow.

Power Automate 
Action items 
My flows 
Al Builder 
Search for helpful resources 
When an item is created 
Get attachments from current item 
Apply to each 
Personal (default) (orqfa 
Save te_9 Flow checker 
Select an Output 
from previous steps 
body x 
Get file content in current item 
Parse 'SON 
Update item 
Add an action 
+ New step

Flow for picking up the attached file from SharePoint item

How Does it Work

Form-Processing extracts data fields and tables from the form. In your Power Apps environment, you can see the AI Builder in the left pane of the Web Studio. From there, you can build your Models like Form Processing, Object Detection, Prediction and other available AI productivity.

AI Builder in Power Apps

In building the  AI Model using the Form-Processing, you need to have a minimum of 5 documents to train your model. The model can detect duplicate document. Since it is a duplicate, it will ignore and will not upload the document.

Information if the document uploaded has a duplicate.

After you are done uploading all the documents, you start to Analyze the document.

Add documents 
5 documents added 
Select form fields 
Model summary 
-F Add documents Get sample documents 
Add document examples 
Were analyzing your documents 
This may take a few minutes 
5 documents added

Analyzing the document uploaded

After analyzing all the documents, now you start to select the fields that you are going to extract from the document.

Select all 
X Clear all 
3 form fields selected 
V Done 
FormProcessingAlModel form 
Selected fields All fields 
Bill To 
table_O (Item, Quantity, Rate, 
Tax (2%) 

Selecting the fields that you want to extract from the document

Once done selecting the fields, you train the model.

Your model is training 
This may take a while. You can close this window and come back later. 
Go to models

Train the model

After training the model, you will then publish it. After publishing the model, you can now start using it in your Power Automate and Power Apps.

Power Automate 
Action items 
My flows 
Al Builder 
P Search for helpful resources 
Share (preview) Settings 
Your model isn't published yet. Publish to use it in apps and flows. 
Models > 
Form Processing (preview) • Not published • Anj Cerbolles 
Selected fields 
Bill To 
Tax (2%) 
See more 
Personal (default) (orqfa 
How your model is used 
Power Automate 
Model isn't published 
Power Apps 
Model isn't published 
Training document 
See details 
Quick test

Your Form-Processing Model

My Thoughts

This is only my own experience and comments with regards to AI Builder Form-Processing. As I went through my testing and configuration, some things need to address and this would be useful as well. As I have some experience with AI application for the extraction of the data through scanning of the documents. Probably this would be helpful as well for this feature.

  1. When you are done uploading the file and done with analyzing the document if you don’t want to use the document that has been analyzed and need to upload a new document. The new document will not be analyzed anymore. You need to start creating a new model again and start analyzing.
  2. There is no option for the trained model to manually re-train it. Let’s say for example you have more than 20 documents that already analyze but then, the document doesn’t properly extract the fields or the tables. You can’t re-train the model.
  3. There is no option for you to split the documents if there are multiple pages. I know part of the best practice is no multiple pages. If these can be extended to multiple pages, this could be a great value to the app. Especially when the form reaches even the two pages only.
  4. You can’t adjust the positioning of the extracted fields or tables. Let’s say, you have a four-column table. S/No, Description, Rates, Remarks are the headers. During the analysis of your model, extracted fields of the Rates are somewhat nearer to the fields of the Remarks. If the table of the document that you are going to extract is not the same alignment for the trained model. There will be a possibility that the Rates will be read as Remarks if the Description field is wider than the trained model.
  5. If there could be a manual training of the model, we can define the unique keyword of the document. From there, you can set manually the coordinates of the keywords. You can identify if the keyword that you are looking are either in the TOP header or in the footer. 

We know that Form-Processing is still in Preview. There is still room for improvement. If this thing can be addressed, AI Builder Form-Processing can be a powerful tool for extracting forms. And because it is under the Power Platform, you can leverage the Power Apps and Power Automate more and it would be easy to build and can leverage more on the business process.

Featured image by Pixabay