<jupyter_start><jupyter_text>Combining Pre-Built & Custom AI ServicesIn this notebook, you will integrate with the Computer Vision API and the Text Analytics API to augment the claims processing capabilities. In the end, you will integrate the API calls to the summarizer and classifier services that your deployed and produce a finished claim report that shows all of the processing applied to the claim text and claim image.  Task 1 - Caption & Tag with the Computer Vision API In the cell bellow, provided the key to your Computer Vision API and run the cell.<jupyter_code>subscription_key = '' #"<your_computer_vision_api_key>"
assert subscription_key<jupyter_output><empty_output><jupyter_text>Construct the endpoint to the Computer Vision API by running the following cell. Notice the last path segment is analyze, which indicates you will use the analyze feature.Be sure to update the value in vision_endpoint below so it matches the Endpoint value you copied from the Azure Portal for your instance of the Computer Vision service. Be sure your value ends in a slash (/)<jupyter_code>vision_endpoint = '' #"<your_computer_vision_api_endpoint>"
vision_base_url = vision_endpoint + "vision/v1.0/"
vision_analyze_url = vision_base_url + "analyze"<jupyter_output><empty_output><jupyter_text>The following cell contains a list of sample images found after performing a simple web search. Feel free to substitute in URLs to the image of your choice.<jupyter_code>fender_bender = "https://www.washingtonpost.com/blogs/innovations/files/2015/02/Stolen_Car_Crash-00aef.jpg"
damaged_house = "https://c2.staticflickr.com/8/7342/10983313185_0589b74946_z.jpg"
police_car = "https://localtvwnep.files.wordpress.com/2015/11/fender-bender.jpeg"
car_with_text = "https://static.buildasign.com/cmsimages/bas-vinyl-lettering-splash-01.png"
car_tow = 'https://i.ytimg.com/vi/wmxJ2FrzTWo/maxresdefault.jpg'<jupyter_output><empty_output><jupyter_text>From the list of images above, select one and assign it to image_url for further processing:<jupyter_code>image_url = car_tow<jupyter_output><empty_output><jupyter_text>Run the following cell to preview the image you have selected.<jupyter_code>from IPython.display import Image, display
display(Image(image_url))<jupyter_output><empty_output><jupyter_text>The following cell builds the HTTP request to make against the Computer Vision API.Run the following cell to retrieve the caption and tags:<jupyter_code>import requests
headers  = {'Ocp-Apim-Subscription-Key': subscription_key }
params   = {'visualFeatures': 'Categories,Description,Tags,Color'}
data     = {'url': image_url}
response = requests.post(vision_analyze_url, headers=headers, params=params, json=data)
response.raise_for_status()
analysis = response.json()
analysis<jupyter_output><empty_output><jupyter_text>As you can see in the above output, the result is a nested document structure. Run the following cells to pull out the caption and top 3 tag results:<jupyter_code>caption = analysis["description"]["captions"][0]["text"].capitalize()
caption
topTags = analysis["description"]["tags"][0:3]
topTags<jupyter_output><empty_output><jupyter_text>Task 2 - Performing OCR In order to perform OCR with the Computer Vision service, you need to target the OCR endpoint.Run the following cell to construct the right URL:<jupyter_code>vision_ocr_url = vision_base_url + "ocr"<jupyter_output><empty_output><jupyter_text>Next, invoke the OCR endpoint with the following code and examine the result:<jupyter_code>headers  = {'Ocp-Apim-Subscription-Key': subscription_key }
params   = {}
data     = {'url': image_url}
response = requests.post(vision_ocr_url, headers=headers, params=params, json=data)
response.raise_for_status()
ocr_analysis = response.json()
ocr_analysis<jupyter_output><empty_output><jupyter_text>We have provided the following code for you to extract the text as a flat array from the results.Run the following cell to extract the text items from the results document:<jupyter_code>import itertools
flatten = lambda x: list(itertools.chain.from_iterable(x))
words_list = [[ [w['text'] for w in line['words']]  for line in d['lines']] for d in ocr_analysis['regions']]
words_list = flatten(flatten(words_list))
print(list(words_list))<jupyter_output><empty_output><jupyter_text>Task 3 - Performing Sentiment Analysis Sentiment Analysis is performed using the Text Analytics API.Update the following cell with the key to your instance of the Text Analytics API and run the cell:<jupyter_code>text_analytics_subscription_key = '' #"<your_text_analytics_key>"
assert text_analytics_subscription_key<jupyter_output><empty_output><jupyter_text>Update the following cell with the correct base URL for your deployed instance of the Text Analytics API and run the cell:<jupyter_code>#"<your_text_analytics_base_url>"
text_analytics_base_url = ''
sentiment_api_url = text_analytics_base_url + "/text/analytics/v2.1/sentiment"<jupyter_output><empty_output><jupyter_text>The following cell has a set of example claims you can use to test the measurement sentiment.Run the cell:<jupyter_code>neg_sent = """We are just devastated and emotionally drained. 
The roof was torn off of our car, and to make matters
worse my daughter's favorite teddy bear was impaled on the street lamp."""
pos_sent = """We are just happy the damaage was mininmal and that everyone is safe. 
We are thankful for your support."""
neutral_sent = """I crashed my car."""
long_claim = """
I was driving down El Camino and stopped at a red light.
It was about 3pm in the afternoon. The sun was bright and shining just behind the stoplight.
This made it hard to see the lights. There was a car on my left in the left turn lane.
A few moments later another car, a black sedan pulled up behind me. 
When the left turn light changed green, the black sedan hit me thinking 
that the light had changed for us, but I had not moved because the light 
was still red. After hitting my car, the black sedan backed up and then sped past me.
I did manage to catch its license plate. The license plate of the black sedan was ABC123. 
"""<jupyter_output><empty_output><jupyter_text>From the above list of claims, select one and assign its variable to claim_text to be used in the call to the Text Analytics API.<jupyter_code>claim_text = long_claim<jupyter_output><empty_output><jupyter_text>The API requires you to submit a document of the following form.Run the cell to build the request document:<jupyter_code>documents = {'documents' : [
    {'id': '1', 'language': 'en', 'text': claim_text}
]}<jupyter_output><empty_output><jupyter_text>Now invoke the Text Analytics API and observe the result.<jupyter_code>headers   = {"Ocp-Apim-Subscription-Key": text_analytics_subscription_key}
response  = requests.post(sentiment_api_url, headers=headers, json=documents)
sentiments = response.json()
sentiments<jupyter_output><empty_output><jupyter_text>To parse out the sentiment score from the response, run the following cell:<jupyter_code>score = sentiments['documents'][0]['score']
score<jupyter_output><empty_output><jupyter_text>You can provide a human-friendly interpretation on this score by running the following cell:<jupyter_code>score_interpretation = "neutral"
if (score < 0.45): 
    score_interpretation = "negative"
elif (score >= 0.55):
    score_interpretation = "positive"
score_interpretation<jupyter_output><empty_output><jupyter_text>Task 4 - Invoking the Azure ML Deployed Services Run the following cell to define a method that will be used to invoke your classifier and summarizer methods deployed using Azure Machine Learning service to Azure Container Instances:<jupyter_code>def invoke_service(ml_service_key, ml_service_scoring_endpoint, ml_service_input):
    headers   = {"Authorization": "Bearer " + ml_service_key}
    response  = requests.post(ml_service_scoring_endpoint, headers=headers, json=ml_service_input)
    result = response.json()
    return result<jupyter_output><empty_output><jupyter_text>Configure the classifier invocation with the key and endpoint as appropriate to your deployed instance:<jupyter_code>classifier_service_key = "" #leave this value empty if the service does not have authentication enabled
#"<your_classifier_scoring_url>"
classifier_service_scoring_endpoint = ''
classifier_service_input = [claim_text]<jupyter_output><empty_output><jupyter_text>Invoke the classifier and observe the result:<jupyter_code>classifier_result = invoke_service(classifier_service_key, 
                                   classifier_service_scoring_endpoint, classifier_service_input)
classifier_result
# Interpret the classifier result
classification = 'Auto Insurance Claim' if classifier_result == 1 else 'Home Insurance Claim' 
classification<jupyter_output><empty_output><jupyter_text>Similarly, configure the key and scoring endpoint as appropriate to your summarizer service:<jupyter_code>summarizer_service_key = "" #leave this value empty if the service does not have authentication enabled
#"<your_summarizer_service_url>"
summarizer_service_scoring_endpoint = ''
summarizer_service_input = claim_text<jupyter_output><empty_output><jupyter_text>Invoke the summarizer service and observe the result:<jupyter_code>summarizer_result = invoke_service(summarizer_service_key, summarizer_service_scoring_endpoint, 
                                   summarizer_service_input)
formatted_result =  summarizer_result[0].replace("\\n", " ").strip() if len(summarizer_result) > 0 else "N/A"
formatted_result<jupyter_output><empty_output><jupyter_text>Task 5 - Summarizing the ResultsIn this final task, you pull together all of the pieces to display the results of your AI based processing.Run the following cell and examine the result.<jupyter_code>from IPython.core.display import HTML

displayTemplate = """
<div><b>Claim Summary</b></div>
<div>Classification: {}</div>
<div>Caption: {}</div>
<div>Tags: {}</div>
<div>Text in Image: {}</div>
<div>Sentiment: {}</div>
<div><img src='{}' width='200px'></div>
<div>Summary: </div>
<div><pre>{} </pre></div>
<div>&nbsp;</div>
<div>Claim:</div>
<div>{}</div>

"""
displayTemplate = displayTemplate.format(classification, caption, ' '.join(topTags), ' '.join(words_list), 
                                         score_interpretation, image_url, formatted_result, 
                                         claim_text)
display(HTML(displayTemplate))<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup