# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] id="130546d4-d72c-46f1-a506-246ace42ad56"
# <img src="https://github.com/pmservice/ai-openscale-tutorials/raw/master/notebooks/images/banner.png" align="left" alt="banner">

# + [markdown] id="a6d32920-c6f8-4df6-91a7-925640a9920a"
# # Working with a custom metrics provider

# + [markdown] id="ab7073b7-baf8-48ba-b9fb-e39e497596aa"
# This notebook demonstrates how to configure the custom monitor and custom metrics deployment by using IBM Watson OpenScale. It requires service credentials for the following services:
#   * Watson OpenScale
#   * Watson Machine Learning
#
#   
# ## Contents
#
# This notebook contains the following parts:
#
#   1. [Set up your environment.](#setup)
#   1. [Create the custom metrics provider - python function.](#provider)
#   1. [Register the custom metrics provider and create a deployment.](#deployment)
#   1. [Configure Watson OpenScale](#config)
#   1. [Create the integrated system for the custom metrics provider.](#custom)
#   1. [Set up the custom monitor definition and instance.](#instance)
#

# + [markdown] id="8c3c7fcc-8701-45f3-ab34-af1b093b590c"
# ## 1. Set up your environment. <a name="setup"></a>
#
# Before you use the sample code in this notebook, you must perform the following setup tasks:

# + [markdown] id="8b4212ee-4116-4a47-a586-f0997df1ec53"
# ### Install the  `ibm-watson-machine-learning` and `ibm-watson-openscale` packages.

# + id="2eb75b61-93b5-4139-96d5-fae98c8c9173"
# !pip install --upgrade ibm-watson-machine-learning   | tail -n 1
# !pip install --upgrade ibm-watson-openscale --no-cache | tail -n 1

# + [markdown] id="cd55a125-14e4-440c-a2ee-c58813d75f9e"
# ### Action: restart the kernel!

# + [markdown] id="2abe81f0-cd47-4fe0-9b5c-505712c17752"
# ### Credentials for IBM Cloud Pak for Data
# To authenticate, in the following code boxes, replace the sample data with your own credentials. Get the information from your system administrator or through the Cloud Pak for Data dashboard.
#
#
# ### Obtaining your Watson OpenScale credentials
#
# You can retrieve the URL by running the following command: `oc get route -n namespace1 --no-headers | awk '{print $2}'` Replace the `namespace1` variable with your namespace.
#
# You should have been assigned a username and password when you were added to the Cloud Pak for Data system. You might need to ask either your database administrator or your system administrator for some of the information.
#

# + id="3f499be0-0ed7-46c9-9bf4-753c2529525f"
############################################################################################
# Paste your Watson OpenScale credentials into the following section and then run this cell.
############################################################################################
WOS_CREDENTIALS = {
    "url": "<CP4D cluster URL>",
    "username": "*****",
    "password": "*****"
}
# -

#
# ### Enter your Watson OpenScale GUID.
#
# For most systems, the default GUID is already entered for you. You would only need to update this particular entry if the GUID was changed from the default.
#

WOS_GUID="00000000-0000-0000-0000-000000000000"

# + id="4515adc7fa8b46ac8e6e9ac59d396b7f"
WML_CREDENTIALS = WOS_CREDENTIALS.copy()
WML_CREDENTIALS['instance_id']='openshift'
WML_CREDENTIALS['version']='4.0'
WML_CREDENTIALS

# + [markdown] id="9cce9ad8f9bf4fb095680d91ad38b2a0"
# ### Define the Watson OpenScale subscription to which the custom metrics have to be sent.
#
# Create a subscription from the Watson Openscale UI or SDK to configure custom metrics. You can configure custom metrics for the subscriptions that have predefined monitors, such as fairness, quality, or drift or without predefined monitors.

# + id="48519343224a4ca582ee954c3a0e6605"
####################################################################
# Paste your Subscription in the following field and then run this cell.
####################################################################

subscription_id = "<update the subscription id here>"


# + [markdown] id="51dab6f2e41045f38ead783922814c0a"
# ## 2. Create the custom metrics provider - Python function. <a name="provider"></a>
#
# The Python function receives the required variables, such as the `datamart_id`, `monitor_instance_id`, `monitor_id`, `monitor_instance_parameters` and `subscription_id` from the Watson OpenScale service when it is invoked by the custom monitor. 
#
# In the Python function, add your own logic to compute the custom metrics in the `get_metrics` method, publish the metrics to the Watson Openscale service and update the status of the run to the `finished` state in the custom monitor instance.
#
# Update the `WOS_CREDENTIALS` in the Python function. 

# + id="e7ec6979ce114f528246ba99e8619090"
#wml_python_function
def custom_metrics_provider():
    
    import json
    import requests
    import base64
    from requests.auth import HTTPBasicAuth
    import time
    import uuid    
    
    #Update the WOS_CREDENTIALS here
    WOS_CREDENTIALS = {
        "url": "<CP4D cluster URL>",
        "username": "*****",
        "password": "*****"
    }
    
    # Get the access token
    def get_access_token():
        token_url = WOS_CREDENTIALS['url'] + '/v1/preauth/validateAuth'
        headers = {}
        headers["Accept"] = "application/json"
        auth = HTTPBasicAuth(WOS_CREDENTIALS['username'], WOS_CREDENTIALS['password'])
        response = requests.get(token_url, headers=headers, auth=auth, verify=False)
        json_data = response.json()
        access_token = json_data['accessToken']
        return access_token
    
    # Get the WOS base url
    def get_wos_url(service_instance_id):
        return WOS_CREDENTIALS['url'] + '/openscale' + '/' + service_instance_id
    
    # construct the auth headers for communication with WOS APIs
    def get_headers(access_token):
        headers = {}
        headers["Content-Type"] = "application/json"
        headers["Accept"] = "application/json"
        headers["Authorization"] = "Bearer {}".format(access_token)
        return headers
    
    #Get the current time 
    def get_current_time():
        import datetime
        date_format = "%Y-%m-%dT%H:%M:%S.%fZ"
        now = datetime.datetime.utcnow()
        timestamp = now.strftime(date_format)
        return timestamp
    
    #Update the run status to Finished in the custom monitor instance
    def update_monitor_instance(base_url, access_token, custom_monitor_instance_id, payload):
        monitor_instance_url = base_url + '/v2/monitor_instances/' + custom_monitor_instance_id + '?update_metadata_only=true'
        
        patch_payload  = [
            {
                "op": "replace",
                "path": "/parameters",
                "value": payload
            }
        ]
        response = requests.patch(monitor_instance_url, headers=get_headers(access_token), json = patch_payload, verify=False)
        monitor_response = response.json()
        return response.status_code, monitor_response
    
    #Add your code to compute the custom metrics. 
    def get_metrics(subscription_id):
        #Add the logic here to compute the metrics. Use the below metric names while creating the custom monitor definition
        dummy_metrics = {"specificity": 1.2, "sensitivity": 0.85,"region": "us-south"}
        
        return dummy_metrics
        
        
    # Publishes the Custom Metrics to OpenScale
    def publish_metrics(base_url, access_token, subscription_id, custom_monitor_id, custom_monitor_instance_id, custom_monitoring_run_id):
        # Generate an monitoring run id, where the publishing happens against this run id
        timestamp = get_current_time()
        measurements_payload = [
                  {
                    "timestamp": timestamp,
                    "run_id": custom_monitoring_run_id,
                    "metrics": [get_metrics(subscription_id)]
                  }
                ]
        measurements_url = base_url + '/v2/monitor_instances/' + custom_monitor_instance_id + '/measurements'
        response = requests.post(measurements_url, headers=get_headers(access_token), json = measurements_payload, verify=False)
        published_measurement = response.json()
     
        return response.status_code, published_measurement
        
    
    def publish( input_data ):
        payload = input_data.get("input_data")[0].get("values")
        data_mart_id = payload['data_mart_id']
        subscription_id = payload['subscription_id']
        custom_monitor_id = payload['custom_monitor_id']
        custom_monitor_instance_id = payload['custom_monitor_instance_id']
        custom_monitor_instance_params  = payload['custom_monitor_instance_params']

        base_url = get_wos_url(data_mart_id)
        access_token = get_access_token()
        
        published_measurements = []
        error_msg = None
        
        custom_monitoring_run_id = custom_monitor_instance_params["run_details"]["run_id"]
        try:
            status_code, published_measurement = publish_metrics(base_url, access_token, subscription_id, custom_monitor_id, custom_monitor_instance_id, custom_monitoring_run_id)
            if int(status_code) in [200, 201, 202]:
                custom_monitor_instance_params["run_details"]["run_status"] = "finished"
                published_measurements.append(published_measurement)
            else:
                custom_monitor_instance_params["run_details"]["run_status"] = "error"
                custom_monitor_instance_params["run_details"]["run_error_msg"] = published_measurement
                error_msg = published_measurement
                    
            custom_monitor_instance_params["last_run_time"] = get_current_time()
            status_code, response = update_monitor_instance(base_url, access_token, custom_monitor_instance_id, custom_monitor_instance_params)
            if not int(status_code) in [200, 201, 202]:
                error_msg = response
                
        except Exception as ex:
            error_msg = str(ex)
        if error_msg is None:
            response_payload = {
                "predictions" : [{ 
                    "values" : published_measurements
                }]

            }
        else:
            response_payload = {
                "error_msg": error_msg
            }
        
        return response_payload
        
    return publish
    

# + [markdown] id="47683fe263944fafb25f856214b1ec70"
# ## 3. Register the custom metrics provider and create a deployment. <a name="deployment"></a>

# + id="d8948f9b03e849d9aba1f74d7e7830c3"
import json
from ibm_watson_machine_learning import APIClient

wml_client = APIClient(WML_CREDENTIALS)
wml_client.version

# + id="15601ba7120748b9902f57ca107067b6"
wml_client.spaces.list(limit=10)

# + id="2f9a983b13d84b40bf2b5544ac11d078"
space_name = "<your space Name>" #update your space id
spaces = wml_client.spaces.get_details()['resources']
space_id = None
for space in spaces:
    if space['entity']['name'] == space_name:
        space_id = space["metadata"]["id"]

print(space_id)
wml_client.set.default_space(space_id)
# -

PYTHON_FUNCTION_NAME = 'Custom Metrics Provider Function' #update your python function name
DEPLOYMENT_NAME = 'Custom Metrics Provider Deployment'#update your deployment name

# ### Remove existing function and deployment.

# +
deployments_list = wml_client.deployments.get_details()
for deployment in deployments_list["resources"]:
    model_id = deployment["entity"]["asset"]["id"]
    deployment_id = deployment["metadata"]["id"]
    if deployment["metadata"]["name"] == DEPLOYMENT_NAME:
        print("Deleting deployment id", deployment_id)
        wml_client.deployments.delete(deployment_id)
        print("Deleting model id", model_id)
        wml_client.repository.delete(model_id)

wml_client.repository.list_functions()



# + [markdown] id="36a1b7cb5d3f48049fd42edcb471d157"
# ### Create the function meta properties.
#

# + id="bd7e0a1e303c41b9b90815e8c2b33894"
software_spec_id =  wml_client.software_specifications.get_id_by_name('default_py3.7_opence')
print(software_spec_id)
function_meta_props = {
     wml_client.repository.FunctionMetaNames.NAME: PYTHON_FUNCTION_NAME,
     wml_client.repository.FunctionMetaNames.SOFTWARE_SPEC_ID: software_spec_id
     }

# + [markdown] id="e7fbbccd0f2344d097433ebdaf04469c"
# ### Store the Python function.

# + id="2d2aef0d0e1e47048cbf4abd94aac631"
function_artifact = wml_client.repository.store_function(meta_props=function_meta_props, function=custom_metrics_provider)
function_uid = wml_client.repository.get_function_id(function_artifact)
print("Function UID = " + function_uid)

# + id="82a7d7df8fd344158108d57be229c5fa"
function_details = wml_client.repository.get_details(function_uid)
from pprint import pprint
pprint(function_details)

# + [markdown] id="8b89b9d948f242ffa77bde26a8e9cb81"
# ### Deploy the Python function.
#

# + id="d1c33df40ea94f4ca7f8de257818b417"
hardware_spec_id = wml_client.hardware_specifications.get_id_by_name('M')
hardware_spec_id

# + [markdown] id="6774445edd764e708ba6b4612edaed71"
# ### Create deployment metadata for the Python function.

# + id="888a3c84fc06461f8ebc2120dfd58b22"
deploy_meta = {
 wml_client.deployments.ConfigurationMetaNames.NAME: DEPLOYMENT_NAME,
 wml_client.deployments.ConfigurationMetaNames.ONLINE: {},
 wml_client.deployments.ConfigurationMetaNames.HARDWARE_SPEC: { "id": hardware_spec_id}
}

# + [markdown] id="3273246ba52a49a1a7e7e98964d2cb63"
# ### Create a deployment.

# + id="7b5a9733edfb484e9a8533d22581bd8d"
deployment_details = wml_client.deployments.create(function_uid, meta_props=deploy_meta)
# -

# ### Get the scoring URL.
#

# + id="bb56417d0e6e444484c25da9e6048aa5"
created_at = deployment_details['metadata']['created_at']
find_string_pos = created_at.find("T")
if find_string_pos is not -1:
    current_date = created_at[0:find_string_pos]
scoring_url = wml_client.deployments.get_scoring_href(deployment_details)
scoring_url = scoring_url + "?version="+current_date
print(scoring_url)


# + [markdown] id="73c85183a81f4997837587b2c64753c6"
# ## 4. Configure OpenScale. <a name="config"></a>
#
# Import the required libraries and set up the Watson OpenScale Python client.

# + id="addc28e79d494e68a5cb6a942d4f8bac"
from ibm_watson_openscale import APIClient
from ibm_watson_openscale.base_classes.watson_open_scale_v2 import MonitorMeasurementRequest
from ibm_watson_openscale.base_classes.watson_open_scale_v2 import MonitorMetricRequest
from ibm_watson_openscale.base_classes.watson_open_scale_v2 import MetricThreshold
from ibm_watson_openscale.supporting_classes.enums import MetricThresholdTypes
from ibm_watson_openscale.base_classes.watson_open_scale_v2 import MonitorTagRequest
from ibm_watson_openscale.base_classes.watson_open_scale_v2 import Target
from ibm_watson_openscale.supporting_classes.enums import TargetTypes
from ibm_watson_openscale.base_classes.watson_open_scale_v2 import IntegratedSystems

from datetime import datetime, timezone, timedelta
import uuid    



# + id="1bb826627adf4a2ba2e2bc20d5ab25a2"
from ibm_cloud_sdk_core.authenticators import CloudPakForDataAuthenticator

from ibm_watson_openscale import *
from ibm_watson_openscale.supporting_classes.enums import *
from ibm_watson_openscale.supporting_classes import *


authenticator = CloudPakForDataAuthenticator(
        url=WOS_CREDENTIALS['url'],
        username=WOS_CREDENTIALS['username'],
        password=<PASSWORD>['password'],
        disable_ssl_verification=True
    )
wos_client = APIClient(service_url=WOS_CREDENTIALS['url'],authenticator=authenticator, service_instance_id=WOS_GUID)
wos_client.version



# + [markdown] id="09b669c1459a4f75ad0d040d433795e4"
# ## 5. Create the integrated system for the custom metrics provider. <a name="custom"></a>
#
#
# Update the custom metrics deployment URL, which is created during the Python function creation in the integrated system. Watson OpenScale invokes the deployment URL at runtime to compute the custom metrics. 
#
# You must define the authentication type based on the communication with custom metrics deployment. Watson OpenScale supports 2 types of authentication: basic and bearer. If custom metrics deployment accepts the `basic` authentication type, then provide `auth_type=basic` otherwise use `auth_type=bearer`.

# +
CUSTOM_METRICS_ENGINE_NAME = "Sample Custom Metrics Engine" # update the metrics engine name here 

auth_type = "bearer" #Supported values are basic and bearer

if auth_type == "basic":
    CUSTOM_METRICS_ENGINE_CREDENTIALS = {
        "auth_type":"basic",
        "username":  "*****",# update the username here 
        "password": "*****"# Update the password here
   }
    
if auth_type == "bearer":
    CUSTOM_METRICS_ENGINE_CREDENTIALS = {
        "auth_type":"bearer",
        "username":"*****",# update the username here 
        "apikey": "*****" #update the apikey of the cpd cluster where custom metrics deployment is created
   }
    
  #if custom metrics deployment is on other cpd cluster or some other cloud then please uncomment and update 
  #the below "TOKEN_INFO" properties to generate the token to communicate to the custom metrics deployment url
  #Here are the sample values given in the token_info
    #TOKEN_INFO = {
    #    "url":  "https://iam.ng.bluemix.net/oidc/token", # update the token generation here 
    #    "headers": { "Content-type": "application/x-www-form-urlencoded" }, # update the headers here 
    #    "payload": "grant_type=urn:ibm:params:oauth:grant-type:apikey&response_type=cloud_iam&apikey=<api_key>", # update the payload here 
    #    "method": "POST" # # update the http method here 
    #}
    #CUSTOM_METRICS_ENGINE_CREDENTIALS["token_info"] = TOKEN_INFO

# -

# ### Remove existing integrated system 

# Delete existing custom metrics provider integrated systems if present
integrated_systems = IntegratedSystems(wos_client).list().result.integrated_systems
for system in integrated_systems:
    if system.entity.type == 'custom_metrics_provider' and system.entity.name == CUSTOM_METRICS_ENGINE_NAME:
        print("Deleting integrated system {}".format(system.entity.name))
        IntegratedSystems(wos_client).delete(integrated_system_id=system.metadata.id)

# +
custom_metrics_integrated_system = IntegratedSystems(wos_client).add(
    name=CUSTOM_METRICS_ENGINE_NAME,
    description=CUSTOM_METRICS_ENGINE_NAME,
    type="custom_metrics_provider",
    credentials= CUSTOM_METRICS_ENGINE_CREDENTIALS,
    connection={
        "display_name": CUSTOM_METRICS_ENGINE_NAME,
        "endpoint": scoring_url
    }
).result

integrated_system_id = custom_metrics_integrated_system.metadata.id
print(custom_metrics_integrated_system)
# -

# ## 6. Set up the custom monitor definition and instance. <a name="instance"></a>
#

###################################################################
# UPDATE your custom monitor name in the following field and then run this cell.
####################################################################
CUSTOM_MONITOR_NAME = 'Sample model performance' 


# + [markdown] id="59b9827e31ab45edbad79789213703d1"
# ### Check for the existence of the custom monitor definition.
#

# + id="14e3f1e2e2f24bfd86f86feee24619e0"
def get_custom_monitor_definition():
    monitor_definitions = wos_client.monitor_definitions.list().result.monitor_definitions
    for definition in monitor_definitions:
        if CUSTOM_MONITOR_NAME == definition.entity.name:
            return definition
    return None   


# + [markdown] id="30720ce70ddf485782ebf767a606366b"
# ### Create the  custom monitor definition.
#
# Update the custom metric names, threshold types (`LOWER_LIMIT`, `UPPER_LIMIT`) and default values as required. You can define the threshold type as lower limit, upper limit, or both.
# -

###################################################################
# Update your custom monitor metrics names in the following field. Use the same metric names for creating the 
# monitor definition and publishing the metrics to Openscale in your python function
####################################################################
CUSTOM_MONITOR_METRICS_NAMES = ['sensitivity','specificity']
#Update the tag values if you want to fetch the metrics by tags
TAGS= ['region']
TAG_DESCRIPTION =['customer geographical region'] 


# + id="df7bfd7b6f274d54897133f94dc14310"
#Update the Threshold types and default values of the metrics
def custom_metric_definitions():
    
    metrics = [MonitorMetricRequest(name=CUSTOM_MONITOR_METRICS_NAMES[0],
                                    thresholds=[MetricThreshold(type=MetricThresholdTypes.LOWER_LIMIT, default=0.8)]),
              MonitorMetricRequest(name=CUSTOM_MONITOR_METRICS_NAMES[1],
                                 thresholds=[MetricThreshold(type=MetricThresholdTypes.LOWER_LIMIT, default=0.6),MetricThreshold(type=MetricThresholdTypes.UPPER_LIMIT, default=1)])]
    #Comment the below tags code if there are no tags to be created
    tags = [MonitorTagRequest(name=TAGS[0], description=TAG_DESCRIPTION[0])]
    
    return metrics, tags


# + id="79e2f275e13948e5834e77ec4dfadeb6"
def create_custom_monitor_definition():
    # check if the custom monitor definition already exists or not
    existing_definition = get_custom_monitor_definition()

    # if it does not exists, then create a new one.
    if existing_definition is None:
        metrics, tags = custom_metric_definitions()
        custom_monitor_details = wos_client.monitor_definitions.add(name=CUSTOM_MONITOR_NAME, metrics=metrics, tags=tags, background_mode=False).result
    else:
        # otherwise, send the existing definition
        custom_monitor_details = existing_definition
    return custom_monitor_details


# + id="8d352bcf09e646d0a6d174e54078f2c6"
custom_monitor_details = create_custom_monitor_definition()
custom_monitor_id = custom_monitor_details.metadata.id
custom_monitor_id


# + [markdown] id="3a0dbf50822c40c680cc9430eb88306c"
# ### Check the existence of custom monitor instance.
#

# + id="7176b942f61f4d0381fb63b92101e76f"
def get_custom_monitor_instance(custom_monitor_id):
    monitor_instances = wos_client.monitor_instances.list(data_mart_id = WOS_GUID, monitor_definition_id = custom_monitor_id, target_target_id = subscription_id).result.monitor_instances
    if len(monitor_instances) == 1:
        return monitor_instances[0]
    return None


# +
# Openscale MRM service invokes custom metrics deployment url during runtime and wait for the default time of 60 second's to 
# to check the run status ie finished/Failed and fetch the latest measurement. Increase the wait time, if the runtime deployment 
# takes more than 60 seconds to compute and publish the custom metrics 

#Update the wait time here.
custom_metrics_wait_time = 60 #time in seconds <update the time here>


# -

# ### Update the custom monitor instance.

def update_custom_monitor_instance(custom_monitor_instance_id):
    payload = [
     {
       "op": "replace",
       "path": "/parameters",
       "value": {
           "custom_metrics_provider_id": integrated_system_id,
           "custom_metrics_wait_time":   custom_metrics_wait_time 
       }
     }
    ]
    response = wos_client.monitor_instances.update(custom_monitor_instance_id, payload, update_metadata_only = True)
    result = response.result
    return result


# + [markdown] id="678df1de54164d6284cdf9f255affd8a"
# ### For the custom monitor definition, create a custom monitor instance.
#

# + id="97624e1c0a8e4648af05162690c96c66"
def create_custom_monitor_instance(custom_monitor_id):
    # Check if an custom monitor instance already exists
    existing_monitor_instance = get_custom_monitor_instance(custom_monitor_id)

    # If it does not exist, then create one
    if existing_monitor_instance is None:
        target = Target(
                target_type=TargetTypes.SUBSCRIPTION,
                target_id=subscription_id
            )
        parameters = {
            "custom_metrics_provider_id": integrated_system_id,
            "custom_metrics_wait_time":   custom_metrics_wait_time 
        }
        # create the custom monitor instance id here.
        custom_monitor_instance_details = wos_client.monitor_instances.create(
                    data_mart_id=WOS_GUID,
                    background_mode=False,
                    monitor_definition_id=custom_monitor_id,
                    target=target,
                    parameters=parameters
        ).result
    else:
        # otherwise, update the existing one with latest integrated system details.
        instance_id = existing_monitor_instance.metadata.id
        custom_monitor_instance_details = update_custom_monitor_instance(instance_id)
    return custom_monitor_instance_details


# + id="ffb159c5c1f54ffe9f1b7a48b727810c"
monitor_instance_details = create_custom_monitor_instance(custom_monitor_id)
custom_monitor_instance_id = monitor_instance_details.metadata.id
print(monitor_instance_details)
# -

# ### Invoke the custom metrics deployment Python function.
#
# Validate the custom metrics provider deployment by providing the correct set of paramaters to generate the custom metrics.

# +
import uuid
parameters = {
    "custom_metrics_provider_id": integrated_system_id,
    "custom_metrics_wait_time":   custom_metrics_wait_time,
    "run_details": {
    "run_id": str(uuid.uuid4()),
    "run_status": "Running"
    }
}

payload= {
    "data_mart_id" : WOS_GUID,
    "subscription_id" : subscription_id,
    "custom_monitor_id" : custom_monitor_id,
    "custom_monitor_instance_id" : custom_monitor_instance_id,
    "custom_monitor_instance_params": parameters
    
}

input_data= { "input_data": [ { "values": payload } ]
            }


deployment_uid=wml_client.deployments.get_uid(deployment_details)
job_details = wml_client.deployments.score(deployment_uid, input_data)
pprint(job_details)

# + [markdown] id="210c3b5835494f67812f0704bad84705"
# ## Congratulations
#
# You have finished configuring Custom Monitor Definition and Monitor instance for your Subscription. You can now run the custom monitor from `Watson OpenScale Dashboard`(https://url-to-your-cp4d-cluster/aiopenscale). Click the tile of your model and select `Evaluate Now` option from `Actions` drop down menu to run the monitor.

StarcoderdataJupyterScriptsDedupFiltered