<jupyter_start><jupyter_text>Predicting Wine Quality by using Watson Machine Learning <NAME>This notebook contains steps and code to create a predictive model to predict Wine Quality and then deploy that model to Watson Machine Learning so it can be used in an application. Learning GoalsThe learning goals of this notebook are:* Load a CSV file into the  Object Storage Service linked to my Data Science Experience * Create an Pandas machine learning model* Train and evaluate a model* Persist a model in a Watson Machine Learning repository 1. SetupBefore you use the sample code in this notebook, you must perform the following setup tasks:* Create a Watson Machine Learning Service instance (a free plan is offered) and associate it with your project* Upload wine quality  data to the Object Store service that is part of your data Science Experience trial  2. Load and explore dataIn this section load the data as Pandas DataFrame and perform a basic exploration.Load the data to the Pandas DataFrame from the associated Object Storage instance.<jupyter_code>import types
import pandas as pd
from ibm_botocore.client import Config
import ibm_boto3

def __iter__(self): return 0

# @hidden_cell
# The following code accesses a file in your IBM Cloud Object Storage. It includes your credentials.
# You might want to remove those credentials before you share your notebook.
client_8c14de1d42f947c2812d9d92c44fc593 = ibm_boto3.client(service_name='s3',
    ibm_api_key_id='<KEY>',
    ibm_auth_endpoint="https://iam.bluemix.net/oidc/token",
    config=Config(signature_version='oauth'),
    endpoint_url='https://s3-api.us-geo.objectstorage.service.networklayer.com')

body = client_8c14de1d42f947c2812d9d92c44fc593.get_object(Bucket='default-donotdelete-pr-ijbvuypsfmigjt',Key='winequality-red.csv')['Body']
# add missing __iter__ method, so pandas accepts body as file-like object
if not hasattr(body, "__iter__"): body.__iter__ = types.MethodType( __iter__, body )

red = pd.read_csv(body)
red.head()
body = client_8c14de1d42f947c2812d9d92c44fc593.get_object(Bucket='default-donotdelete-pr-ijbvuypsfmigjt',Key='winequality-white.csv')['Body']
# add missing __iter__ method, so pandas accepts body as file-like object
if not hasattr(body, "__iter__"): body.__iter__ = types.MethodType( __iter__, body )

white = pd.read_csv(body)
white.head()<jupyter_output><empty_output><jupyter_text>Explore the loaded data by using the following Pandas DataFrame methods:* print dataframe info* print head records* count tail records* print sample records* describe dataframe* check isnull values in dataframe<jupyter_code># Print info on white wine
print(white.info())

# Print info on red wine
print(red.info())
# First rows of `red` 
red.head()
# Last rows of `white`
white.tail()
# Take a sample of 5 rows of `red`
red.sample(5)
# Describe `white`
white.describe()

# Double check for null values in `red`
pd.isnull(red)<jupyter_output><empty_output><jupyter_text>3 Interactive Visualizations with Matplotlib and Numpy 3.1: Visualize Alcohol vs FrequencyDistribution of Alcohol in % Vol for red and white wines.<jupyter_code>import matplotlib.pyplot as plt

fig, ax = plt.subplots(1, 2)

ax[0].hist(red.alcohol, 10, facecolor='red', alpha=0.5, label="Red wine")
ax[1].hist(white.alcohol, 10, facecolor='white', ec="black", lw=0.5, alpha=0.5, label="White wine")

fig.subplots_adjust(left=0, right=1, bottom=0, top=0.5, hspace=0.05, wspace=1)
ax[0].set_ylim([0, 1000])
ax[0].set_xlabel("Alcohol in % Vol")
ax[0].set_ylabel("Frequency")
ax[1].set_xlabel("Alcohol in % Vol")
ax[1].set_ylabel("Frequency")
fig.suptitle("Distribution of Alcohol in % Vol")

plt.show()<jupyter_output><empty_output><jupyter_text>3.2: Print histograms of alcohol using NumpyHistogram of alcohol for red and white wines.<jupyter_code>import numpy as np
print(np.histogram(red.alcohol, bins=[7,8,9,10,11,12,13,14,15]))
print(np.histogram(white.alcohol, bins=[7,8,9,10,11,12,13,14,15]))<jupyter_output>(array([  0,   7, 673, 452, 305, 133,  21,   8]), array([ 7,  8,  9, 10, 11, 12, 13, 14, 15]))
(array([   0,  317, 1606, 1256,  906,  675,  131,    7]), array([ 7,  8,  9, 10, 11, 12, 13, 14, 15]))<jupyter_text>3.3: Visualize Quality vs SulphatesQuality vs Sulphates for red and white wines.<jupyter_code>import matplotlib.pyplot as plt

fig, ax = plt.subplots(1, 2, figsize=(8, 4))

ax[0].scatter(red['quality'], red["sulphates"], color="red")
ax[1].scatter(white['quality'], white['sulphates'], color="white", edgecolors="black", lw=0.5)

ax[0].set_title("Red Wine")
ax[1].set_title("White Wine")
ax[0].set_xlabel("Quality")
ax[1].set_xlabel("Quality")
ax[0].set_ylabel("Sulphates")
ax[1].set_ylabel("Sulphates")
ax[0].set_xlim([0,10])
ax[1].set_xlim([0,10])
ax[0].set_ylim([0,2.5])
ax[1].set_ylim([0,2.5])
fig.subplots_adjust(wspace=0.5)
fig.suptitle("Wine Quality by Amount of Sulphates")

plt.show()<jupyter_output><empty_output><jupyter_text>3.4: Visualize Quality vs Volatile Acidity vs AlcoholQuality vs volatile acidity vs alcohol for red and white wines.<jupyter_code>import matplotlib.pyplot as plt
import numpy as np

np.random.seed(570)

redlabels = np.unique(red['quality'])
whitelabels = np.unique(white['quality'])

import matplotlib.pyplot as plt
fig, ax = plt.subplots(1, 2, figsize=(8, 4))
redcolors = np.random.rand(6,4)
whitecolors = np.append(redcolors, np.random.rand(1,4), axis=0)

for i in range(len(redcolors)):
    redy = red['alcohol'][red.quality == redlabels[i]]
    redx = red['volatile acidity'][red.quality == redlabels[i]]
    ax[0].scatter(redx, redy, c=redcolors[i])
for i in range(len(whitecolors)):
    whitey = white['alcohol'][white.quality == whitelabels[i]]
    whitex = white['volatile acidity'][white.quality == whitelabels[i]]
    ax[1].scatter(whitex, whitey, c=whitecolors[i])
    
ax[0].set_title("Red Wine")
ax[1].set_title("White Wine")
ax[0].set_xlim([0,1.7])
ax[1].set_xlim([0,1.7])
ax[0].set_ylim([5,15.5])
ax[1].set_ylim([5,15.5])
ax[0].set_xlabel("Volatile Acidity")
ax[0].set_ylabel("Alcohol")
ax[1].set_xlabel("Volatile Acidity")
ax[1].set_ylabel("Alcohol") 
#ax[0].legend(redlabels, loc='best', bbox_to_anchor=(1.3, 1))
ax[1].legend(whitelabels, loc='best', bbox_to_anchor=(1.3, 1))
#fig.suptitle("Alcohol - Volatile Acidity")
fig.subplots_adjust(top=0.85, wspace=0.7)

plt.show()<jupyter_output><empty_output><jupyter_text>4. Create Pandas machine learning modelIn this section I prepare data, create and train Pandas machine learning model. 4.1: Prepare dataIn this subsection data is joined and prepared: labels are separated from the features.<jupyter_code># Append `white` to `red`
wines = red.append(white, ignore_index=True)
wines.shape
# Isolate target labels
y = wines.quality

# Isolate data
X = wines.drop('quality', axis=1)<jupyter_output><empty_output><jupyter_text>4.2: Visualize data using Seaborn heatmapHeatmap<jupyter_code>import seaborn as sns
corr = wines.corr()
sns.heatmap(corr, 
            xticklabels=corr.columns.values,
            yticklabels=corr.columns.values,
            cmap="YlGnBu")<jupyter_output><empty_output><jupyter_text>4.3: Preprocess DataStandardize features by removing the mean and scaling to unit variance<jupyter_code># Import `StandardScaler` from `sklearn.preprocessing`
from sklearn.preprocessing import StandardScaler
import numpy as np

# Scale the data with `StandardScaler`
X = StandardScaler().fit_transform(X)<jupyter_output><empty_output><jupyter_text>4.4: Creating modelCreating model using K fold validation partitions<jupyter_code>import numpy as np
from sklearn.model_selection import StratifiedKFold
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import SGD, RMSprop

seed = 7
np.random.seed(seed)

kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=seed)
for train, test in kfold.split(X, y):
    model = Sequential()
    model.add(Dense(128, input_dim=11, activation='relu'))
    model.add(Dense(1))
    
#     rmsprop = RMSprop(lr=0.001)
    sgd=SGD(lr=0.01)
    model.compile(optimizer=sgd, loss='mse', metrics=['mae'])

#     model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])
    model.fit(X[train], y[train], epochs=10, verbose=1)<jupyter_output>Epoch 1/10
5195/5195 [==============================] - 0s - loss: 2.3357 - mean_absolute_error: 1.0516     
Epoch 2/10
5195/5195 [==============================] - 0s - loss: 0.7159 - mean_absolute_error: 0.6542     
Epoch 3/10
5195/5195 [==============================] - 0s - loss: 0.5845 - mean_absolute_error: 0.5949     
Epoch 4/10
5195/5195 [==============================] - 0s - loss: 0.5359 - mean_absolute_error: 0.5701     
Epoch 5/10
5195/5195 [==============================] - 0s - loss: 0.5145 - mean_absolute_error: 0.5563     
Epoch 6/10
5195/5195 [==============================] - 0s - loss: 0.5057 - mean_absolute_error: 0.5530     
Epoch 7/10
5195/5195 [==============================] - 0s - loss: 0.4992 - mean_absolute_error: 0.5495     
Epoch 8/10
5195/5195 [==============================] - 0s - loss: 0.4900 - mean_absolute_error: 0.5453     
Epoch 9/10
5195/5195 [==============================] - 0s - loss: 0.4872 - mean_absolute_error: 0.5442     
Epoch 10/10
5195/51[...]<jupyter_text>4.5: Evaluate modelEvaluate model by checking  Mean Squared Error (MSE) and the Mean Absolute Error (MAE) and R2 score or the regression score function<jupyter_code>mse_value, mae_value = model.evaluate(X[test], y[test], verbose=0)

print(mse_value)
print(mae_value)
from sklearn.metrics import r2_score
y_pred = model.predict(X[test])
r2_score(y[test], y_pred)
y_pred
# Model output shape
model.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_29 (Dense)             (None, 128)               1536      
_________________________________________________________________
dense_30 (Dense)             (None, 1)                 129       
=================================================================
Total params: 1,665
Trainable params: 1,665
Non-trainable params: 0
_________________________________________________________________<jupyter_text>4.6: Compare the resultsCheck the predictions<jupyter_code>y_pred = y_pred.astype(int)
predictions = np.column_stack((y[test], y_pred));
print(predictions[:15])<jupyter_output>[[5 5]
 [5 4]
 [7 5]
 [5 5]
 [4 4]
 [5 5]
 [6 5]
 [5 5]
 [6 5]
 [5 5]
 [5 4]
 [5 5]
 [5 6]
 [6 6]
 [5 4]]<jupyter_text>5. Persist modelImport client libraries.<jupyter_code>wml_credentials={
   "url": "https://ibm-watson-ml.mybluemix.net",
   "username": "0faaa0df-0f3a-4aa7-835d-8929d6cda36e",
   "password": "<PASSWORD>",
   "instance_id": "2074d379-2141-42dc-9340-a78bf626de46"
 }
from watson_machine_learning_client import \
  WatsonMachineLearningAPIClient
client = WatsonMachineLearningAPIClient(wml_credentials)
h5file = 'winemodel-keras.h5'
gzfile = 'winemodel-keras.tar.gz'
model.save(h5file)
import tarfile
with tarfile.open(gzfile, 'w:gz') as tf:
    tf.add(h5file)
metadata = {
         client.repository.ModelMetaNames.NAME: 'Wine Quality Prediction Model',
         client.repository.ModelMetaNames.FRAMEWORK_NAME: 'tensorflow',
         client.repository.ModelMetaNames.FRAMEWORK_VERSION: '1.3',
         client.repository.ModelMetaNames.RUNTIME_NAME: 'python',
         client.repository.ModelMetaNames.RUNTIME_VERSION: '3.5',
         client.repository.ModelMetaNames.FRAMEWORK_LIBRARIES: [{'name':'keras', 'version': '2.1.3'}]
}
published_model = client.repository.store_model(model=gzfile, meta_props=metadata)
published_model<jupyter_output><empty_output><jupyter_text>6. Load model to verify that it was saved correctlyYou can load your model to make sure that it was saved correctly.<jupyter_code>import json
import requests
from base64 import b64encode

token_url = service_path + "/v3/identity/token"

# NOTE: for python 2.x, uncomment below, and comment out the next line of code:
#userAndPass = b64encode(bytes(username + ':' + password)).decode("ascii")
# Use below for python 3.x, comment below out for python 2.x
userAndPass = b64encode(bytes(username + ':' + password, "utf-8")).decode("ascii")
headers = { 'Authorization' : 'Basic %s' %  userAndPass }

response = requests.request("GET", token_url, headers=headers)

watson_ml_token = json.loads(response.text)['token']
print(watson_ml_token)<jupyter_output>eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXVCJ9.eyJ0ZW5hbnRJZCI6IjIwNzRkMzc5LTIxNDEtNDJkYy05MzQwLWE3OGJmNjI2ZGU0NiIsImluc3RhbmNlSWQiOiIyMDc0ZDM3OS0yMTQxLTQyZGMtOTM0MC1hNzhiZjYyNmRlNDYiLCJwbGFuSWQiOiIzZjZhY2Y0My1lZGU4LTQxM2EtYWM2OS1mOGFmM2JiMGNiZmUiLCJyZWdpb24iOiJ1cy1zb3V0aCIsInVzZXJJZCI6IjRhMGU2NzRkLTBmYzUtNDkzMi05NGExLTgxZDc4MGE0NGNlZSIsImlzcyI6Imh0dHBzOi8vdXMtc291dGgubWwuY2xvdWQuaWJtLmNvbS92My9pZGVudGl0eSIsImlhdCI6MTU1MTE5MTg1OCwiZXhwIjoxNTUxMjIwNjU4LCJjcmVhdGVkVGltZSI6MTU1MTE5MTg1OH0.BU28YSSv2P2iFBwINWKlkN9NkorRkXqLU0hS9uxmuRWKpNRPx1V-cx8Ix3xFrxkicMTqzEBwUXi0UP8iB5KyjRNrZO_OqwBoqmn04Jyf6tPIzJARNOVfkIzr2_bazSK6mpYWboJWsOYlnFq37R_mQRQGdmxLaExwe1W8leohKSJwpdVMszlcOHfS7nXLo2j1kGPaGSCbKJCj0lPcN4-Od4QBeRmlhy4mA9ucGfIkAmKDbryV5GWQR7McyVYB4izcBeJdaZHbLVN073_myD-fjRkMq13cOemoa9ynFQYH_Jy4I0YSmsVIowJk6c5kZb0UNCrRhq-CWW19-t9xA9GRLg<jupyter_text>6.2 Preview currenly published models<jupyter_code>model_url = service_path + "/v3/wml_instances/" + instance_id + "/published_models"

headers = {'authorization': 'Bearer ' + watson_ml_token }
response = requests.request("GET", model_url, headers=headers)

published_models = json.loads(response.text)
print(json.dumps(published_models, indent=2))<jupyter_output>{
  "limit": 1000,
  "first": {
    "url": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/2074d379-2141-42dc-9340-a78bf626de46/published_models?limit=1000"
  },
  "resources": [
    {
      "metadata": {
        "created_at": "2019-02-21T13:31:38.699Z",
        "url": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/2074d379-2141-42dc-9340-a78bf626de46/published_models/4116080a-a8f8-4147-9cdd-79bec04900ea",
        "modified_at": "2019-02-21T13:38:58.094Z",
        "guid": "4116080a-a8f8-4147-9cdd-79bec04900ea"
      },
      "entity": {
        "latest_version": {
          "created_at": "2019-02-21T13:31:38.812Z",
          "url": "https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/4116080a-a8f8-4147-9cdd-79bec04900ea/versions/1cea7dd4-1637-4173-8c7f-c17224f39ac1",
          "guid": "1cea7dd4-1637-4173-8c7f-c17224f39ac1"
        },
        "runtime_environment": "spark-2.1",
        "name": "Heart Failure Prediction Model",
        "learning_configuration_url": "http[...]<jupyter_text>Read the details of any returned models<jupyter_code>print('{} model(s) are available in your Watson ML Service'.format(len(published_models['resources'])))
for model in published_models['resources']:
    print('\t- name:        {}'.format(model['entity']['name']))
    print('\t  model_id:    {}'.format(model['metadata']['guid']))
    print('\t  deployments: {}'.format(model['entity']['deployments']['count']))<jupyter_output>3 model(s) are available in your Watson ML Service
	- name:        Heart Failure Prediction Model
	  model_id:    4116080a-a8f8-4147-9cdd-79bec04900ea
	  deployments: 1
	- name:        Wine Quality Prediction Model
	  model_id:    213f75a7-406f-4c9d-b1ca-6f07edc7bbca
	  deployments: 2
	- name:        Wine Quality Prediction Model
	  model_id:    419dc1fe-812e-4bc9-9d6c-ee61c1febced
	  deployments: 1<jupyter_text>Create a new deployment of the Model<jupyter_code># Update this `model_id` with the model_id from model that you wish to deploy listed above.
model_id = '213f75a7-406f-4c9d-b1ca-6f07edc7bbca'

deployment_url = service_path + "/v3/wml_instances/" + instance_id + "/published_models/" + model_id + "/deployments"

payload = "{\"name\": \"Wine Quality Prediction Model Deployment\", \"description\": \"First deployment of Wine Quality Prediction Model\", \"type\": \"online\"}"
headers = {'authorization': 'Bearer ' + watson_ml_token, 'content-type': "application/json" }

response = requests.request("POST", deployment_url, data=payload, headers=headers)

print(response.text)
deployment = json.loads(response.text)

print('Model {} deployed.'.format(model_id))
print('\tname: {}'.format(deployment['entity']['name']))
print('\tdeployment_id: {}'.format(deployment['metadata']['guid']))
print('\tstatus: {}'.format(deployment['entity']['status']))
print('\tscoring_url: {}'.format(deployment['entity']['scoring_url']))<jupyter_output>Model 213f75a7-406f-4c9d-b1ca-6f07edc7bbca deployed.
	name: Wine Quality Prediction Model Deployment
	deployment_id: 7de2f5a3-44a1-439d-ada8-ce9e4c73eb60
	status: DEPLOY_SUCCESS
	scoring_url: https://ibm-watson-ml.mybluemix.net/v3/wml_instances/2074d379-2141-42dc-9340-a78bf626de46/published_models/213f75a7-406f-4c9d-b1ca-6f07edc7bbca/deployments/7de2f5a3-44a1-439d-ada8-ce9e4c73eb60/online<jupyter_text>Monitor the status of deployment<jupyter_code># Update this `deployment_id` from the newly deployed model from above.
deployment_id = "7de2f5a3-44a1-439d-ada8-ce9e4c73eb60"
deployment_details_url = service_path + "/v3/wml_instances/" + instance_id + "/published_models/" + model_id + "/deployments/" + deployment_id

headers = {'authorization': 'Bearer ' + watson_ml_token, 'content-type': "application/json" }

response = requests.request("GET", deployment_url, headers=headers)
print(response.text)
deployment_details = json.loads(response.text)

for resources in deployment_details['resources']:
    print('name: {}'.format(resources['entity']['name']))
    print('status: {}'.format(resources['entity']['status']))
    print('scoring url: {}'.format(resources['entity']['scoring_url']))<jupyter_output>name: Heart Failure Prediction Model Deployment
status: DEPLOY_SUCCESS
scoring url: https://ibm-watson-ml.mybluemix.net/v3/wml_instances/2074d379-2141-42dc-9340-a78bf626de46/deployments/1bf4866e-6be7-4bca-be6d-18db9419e17f/online
name: Wine Quality Prediction Model Deployment
status: DEPLOY_SUCCESS
scoring url: https://ibm-watson-ml.mybluemix.net/v3/wml_instances/2074d379-2141-42dc-9340-a78bf626de46/deployments/7de2f5a3-44a1-439d-ada8-ce9e4c73eb60/online<jupyter_text>6.3 Invoke prediction model deploymentDefine a method to call scoring url. Replace the **scoring_url** in the method below with the scoring_url returned from above.<jupyter_code>def get_prediction_ml(fa, va, ca, rs, ch, fsd, tsd, d, p, s, a):
    scoring_url = 'https://ibm-watson-ml.mybluemix.net/v3/wml_instances/2074d379-2141-42dc-9340-a78bf626de46/deployments/7de2f5a3-44a1-439d-ada8-ce9e4c73eb60/online'
    scoring_payload = { "fields":["fixed acidity", "volatile acidity", "citric acid", "residual sugar", "chlorides", "free sulfur dioxide", "total sulfur dioxide", "density", "pH", "sulphates", "alcohol"],"values":[[fa, va, ca, rs, ch, fsd, tsd, d, p, s, a]]}
    header = {'authorization': 'Bearer ' + watson_ml_token, 'content-type': "application/json" }
    scoring_response = requests.post(scoring_url, json=scoring_payload, headers=header)
    print(scoring_response.text)
    return (json.loads(scoring_response.text).get("values")[0][0])
data = np.array([7, 0.27, 0.36 ,20.7 ,0.045 ,45 ,170 ,1.001 ,3 ,0.45 ,8.8]).reshape(-1, 1)
data = StandardScaler().fit_transform(data)
data.T

# print('What is quality of wine with such characteristics?: {}'.format(get_prediction_ml(0.45, 3.28, -2.19, -0.59, 1.19, -0.31, -0.86, 0.7, -0.11, 0.99, -0.58)))<jupyter_output><empty_output>