<jupyter_start><jupyter_text>Manifold**Manifold** is a tool for model-agnostic evaluation with visual support developed by Uber — you can find its repo [here](https://github.com/uber/manifold).In this quick demo, the [California housing dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_california_housing.html) (regression) and part of the setup available in this [example](https://scikit-learn.org/stable/auto_examples/inspection/plot_partial_dependence.html) on the [scikit-learn](https://scikit-learn.org/stable/index.html) website are used.The main idea for this demo is to test **Manifold**'s [integration with Jupyter Notebook](https://eng.uber.com/manifold-open-source/) and the [Geo Feature View](https://github.com/uber/manifoldgeo-feature-view) map.<jupyter_code>from sklearn.datasets import fetch_california_housing

import pandas as pd
import numpy as np

# Installation: https://github.com/uber/manifold/tree/master/bindings/jupyter
from mlvis import Manifold

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import QuantileTransformer
from sklearn.pipeline import make_pipeline
from sklearn.neural_network import MLPRegressor
california_housing = fetch_california_housing()
california_housing.data.shape
california_housing.feature_names
california_housing.target.shape
california_housing.keys()
X = pd.DataFrame(california_housing.data, columns=california_housing.feature_names)

y = california_housing.target
# The Latitude feature must be called "lat".
# The Longitude feature must be called "lng".

X.rename(columns={'Latitude':'lat', 'Longitude': 'lng'}, inplace=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=0)
est = make_pipeline(QuantileTransformer(),
                    MLPRegressor(hidden_layer_sizes=(50, 50),
                                 learning_rate_init=0.01,
                                 early_stopping=True))

est.fit(X_train, y_train)

print("Test R2 score: {:.2f}".format(est.score(X_test, y_test)))
yPred = est.predict(X_test)
# Mapbox access token: https://docs.mapbox.com/help/how-mapbox-works/access-tokens/

# It must be replaced with a valid token.
TOKEN = "<KEY>"
# Props: https://github.com/uber/manifold/blob/master/bindings/jupyter-modules/jupyter-manifold/src/manifold.js
# Classification: https://github.com/uber/manifold/blob/master/bindings/jupyter/notebooks/manifold.ipynb

Manifold(props={'data': {
    'x': X_test[['lat', 'lng']],
    'yPred': [pd.DataFrame(yPred, columns=["Target"])], # Each element in this list contains the predictions for a model.
    'yTrue': pd.DataFrame(y_test, columns=["Target"])
    },
    'mapboxAccessToken': TOKEN,
    'width': 1000,
    'height': 700
})<jupyter_output><empty_output>