<jupyter_start><jupyter_text>Fit Kmeans Model<jupyter_code>from pyspark.ml.clustering import KMeans
kmeans = KMeans().setK(2).setSeed(42)
model = kmeans.fit(final_data)
model.predictionCol<jupyter_output><empty_output><jupyter_text>Calculate Within Set Sum of Squared Errors (WSSSE)<jupyter_code># within set sum of squared errors
wssse = model.computeCost(final_data)
wssse<jupyter_output><empty_output><jupyter_text>Where Are the Centroids ?<jupyter_code>centers = model.clusterCenters()
centers<jupyter_output><empty_output><jupyter_text>Which Cluster Is Each Sample In ?<jupyter_code>results = model.transform(final_data)
results.show()<jupyter_output>+--------------------+----------+
|            features|prediction|
+--------------------+----------+
|           (3,[],[])|         1|
|(3,[0,1,2],[0.1,0...|         1|
|(3,[0,1,2],[0.2,0...|         1|
|(3,[0,1,2],[9.0,9...|         0|
|(3,[0,1,2],[9.1,9...|         0|
|(3,[0,1,2],[9.2,9...|         0|
+--------------------+----------+
StarcoderdataJupyterStructuredCleanDedup