  Comparing ISPs on broadband speed is challenging, since measurements can vary
due to subscriber attributes such as operation system and test conditions such
as access capacity, server distance, TCP window size, time-of-day, and network
segment size. In this paper, we draw inspiration from observational studies in
medicine, which face a similar challenge in comparing the effect of treatments
on patients with diverse characteristics, and have successfully tackled this
using "causal inference" techniques for {\em post facto} analysis of medical
records. Our first contribution is to develop a tool to pre-process and
visualize the millions of data points in M-Lab at various time- and
space-granularities to get preliminary insights on factors affecting broadband
performance. Next, we analyze 24 months of data pertaining to twelve ISPs
across three countries, and demonstrate that there is observational bias in the
data due to disparities amongst ISPs in their attribute distributions. For our
third contribution, we apply a multi-variate matching method to identify
suitable cohorts that can be compared without bias, which reveals that ISPs are
closer in performance than thought before. Our final contribution is to refine
our model by developing a method for estimating speed-tier and re-apply
matching for comparison of ISP performance. Our results challenge conventional
rankings of ISPs, and pave the way towards data-driven approaches for unbiased
comparisons of ISPs world-wide.
