  Data integration between web sources and relational data is a key challenge
faced by data scientists and spreadsheet users. There are two main challenges
in programmatically joining web data with relational data. First, most websites
do not expose a direct interface to obtain tabular data, so the user needs to
formulate a logic to get to different webpages for each input row in the
relational table. Second, after reaching the desired webpage, the user needs to
write complex scripts to extract the relevant data, which is often conditioned
on the input data. Since many data scientists and end-users come from diverse
backgrounds, writing such complex regular-expression based logical scripts to
perform data integration tasks is unfortunately often beyond their programming
expertise.
  We present WebRelate, a system that allows users to join semi-structured web
data with relational data in spreadsheets using input-output examples.
WebRelate decomposes the web data integration task into two sub-tasks of i) URL
learning and ii) input-dependent web extraction. The first sub-task generates
the URLs for the webpages containing the desired data for all rows in the
relational table. WebRelate achieves this by learning a string transformation
program using a few example URLs. The second sub-task uses examples of desired
data to be extracted from the corresponding webpages and learns a program to
extract the data for the other rows. We design expressive domain-specific
languages for URL generation and web data extraction, and present efficient
synthesis algorithms for learning programs in these DSLs from few input-output
examples. We evaluate WebRelate on 88 real-world web data integration tasks
taken from online help forums and Excel product team, and show that WebRelate
can learn the desired programs within few seconds using only 1 example for the
majority of the tasks.
