Daniel Russell knows how to find the answers to questions you can't get to with a simple Google query. In his weekly Search Research column, Russell issues a search challenge, then follows up later in the week with his solution—using whatever search technology and methodology fits the bill. This week's challenge: How much did Thomas Jefferson think about wine?

Short answers:
1. Dr. Vine Utley of Lyme, CT was the correspondent.
2. The house is (as of Sep 2009) white.
3. Jefferson wrote the word "wine" 267 times in his lifetime.

This was a question in three parts. Let's deal with each at a time...

1. (Easy) Who was Jefferson writing to when he wrote this famous line about "I have lived temperately… I double the Doctor's recommendation…"? What was the correspondent's name?
As most people found, this wasn't that hard. A simple search for a quoted phrase:

[ Thomas Jefferson "and even treble it with a friend" ]

Why that particular phrase to search? Answer: I knew it would have to be Jefferson, and this particular construction "half of wine" and "even treble it with a friend" would be idiosyncratic enough to pop out on the search.

Thing is, when I did that, I found a bunch of "wine quotes," which suggest that Jefferson did say it, but it took a little more digging to find an authoritative source (and the rest of the letter, so I could see who he was writing to).

So I thought I'd search on Google Books and see if there wasn't a collection of his letters. Sure enough, there's a very nice collection: Memoirs, Correspondence and Private Papers of Thomas Jefferson, Volume 4 (published by Colburn & Bentley, 1829)—that contains the text of Jefferson's letter to Dr. Vine Utley on March 21, 1819 from Monticello.

Answer: Dr. Vine Utley

2. (Medium) Amazingly enough, more than 200 years later, the correspondent's house still stands. What color was that person's house in 2009?

Okay, so who IS this Dr. Utley person? And where did he live?

Since I'd never heard of Vine Utley before, I started with a simple query on his name. After all, how common can "Vine Utley" be?

[ Vine Utley ]

Much to my delight, I found many references to Dr. Utley, and quickly found a WikiMapia site that gave the lat/long for his house in Lyme, CT. (And to be sure, I did a bunch of additional reading about Vine Utley to convince me that he lived in this house during the time he corresponded with Jefferson.)

With the lat/long in hand (41°19'53"N 72°13'32"W) and a bit of Google Maps + Streetview, it didn't take long to get a nice picture of the house from street and verify that it is (as of Sept 2009) a white house.

3. (Hard) How many times did Jefferson refer to wine in ALL of his writings?

As I said, this is a bit more difficult.

I thought I'd start by finding books of Jefferson's correspondence in Google Books—and yes, I can find them easily enough—but it's difficult to be sure you've got them ALL of his collected writings. Even then, Google Books often is inconsistent about it's "Find within this book" feature. For instance, if you search within this book (Volume 4 of his collection) for the word "wine," it will show you one instance of the word in the text. But simple browsing around within the text will show that this isn't the case.

So we must go deeper.

The simplest solution I could think of was to find a collection of Jefferson's writings and then do a regular text search through that.

This comes under the category of downloading and scraping content from web sites (which we'll discuss later in more detail), but for this problem here's what I did.

a. Find a collection of ALL of Jefferson's writings in plain text.
b. Download all of those texts and concatenate them into one big file. (Remember that this is someone writing with quill and ink—he didn't write THAT much text in the course of his life.)
c. Do a textual search for "wine" throughout his collected text (by using Control-F, Command-F, or if you're geeky, grep).

One step at a time:

a. Find a collection: Easy. [ Thomas Jefferson collected writings ] quickly leads you to the marvelous History1700s.com, a site with lots of collected works—and in our case, the works of Jefferson in 19 handy plain text files.

b. Download. Since there are only 19 files, you could just click and save each file. (Or, if you know how to use the Linux command wget, you can say:
wget -r -l1 http://http://www.history1700s.com/etext/bltjefferson.shtml
which will grab all of those text files in one operation. Once you have all 19 files, you can just concatenate them together (using the Linux cat operation) to create one big file. I called in Jeff-full.txt, and it's 14.3Mb long.

c. Text search. We'll talk about grep later, but for now, the simplest thing is to open the file in your Chrome browser and do a search for { wine }, which shows there are 273 references to the text "wine" in all of his collected writings. (But be slightly careful: 5 of them are actually to "twine" and one is to "swine," which means he meant wine, the drink only 267 times.)

You'd think that's it... but if you READ all of the references, you'll find that many of them are discussions of wine in regulatory terms (e.g., the thrilling prose "If of wine, it must contain 31 1/2 gallons by the statue 2 Henry VI. c. II, and 1 Rich. III. c. 15.")

But, it's there, in his writings, so we need to count those prosaic references as well.

As you can see by the comments in yesterday's post, there were many answers to this question. Of course, there's really only one answer—but bear in mind that even that might change over time. I can easily imagine that there's a trove of Jeffersonian letters that has yet to be discovered.
So we'll stop here, with our discovery that Jefferson seems to have written the word "wine" at least 267 times throughout his life.

Search lessons: Sometimes you have to download the data and do a bit of processing yourself. In such cases, you really need to have a basic set of tools to help you find what you need, and then process it in some useful way. We'll talk more about this later, but remember that although the web is a wonderful place, not all answers are out there to be picked up like beautiful stones on the beach. You still need to dig for the information clams!

Search on.. and digging...

Answer: How much did Jefferson think about wine? | SearchReSearch


Daniel M. Russell studies the way people search and research—an anthropologist of search, if you will. You can read more from Russell on his SearchReSearch blog, and stay tuned for his weekly challenges (and answers) here on Lifehacker.
http://lifehacker.com/5927473/answer-how-much-did-thomas-jefferson-think-about-wine
lifehacker.com
crawl=CC-MAIN-2013-20
48112
application/http; msgtype=response
sha1:MLX7V6O6434JZ7B3TWY3LEUQH3RV3B7O
<urn:uuid:431fac8d-e085-4a9e-a55d-66b034cc34cb>
2013-05-21T22:23:46Z
199.27.72.129
sha1:SL5LMRC2VPNT4NUTRY276UAQ7Q4IYMUQ
<urn:uuid:7e7b5b59-427a-4673-a051-efbcc4f704fe>
http://lifehacker.com/5927473/answer-how-much-did-thomas-jefferson-think-about-wine
response
<urn:uuid:077471ea-356f-4394-84d6-9f232a9d169e>
robots: classic
hostname: ip-10-60-113-184.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2013-20
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for Spring 2013
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf