Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

July 29 2011

14:44

Meet the User – Claire Miller

Here at ScraperWiki we like programmers and journalists. We’re also interested in helping to bridge the gap, so with no further ado let me present to you: Claire Miller. She’s a reporter and data journalist for Media Wales.

She’s come to ScraperWiki from the non-coding side but with a journalistic eye for the story. She’s new to the ScraperWiki way of doing things but says “I’m just starting to appreciate how handy a tool ScraperWiki can be for journalists. At the moment when I come across a dataset on a government website, like the Food Standards Agency ratings or jobcentre vacancies, I’ve taken to checking ScraperWiki to see if someone is trying to scrape them – this is how I ended up with a collection of scrapers gathering up ratings of dodgy takeaways in Wales.”

She’s hoping there will be some stories for the paper but has gone over to the ScraperWiki side and wants to make the data useful for people living in Wales. Like me and many others in the data journalism fold she’s decided to just jump in by forking useful looking scrapers and pointing them at Welsh datasets.

I’m interested in how I can use ScraperWiki to find data I can use to find stories. I’ve already used some from a series of scrapers gathering data from the jobcentre vacancies search to analyse the sort of vacancies that are offer for people in Wales. I’m also working on gathering up lots of data on Welsh schools and found the Welsh School Finder which saved me so much time in linking census and financial data to addresses and locations.

Her ultimate goal is to start scraping PDFs as FOI requests are constantly being given in that most evil of formats. We’re working on our documentation and tutorials and PDFs are most definitely on our list. For the budding data journalists out there, I’d say walk before you can run. PDFs are hard. So start with html web scraping and CSVs. But remember, where there’s a ScraperWiki digger, there’s a way!


Don't be the product, buy the product!

Schweinderl
http://smarques.soup.io/tag/Claire%20Miller
smarques.soup.io
crawl=CC-MAIN-2019-47
35162
application/http; msgtype=response
sha1:ADYF4GXY6SSQ35NHVQPQHDIXJJ3Z72X4
<urn:uuid:99525921-7614-4be3-a975-75ba0a488ca1>
2019-11-13T16:54:49Z
45.153.143.247
application/xhtml+xml
sha1:XVAPYLD64DTAQ7JGQYYSHN7B742TYY4G
<urn:uuid:9c706b3b-e67a-458a-b4fc-4e0c96654715>
http://smarques.soup.io/tag/Claire%20Miller
response
<urn:uuid:54977975-1be4-4401-aea5-8bb794a8d1a6>
isPartOf: CC-MAIN-2019-47
publisher: Common Crawl
description: Wide crawl of the web for November 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-22.ec2.internal
software: Apache Nutch 1.16 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.1-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/