Repurposing Wikipedia: Wikipedia as data set and analytical device

7,830 views

Published on

Tutorial at the Digital Methods Summer School 2013

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,830
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Repurposing Wikipedia: Wikipedia as data set and analytical device

  1. 1. wikipedia as a data set & analytical device Erik Borra, June 2013
  2. 2. DMI research with Wikipedia reference work bureaucracy scandal machine repurposing Wikipedia as a ... vigilant community
  3. 3. Medium-specific outlook “follow the medium” (Rogers 2009)
  4. 4. The Anatomy of a Wikipedia page
  5. 5. DMI research with Wikipedia reference work bureaucracy scandal machine repurposing Wikipedia as a ... vigilant community
  6. 6. DMI research with Wikipedia repurposing Wikipedia as a ... reference work <=> cultural reference bureaucracy scandal machine vigilant community
  7. 7. www.nature.com/nature/journal/ v438/n7070/extref/438900a-s1.doc
  8. 8. Burial of 465 identified Bosniaks, Potočari, 2007. Map of the Srebrenica military operations, made by the U.S. Central Intelligence Agency, with green arrow showing the route of the Bosnian forces. Map of the location of Srebrenica, the Republika Srpska, Bosnia-Herzegovina. Srebrenica-Potočari Memorial and Cemetery, Bosnia-Herzegovina. Grave of a 13-year old Bosniak boy. Ratko Mladic. An exhumed body with blindfold and hands tied behind his back. As of September 2012, the photo has been removed from Wikipedia article. Exhumed grave of victims, 2007. Podrinje Identification Project's facility for storing, processing, and handling exhumed remains.. "UN left 8,000 to die in Bosnia." Headline in The Independent, 30 October 1995. Satellite photo of Nova Kasaba mass grave. International Criminal Tribunal for the Former Yugoslavia, Den Haag, the Netherlands. DUTCH ENGLISH BOSNIAN CROATIAN SERBIAN SERBO- CROATIAN Tool: Wikipedia Cross-Lingual Image Analysis
  9. 9. Tool: triangulate
  10. 10. National Point of View Neutral Point of View Linguistic Point of View manypedia.com - comparing linguistic points of view (LPOV) omnipedia.northwestern.edu - making Wikipedia articles in diﬀerent languages comparable
  11. 11. DMI research with Wikipedia repurposing Wikipedia as a ... reference work bureaucracy scandal machine vigilant community <=> socio-technical device
  12. 12. Wikipedia has been described in terms of open source intelligence(Stalder and Hirsh, 2002), wisdom of crowds(Surowiecki, 2004; Kittur and Kraut, 2008), many minds(Sunstein, 2006), collaborative knowledge(Poe, 2006, McKenzie Wark 2007), an army of volunteers(Jenkins, 2006), mass collaboration(Tapscott, 2007), distributed collaboration(Shirky, 2008), produsage(Bruns, 2008), crowdsourcing(Economist, 2008), and mentioned in the context of free labour(Deuze, 2006), and the cult of the amateur(Keen, 2007).
  13. 13. Tool: Wikipedia Edits Scraper and IP Localizer + screen recording software
  14. 14. Govcom.org, 2008. http://govcom.org/ Analysis by Zachary Devereaux, Sabine Niederer, Richard Rogers, Bram Nijhof and Michael Stevenson. © 2008 Wikipedia editing by bots and users: Overall percentage of Bot activity of all edit activity________________________________ 08August Wikipedia editing by bots and users: Overall percentage of Bot activity of all edit activity Visualization: Rosa Menkman. Percentage of bot edits All wikipedia edits
  15. 15. Digital Methods Initiative, 2008. www.digitalmethods.net Analysis by Zachary Devereaux, Sabine Niederer, Richard Rogers, Bram Nijhof and Michael Stevenson. © 2008 Human, Bot and Software Assisted Activity on Top Twenty EN Wikipedia Articles_________________________________________ 0August Human, Bot and Software Assisted Activity on Top Twenty EN Wikipedia Articles 46444644 Software Assisted Edits 30673067 Bot Edits 251378251378 Human Edits Maps by: Rosa Menkman, Auke Touwslager and Marieke van Dijk
  16. 16. Govcomorg Jubilee, 2008. www.govcom.org Analysis by Zachary Devereaux, Sabine Niederer, Richard Rogers, Bram Nijhof and Michael Stevenson Visualization by Auke Touwslager © 2008 Human, Bot and Software Assisted Activity on Top Twenty EN Wikipedia Articles_________________________________________ 0August Percentage of bot activity Language scaled by Wikipedia size Cornish Võro Manx Ladino Friulian Romansh Aromanian Sanskrit Upper Sorbian Corsican Scottish Gaelic Samogitian Chuvash Aragonese Occitan Breton Endangered Languages Revived Languages Hawaiian Cornish Manx West Frisian Belarusian Basque Galician Estonian Hebrew Czech Catalan Wikipedia Bot Activity in Endangered and Revived Languages What is the level of bot activity per language, looking at endangered and revived languages? stats.wikimedia.org + bubble lines
  17. 17. Govcomorg Jubilee, 2008. www.govcom.org Analysis by Zachary Devereaux, Sabine Niederer, Richard Rogers, Bram Nijhof and Michael Stevenson Visualization by Auke Touwslager © 2008 Human, Bot and Software Assisted Activity on Top Twenty EN Wikipedia Articles_________________________________________ 0August Javanese Urdu Tamil Hindi Marathi Bengali Telugu Korean Vietnamese Arabic Chinese Russian Spanish Japanese Italian Portuguese German French English Percentage of bot activity Language scaled by Wikipedia size Wikipedia Bot Activity in Most-Used Languages Worldwide What is the level of bot activity per language, looking at top 20 of most-used languages? stats.wikimedia.org + Dorling
  18. 18. DMI research with Wikipedia repurposing Wikipedia as a ... reference work bureaucracy scandal machine <=> place of edits vigilant community
  19. 19. DMI research with Wikipedia repurposing Wikipedia as a ... reference work bureaucracy <=> controversy diagnostics machine scandal machine vigilant community
  20. 20. Administrative apparatus: Wikipedia’s procedural policies, guidelines and essays ... with the purpose of reaching consensus
  21. 21. Core content policies
  22. 22. Wikipedia collaboration and conflict studies: Cooperation and conflict (Viegas, 2004) Conflict and coordination (Kittur, 2007) Revert Graph (Suh et al, 2007) Mutual reverts (Brandes, 2007) Argument identification (Rad & Barosa, 2011) Edit wars (Sumi, 2011) Evolution of discussions (Kaltenbrunner & Laniado, 2012)
  23. 23. medium-specific approach: repurpose how consensus principles (as method of the medium) act on the objects of the medium
  24. 24. templates links images references interlanguage links timestamps comments Wikipedia objects author
  25. 25. contropedia.net
  26. 26. DMI research with Wikipedia reference work <=> cultural reference bureaucracy <=> controversy diagnostics machine scandal machine <=> places of edits repurposing Wikipedia as a ... vigilant community <=> socio-technical system
  27. 27. DMI projects mentioned R. Rogers, Digital Methods , Cambridge, MA: MIT Press, 2013. Chapter 5. https://wiki.digitalmethods.net/Dmi/DebottingWikipedia https://www.digitalmethods.net/Digitalmethods/TheNetworkedContent Contropedia, contropedia.net S. Niederer and J. van Dijck, " Wisdom of the Crowd or Technicity of Content? Wikipedia as Socio-technical System," New Media & Society, 12, 8, 2010, 1368-1387.
  28. 28. tools.digitalmethods.net
  29. 29. Tools native to Wikipedia Page information Search revision history Contributors per article, ordered by number of edits Page view statistics stats.wikimedia.org - Data is published in exportable, computable format, eg csv. Stats look at all wikis hosted by the foundation, largest are the Wikipedias. User edits searches, for all the edits of a specific user Number of watchers
  30. 30. Other useful tools manypedia.com - puts two language versions of an article side by side / computes concept similarity omnipedia.northwestern.edu - making Wikipedia articles in diﬀerent languages comparable wikiscanner (defunct) history flow (defunct) http://vs.aka-online.de/cgi-bin/wppagehiststat.pl builds an edit history overview page for the article http://sonetlab.fbk.eu/wikitrip to see geographical statistics for anonymous edits and gender
  31. 31. Academic literature on Wikipedia http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia Okoli et al. (2012). The people's encyclopedia under the gaze of the sages: A systematic review of scholarly research on Wikipedia
  32. 32. www.digitalmethods.net sabine@digitalmethods.net thank you. erik@digitalmethods.net

×
https://www.slideshare.net/digitalmethods/repurposing-wikipedia-dm13
www.slideshare.net
crawl=CC-MAIN-2017-30
209393
application/http; msgtype=response
sha1:CEKHIGTO2D7OG6UZSYXB5ZFCEIJQVO5S
<urn:uuid:08794107-f7bf-4bdc-b728-f1e557038e5c>
2017-07-26T19:13:35Z
108.174.10.19
text/html
sha1:ELUY7HBHRRQ6KGOKYSERQBTHKC6CBY2T
<urn:uuid:29d824f9-f00a-439c-a525-d8d8d46d5656>
https://www.slideshare.net/digitalmethods/repurposing-wikipedia-dm13
length
response
<urn:uuid:258afee0-47fd-4945-834b-15c45b91567e>
robots: classic
hostname: ip-10-186-218-81.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2017-30
operator: Common Crawl Admin
description: Wide crawl of the web for July 2017
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf