You are here: Foswiki>Dmi Web>GovcomorgJubilee>NetworkedContent (23 Jan 2010, issuecrawler14)Edit Attach

Networked Content

Team

Zachary Devereaux, Sabine Niederer, Bram Nijhof, Richard Rogers, Auke Touwslager (Design)

Questions & Context: The Vigilance of Wikipedians

In the well-known video about the history of a Wikipedia entry (the heavy metal umlaut), the narrator, Jon Udell, is astounded at the speed by which a vandalized page is corrected by Wikipedia's users. A vandalized page is reverted to the previous version of the entry within a minute. "Wikipedians" appear extremely vigilant.

An underinterrogated aspect of Wikipedians' vigilance are the robots, or bots -- software that often automatically monitors wikipedia pages for changes. When a change is made to a page, a bot will come calling, examining the change. Perhaps it will add a link to another wikipedia (one in a different language). Perhaps it will send an alert to a user, telling the wikipedian about a reversion or any other manner of activity. Thus changes to wikipedia pages trigger bot visits. Vigilance is often bot-driven.

In this project we are interested in the reliance of wikipedia on bots in order for the collectively-authored, user-generated encyclopedia to maintain itself.

What is the degree of dependency of Wikipedia on bots?

In order to answer this question, we looked into the amount of activity in wikipedia by human users and by bots (as well as by human users assisted by monitoring software). Remarkably, the most active wikipedians are bots, followed by human wikipedians, many of whom, presumably, use the wikipedia monitoring tools and are alerted when changes are made to wikipedia pages. See

Thus, initially, we interested in the overall picture. How many edits are made by humans, and how many by bots? According to Wikipedia bot activity data (which we later found to be incomplete), overall, 39% of all edits to all wikipedias (that is, to all the different language versions) are made by bots. Adding the English-language data (which was missing) we found that overall share of edits by bots in all wikipedias is 19%.

Having noticed the great discrepancy between bot activity in English and the other language versions of Wikipedia, we became interested in the level of bot activity per language. There are currently 264 language versions of wikipedia. The English-language wikipedia is the largest in size, followed by the German in the number two position. Bot activity varies greatly by language version. One small wikipedia has seen 97% of its edits made by bots.

The graphics below show the findings with respect to the share of edits by bots versus humans, as well as per language. In the final graphic, we explore the radical idea of the dependency of endangered languages on bots. (It is not so much that the endangered languages rely on bots for their survival as much as it is that bot activity is remarkably high in endangered languages, suggesting not only few humans in those language spaces, which we already know. Rather, the bots are creating links and otherwise integrating the languages into the larger wikipedia undertaking.

Findings in the form of graphics

1. Wikipedia editing by bots and humans: Overall percentage of Bot activity of all edit activity

Wikipedia_editing_by_bots_and_users_Overall_percentage_of_Bot_activity_of_all_activity.png

2. Shares of human, bot and software-assisted human edits in the top twenty most active Wikipedia entries in the English-language version.

HumanBotandSoftwareAssistedActivityonTopTwentyENWikipedia.png

3. Share of Wikipedia bot activity in the twenty most used languages worldwide. wikipedia_bot_activity_top20_languages.png

4. Wikipedia Bot Activity in Endangered and Revived Languages
wikipedia_bot_activity_endangered_revised_languages.jpg
Topic attachments
I Attachment Action Size Date Who CommentSorted ascending
HumanBotandSoftwareAssistedActivityonTopTwentyENWikipedia.pdfpdf HumanBotandSoftwareAssistedActivityonTopTwentyENWikipedia.pdf manage 127 K 15 Aug 2008 - 14:32 Main.r00s  
Wikipedia_editing_by_bots_and_users_Overall_percentage_of_Bot_activity_of_all_activity.pngpng Wikipedia_editing_by_bots_and_users_Overall_percentage_of_Bot_activity_of_all_activity.png manage 137 K 15 Aug 2008 - 14:42 Main.r00s  
wikipedia_bot_activity_endangered_revised_languages.jpgjpg wikipedia_bot_activity_endangered_revised_languages.jpg manage 55 K 19 Aug 2008 - 15:02 Main.issuecrawler14  
wikipedia_bot_activity_top20_languages.pngpng wikipedia_bot_activity_top20_languages.png manage 51 K 15 Aug 2008 - 14:45 Main.issuecrawler14  
A_day_of_edits_in_the_Cornish_Wikipedia_kw.pngpng A_day_of_edits_in_the_Cornish_Wikipedia_kw.png manage 230 K 14 Aug 2008 - 14:29 MichaelStevenson Cornish Wikipedia Edits
HumanBotandSoftwareAssistedActivityonTopTwentyENWikipedia.pngpng HumanBotandSoftwareAssistedActivityonTopTwentyENWikipedia.png manage 133 K 15 Aug 2008 - 12:55 Main.r00s Human Bot and Software Assisted Activity on Top Twenty EN Wikipedia
Wikipedia_editing_by_bots_and_users_Overall_percentage_of_Bot_activity_of_all_activity.pdfpdf Wikipedia_editing_by_bots_and_users_Overall_percentage_of_Bot_activity_of_all_activity.pdf manage 137 K 15 Aug 2008 - 14:42 Main.issuecrawler14 Overall percentage of bot activity
Topic revision: r18 - 23 Jan 2010, issuecrawler14
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback
https://digitalmethods.net/Dmi/NetworkedContent?sortcol=6;table=1;up=0
digitalmethods.net
crawl=CC-MAIN-2019-26
38402
application/http; msgtype=response
sha1:XYPC662AV3MTODCXVJX4BNYMAOBE4332
<urn:uuid:2ef7c7f4-23d1-4918-b937-136660fc6247>
2019-06-20T23:20:32Z
37.97.130.157
text/html
sha1:VFRF5WQMGWNRDSOZUKR6FFEZBOPUMUUL
<urn:uuid:0d70c851-e473-4717-93c2-ec5c67f1242e>
https://digitalmethods.net/Dmi/NetworkedContent?sortcol=6;table=1;up=0
response
<urn:uuid:9b460367-eb3b-4ef3-a640-49e16d38013b>
isPartOf: CC-MAIN-2019-26
publisher: Common Crawl
description: Wide crawl of the web for June 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-81-161-154.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.1-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/