ArsTechnica

Scientific Method / Science & Exploration

Study concludes Wikipedians are a bunch of grumpy introverts

A study making the rounds suggests that Wikipedians may feel at home online, …

Behavioral studies of specific populations are extremely challenging things, as recruiting a set of participants that represent a true cross-section of the larger population is never easy. The mere willingness to participate in a study involves a necessary degree of self-selection within this population, which can seriously complicate behavioral studies—after all, it's entirely possible that a willingness to take surveys is the product of one of the behavioral traits under examination. That's why, even though a small survey can produce results with a high statistical confidence, it still may represent nothing more than a robust result within a non-representative group. All of that should serve as a precautionary background on a new survey of Wikipedia participants.

The short publication that describes these results is entitled, "Personality Characteristics of Wikipedia Members," and the news isn't especially good. Those contributing online feel that their true personality comes across better in the virtual environment; they also scored lower on a measure of agreeableness, and aren't especially open to new ideas.

It's easy to rationalize the first of these findings: if someone is more comfortable online, they're more likely to contribute to an online community. The other two, however, require a bit more in the way of mental gymnastics, given Wikipedia's relative newness and its focus on building a consensus about what constitutes relevant, unbiased information. In one press report on the findings, an author suggests that Wikipedia might provide a way for these individuals to compensate for real world inadequacies.

Credit to whoever did that reporting for noting the number of Wikipedians who were surveyed to produce these results: a grand total of 69, all from Israel. For some unfathomable reason, however, the reporter neglected to consider just how staggeringly unrepresentative those 69 are of Wikipedia at large. Wikipedia's own survey of its contributors racked up roughly 12,000 original contributors (and another 32,000 sporadic contributors). Given that there were responses from about 400 Israelis, that survey probably included about 175 Israeli contributors. And Israel appears to constitute a small corner of the Wikiverse, as there were roughly 20,000 responses each from the US and Germany; another five countries provided over 5,000 survey responses.

So, the 69 individuals surveyed appear to represent little more than the Israeli contributors that were willing to complete a survey. There's reason to be a bit skeptical about the survey, as well. The evaluation of personality traits appears to have used a standard series of questions that have been validated by others in the field. But the questions about where one best expresses their real identity were derived from a "Real Me Questionnaire" that consisted of only four questions.

There's nothing wrong with the study per se; it's a reasonable first attempt at addressing a recent social phenomenon that hasn't been extensively explored. But the survey size and geographic limits indicate that it should be interpreted very cautiously and not viewed as a generalized portrait of Wikipedia contributors. Unfortunately, the authors of the paper have done just that, both in the paper itself and in the interview material.

Of course, the paper itself probably would have only reached an audience of people trained to identify the limitations of research if it weren't for the press coverage. The fact that the author of that coverage was clearly aware of the issue (at least to the degree that he typed the numbers) but didn't bother to register its importance is pretty discouraging.

CyberPsychology & Behavior, 2009. DOI: 10.1089/cpb.2007.0225

Listing image by Image from Wikimedia Commons; Oscar the Grouch character

Expand full story

44 Reader Comments

  1. An interesting study. But definitely important to note that respondents are self-selected and from a country in great turmoil. There's meaning in the results, although not necessarily generalizable to more than the narrow demographics of those involved.

    It was great to see this article critique the study in this manner. Smile
    1790 posts | registered
  2. CITATION NEEDED
    395 posts | registered
  3. all i can say is LOL
    1 post | registered
  4. quote:
    But the results need to be interpreted very cautiously, as they were based on only 69 contributors from a single nation, a tiny drop in the Wikipedian ocean.


    But the results need to be completely ignored as they are based on a statistically unsupportable minuscule number of alleged "contributors".

    Fixed that for you.
    654 posts | registered
  5. does anyone actually think that the contributors to the encyclopedia of britannica are anything less than ill-adjusted bookworms that are afraid of anything that isn't presented in tabulated data on a dead-tree format? (such as, but not including, going outside?)

    Newsflash: Encyclopedia are made by nerds.
    14 posts | registered
  6. quote:
    does anyone actually think that the contributors to the encyclopedia of britannica are anything less than ill-adjusted bookworms that are afraid of anything that isn't presented in tabulated data on a dead-tree format?

    I came here to post this.
    21024 posts | registered
  7. There's a whole ecosystem of personalities on there. Who here hasn't experienced the Divine Reverter, who comes in and reverts all your changes because it doesn't fit with how they see the article? Or the Tidier, who does nothing but fix typos and move small words around. And so on.
    1392 posts | registered
  8. darkowl, is there a list somewhere of these people? I find myself curious to know more.

    And my experience with wikipedia is that if it isn't a mainstream or science topic, it's unlikely to even be spell checked...let alone accurate enough to be worth using. The science (includes math, cs, physics, chem) topics tend to be fairly good, encyclopedia style entries. Enough to sketch in a broad overview and lead you in the direction of more information, but written so that anyone with a passing familiarity will be able to follow, given a little effort.

    These science articles tend to read as if written by a good lecturer...which usually means an academic...which usually means a grumpy introvert. You almost have to be to get through grad school, after all. I'd say the grumpy introvert is quite well represented on wikipedia, but not the dominant group, if I had to guess.
    1035 posts | registered
  9. CyberPsychology & Behaviour must be really desperate for papers. I would rate this at the same level as an undergraduate term project.
    3462 posts | registered
  10. darkowl is not exaggerating: see the Wiki Fauna:

    http://en.wikipedia.org/wiki/Template:Wikipedia_fauna

    And I agree with Doubter the results of the study should be ignored (although it is still fine to critique the study). We're not in a good situation if journalists credulously report shoddy research and tack on a disclaimer that the results may not be statistically valid. The study should either be ignored, or if it is being reported on elsewhere, the real story is the low standards of journalists looking for a good headline.

    This comment was edited by Massu on June 30, 2009 13:52
    13 posts | registered
  11. > The science (includes math, cs, physics, chem) topics tend to be fairly good

    Oh geez, I hate them, at least the physics ones. They rarely explain the science itself, jump directly into the math, and use all sorts of jargon. When you try to understand the jargon, it leads you to a similar article.

    > if it isn't a mainstream or science topic, it's unlikely to even be spell checked

    Your experience is far from typical. The vast majority of better the articles are on historical topics.

    Maury
    2028 posts | registered
  12. Anyone truly surprised? Wink They could have added "with IT, anime, and sci-fi interests" too, judging by the extreme depths of these articles compared to those in, say, botany.
    144 posts | registered
  13. "Those contributing online feel that their true personality comes across better in the virtual environment; they also scored lower on a measure of agreeableness, and aren't especially open to new ideas."

    Without looking at the study and regardless of the statistics, I would suggest the following analysis. A fundamental dimension to variations in human behavior has to do with group participation.
    At one end of the spectrum you have people whose life is totally oriented towards participation in a group. Their thoughts are the group's thoughts. Their concern is the group's concern. They are open to whatever the group currently finds trendy. They feel alive when the are interacting face to face with other members of the group and building their role in the group.
    At the other end of the spectrum you find people who are centered more in their own individuality. They are resistant to the imposition of the group's trends. They have less interest and more difficulty in establishing roles in groups. They have more interest in their own thoughts.
    It is quite possible and even likely that the second group of people spend more time online. But value judgements about the quality of each group's experience are just that. The fact that the average social bee feels threatened by anyone who bothers to spend some time thinking does not prove that thought is valueless. The fact that someone is not so quick to change the length of their skirt does not prove that they are resistant to innovations.
    One reality is that the internet is a large innovation. Another reality is that online interaction like other forms of automation extends human experience. Those who are not comfortable with it are the ones who are rejecting change. Of course, another reality is that those more skilled online and less skilled in tradtional group interactions benefit in the competitive world from an opportunity that favors their abilities. Those who are not able to do as well online have an interest in demeaning the experience of those who do.
    The big reality is that human experience will continue to center around social groups and skill in participating in those groups will continue to be a very important competitive factor for people making their way in the world. But the online world is also here to stay and likely to grow. Those who are comfortable with it are not likely to worry about the social trend setter wannabes who are anxious to make capital in the media by denigrating their experience.
    2040 posts | registered
  14. Has anyone here actually read the report?

    I'd heard of this study earlier, and it seems to be one of those studies that's interesting in theory, but probably useless in practice. Seeing that people outside the Wikipedia community are interested in it, I decided to take a look.

    The first thing that jumps out at me is that the report does not describe how the participants were found—the closest thing to a hint is that "[t]he data were collected via online questionnaires". This does not inspire confidence. From the news I've heard, it's my understanding that it collected (mostly?) Israelis; this calls into question whether Israelis, Israeli Wikipedians, or Israeli non-Wikipedians are representative of the wider community (and non-community). I suspect that it is not representative, not to mention the small sample size.

    The second thing that jumps out at me is that if you look at the data table provided in the report, Wikipedians scored higher on the openness trait. The table gives a mean score of 3.75 for male Wikipedians (SD 0.63) compared with a mean score of 3.55 for male non-Wikipedians (SD 0.51), and a mean score of 3.92 for female Wikipedians (SD 0.38) compared with 3.64 (SD 0.59) for female non-Wikipedians. There's clearly a major error here, given that the paper is claiming, in its conclusion, that Wikipedians have less openness.

    This may have been caused by what appears to be a copy-and-paste error evident in the text. Below the text reading "the average of the agreeableness trait among the Wikipedia members is significantly lower as compared with that of participants who are not Wikipedia members" the text mentions "In addition, a significant difference was found in the openness trait […]" followed by a verbatim copy of the previous text regarding agreeableness—an incongruous phrasing, to say the least! Smile

    This paper looks to be a trainwreck: when the conclusion is not supported by the data published in the paper, with such an elementary mistake in the text, the paper fails to be convincing. The fact that the sample is probably not representative of the population to be studied is also troubling.

    I'd find it interesting if Wikipedians did overall score much differently—and I'd find it further interesting if Wikipedians were markedly different from the denizens of other Internet fora—but this study simply doesn't inspire confidence at a basic level, let alone one worthy of rigorous peer review.

    For disclosure, let me note that I am a volunteer administrator on Wikipedia. (I can foresee comments saying "oh, look, a Wikipedian is attacking the study; obviously he must be 'grumpy'", so please, do me the respect of not using that catch-22; I'm friendly enough. Smile )

    I originally posted this comment at http://bit.ly/LfLaZ .
    3 posts | registered
  15. quote:
    Originally posted by Maury Markowitz:
    > The science (includes math, cs, physics, chem) topics tend to be fairly good

    Oh geez, I hate them, at least the physics ones. They rarely explain the science itself, jump directly into the math, and use all sorts of jargon. When you try to understand the jargon, it leads you to a similar article.

    Maury


    Well, that's how we physicists think. In math. And jargon. Personally, I've always had the exact opposite problem, that the articles weren't detailed enough.

    Anyway, bravo Ars. You finally dissected a scientific study and looked for explanations other than the ones the researchers gave. I just wish it wasn't something as incredibly minor as showing that people online are angry introverts. Just a quick stroll through the Something Awful or Fark forums will show you that.

    Honestly, I don't know why anyone trusts behavioral scientists anymore. The big experiments that get reported on are always so poorly constructed. I realize that the big "results" are usually so hyped by the media that they bear little resemblance to the original results, but that doesn't excuse sloppy experimental design. Especially the ones they do on dog/human interactions, where it seems every researcher forgets that pack dynamics are hugely important with dogs, so a dog is going to behave much differently if you put him in a room with a stranger than he would if you put him in a room with his pack mates (owners).
    186 posts | registered
  16. this is one of those things i didn't need a study to tell me.
    217 posts | registered
  17. "Introvert" is not pejorative.
    78 posts | registered
  18. Yeah, after spending a lot of time for several years editing Wikipedia as a hobby, I know
    1 I myself am grumpy introvert.
    2 I was only twice invited to collaborate on an article, something I expected to happen A LOT more.
    3 Many, many times "contributors" seemed delighted to delete someone's work, including entire articles, and not want further interaction or discussion.
    So whether the study is trash or not, it certainly reflects my experience and my frustrations. I'm really disappointed that knol.com and the wikipedia alternatives haven't become big, or the h2g2 didn't rise to the next level.
    Of course, I'd be frightened to find out the personality traits of Ars.com posters. You will never find a more wretched hive of scum and villany!!!

    This comment was edited by cuvtixo on June 30, 2009 15:39
    746 posts | registered
  19. quote:
    Originally posted by strider_mt2k:
    CITATION NEEDED


    The citation is at the bottom of the story, as it has been since publication.
    18454 posts | registered
  20. Grumpy introverts, represent!
    791 posts | registered
  21. quote:
    Originally posted by RRGG:
    this is one of those things i didn't need a study to tell me.


    ++
    13594 posts | registered
  22. quote:
    Originally posted by thenino85:
    quote:
    Originally posted by Maury Markowitz:
    > The science (includes math, cs, physics, chem) topics tend to be fairly good
    Oh geez, I hate them, at least the physics ones. They rarely explain the science itself, jump directly into the math, and use all sorts of jargon.

    Well, that's how we physicists think...
    Honestly, I don't know why anyone trusts behavioral scientists anymore...

    Are you purposely trying to illustrate the conclusions of the paper, or is the Pot-Kettle-Black routine spontaneous? Wink Big Grin
    746 posts | registered
  23. Maybe next they can prove that most online debates involve comparisons to Hitler and/or the Nazi party.
    2879 posts | registered
  24. Silly survey, silly article. Why write anything about a tiny, ultimately meaningless survey?
    813 posts | registered
  25. quote:
    Originally posted by darkowl:
    There's a whole ecosystem of personalities on there. Who here hasn't experienced the Divine Reverter, who comes in and reverts all your changes because it doesn't fit with how they see the article? Or the Tidier, who does nothing but fix typos and move small words around. And so on.


    I fall into this crowd. The only two contributions I have made were one where someone spammed an article with "hiya!" all over the place and the other was someone put in that the 1600s were during the 16th century.

    I'm not "expert" enough about anything to really go in and rewrite someone else's work.
    9393 posts | registered
  26. quote:
    Originally posted by Laner:
    quote:
    Originally posted by RRGG:
    this is one of those things i didn't need a study to tell me.


    ++


    ++++
    1373 posts | registered
  27. quote:
    Originally posted by Milio:
    Grumpy introverts, represent!

    Plus plusses.

    Also, I seem to be a WikiGnome.
    14133 posts | registered
  28. The important question is, how much did this study cost (and where did they get the money)?
    1058 posts | registered
  29. WikiOtter, repressing his inner WikiDragon.
    12590 posts | registered
  30. quote:
    Originally posted by RRGG:
    this is one of those things i didn't need a study to tell me.


    Yeah, the headline didn't come as a shock to me either. Methodological flaws notwithstanding, it kind of has the aura of truthiness about it doesn't it?
    1500 posts | registered
  31. Maury is right.

    The science content, even slightly off the beaten path, completely fails being an encyclopedia, that is clearly explaining and summarizing a topic for non-experts. I'm sure most physics, chem etc articles are factually correct, but they are obtuse.
    194 posts | registered
  32. Heh.

    People on the internet are introverts. News at 11!
    4885 posts | registered
  33. quote:
    Originally posted by cuvtixo:
    Yeah, after spending a lot of time for several years editing Wikipedia as a hobby, I know
    1 I myself am grumpy introvert.
    2 I was only twice invited to collaborate on an article, something I expected to happen A LOT more.
    3 Many, many times "contributors" seemed delighted to delete someone's work, including entire articles, and not want further interaction or discussion.
    So whether the study is trash or not, it certainly reflects my experience and my frustrations. I'm really disappointed that knol.com and the wikipedia alternatives haven't become big, or the h2g2 didn't rise to the next level.

    I don't know, I've had far more positive experiences with editing wikis - albeit specific wikis, not the Wikipedia. On a more focused wiki, you get more likeminded people who are more likely to contribute positively. Sure, I've seen hit-and-run deleters and vandals, but they are the minority, and since reverting their edits is trivial, those idiots have zero effect and are just wasting their own time.

    The "real" Wikipedia holds less interest for me, perhaps because it gets a larger audience so the chances of a moron shitting all over your article is higher. I've done some minor edits here and there, but mostly of the typo/sauce fixing kind.

    quote:
    Originally posted by cuvtixo:
    Of course, I'd be frightened to find out the personality traits of Ars.com posters. You will never find a more wretched hive of scum and villany!!!

    At least you can hold an intelligible discussion here, as opposed to on, say, 4chan.
    166 posts | registered
  34. quote:
    Originally posted by TheFerenc:
    darkowl, is there a list somewhere of these people? I find myself curious to know more.
    http://en.wikipedia.org/wiki/Template:Wikipedia_fauna

    EDIT: Oh woops. That'll teach me to post without reading all the comments first.
    3893 posts | registered
  35. quote:
    Originally posted by asolipsist:
    Maury is right.

    The science content, even slightly off the beaten path, completely fails being an encyclopedia, that is clearly explaining and summarizing a topic for non-experts. I'm sure most physics, chem etc articles are factually correct, but they are obtuse.
    ++++

    My roomate and I always argue about this. He says, "No! All the esoteric concepts and terms are linked, so you can just read about them and come back and finish the article!" He doesn't seem to realize that these math and physics articles follow an infinite regress of utterly unintelligible articles. I once spent like 6 hours trying to read about topology and set theory. I learned some things, but I must have gone through over 15 articles trying to understand the basics, and I still couldn't get through a sentence without running into some technical term that wasn't defined, and had to go read another article, just as opaque. I'm not dumb, and my math goes up through calculus (more of a liberal arts guy), but man if it wasn't frustrating.

    Of course, my roomate is an admin on Wiktionary. Now THOSE guys are grumpy! Just try breaking that cabal. It makes Wikipedia look like a free love commune.
    3893 posts | registered
  36. quote:
    Originally posted by Shades047:
    quote:
    Originally posted by Laner:
    quote:
    Originally posted by RRGG:
    this is one of those things i didn't need a study to tell me.


    ++


    ++++


    Plus etc. I didn't realize anyone needed to do an actual study to come to this conclusion.

    As others have said, you've got roughly three main types of articles. You've got the truly random stuff that is full of spelling mistakes and horrible grammar that are borderline useless. You've got your "pet" articles patrolled by the Wikipedia elite who seemingly never sleep and constantly revert everyone's edits. Then you've got the articles that would likely be superb scientific/mathematical/etc. resources if they weren't full of nothing but industry lingo and jargon in order to make the author feel smart.

    It's quickly devolved into the same kind of community as your random general purpose forum. Useful content only exists at the whims of the supposed elite, which sucks considering the idea being the project.
    796 posts | registered
  37. quote:
    Originally posted by rotational:
    quote:
    Originally posted by Shades047:
    quote:
    Originally posted by Laner:
    quote:
    Originally posted by RRGG:
    this is one of those things i didn't need a study to tell me.


    ++


    ++++


    Plus etc. I didn't realize anyone needed to do an actual study to come to this conclusion.

    As others have said, you've got roughly three main types of articles. You've got the truly random stuff that is full of spelling mistakes and horrible grammar that are borderline useless. You've got your "pet" articles patrolled by the Wikipedia elite who seemingly never sleep and constantly revert everyone's edits. Then you've got the articles that would likely be superb scientific/mathematical/etc. resources if they weren't full of nothing but industry lingo and jargon in order to make the author feel smart.

    It's quickly devolved into the same kind of community as your random general purpose forum. Useful content only exists at the whims of the supposed elite, which sucks considering the idea being the project.
    I disagree with your conclusion. Imagine life without Wikipedia. It's nothing short of incredible. There are problem areas, no doubt, but I think your characterization misses a lot of really good articles that are quite informative and well-written.
    3893 posts | registered
  38. Unecessary article on a statistically worthless survey about a banal topic.
    427 posts | registered
  39. <joke>It is a Palestinian plot to make Isreal look bad in the eyes of the world</joke>
    50 posts | registered

You must to comment.

   

You May Also Like
http://arstechnica.com/science/2009/06/wikipedians-grumpy-introverts-best-not-to-generalize/?comments=1
arstechnica.com
crawl=CC-MAIN-2015-48
143256
application/http; msgtype=response
sha1:BRYXNETOKRPCXWKEI4W7OZE4YQ2GJAR3
<urn:uuid:602222ca-bd40-4bb1-8b4b-b1c609e5eb9e>
2015-11-29T13:33:50Z
50.31.151.33
sha1:5SXW7MEQ22XAHC22IKL24OGSNPVBJEO7
<urn:uuid:7789f846-5366-49cc-a35f-db7af98a2030>
http://arstechnica.com/science/2009/06/wikipedians-grumpy-introverts-best-not-to-generalize/?comments=1
length
response
<urn:uuid:05b0597f-2d0e-40e7-99f6-f7cbf1aaa7c1>
robots: classic
hostname: ip-10-71-132-137.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2015-48
operator: CommonCrawl Admin
description: Wide crawl of the web for Nov 2015
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf