Quality review

From mediawiki.org

The open nature of wikis makes quality management a serious challenge. This page gives a general overview of quality review systems and their characteristics.

Review processes by types[edit]

A review process can be:

  • internal: the process is mainly handled locally by the wiki's community, and the data is stored in the wiki's pages or its database;
  • external: the process is mainly handled by an external group, and the data is stored outside the wiki. Integration with the wiki is optional.
  • mixed: e.g., an external group posts reviews on the wiki.

Some authority-based review processes distinguish between "experts" and others. In this case, reviewers can be classified whether they are:

  • "non-experts"
  • people self-identifying as experts (no credentials verification)
  • people identifying as experts whose credentials have been verified.

Example review processes[edit]

A more detailed description is available for some projects.
  1. People with academic degrees or publications in the relevant science (RNA/protein biology)
  2. Reviews are filtered a posteriori based on some basic level information provided by reviewers about their education level and whether they are a member of APS or not.

Review systems[edit]

  1. Exclusive: only one master review/rating possible; cumulative: multiple reviews/ratings possible.
  2. For example to select articles for publication in other media like books.

User research[edit]

In the context of quality review, there are two kinds of users: reviewers and review readers.

From a reviewer point of view

  • What system, how it works, what the goals are, what they expect
  • Do they want to communicate with review readers?

From a review reader point of view

  • What are their goals by reading a review? (i.e. what is useful for them)
  • What do they expect from a review?
  • Do they want to communicate with reviewers?

People who have already created review processes might be able to explain:

  • what were their goals when they created these tools & processes
  • whether some design decisions were intentional (e.g. type/scale of rating)

Domain research[edit]

Sources:
https://www.mediawiki.org/wiki/Quality_review
www.mediawiki.org
crawl=CC-MAIN-2022-27
51865
application/http; msgtype=response
sha1:UH6KDSU34KLZU2UJFSKNIGRHT4KKQ7XO
<urn:uuid:f6f8d24e-6d58-4788-9579-be46d143d441>
2022-07-03T21:32:07Z
208.80.154.224
text/html
sha1:OXSX7XBJK6LN7EVHLZNDGROOWRBM4Y6M
<urn:uuid:0416ec97-7c14-4ed5-8063-9a99bcfd0f34>
https://www.mediawiki.org/wiki/Quality_review
response
<urn:uuid:a1c75727-9bae-44b9-8e7c-8c1de8425fad>
isPartOf: CC-MAIN-2022-27
publisher: Common Crawl
description: Wide crawl of the web for June/July 2022
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-171
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.3-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/