First Monday

Aggregated trustworthiness: Redefining online credibility through social validation by Johan Jessen and Anker Helms Jorgensen



Abstract
This article investigates the impact of social dynamics on online credibility. Empirical studies by Pettingill (2006) and Hargittai, et al. (2010) suggest that social validation and online trustees play increasingly important roles when evaluating credibility online. This dynamic puts pressure on the dominant theory of online credibility presented by Fogg and Tseng (1999). To remedy this problem we present a new theory we call “aggregated trustworthiness” based on social dynamics and online navigational practices.

Contents

Introduction
Defining online credibility
Aggregated trustworthiness
Explanatory power
Conclusion

 


 

Introduction

Credibility of online information has always been a key factor in our understanding of the Web at large. With the recent rise in social media and particularly user–generated content, our need for a solid understanding of online credibility is becoming ever more important (Gillmor, 2008). Recent empirical studies suggest that we put more emphasis on social validation than traditional expert sources when assessing online information (Hargittai, et al., 2010; Pettingill, 2006). Yet, our current theories fail to explain this dynamic, leaving us with an insufficient framework to analyze and explain the workings of Web sites such as Twitter and Wikipedia.

The point of departure for this paper is a discrepancy between the leading theory of online credibility and our actual online behavior. We argue that the social dynamics online deeply impact our evaluation process by incorporating other people’s evaluation and the navigation process itself. The unique characteristics of online information seeking routines and navigational practices presents a more dynamic theory of online credibility evaluation. We call this theory “aggregated trustworthiness.”

 

++++++++++

Defining online credibility

Several theories have tried to pinpoint the concepts involved and the workings of online credibility. Fogg and Tseng’s (1999) paper on ‘computer credibility’ is broadly cited and generally offers the best theory of online credibility. However, their core theory was developed over a decade ago, and as we will argue, crucial changes in both technological reach and online practices have made their theory of online credibility inadequate for our present day online environment.

To understand why it is necessary to explain Fogg and Tseng’s theory of computer credibility. They stress that credibility is always a perceived quality and not a property residing in any human or computer product. Rhetoricians use the term ethos to explain how the subjective quality of credibility evaluation (also known as trustworthiness) is primarily established by the receiver of the message, incorporating emotional sentiment and situational context (Bitzer, 1968; Warnick, 2004; boyd, 2008). In other words, we cannot design credibility itself, only design for credibility. Fogg uses a simple metaphor to illustrate credibility; like beauty, credibility is in the eye of the beholder and “it exists only when you make an evaluation of a person, object, or piece of information.” [1].

That said, evaluating credibility is not random. Certain qualities exists to help lay the basis for a Web site to be considered credible (Fogg, et al., 2002). Fogg and Tseng (1999) calls this property of credibility for believability [2]. In its simplest form credibility is the persuasive nature of the medium (Fogg and Tseng, 1999; Fogg, 2003). Thus, if a Web page is believable it is also considered credible. Fogg and Tseng argue that from the dozen or more elements that contribute to credibility evaluation, there are just two key dimensions of credibility: trustworthiness and expertise (Fogg and Tseng, 1999; Fogg, 2003).

 

Figure 1: Model of the key dimensions of credibility
Figure 1: Model of the key dimensions of credibility (Fogg, 2003).

 

Fogg defines trustworthiness in the terms well–intentioned and unbiased, and expertise in the terms of perceived knowledge, skill, and experience (Fogg, 2003). According to Fogg and Tseng’s theory a high level of credibility incorporates both a high level of trustworthiness and expertise. Hence, according to their theory, a Web site cannot be considered credible if it does not entail both of these concepts. This construct is the basis of their theory on online credibility.

 

++++++++++

Aggregated trustworthiness

Moving onwards from this theory, online users would tend to look for both trustworthiness and expertise cues to establish a level of credibility of the information in question — effectively mimicking the traditional routine of source evaluation. In this routine a user would look at who wrote some information to assess whether the author is trustworthy and an authority on the matter. If so, strong credentials and objectivity would serve as strong credibility cues. In most off–line settings this approach is very robust since verifying the identity and credentials of the source are much easier than evaluating the accuracy of their claims.

The problem, however, is that a great deal of information online is detached from these credential and authority cues. Particularly in user–created content platforms like wikis, review and rating Web sites, blogs, forums, Twitter etc. we find very few, if any, direct cues of expertise. Following Fogg and Tseng, this dynamic severely lowers the perceived credibility of online information, since we are not able to easily identify authors and hence evaluate their expertise.

Yet, recent empirical studies suggest that youth do in fact perceive information without an identified author as credible (Pettingill, 2006; Hargittai, et al., 2010), especially if some sort of collective judgment of the information is available (Lankes, 2008; O’Byrne, 2009). This means that the feedback of others is crucial when assessing the credibility of online information (Weinschenk, 2009; Ljung and Wahlforss, 2008). An extension of this dynamic is the reliance on trustees (Pettingill, 2006). Trustees often act as a form of authority and provides a baseline of trustworthiness. Trustees are not necessarily experts of the specific topic, yet are important fixtures in the evaluation process and the overall dynamic of establishing credibility (Wang and Emurian, 2005). In addition, the navigation process itself affects the perceived credibility by highlighting search rankings, topic correlations, and brand exposure (Hargittai, et al., 2010).

The social element attached to information is thus key when users evaluate its credibility. Enabling vote–like behavior such as comments, “Likes”, ratings, and even links simply provide a much broader spectrum of validation than possible in any off–line setting. Collecting multiple streams of trustworthiness cues to form an aggregate of credibility is at the root of this dynamic. We call this theory of online credibility, aggregated trustworthiness. The illustration below shows the factors and dynamic of aggregated trustworthiness (see Figure 2).

 

Figure 2: Illustration of aggregated trustworthiness
Figure 2: Illustration of aggregated trustworthiness.

 

On the right side of the solid arrow is ‘perceived credibility’ as the degree to which we believe the information presented to us. On the left side of the solid arrow, there are three main factors. ‘Social validation’ includes the large–scale verifications made by others (e.g., comments, Facebook Likes, shares, social bookmarks, ratings, etc.). Social validation may include profiles, but are not constrained to them. In our theory, social validation simply means that the more people acknowledge a certain piece of information the more trustworthy it is perceived. ‘Profile’ provides the baseline for identity online as well as adding a fixture of the evaluation (e.g., LinkedIn profile, Twitter stream, personal Web site or blog). Having a known identity can be critical when assessing important information. ‘Authority & trustee’ includes the known brand or authority on the matter [3] (e.g., New York Times, Stanford University, etc.), but also trustees verifying lesser known sources (e.g., social network ‘friends’, Wikipedia references, Twitter personas, etc.). These three factors are dependent on each other and are thus vetted into a larger system of navigation (the dotted arrows). This dynamic includes basic search and navigational processes (e.g., search context such as history, ranking, lookups, links, etc.) Together the model illustrates how social validation may provide verification of an authority, which in turn may provide verification of a specific profile, focusing our evaluation process and establishing the level of perceived credibility of the information.

However, we are not suggesting that quantity metrics such as Facebook Likes or Google’s PageRank can or should substitute a critical analysis of online information. Yet, we do argue that the theory of aggregated trustworthiness explains a dynamic unique to the Web and is made possible by factors not attainable in any off–line setting, such as large–scale feedback systems. These elements shift the perception of credibility from necessitating a fixture of traditional expertise cues, to a process which is inherently more dynamic and flexible, not hinging on any root authority.

Changing the dynamics this way effectively spreads out the risk of being mislead from relying on a few stable sources to many, albeit, less stable sources. The key here is how the dynamic functions without cues of root authority, and hence perceived expertise of the author of the information — a dynamic in stark contrast to our current theory of online credibility formulated by Fogg and Tseng (1999). Instead of factoring in the perceived expertise of the sender of information, we instead leverage social feedback and collective judgment to assess its credibility. Joining many untrustworthy pieces of information to stitch together a patchwork of credibility is simply easier in an online setting where root authority is much harder to establish (Lankes, 2008; Shirky, 2009).

We have based our theory of aggregated trustworthiness on two major studies of young adults’ evaluation of Web content and information–seeking routines (Hargittai, et al., 2010) and youth’s online research practices (Pettingill, 2006). Both studies demonstrate how youth gather credibility cues from a broad spectrum of sources, not confined to expert sources. Hargittai, et al.’s (2010) study even demonstrate how users not exposed to source credentials or traditional expertise cues still manage to successfully complete their given information–seeking tasks. Additionally, a group in one study placed great emphasis on trustees when accessing the quality of information (Pettingill, 2006). A trustee is typically a guiding person they know from an off–line setting like a teacher or a parent. Yet, youth involved in social networking had expanded these trustee roles to include members of their online network, leveraging the flexibility and reach of online social networks when evaluating online credibility. As Pettingill explains: “While subjects were ambivalent about the use of Wikipedia generally, those engaging in social networking sites daily were more likely to cite Wikipedia as a trusted source for information.” [4] The evaluations made by others are thus a key cue to determine the credibility of the information in question (O’Byrne, 2009). Aggregating a wealth of trustworthiness cues provide the most robust form of evaluation, when author credentials are hard or impossible to come by.

Understanding the social dynamics online is thus far more important in credibility evaluation than the static methods of traditional source critique. Traditional theories on online credibility, such as the one proposed by Fogg and Tseng, give very little attention to this fact. Root authority and source critique are still important factors (e.g., when analyzing author intent), but it is not the preferred method of evaluating online credibility. Using our theory we can explain why a number of Web platforms from Wikipedia to Twitter are perceived as credible, despite the lack of known authorities and traditional expertise cues.

Examining other individuals’ evaluations (e.g., Likes, shares, comments, etc.) or the aggregate of these activities (e.g., search ranking, Twitter trending topics, etc.) provides a baseline of judgement and helps our own evaluation (Shirky, 2009; Weinschenk, 2009). Key in this dynamic is how users use the built–in filtering mechanisms of the Web, such as search and social recommendations, putting greater emphasis on navigational tools and processes than expertise (Lankes, 2008; Shirky, 2008; Hargittai, et al., 2010). The core values are the same, but the process is different. In the following we will illustrate this notion’s explanatory power by using Wikipedia as an example.

 

++++++++++

Explanatory power

Aggregated trustworthiness helps explain the success of a number of online platforms like Wikipedia, eBay, Twitter, LinkedIn and many more. Wikipedia has proven exceptionally difficult to explain by traditional theories of credibility (Warnick, 2004), and hence it offers a prime example to explain and highlight the explanatory power of this notion of aggregated trustworthiness.

The platform of Wikipedia has brought with it an array of obstacles in terms of evaluating credibility, verifiability, and consensus of interpretation. Focusing only on credibility of edits we are faced with the following scenario. Normally, in an off–line setting, the author of a text is known and the time of publication is fixed, but not exactly on Wikipedia. What makes Wikipedia possible in the first place — its networked structure and review process — is also what makes it practically impossible to critically evaluate it using traditional methods of source evaluation [5] (Standler, 2004; Stvilia, et al., 2005). Evaluating a Wikipedia article by looking for a known author or time of publication is essentially meaningless due to its format and the dynamic nature of the medium (Warnick, 2004). Since articles are written and rewritten by multiple contributors, very few of which may be said to be a known authority on the matter (or visible to anyone outside the Wikipedia community), the traditional approach of evaluating author credentials and point–of–origin simply breaks down. The collaborative nature of Wikipedia is thus limiting for the traditional method of credibility assessment, but actually advantageous for the aggregated kind.

The vast majority of the editors and contributors on Wikipedia are anonymous. Yet combined and added together, the sum of their edits and re–edits seems to justify the inherent lack of root authority and identity. Seemingly, we do not trust the individual anonymous user, but do we trust a lot of them (Lankes, 2008). One explanation lies in the fact that it is extremely cumbersome to individually coordinate or influence the actions of a very large number of people (Shirky, 2008), especially, when all their actions are transparent to one another. It is simply unlikely that a contributor on Wikipedia had bribed all the other editors; it is much more likely that they independently from each other found the original contribution acceptable. According to Shirky, it is the incredible amount of social media that necessitates this post–hoc filtering approach: “The expansion of social media means the only working system is publish–then–filter.” [6].

The dynamic of Wikipedia’s post–hoc filtering system is dependent on social cues, broadly defined as our reactions to certain behavior characteristics, social validation principles, and general reinforcements through other people’s judgements (Resnick, et al., 2003; Weinschenk, 2009). By consulting what (a lot of) other people think we are presented by a form of transparency normally unavailable — be it news articles, forum posts, or blog comments. The opinions of others matter to us and studies done on whether online recommendations influence buying decisions showed a volume increase of 20 percent for items with recommendations over items without them (de Vries and Pruyn, 2008).

Since we react favorably to social validation, other people’s collective judgement is perceived as a trustworthiness metric. We even seem to react to artificial mimicry of social cues as a way to evaluate trustworthiness cues directly from computers (Mui, 2002; Nass and Reeves, 1996). This tendency of social validation and reliance of social cues is a key factor in the dynamic of wiki edits and a founding principle in aggregated trustworthiness.

Researchers at the University of California, Santa Cruz have even taken this dynamic a step further and developed an algorithm called WikiTrust [7]. Critics of Wikipedia have argued that there is no apparent way to see which articles, and more importantly what part of the article, is credible and which is not (Stvilia, et al., 2005). To counter this criticism and offer a solution using the process of multiple edits, social validation principles, and post–hoc filtering, these researchers have developed WikiTrust. The algorithm works by color coding every word in every article based on the reliability of its author and the length of time the edit has persisted on the page. The algorithm counts the number of silent edits any word has undergone and assigns a value to the author. The more unedited pieces of text an author has on Wikipedia, the higher the reputation [8] within the algorithm that the author will earn. Newly added edits from questionable sources are highlighted in bright orange; the longer information remains on the page, literally surviving multiple edits, it will turn from lighter shades of orange eventually to white (see Figure 3).

 

Figure 3: An example from WikiTrust
Figure 3: An example from WikiTrust: The Wikipedia page “Politics of Denmark”. Notice the error “Anders Fjogh Rasmussen” highlighted in bright orange. The word Fjogh is a play on the Prime Minister’s real middle name Fogh and literally means fool in Danish. Screen capture: http://wikitrust.collaborativetrust.com/screenshots, accessed 29 July 2010.

 

Wikipedia and particularly the WikiTrust algorithm are clear examples of the explanatory power of aggregated trustworthiness. Embedding the social verification into the edit and navigation process itself, we see how users are able to circumvent a credibility evaluation based on expertise. WikiTrust is just one example of a service letting people assess the credibility of online information based on the collective judgments made by other people.

 

++++++++++

Conclusion

Aggregated trustworthiness provides a more adequate explanation of online credibility. Incorporating social validation, online trustees, and profile–based Web sites, this notion is a first step towards better explaining the processes of credibility evaluation of online information and platforms lacking traditional expert cues. Highlighting recent empirical studies we have argued that information without explicit cues of authorship and expertise are in fact, and contrary to our present theories, perceived as credible. To illustrate the explanatory power of aggregated trustworthiness we have focused on the social dynamics of Wikipedia and how these dynamics mitigate the need for expertise to establish credibility. Aggregated trustworthiness is based on this social dynamic and points to a new direction for future research of online credibility. End of article

 

About the authors

Johan Jessen (M.Sc., IT University of Copenhagen) has studied online trust and credibility at the Department of Communication at Stanford University in 2009–2010 and is attributing author to Howard Rheingold’s book on digital literacy (in press). Direct comments to juje [at] itu [dot] dk

Anker Helms Jørgensen (Ph.D., University of Copenhagen) is associate professor at the IT University of Copenhagen with a primary research interest in human–computer interaction.

 

Notes

1. Fogg, 2003, p. 122.

2. A point worth noting is how Fogg and Tseng (1999) argue that the concepts of ‘trust’ and ‘credibility’ are not the same, and have unfortunately been used imprecisely and inconsistently in much of the relevant academic literature. They stress that ‘trust’ should be viewed as dependability and ‘credibility’ should be viewed as believability (Fogg and Tseng, 1999; Fogg, 2003). Related, yet not interchangeable. For the sake of brevity and clarity we will not challenge their definitions. Throughout the article we will refer to credibility as the degree to which we believe the information presented to us.

3. An interesting point of view is how authority and trustees do not have to be human. In a speculative blog post, Shirky argues that computer algorithms are increasingly exhibiting the same traits as a human authority and, more importantly, also increasingly being perceived as authority (Shirky, 2009). This point echoes much of the academic literature on the subject (Mui, 2002; Nass and Reeves, 1996).

4. Pettingill, 2006, p. 6.

5. A study published in Nature demonstrated that Wikipedia was just shy of being as accurate as Encyclopædia Britannica (EB). Based on 42 articles Nature found 123 factual errors in EB and 162 in Wikipedia. EB refuted the findings, among other arguments over the study claiming greater severity of errors in Wikipedia, but Nature did not alter its original analysis. See http://www.nature.com/nature/journal/v438/n7070/full/438900a.html, accessed 22 February 2010.

6. Shirky, 2008, p. 98.

7. http://wikitrust.soe.ucsc.edu/, accessed 27 June 2010.

8. An important point to note about the WikiTrust algorithm is how it does not measure the quality of the edits or the actual claims within the text. WikiTrust only measures credibility derived from consensus. If the majority of editors on a given topic do not correct an erroneous statement, it remains, building a ‘credibility score’.

 

References

Lloyd F. Bitzer, 1968. “The rhetorical situation,” Philosophy & Rhetoric, volume 1, number 1, pp. 1–14.

danah boyd, 2008. “Taken out of context: American teen sociality in networked publics,” Ph.D. dissertation, School of Information, University of California, Berkeley, at http://www.danah.org/papers/TakenOutOfContext.pdf, accessed 27 April 2010.

Peter de Vries and Ad Pruyn, 2008. “Source salience and the persuasiveness of peer recommendations: The mediating role of social trust,” In: Yvonne de Kort, Wijnand IJsselsteijn, and Cees Midden (editors). Persuasive technology: Second International Conference, Persuasive 2007 (26–27 April, Palo Alto, Calif.). Lecture Notes in Computer Science, volume 4744. Berlin: Springer–Verlag, pp. 164–175.

B.J. Fogg, 2003. Persuasive technology: Using computers to change what we think and do. Boston: Morgan Kaufmann.

B.J. Fogg, Cathy Soohoo, David Danielson, Leslie Marable, Julianne Stanford, and Ellen R. Tauber, 2002. “How do people evaluate a Web site’s credibility? Results from a large study,” at http://www.consumerwebwatch.org/pdfs/stanfordPTL.pdf, accessed 15 January 2010.

B.J. Fogg and Hsiang Tseng, 1999. “The elements of computer credibility,” CHI ’99: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 80–87.

Dan Gillmor, 2008. “Principles for a new media literacy,” Berkman Center for Internet & Society, Harvard University, at http://cyber.law.harvard.edu, accessed 27 April 2010.

Eszter Hargittai, Lindsay Fullerton, Ericka Menchen–Trevino, and Kristin Yates Thomas, 2010. “Trust online: Young adults’ evaluation of Web content,” International Journal of Communication, volume 4, pp. 468–494, and at http://ijoc.org/ojs/index.php/ijoc/article/view/636, accessed 22 December 2011.

R. David Lankes, 2008. “Trusting the Internet: New approaches to credibility tools,” In: Miriam J. Metzger and Andrew J. Flanagin (editors). Digital media, youth, and credibility. Cambridge, Mass.: MIT Press, pp. 101–122.

Alexander Ljung and Eric Wahlforss, 2008. People, profiles & trust: On interpersonal trust in Web–mediated social spaces. Berlin: Lulu.

Lik Mui, 2002. “Computational models of trust and reputation: Agents, evolutionary games, and social networks.” Ph.D. dissertation, Department of Electrical Engineering and Computer Science, MIT, at http://groups.csail.mit.edu/medg/people/lmui/docs/phddissertation.pdf, accessed 15 January 2010.

Cliff Nass and Byron Reeves, 1996. The media equation: How people treat computers, television, and new media like real people and places. Cambridge: Cambridge University Press.

W. Ian O’Byrne, 2009. “Facilitating critical thinking skills through content creation,” at http://www.scribd.com/doc/47085454/Facilitating-Critical-Thinking-Skills-Through-Content-Creation, accessed 25 June 2011.

Lindsay Pettingill, 2006. “Trust without knowledge: How young persons carry out research on the Internet,” GoodWork Project Report Series, Harvard University, number 48, at http://pzweb.harvard.edu/ebookstore/pdfs/goodwork48.pdf, accessed 22 December 2011.

Paul Resnick, Richard J. Zeckhauser, John Swanson, and Kate Lockwood, 2003. “The value of reputation on eBay: A controlled experiment,” Working paper, Division of Research, Harvard Business School, number 03–205.

Clay Shirky, 2009. “A speculative post on the idea of algorithmic authority,” at http://www.shirky.com/weblog/2009/11/a-speculative-post-on-the-idea-of-algorithmic-authority/, accessed 15 January 2010.

Clay Shirky, 2008. Here comes everybody: The power of organizing without organizations. New York: Penguin Press.

Ronald B. Standler, 2004. “Evaluating credibility of information on the Internet,” at http://www.rbs0.com/credible.pdf, accessed 27 April 2010.

Besiki Stvilia, Michael B. Twidale, Les Gasser, and Linda C. Smith, 2005. “Information quality discussions in Wikipedia,” at http://mailer.fsu.edu/~bstvilia/papers/qualWiki.pdf, accessed 25 June 2011.

Ye Diana Wang and Henry H. Emurian, 2005. “An overview of online trust: Concepts, elements, and implications,” Computers in Human Behavior, volume 21, number 1, pp. 105–125.http://dx.doi.org/10.1016/j.chb.2003.11.008

Barbara Warnick, 2004. “Online ethos: Source credibility in an ‘authorless’ environment,” American Behavioral Scientist, volume 48, number 2, pp. 256–265.http://dx.doi.org/10.1177/0002764204267273

Susan M. Weinschenk, 2009. Neuro Web design: What makes them click? Berkeley, Calif.: New Riders.

 


Editorial history

Received 12 August 2011; accepted 23 December 2011.


Creative Commons License
“Aggregated trustworthiness: Redefining online credibility through social validation” by Johan Jessen and Anker Helms Jørgensen by is licensed under a Creative Commons Attribution–NonCommercial 3.0 Unported License.

Aggregated trustworthiness: Redefining online credibility through social validation
by Johan Jessen and Anker Helms Jørgensen.
First Monday, Volume 17, Number 1 - 2 January 2012
https://journals.uic.edu/ojs/index.php/fm/article/download/3731/3132
doi:10.5210/fm.v17i1.3731
https://journals.uic.edu/ojs/index.php/fm/article/download/3731/3132?inline=1
journals.uic.edu
crawl=CC-MAIN-2022-49
34181
application/http; msgtype=response
sha1:4T7R7I24V4D7FKETC44DXFMOLC6FMCM5
<urn:uuid:ea768ebd-f281-43a2-8f1c-25079d449414>
2022-12-07T19:50:32Z
131.193.153.28
text/html
sha1:C4X3JGZTCRUYSJEGRXUGO4JDLSM3FOPO
<urn:uuid:f3c24610-dec3-4ba9-80c7-98937be58267>
https://journals.uic.edu/ojs/index.php/fm/article/download/3731/3132?inline=1
response
<urn:uuid:97c3102f-223c-4ee0-9cce-5e3e709cd343>
isPartOf: CC-MAIN-2022-49
publisher: Common Crawl
description: Wide crawl of the web for November/December 2022
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-177
software: Apache Nutch 1.19 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.4-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/