dark        
Jinx 01 blog header photo
Jinx 01's c-blog
Fronts 7Posts 776Blogs 41Following 0Followers 51


 
 

LONG BLOG

​The Impossible Ethics of Review Events

   0

I wasn’t very interested in Titanfall back when it came out. It was impossible to avoid the game, however, with all the press it got. Despite my lack of interest, I somehow got a little into the hype and watched the reviews come in on release night.

As I read them, I noticed that many had disclosures referencing a “review event” EA and Microsoft had thrown. I’d never heard of these before, and the idea seemed bizarre. Why were pre-release download codes not sufficient? While I applaud that sites are trying to be more up-front about their reviewing and ethics policies, when I see a disclosure like Polygon's at the beginning of a review I'm not sure what I'm supposed to think:

"Note: This review involved a two-day review event hosted by Microsoft and EA, playing on Xbox One consoles and "retail servers" on March 3 and 4, as well as time spent playing "retail" downloadable copies prior to launch by multiple Polygon editors. Staff also participated in Titanfall's closed alpha and beta in January and February respectively. This review will be updated to reflect any extended server issues, should they arise. You can find additional information about Polygon's ethics policy here."
Polygon's review 9/10

When you need to put a huge disclaimer on your review assuring people that you were not unduly influenced there is already a problem. Reviewers shouldn’t be doing things so questionable that an entire paragraph is needed by way of explanation. Never mind that EA had the gall to call this a "review event", implying that reviews should be based solely on the impressions journalists received in a tightly controlled environment. Quite the arrogance on their part, to try dictating conditions like that, but it’s exactly what some reviewers let them do.

"Titanfall was played for review at a two-day event in London, hosted by EA. Eurogamer paid travel and accommodation expenses."
Eurogamer's review (score withheld until servers tested) (8/10)

"Editor’s Note: These review impressions were formed at a private two-day EA event, exclusively on Xbox One. Due to insufficient play time on public servers, we have decided to withhold our final judgment until we can play under real-world circumstances. Look for our final review and scores for both the Xbox One and PC versions – complete with any updated impressions – in the coming days."
IGN's review (8.9/10)

"This review is based on Xbox One version of Titanfall, played at an Electronic Arts review event on live servers. The Xbox One retail version was also tested on live servers. Titanfall is not playable offline. Images: Electronic Arts" (Why take your own screenshots when you can use EA's bullshots?)
Joystiq's review 4.5/5.0 Stars

"Chris Watters played Titanfall for two days straight at a review event run by EA, where he encountered server issues allegedly caused by small player population. Over the weekend preceding publication of the GameSpot review, he played from home as the servers went up and was able to reliably enter lag-free matches in the most populated lobby."
Gamespot's review (9/10)

"We reviewed a prerelease version of Titanfall on the Xbox One at a dedicated press event hosted by Electronic Arts, who also provided GamesBeat with a downloadable copy of the game for the purpose of this review."
Venturebeat's review (82/100)

Destructoid didn’t mention the event so I asked about it in the review’s comment section. Chris Carter, who wrote the review, replied: "We didn't attend the review event. It's against our policy to score games based on those." A few other sites didn’t mention the event, so it's hard to say whether they didn’t go or simply neglected to disclose that they had.

Some of these disclosures mention that the game was played under other conditions, even though that’s not something that needs to be disclosed. So why is it there? Because the reviewers themselves felt that their impressions from that EA event could not be trusted. Because even they knew it looked bad.

I don't recall the event ever being mentioned previously on any of the sites I frequent before release. One wonders if this was a matter of MS/EA including their attendance as part of the review blackout. Or perhaps sites were just avoiding the walk of shame until the day of the review when they were forced to come clean because of their own ethics policies. The lack of coverage of the event itself makes it hard for us to judge how much it might have influenced reviewers. I couldn’t find any photos from it, but a two-day event for a single game seems excessive. I did find photos from a couple other press events, though, and it seems safe to assume the review event was at least as cushy as these in terms of giant monitors, expensive headsets, and conditions otherwise made optimal.

Titanfall Press Event (story posted 2/13/14)

"TiX was kindly invited to a very exclusive hands on preview event at a secret location in London for Titanfall held by Respawn Entertainment and Xbox UK."
This is Xbox London Hands-On Event (story posted 2/12/14)

“Secret location”, eh? That doesn’t sound shady at all!

There was also a "launch event" to which Eurogamer and others were invited. Eurogamer even live-streamed it. Their promotion of it raises further ethical concerns. Were they compensated for that additional advertising? Even if not, it was a partnership that resulted in additional coverage and hype for the game.

When Disclosure Is Not Enough

I do think sites and reviewers deserve praise for trying to address these issues by disclosing their attendance of these events. But have no doubt – Microsoft and EA spent a lot of money on that event, and they surely expected a return on their investment. What did they get from it? A lot, regardless of how ethical the attendees tried to be. For example, some sites like Eurogamer tried to be clear that they paid for their own travel and lodging. Yet it might have been better if journalists’ expenses were paid by MS/EA. Any site that invests money to send journalists to the event will want, themselves, to get something out of that trip – in the form of game coverage that will net them further advertising revenue. And how could going to a cushy event like that not affect their perception of the game? How could it not make them feel obligated toward the people who had hosted them?

I am certainly not suggesting that reviewers were "bought". It seems clear that most reviewers were, in fact, doing their best to avoid conflicts of interest, remain objective, and disclose anything that might have influenced their review. The problem is that going to a review event at all is accepting undue influence. Both the publisher and the review sites are making an investment of time and money with those events – and the optimal return on that investment for both parties is a large amount of positive coverage of the game.

You also have to wonder about the "preview events" I referenced earlier. Websites might not feel obligated to disclose attending those. Yet they would have, like the review events, encouraged further positive coverage. In fact, the game got so much coverage that readers were often seen mocking websites for the ridiculous number of Titanfall articles they were posting. Was it due to these review and preview events? It's hard to say. But as I said, going to them is an investment of time and money that websites would want to recoup in the form of coverage.

In the case of Titanfall, did it work? I’d say yes. In addition to receiving an excess of preview coverage, critics’ review scores were rather high in comparison to the lukewarm response the game got from actual gamers. After months of intense coverage and hype, the game was pretty much forgotten by everyone shortly after release, and was receiving deep discounts on PC just weeks after launch.

Being Realistic

Conflicts of interest and outside influences can’t be totally avoided when it comes to covering and reviewing games. Journalists need to attend events like E3 to learn about upcoming games, and direct contact with smaller developers is important for learning about their games. It’s also a “small” industry. People in the press become friends with others in the industry, that’s natural. All of these things will influence journalists, but not in any way a reader wouldn’t expect. And journalists, of course, should be experienced at separating themselves from all of this when it comes time to write an actual review.

These review events, though, seem like a step too far. You can’t go to a cushy two-day LAN party thrown by a publisher and think just disclosing that is enough to satisfy a sense of ethics. The event is going to create both an idealized experience of the game and a sense of gratitude and obligation toward the publisher. Their views on the game are unavoidably influenced in a way that I feel should ethically preclude them from reviewing the game. Game journalists will always be walking a fine line between maintaining access and maintaining objectivity, but this is one situation where there may be no middle ground.

Login to vote this up!

LOOK WHO CAME:


Jinx 01   
Larx   36
Floppablecat   31
GodEmperor Paige   14
TheBlondeBass   11
Wes Tacos   10
Malthor   8
Scrustle   7
Elsa   5
absolutfreak   3
Gajknight   2
CharlieWhistle   2
Agent9   2
SUPERHITLER   2
Luckrequired   1
Dark Chao   1
Gamemaniac3434   1
Gortexfogg   1


 
 

  0 COMMENTS

Please login (or) make a quick account (free)
to view and post comments.



 Login with Twitter

 Login with Dtoid

Three day old threads are only visible to verified humans - this helps our small community management team stay on top of spam

Sorry for the extra step!

 

About Jinx 01one of us since 4:26 PM on 11.18.2011

I love gaming, and I have followed the industry and its technology since I was a kid in the 80's. I have gamed primarily on PC since 2000, though I still follow console news and hardware as well.

I worked as a mapper and beta tester for the mod Action Half-Life. I also make custom images for Steam's grid view, those are over on my website.
Steam ID:cryotek


 

Around the Community

Featured

Posts

Blogs

Support

 
https://other.destructoid.com/--350140.phtml
other.destructoid.com
crawl=CC-MAIN-2022-33
46444
application/http; msgtype=response
sha1:SAIVZ3IYTWRI7BZ3F224SHIZBOO7P5O3
<urn:uuid:d3a84903-2b88-4bea-9a53-4d4cc29951d4>
2022-08-11T09:11:57Z
172.66.43.42
text/html
sha1:CFQUVALMJXILZ7FE3FPT6SV64LA5VEKL
<urn:uuid:3f721c4c-5a32-44a3-bf12-cb066dcafcfa>
https://other.destructoid.com/--350140.phtml
response
<urn:uuid:76be7a41-0a8b-4cd0-8ccb-bee51a60005e>
isPartOf: CC-MAIN-2022-33
publisher: Common Crawl
description: Wide crawl of the web for August 2022
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-209
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.4-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/