49

We are building a large community that will be massive and difficult to monitor. I admire Stack Overflow, because it has such high-quality content/comment community-based websites. I see Discourse has some similar community monitoring systems in place, and I know they are related, but are they using similar code bases?

We would like to have several million listings with discussion threads, but monitoring them will be difficult if not impossible. I understand CAPTCHA works to some extent, but there are still trolls and computer intelligence has basically broken it. I know Google has came out with a new version, but that still doesn't stop the endless horde of trolls.

I see there is more a community-based effort in cleaning up this site: in which ways does Stack Overflow control content? I also see bots from time to time, but overall it seems to be a voting system. Are the higher-up members assigned more weight to their votes, and what does it take to be a moderator? Can one moderator close a question? What type of algorithm goes into a revolutionary website like this?

I understand there are probably trade secrets, but if we were to build such a system what is its basis, and is it already incorporated into Discourse?

2
  • 3
    see the help page on privileges to see what moderation powers users have, and at what rep level: stackoverflow.com/help/privileges
    – samgak
    Commented Jan 30, 2015 at 2:51
  • 37
    Internet points, valuable valuable internet points.
    – user764357
    Commented Jan 30, 2015 at 5:10

6 Answers 6

121

I don't know anything about how Discourse does it - but they're pretty open, if you're willing to do some research.

We rely on people. Lots and lots and lots of spam-hating people. The energy of their raw, combined hatred, channeled through flags and focused by the system, vaporizes spam nearly instantly, leaving only a pink, faintly pork-scented vapor behind.

Tim writes a bit more about this here: Recent Mass Football Spam

7
  • 2
    excellent people. i for one hate spam and will flag it to the authorities as soon as i see something that is clearly spam. spam increases the noise level.
    – Ryan
    Commented Jan 30, 2015 at 4:40
  • 8
    Very poetic answer... Blocking spam, it's all the rage! ;) Commented Jan 30, 2015 at 4:49
  • 47
    "We rely on people... spam-hating people." Tabloid headline: Stack Exchange relies on hate.
    – yannis
    Commented Jan 30, 2015 at 13:45
  • 3
    Now we just need to weaponize our raw spam-hatred, and we can take over the world! Commented Jan 30, 2015 at 17:42
  • 5
    There is so much hate I never see spam on SO.
    – Raedwald
    Commented Jan 30, 2015 at 19:04
  • 1
    @Yannis Other tabloid headline: "Spam found to be a source of renewable energy" Commented Feb 1, 2015 at 14:12
  • 2
    @Yannis, don't underestimate the power of hate. :P
    – Jaydles
    Commented Feb 2, 2015 at 17:09
23

It's because Stack Exchange attracts the right kind of folks.

I've been here for a good five years. The first 3.5 actively answering, editing, etc. Through that time I've seen some awesome stuff happen.

It's not just the spam that is kept out, but, as you put it, we're "clean". We have good content that people are willing to spend their time to maintain.

There's no "trade secret" It's all here: https://stackoverflow.com/help/privileges

The folks at Stack Exchange worked with developers right on this here ground..erm site and talked/listened and worked out many spam fighting, quality maintaining systems. Then we also have http://stackapps.com and https://data.stackexchange.com/ ... .....all this attracts the geeks* you need who will strive to work hard for precious Internet points!

*I for one took my entire team and coworkers from work for lunch the day I reached that sweet pinnacle of 10k.

8

Spam is fought by a series of flag votes. If a post gets too many drastic flags (there is a set of flags which range from mediocre to harsh) then the autobot (aka Community) takes action against the post.

For posts which do not receive an abundance of the worst type of flags then a mighty sorcerer (moderator) steps in and handles it.

8

"Users who care" is the most important factor. And other answers have covered that well already.

But there are a couple of other things worth mentioning:

We leverage what all those users tell us with systems that use that data to recognize similar patterns when they pop up again.

Without getting into the details, when users flag things as spam, we save that information and use it to automatically identify similar activity in the future that is likely to be spam-esque.

The way some privileges are unlocked only after you've demonstrated some commitment to the site is a big factor in keeping spam at bay:

And understanding its role in keeping the site clean helps to explain a part of the system that often confuses new users, or feels elitist:

  • Comments are one of the best examples. Because new comments don't bump posts to the front page where a lot of eyes are on em, they'd be a big spam risk if any new account could post em. the fact that spammers can't post them without first earning some rep is a big help at the top of the funnel, and keeps a ton of crap out from the get-go.
  • Bumping a post whenever there's a new answer helps here, too. Since the only place a new user can put words (without queue-based review) is a question or answer, bumping ensures anything piggy gets seen right away.

Those are just a couple of examples, but much of the unlocking system is optimized to ensure that you only get the ability to spray nonsense around once you've invested too much time to make the approach a viable spam strategy.

Which is good. Because, while we take great pride in the number of users who care enough to help flag and clean up spam, we'd rather let them save their valuable time for answering questions and increasing knowledge.

3

Stack Exchange has a site for this.

Community Building
https://communitybuilding.stackexchange.com/

Check what’s on-topic there.

2

Users. A lot of enthusiastic users. Flagging is the start of it, but there is more. You might be interested in some chat rooms that deal with a lot of stuff. You have close-review rooms for example, but you even have community-broad chat rooms that deal with a lot of spam. Some users even write their own bot that crawl the front pages from all sites for spam and low-quality content.

Some resources:

Beside that, there a just a few moderators, but they do a lot of work too. They do the follow-up on spam, for example by deleting the user and communicating with SE to block IP addresses.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .
https://meta.stackoverflow.com/questions/284902/how-does-stack-overflow-keep-such-a-clean-community-and-keep-out-spam-algorithmi
meta.stackoverflow.com
CC-MAIN-2024-33
180823
application/http; msgtype=response
sha1:X2VGE4UH5IFQB7ZN7JW7TVSTXMVICE3K
<urn:uuid:ddd1eedb-3c3f-4084-8342-e84b03bcee84>
2024-08-14T23:32:25Z
104.18.32.7
text/html
sha1:5ZORNAWBORDH5LQQNBU5UJVXI2LHJ5RW
<urn:uuid:a5a75517-b0d4-4530-bc7e-9db4d99b290f>
https://meta.stackoverflow.com/questions/284902/how-does-stack-overflow-keep-such-a-clean-community-and-keep-out-spam-algorithmi
response
<urn:uuid:6ade5653-76fb-48c0-824f-21715803b442>
isPartOf: CC-MAIN-2024-33
publisher: Common Crawl
description: Wide crawl of the web for August 2024
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-150
software: Apache Nutch 1.20 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.5-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/