45

Almost every user has the ability to flag posts, yet it's clear that many do not understand what flags are for or what moderators can do about them. Even very active users can be unaware of subtleties around flag handling, like the fact that even moderators can't migrate posts more than 60 days old.

To date, moderators have used declined flags to try to educate flaggers as to what kind of flags are inappropriate to use on content. This has been enhanced by the timed warnings and bans that the system now imposes after a significant portion of flags are declined.

However, this can also lead to some consternation on the part of the flagger (which we occasionally see on Meta) and does nothing for the many new users who fire off one or two "plz anser thz urgent" flags and then stop.

I propose to use time-of-flagging heuristics to detect potentially bad "other" flags as they're being written and to warn people before those flags are submitted. Ideally, this would catch flags we can't act on before they were cast and would educate a flagger without the disappointment of a declined flag.

I imagine a warning similar to the one that was tried on database-related tags:

SQL Tag Message

and that would pop out of the flag prompt as you were typing, if a heuristic was tripped. While I recognize that many people ignore text that is placed in front of them, these kinds of warnings do catch your eye and people did seem to pay attention to them when used for the database tags.

I envision a few heuristics to start with:

Migration requests for content more than 60 days old

Even moderators can't migrate questions that are more than 60 days old, so we decline almost all of these flags. If a question is more than 60 days old, and someone flags it with "migrat" or "mov" somewhere in the flag, they should be presented with a warning like:

This question is too old to be migrated, even by moderators. Any flag to request this will be declined.

Flags indicating incorrect answers

Moderators are not here to judge the correctness of answers, yet we are often flagged by people stating that one answer or another is incorrect. If someone flags an answer using the words "incorrect" or "wrong", they should see a warning of

Flags should not be used to indicate technical inaccuracies, or an altogether wrong answer.

(which is a built-in flag decline reason at present)

Flags asking moderators to answer their question

These are perhaps the most infuriating flags that we see, because of the often extremely selfish nature of them and the fact that they are nowhere near an appropriate use of flags. Interestingly, no other site in the Stack Exchange network sees these, just Stack Overflow. Some examples:

Please help me to solve this problem... There are few more errors
Need urgent help. I need to do this for a college homework
Please I need answers to my question. I'm yet to resolve this challenge. 
please Help for me.......... 
Need urgent help!!
Expecting a solution
need urgently
Solution..
m.m,ms laxkamx;lamx;lax;la;lxa;lx
help me for
i am flagging this question beacuse i dont find an answer as i aspact

I've been analyzing the ones of these that I've handled over the last few months and have seen some patterns in them. In 580 flags like this that I've handled, the word "urgent" appears 121 times, the phrase "help me" 92 times, "asap" 11 times, and I'm still doing data mining to identify other likely indicators of flags like this on questions. Words like "urgent" and "asap" almost never appear on good flags, and "help me" rarely does.

If you'd like to take a stab at data mining these yourself, and don't mind staring into the abyss, I've extracted and anonymized over 400 of these terrible flags in this gist.

For flags on questions that meet these heuristics, a warning like the following could be presented:

Flags are not to be used to request answers. Moderators do not answer flagged questions.

I'm open to any other suggestions for heuristics on bad flags that other moderators or users could provide in answers below, and I'll edit this with others as I think of them.

For now, I'm posting this on Meta.SO because we have a different class of bad flags than other sites do, so these heuristics may not apply across the network.

To be clear: I'm not asking for a ban on casting flags that meet these heuristics, because we've had "porblems" with banning keywords in the past. I think a pop-up warning in line with the flag text box would catch the eye of flaggers and could stop a number of these before they are filed. Even those leaving "answer my question" flags seem to stop after being told this is inappropriate.

I think it's worth a try to potentially reduce the flag workload and educate flaggers in a gentler way.

4
  • 4
    For most of these, we should be able to reuse the existing blacklist/warning system - worth a try at least. Migration would be a bit more involved, so I think we should try the pure keyword-based ideas first.\
    – Shog9
    Nov 9 '15 at 19:33
  • "Interestingly, no other site in the Stack Exchange network sees these, just Stack Overflow" isn't correct. We see them too on Mathematics. But not as frequently as you. Nov 9 '15 at 22:42
  • 3
    Do you have a similar set of anonymized known-good flags to mine in comparison? It seems like that would be necessary for anything more sophisticated (i.e., data-driven) than "'please help' looks vampire-ish" manual checks. Nov 9 '15 at 23:13
  • 1
    Three heuristics that were not mentioned: flag text is any of 1) title of question 2) substring of body in question 3) tags
    – BoltClock
    Jul 14 '16 at 3:56
14

After 6-8 weeks months somethings, we finally have this in place, but it's not quite turned on yet.

As Shog9 mentioned in his comment, we leveraged the existing blacklist/warning system to warn and/or block users from using the custom other moderator flag...incorrectly.

While it's in place, it's not yet fully implemented because we need to make sure that we aren't preventing valid flags, so we're doing a bit of testing and fine-tuning before we fully release this. We've selected a few of a really, really bad flags 1, 2 that moderators get.

If those work, then we'll look at expanding this to more custom flag comments/content.

10

Well, here's some data to help out with creating heuristics.

I pulled the linked gist of bad flags into a file, I wrote this script to analyse what combinations of words come up together most often, and I got some results.

The most common pairs of words:

'help-me' : 79
'please-help' : 78
'urgent-urgent' : 70
'i-need' : 56
'i-am' : 47
'i-have' : 31
'this-question' : 28
'need-help' : 22
'to-solve' : 21
'this-problem' : 21
'this-issue' : 19
'solve-this' : 18
'my-question' : 18
'on-this' : 17
'me-to' : 15
'not-getting' : 13
'a-solution' : 13
'no-one' : 13
'this-is' : 12
'an-answer' : 12
'my-project' : 12
'for-this' : 12
'need-to' : 11
'need-the' : 10
'it-is' : 10
'me-out' : 10
'to-this' : 10
'the-answer' : 10
'any-one' : 10
'need-a' : 10

And the most common triplets of words:

'urgent-urgent-urgent' : 69
'please-help-me' : 49
'to-solve-this' : 14
'help-me-to' : 13
'i-need-help' : 11

I haven't done anything above that because the script was only simple, but that should be a start in identifying phrases to ban (or at least put at the bottom of the queue).

3
  • Well, this flag-message fits right in: "This question has a bounty but is a duplicate of my question here. Thus I need help dupe-closing it." (The problem is that we also need a representative dump of some good flags.) Nov 9 '15 at 23:30
  • @Deduplicator aye, run the same script on a good sample and it'll give a list of phrases not to ban.
    – ArtOfCode
    Nov 9 '15 at 23:50
  • this question jumps out at me as an attractor of false positives.  My most recent two custom flags (on Super User) contained the phrases "this question" and "the question" (e.g., "This question ___, but I can't VTC because it has a bounty.") and they were both marked as "helpful".  Consider also "This question might be better suited on (other site)", "This question is not in English" (a frequent problem on Super User), and "The tone of the this question borders on the abusive."
    – Scott
    Jul 20 '16 at 3:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .
https://meta.stackoverflow.com/questions/309732/provide-context-specific-warnings-at-time-of-flagging-for-incorrect-flag-use?noredirect=1
meta.stackoverflow.com
crawl=CC-MAIN-2021-49
132154
application/http; msgtype=response
sha1:BQFJ4HDMRZVRYXKI7IN62I5VEHNYLHHF
<urn:uuid:5560c7b2-10c1-4cd2-a1a3-f3a0e1c07b62>
2021-12-09T04:43:03Z
151.101.193.69
text/html
sha1:ZZEMNT4MYMVZIUQFJSEU4R4ISCEMPZWV
<urn:uuid:41b02a37-7b2e-4c6c-99c2-4ddaf2d991e0>
https://meta.stackoverflow.com/questions/309732/provide-context-specific-warnings-at-time-of-flagging-for-incorrect-flag-use?noredirect=1
response
<urn:uuid:abaeedc7-7f3b-4635-8408-d4d61b4eff21>
isPartOf: CC-MAIN-2021-49
publisher: Common Crawl
description: Wide crawl of the web for November/December 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-133
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.3-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/