Rethinking flagging

@BrokenEagle98: For myself, I think the current system works...

2018-12-06T21:08:41-05:00

For myself, I think the current system works well enough.

The following are my thoughts on the current and proposed systems.

Status quo

Pros:

Multi-approver system guarantees all approved images will meet the lowest standards of at least one of the approvers, so it's very permissive

This also includes both pending and flagged posts

Appeal/Flag system provides a formal structured mechanism for any user to get a post's status reversed
The high penalty of post deletion is a great motivator/demotivator for image quality

Cons:

Moderators are needed to maintain the list of current approvers, adding or pruning as needed

Perception from some that the mod queue (list of approvers / unlimited uploaders) doesn't work, as image A gets approved, but image B doesn't

Perceived slighting, whether real or not, of a user's judgement when an image is flagged/unapproved

Either:

Depends on user's POV

Deleted posts hidden by default

Proposed

Pros:

Any user can change the quality tags

Cons:

Any user can change the quality tags

Retarded edit wars, no formal structured mechanism for changing a quality tag

Vast potential for vandalism, especially older less-monitored posts

Perceived slighting, whether real or not, of a user's judgement when an image is tagged

Medium/low penalty of post tagged with bad quality, providing a "meh" motivator/demotivator for image quality

Either:

Depends on user's POV

Bad quality hidden with a blacklist instead

Final

I prefer the current system which is well-defined and structured, providing formal mechanism to change a post's status. The proposed system seems a lot more chaotic, and in my opinion it is ultimately more likely to increase the amount of drama rather than decreasing it as claimed.

@Dalamar: > nonamethanks said: > > We already have that...

2018-12-06T19:45:50-05:00

nonamethanks said:

We already have that as status:deleted.

And there's already a booru that doesn't delete bad stuff - it's called Gelbooru.

Gelbooru also doesn't put much effort in tagging. Sankaku is also better than gelbooru but they're still both pretty bad. Also, many unapproved posts only have minor quirks, rather than being straight terrible like what you see on TBIB etc.

And "good" and "bad" are both rather subjective. Certain elitists are always raising stinks about bad art and then posting stuff like: /posts/3339569 *cough*
I can find problems with the vast majority of top tier art(ists) if I wanted to!

Either way I'm really fed up with the mod queue problem.
More proof it doesn't work, this has been pending for 2 days for example: /posts/3338011

iridescent_slime said:
Also, I don't think tags work the way you think they do, i.e "won't show unless actually searched for". We don't have any tags that are hidden from everyone by default. It probably wouldn't take too much effort to code a special type of tag that does this, but I don't see a reason for this when we already have status:deleted.

I could have sworn status:deleted didn't work for normal members, but it just showed in incognito tab. Either way my point above still stands.

@iridescent_slime: > Dalamar said: > > Crap has always gotten...

2018-12-06T18:11:05-05:00

Dalamar said:

Crap has always gotten approved, and good things have always been deleted.

Crap getting approved is why we have flags. Good things getting deleted is why we have appeals. You're welcome to do both.

Making bad_quality act as a "deleted" post that can be manually searched is just fine. It's not like it would consume more space since nothing is deleted anyways as it is.

We don't have a "bad_quality" tag. There was a fairly subjective poorly_drawn tag once, but it was removed once it was decided that badly drawn art should just get flagged instead. Questions of quality are better left to trusted approvers than put in the hands of any user who can add or remove tags on a whim.

Also, I don't think tags work the way you think they do, i.e "won't show unless actually searched for". We don't have any tags that are hidden from everyone by default. It probably wouldn't take too much effort to code a special type of tag that does this, but I don't see a reason for this when we already have status:deleted.

@nonamethanks: We already have that as status:deleted. And...

2018-12-06T17:50:14-05:00

We already have that as status:deleted.

And there's already a booru that doesn't delete bad stuff - it's called Gelbooru.

@Dalamar: Crap has always gotten approved, and good...

2018-12-06T17:43:49-05:00

Crap has always gotten approved, and good things have always been deleted. Why so much elitism?

Making bad_quality act as a "deleted" post that can be manually searched is just fine. It's not like it would consume more space since nothing is deleted anyways as it is.

@nonamethanks: > Dalamar said: > > Only if it doesn't require...

2018-12-06T08:57:18-05:00

Dalamar said:

Only if it doesn't require you to be using a computer, oops, it does. Time is work. Nobody is getting paid. Good things get deleted.

It takes a single click to reapprove a flagged picture. It's in fact more work to flag it, or even to upload anything. Why not just have automated uploads as well?

The answer is the same: because otherwise crap will end up on the site's main gallery.

@Dalamar: > nonamethanks said: > > Flagging is already...

2018-12-06T08:52:59-05:00

nonamethanks said:

Flagging is already zero work for approvers.

Only if it doesn't require you to be using a computer, oops, it does. Time is work. Nobody is getting paid. Good things get deleted.

@nonamethanks: > Dalamar said: > > Probably less work for mod...

2018-12-06T03:48:14-05:00

Dalamar said:

Probably less work for mod queue if it were done that way too.

Flagging is already zero work for approvers.

@Dalamar: Deleted images aren't deleted anyways, I think...

2018-12-05T23:25:44-05:00

Deleted images aren't deleted anyways, I think the drama would go away if you'd just treat them as bad_quality (won't show unless actually searched for, not just hidden on the page like blacklist) rather than "deleted".

Current system is terrible.

Probably less work for mod queue if it were done that way too.

@Hat_Vangart: So anyway, I was digging into the forum...

2018-08-27T03:16:13-04:00

So anyway, I was digging into the forum archives and my old messages, and now I understand why everybody thought the lottery approver thing was a bad idea.

Test janitor mail I received by albert:

Janitors on Danbooru are responsible for helping maintain a high level of quality on the site. They approve uploads from other users and help with other moderation efforts. You would be expected at a minimum to approve a few posts a week. If you are interested, please respond to this message.

And the approval message:

DanbooruBot said:

You have been selected as a test janitor. You can now approve pending posts and have access to the moderation interface. You should reacquaint yourself with the howto:upload guide to make sure you understand the site rules.

Over the next several weeks your approvals will be monitored. If the majority of them are not quality uploads you will fail the trial period and lose your approval privileges. You will also receive a negative user record indicating you previously attempted and failed a test janitor trial.

There is a minimum quota of 1 approval a month to indicate that you are being active. Remember, the goal isn't to approve as much as possible. It's to filter out borderline-quality art.

This is ALL I received to get me started.

Here's what should've been added:

Here's an approval role. We can *always* take it away on short notice for any reason, even after the unspecified test period of several weeks is over.

Read the rules. ALL of them. We're not going to point you to any specific rules set for approvers, it's your responsibility to figure everything out yourself on this massively complex system and the incredibly impersonal and unwelcoming forums.

A degree in art and anatomy is also mandatory for any good approver.

If you don't understand anything, ask someone.
Don't know anybody? Don't know *what* to ask?
Your problem. We're not your friend.

@iridescent_slime: I don't think I've used flags all that much,...

2018-08-26T00:07:42-04:00

I don't think I've used flags all that much, only about once or twice a month on average, and mostly for off-topic posts and rule-breaking content. If flagging purely for poor quality went away, I probably wouldn't be affected much. It would still be nice to have the option of doing so, however, for those times when an approver or unrestricted uploader makes a mistake and a bad image makes it into the gallery. I'm not talking about images with borderline proportions or an easily-overlooked anatomy error (this is what error and related tags are for), I'm talking about glaringly obvious defects like visible compression artifacts or generally amateurish/sloppy artwork. Flags are, for now, our best means of dealing with posts that the moderation queue failed to keep out.

Also, eliminating quality flags as a means of preventing drama seems a bit... naive, I guess. All you're doing is sweeping problems under the rug; the bad attitudes and hostility will remain, and the people who would use flags for harassment and personal grudges will just find some other tactic to pursue their agendas, like creating forum topics to call out other users. If some tiny minority of users are abusing the flag system, the proper solution is to deal with them on an individual basis, not to deprive everyone else of a valuable tool.

albert said:

Regarding unlimited uploaders:

Flagging is just a bandaid solution. I don't think a dozen deletions a week is a real deterrent to heavy uploaders. If you perceive one of the unlimited uploaders to be bad, then you should message me and we can discuss whether it makes sense to revoke the privilege. It's usually the case that while their average quality is low, and you can always point to a few egregiously bad examples, their uploads are a net positive for the site. But that's an assumption I'm making. It should be handled on a case by case basis.

Do you really see deletion as a mere deterrent?. To me, it's a matter of good housekeeping. New users and prospective uploaders who browse the gallery and find things like this or this might be left with the impression that the posting of such images is condoned, or at least tolerated. It brings down the overall quality of the gallery. Cleaning up such posts sends the message that "high-quality anime-style art and doujinshi" isn't just a meaningless phrase, and that we're serious about holding all users to the same standard.

Squishy said:

Since flagging is a crucial part of the curation process, I don't think it is a good idea to neuter the flagging system from cleaning up images for being poorly drawn.

Especially not without an replacement process to take its place. Even then, I can't think of a better system than what we have now.

Back before it got purged, you could have blacklisted poorly drawn in order to filter some of the worst examples of art quality your search results. I wouldn't call this a "better" system by any means, mind you. The tag was extremely subjective and there were no checks on its use, unlike the system of flags and appeals. But if you wanted a "replacement process" for flags, it's the closest thing we've ever had.

One of the stated reasons for getting rid of this tag, in fact, was that anything deserving of the tag could just be flagged instead. In the event that poor quality ceases to be a legitimate reason for flagging, maybe this tag will make a resurgence? How ironic.

@Hat_Vangart: Oh well. I'm fairly sure that the users will...

2018-08-25T23:32:05-04:00

Oh well.

I'm fairly sure that the users will be much happier with the new and improved quality of the site. Now that you removed the weakest link (based purely on a minor number of downvotes), all the flagging drama is taken care of. Good job.

@nonamethanks: Unique downvotes are found in the weekly...

2018-08-25T22:58:20-04:00

Unique downvotes are found in the weekly reports.

Single post downvotes are found in the post's json (example, ctrl+F "down_score").

@Hat_Vangart: > nonamethanks said: > > A random selection of...

2018-08-25T22:51:38-04:00

nonamethanks said:

A random selection of posts you approved with several downvotes, from your last 200 posts (fetched via the api):

How many downvotes did each one of those get?
How can I check the unique downvotes of my approvals.
How come nobody told me that downvotes can risk my approval privilege?

I don't know how you could have possibly missed the whole ordeal with people questioning the quality of random approvers' approvals, given that it started as soon as the promotions were handed out (over a year ago),

I don't frequent the forums, so I'm afraid that I completely missed it.

@nonamethanks: > Hat_Vangart said:A random selection of posts...

2018-08-25T22:19:06-04:00

Hat_Vangart said:

A random selection of posts you approved with several downvotes, from your last 200 posts (fetched via the api):

post #3217919
post #3215196
post #3215193
post #3200740 (this one, I'll admit, I've downvoted myself earlier before entering this thread, so you might discard it if you want)
post #3199519
post #3193703
post #3193702
post #3145751
post #3136131

Just because the score is zero or above doesn't mean it's well received - sometimes it's just an unusually large amount of downvotes balanced by an average amount of upvotes. In fact, there's 2.2M posts of score above >2, which shows well enough that positive score means nothing in the face of fetishes.

I don't know how you could have possibly missed the whole ordeal with people questioning the quality of random approvers' approvals, given that it started as soon as the promotions were handed out (over a year ago), and it involved several feedback exchanges and forum topics, though your partial inactivity might have been the cause, but the whole issue was with the fact that you were given the privilege at (almost) random. Not sure if it was a matter of matching favorites with albert, or something similar, but there was no on-site merit associated to it. You can choose to discard what I wrote in previous posts if you want, but it doesn't change the fact that it shows your approvals were very sub-par compared to the rest of the approval team.

That said, I don't want to take part in a name-calling contest, so I'll just leave it at that.

@Hat_Vangart: > albert said: > > I appreciate everyone's...

2018-08-25T21:03:32-04:00

albert said:

I appreciate everyone's sincerity and passion. I am listening and reading and I am keeping an open mind about things.

"sincerity"

How mighty hypocritical of someone who only listens to raw numbers and doesn't even bother to take the actual person into account.

If ANYONE had told me: "Dude, your numbers are not so great. Try to step up your game within a month, or we'll have to take your approval rights.", I would've been cool with it.

I've been a member of this site for over 10 years and I believe I've come to know a tiny little bit about quality. I might've been a bit more lenient with my approvals, but I always try to be fair and base them on how the artist generally fairs, if it's part of a pool, etc. I never think about "Hmm, how many upvotes would I get for this image?"

Case in pont:
approver:Hat_Vangart score:0 age:<1y

Out of 23 images,

8 images are part of a cute DBZ sketch series.
3 are each part of a well liked pool
And the rest were simply too unfortunate to be popular.

As for the THREE negative scores:
approver:Hat_Vangart score:<0 age:<1y

post #2894962: A child of another post with a score of 7.
post #2864882: Looks just like the rest of the artist average-hanzo.
post #2850158: Same case inhabituels_estudios

And finally:

post #3217919: The only post where I fucked up, I agree.

I had no idea you folks are being THIS strict about it.

That said, if all you care about are strict numbers like scientists handling sensitive data, or worse, marketing strategists, then I won't want to be a part of your so called quality control anymore.

Squishy:

Otherwise, users who could have improved if they were given a chance will be unfairly blindsided, like Akaineko and Hat Vangart were. At least in my opinion I think it was unfair.

Thanks, at least someone is being reasonable here.

@Squishy: > nonamethanks said: > > I'm sorry, but this...

2018-08-25T15:33:46-04:00

nonamethanks said:

I'm sorry, but this is not a good analysis of the public data available. First of all, you can do a weighted median downvote percentage, and you'll notice that in nearly all of the reports, excluding the two months I've mentioned earlier, Aikaineko in particular is way above any possible linear fit. In short, he's the most outlier among all approvers in terms of how well received his approvals are by gold+ users. In fact, I'll go as far as to say that the only two months where his downvotes are below the average are also months where the average downvote ratio dropped to half, such as for 2018-06-01, for example, when the average downvote per approval was 1.14, down from the observed ~1.60 to 2.10 of other timeframes.

I'm not disputing they were consistent outliers when it came to unique downvotes on their approvals.
The use of weighted median to demonstrate their deviations wasn't mentioned in your initial analysis, nor do I think it necessary since you made a very clear case already.

What I was pointing out was how the latest report (that received the most in-depth analysis) also coincidentally had these users' worst downvote-to-approval ratios in their career as approvers. Did this record breaking magnitude had a more alarming effect on the immediate decision to demote them?

Or perhaps, the fact that they were constant outliers by a large margin was reason enough. The massive spike in unique downvotes for the latest report just icing on the cake.

I used unique downvotes because

they're unique users, so there cannot be an argument to be made for mass downvoting

people downvote much more rarely than they do upvote

Trying to plot data by average upvote or score makes no sense, especially in a site where most of the highest scored posts are fetish-centered. Do male_focus and scenery posts have lesser quality than tits&ass? The data would suggest that yes, big tits have a much higher value than both those other two categories combined, and in general bikini pictures are much better than anything else on the site. That's not really a valid line of reasoning. An even stronger argument can be made if you consider comic uploads. They're extremely popular outside of danbooru, and linked all over the place, but people who read them typically have no account to vote with. And yet one of the reasons danbooru is so beloved, along with the high quality of the active gallery, is its dedicated translators. There's tons of series that were partially popularized because of danbooru's translations (just look at pop team epic or all the yuri stuff), and yet never make it past a score of 10 despite having thousands of views.

Hence why the only other way to determine how well received posts are is by determining how much pushback there is against certain approvers. And unique downvotes and the ratio of flags per posts is the only public available data that stands the scrutiny, given that the ratio of those two users didn't even improve after ceres was banned for flag abuse.

In particular for the second argument, people upvote nearly anything that tickles their interest, but people only downvote if something is so hideous they cannot go out of their way to ignore it - and the same can be said for flags. That's why downvotes are typically outnumbered by upvotes, and why they make a good indication of when something's gone wrong: they're so rare, that if someone is consistently an outlier in them, and receives a consistent higher number of them than the low average, there's something very wrong going on.

You make a very convincing case here, I can see how the gravitas behind a unique downvote far outweighs that of an upvote.

I am one of those people who read the translated images and comics on Danbooru on a frequent basis (the amount of positive feedback I give out to translators is testament to this) and I've always pondered why comics that have an enthusiastic fanbase often have meh scores. Or why a skillfully designed scenic piece with landscapes and architecture constantly lose out to explicit smut.

I guess upvotes are mostly from the lowest common denominator and aren't worth much?

In short, Albert asked for numerical proof that indeed these approvers were a bad choice, and the numbers show some pretty damning evidence. I understand that this kind of talk is not exactly understandable by people who have little practice in statistical analysis (got knows how much I hate it but you gotta do what you gotta do), but there's no other way to explain it other than "downvotes and reflags are the best available way to determine how good an approver you are, and by all factors your numbers look very bad". This is even more true when their percentages are much worse than people who have approved as much as ten to twenty times more.

My main concern was that I thought that the scope of factors was too focused on what these two approvers did poorly in (downvotes, reflags), while not considering some of aspects they did better at (negative scores, upvote medians, etc).
I understand now why unique downvotes was the point of contention here.

However, I still think the demotion was too abrupt.

As far as I can tell, this was the first time people were demoted based on data from the Reportbooru. Unique downvotes can only be found in those reports and I don't know how many people make it a habit to track their performance there. There needs to be some kind of procedure to warn people to step up their game and pay attention to their numbers from the user reports. Probably a DM or feedback citing the report and where they're doing poorly.

Otherwise, users who could have improved if they were given a chance will be unfairly blindsided, like Akaineko and Hat Vangart were. At least in my opinion I think it was unfair.

I'll stick my neck out and mention that Vangart didn't even get that many flags during his tenure either, so he had little to go on to think he was doing badly.
I think he had good reason to ask why he wasn't warned and for advice on how he could have improved.

Speaking of which, I'll pay more attention to these numbers and see what I can do better as an appprover.

Thanks for your explanation and patience with me.

@nonamethanks: > Squishy said: > > Thoughts on Unique...

2018-08-25T12:11:56-04:00

Squishy said:

Thoughts on Unique Downvotes determining unfit approvers

Are we only going to look at unique downvotes for this?

Akaineko and Hat Vangart getting demoted over one perfomance indicator strikes to me as unfair.
Unique downvotes are only one metric of approval quality in terms of post scores, after all. Shouldn't more than one aspect be considered for before coming to a decision?

@albert You mentioned percentage of negative scores and median scores:

I analyzed how Akaineko and Hat Vangart did with regards to these two metrics.

Negatively scored approvals:

The percentage of negatively scored posts is below 2% for Akaineko and below 4% for Hat Vangart

For Akaineko

Out of 3227 approvals within the last year:
53 posts were at score 0 approver:Akaineko score:0 age:<1y - that's only 1.6% posts
2 posts were negatively scored approver:Akaineko score:<0 age:<1y - that's about 0.06%

For Hat Vangart

Out of 671 approvals within the last year:
24 posts were at score 0 approver:Hat_Vangart score:0 age:<1y - that's 3.57% posts
6 posts were negatively scored approver:Hat_Vangart score:<0 age:<1y - only 0.89%

So in terms of negative scores, neither user is coming close to 10% negative scores across their approvals within the last year.

Median scores:

Both users do not deviate more than 2 points from the weekly average.
Nor they never come close the lowest median score of the week.

Median Score comparisons by week

Week Average Median Score
across all users Max Median Score
across all users Min Median Score
across all users Median Score
Akaineko Median Score
Hat Vangart

2018-04-01 7.45 21 2 7 N/A

2018-04-08 7.25 17 2 8 5

2018-04-15 7.23 16 2 8 N/A

2018-04-22 7.53 17 2 6 N/A

2018-04-29 7.18 17 1 7 N/A

2018-05-06 7.33 17 1 8 N/A

2018-05-13 6.88 15 1 7 N/A

2018-05-20 6.43 15 1 8 7

2018-05-27 6.14 15 1 6 6

2018-06-03 6.88 15 1 9 7

2018-06-10 7.25 22 1 9 7

2018-06-17 7.17 17 1 6 N/A

2018-06-24 6.85 18 1 8 N/A

2018-07-01 6.72 17 1 7 N/A

2018-07-08 7.11 16 2 7 N/A

2018-07-15 7.35 23 2 7 9

2018-07-22 7.64 26 1 5 7

2018-07-29 7.34 26 1 4 5

2018-08-05 7.44 24 1 5 6

2018-08-12 7.54 25 1 7 5

2018-08-19 6.88 17 1 7 7

As far as I can tell, both users stayed closed to the average baseline. They are certainly never outliers.

Unique downvotes:

I think the timing was rather unfortunate for both users.
They had a particularly bad week. In fact this week was their highest downvote percentage ever since April

Unique downvote comparisons by week

Week Average downvote percentage
across all users Max downvote percentage
across all users Min downvote percentage
across all users Downvote percentage
Akaineko Downvote percentage
Hat Vangart

2018-04-01 5.16% 30.77% 0.00% 13.79% N/A

2018-04-08 4.59% 20.00% 0.00% 9.28% 8.00%

2018-04-15 5.30% 20.00% 0.00% 12.77% N/A

2018-04-22 5.15% 22.22% 0.00% 22.22% N/A

2018-04-29 5.74% 20.00% 0.89% 9.52% N/A

2018-05-06 5.60% 21.62% 0.00% 9.52% N/A

2018-05-13 5.58% 27.59% 0.00% 9.38% N/A

2018-05-20 5.55% 60.00% 0.00% 8.11% 9.09%

2018-05-27 3.68% 15.38% 0.00% 15.38% 6.94%

2018-06-03 4.49% 23.08% 0.00% 16.22% 8.57%

2018-06-10 4.04% 17.65% 0.00% 17.65% 9.84%

2018-06-17 4.27% 15.38% 0.00% 14.63% N/A

2018-06-24 3.64% 14.00% 0.00% 2.94% N/A

2018-07-01 3.25% 12.50% 0.00% 3.57% N/A

2018-07-08 3.96% 15.25% 0.00% 2.08% N/A

2018-07-15 4.32% 27.27% 0.00% 3.70% 6.45%

2018-07-22 4.46% 28.57% 0.00% 7.59% 5.56%

2018-07-29 5.05% 21.43% 0.00% 11.63% 8.57%

2018-08-05 5.73% 21.74% 0.00% 21.74% 7.32%

2018-08-12 6.42% 25.00% 0.00% 17.86% 15.58%

2018-08-19 5.83% 33.33% 0.00% 33.33% 16.22%

NNT is correct that these two users are generally higher than the weekly average, and I think he's got it right by keeping the comparisons with context of approvers with similar approval numbers.

As NNT pointed out, these users have relatively few approvals and generally take breaks (Weeks for Akaineko, Months for Hat Vangart).
Given that reports operate on the last 30 days, if an approval gets downvoted, it takes four reports for it to fall off. This has particularly high impact on low volume approvers.

I decided to look closer into what kind of images they've been approving:

Akaineko's spike on the start of August occurs during these uploads: approver:Akaineko date:>2018-07-22...<2018-07-29

Since Akaineko didn't approve anything in the following week approver:Akaineko date:>2018-07-29...<2018-08-05, the 5 unique downvotes he got is going to keep hurting.

While the second spike is still going, somewhere in this batch approver:Akaineko date:>2018-08-12...<2018-08-19 he racked up another 3 unique downvotes, which breaks his previous percentage record by almost double. He's at 8 downvotes for 24 approvals in the last 30 days.

Hat Vangart had a pretty good start on July 15, being mostly within 2% of the average. But somewhere in the previous week approver:Hat_Vangart date:>2018-08-05...<2018-08-12 he approved a certain Miku post that earned him at least 5 or more downvotes. This sets a new high record by a fairly large margin. He's in a bad spot on August 12th and onward.

Right now, both of these users are having highest the downvote / approval ratios of their careers by almost a double amount.
Then this thread occurs and it just so happens that downvotes are analyzed.

I think it'd be fair to take into account additional performance indicators as well, such as deletion ratios, scores, etc before considering to write them off.
And since nobody has talked to these users, given them feedback or dmailed them (to my knowledge), they never got the chance to step up their game. Their demotion is quite sudden.

Finally, last thing to consider about using votes as a measure:
I've been told many times that scores and favourites are not a good indicator of what images belong on Danbooru.
The amount of highly scored and favourite imagesd that are deleted speaks to that score:>10 status:deleted
The opposite is true as well score:<0 status:active

So why are post votes now being used to screen approvers here?
Even unrestricted uploaders aren't subject to the amount of votes their uploads garner, nor is it a requirement to be promoted unrestricted and approver status.

Unless, that is changing too?

I'm sorry, but this is not a good analysis of the public data available. First of all, you can do a weighted median downvote percentage, and you'll notice that in nearly all of the reports, excluding the two months I've mentioned earlier, Aikaineko in particular is way above any possible linear fit. In short, he's the most outlier among all approvers in terms of how well received his approvals are by gold+ users. In fact, I'll go as far as to say that the only two months where his downvotes are below the average are also months where the average downvote ratio dropped to half, such as for 2018-06-01, for example, when the average downvote per approval was 1.14, down from the observed ~1.60 to 2.10 of other timeframes.

I used unique downvotes because

they're unique users, so there cannot be an argument to be made for mass downvoting
people downvote much more rarely than they do upvote

Trying to plot data by average upvote or score makes no sense, especially in a site where most of the highest scored posts are fetish-centered. Do male_focus and scenery posts have lesser quality than tits&ass? The data would suggest that yes, big tits have a much higher value than both those other two categories combined, and in general bikini pictures are much better than anything else on the site. That's not really a valid line of reasoning. An even stronger argument can be made if you consider comic uploads. They're extremely popular outside of danbooru, and linked all over the place, but people who read them typically have no account to vote with. And yet one of the reasons danbooru is so beloved, along with the high quality of the active gallery, is its dedicated translators. There's tons of series that were partially popularized because of danbooru's translations (just look at pop team epic or all the yuri stuff), and yet never make it past a score of 10 despite having thousands of views.

Hence why the only other way to determine how well received posts are is by determining how much pushback there is against certain approvers. And unique downvotes and the ratio of flags per posts is the only public available data that stands the scrutiny, given that the ratio of those two users didn't even improve after ceres was banned for flag abuse.

In particular for the second argument, people upvote nearly anything that tickles their interest, but people only downvote if something is so hideous they cannot go out of their way to ignore it - and the same can be said for flags. That's why downvotes are typically outnumbered by upvotes, and why they make a good indication of when something's gone wrong: they're so rare, that if someone is consistently an outlier in them, and receives a consistent higher number of them than the low average, there's something very wrong going on.

I think it'd be fair to take into account additional performance indicators as well, such as deletion ratios, scores, etc before considering to write them off.

This was already brought up, Akaineko has an 89% percent deletion ratio for reflagged approvals.

Hat Vangart is in a slightly better position, but only because he did not approve enough in the previous months to end up in the ranking, and because he has much less approvals. In the months were he did approve, he inevitably ended up being an outlier. Notice how he immediately jumped as fifth as soon as he surfaced in the last batch of reports where his name is featured, and stayed there well above the limit while everyone else who was not a lottery approver or someone with four times less approvals cycled. There's no argument to be made for singular approvals ruining the score because this is a six-week timeframe.

Even if you consider some random months, he always stays at the top and has several times more unfitness than any other manually selected approver.

In fact his promotion was put into question the very first month he was promoted (along with akaineko) (forum #131033, though this is more about akaineko than hat vangart).

If you don't trust me you can plot the data yourself, representing each approver as a function in a 2D graph and seeing that, while hand-picked approvers have ups and downs with a mostly average score, random approvers end up staying distinctly separated from the rest. It could be further improved by doing a weighted median in depth, but that would take quite some time.

In short, Albert asked for numerical proof that indeed these approvers were a bad choice, and the numbers show some pretty damning evidence. I understand that this kind of talk is not exactly understandable by people who have little practice in statistical analysis (got knows how much I hate it but you gotta do what you gotta do), but there's no other way to explain it other than "downvotes and reflags are the best available way to determine how good an approver you are, and by all factors your numbers look very bad". This is even more true when their percentages are much worse than people who have approved as much as ten to twenty times more.

@Astolfo: > nonamethanks said: > > The rationale of...

2018-08-25T11:38:03-04:00

nonamethanks said:

The rationale of flagging those posts is to keep danbooru's active gallery curated. And, I want to add, to make sure new users know what's acceptable or not. I've seen plenty of newcomers trying to 1up horrible but active old posts with pixiv versions they found, only to see theirs deleted and end up complaining in the forums about it. I don't think having posts that the majority of approvers consider bad by today's standards be deleted (which is what flaggers so far have been doing) is bad. Deleted posts aren't expunged, so we're not losing anything.

To add on to that, I'd also mention the relatively frequent use of "there's worse things on the site" as a justification to upload and/or defend barely passable posts.

@Squishy: > Flags and appeals are a perennial source of...

2018-08-25T11:26:03-04:00

Flags and appeals are a perennial source of drama on Danbooru.

The way I see it, the drama that surrounds flags and appeals is a problem stemming from the users.
The tools are not at fault if people take things personally to the point of openly antagonizing users across the site.

I think a large part of it is disputes that are subjective in nature with regards to art quality.

Poorly drawn is not a binary decision. There's a gradient. My opinion is that if you think an upload is ugly, you should either ignore it or down vote it.

This was my stance as a normal user since I signed up for the site: Enjoy what catches my eye and move on if I don't see anything I fancy.
However, as kittey said, I mainly use Danbooru because I do not have to scroll through several pages in other sites to find worthwhile images (as well as excellent tagging and source consolidation). I assume those are the primary reasons why the userbase browses Danbooru and not elsewhere.

After spending a few months as an approver and being given the responsibility of maintaining the gallery, I've realized that the multiple entry/exit manual curation is responsible for Danbooru's high quality galleries.

The approval queue, flagging system and appeal system are all important parts of this curation. Due to the subjective nature of art quality, I think it is a good setup by having several checks and counterchecks to strike that delicate balance.

I suggest we remove poor quality as a valid reason for flagging.

Since flagging is a crucial part of the curation process, I don't think it is a good idea to neuter the flagging system from cleaning up images for being poorly drawn.

Especially not without an replacement process to take its place. Even then, I can't think of a better system than what we have now.

With regards to the drama, changing what flag reasons are valid will not solve it.
The actual problem is when users who go out of their way to attack other users over arguments about art quality.

This is a problem that can only be handled by moderators.
And as far as I know, the people who do this have been warned and banned for going beyond disagreement to outright hostility.

------------------

Thoughts on Unique Downvotes determining unfit approvers

Are we only going to look at unique downvotes for this?

Akaineko and Hat Vangart getting demoted over one perfomance indicator strikes to me as unfair.
Unique downvotes are only one metric of approval quality in terms of post scores, after all. Shouldn't more than one aspect be considered for before coming to a decision?

@albert You mentioned percentage of negative scores and median scores:

But pointing to one or two bad uploads isn't sufficient. Everyone approves bad stuff once in awhile, us included. You need to prove that a large fraction of their approvals are bad. I'd say at least 10%. Stuff like a high percentage of negative scores is good evidence, or even a low median score.

I analyzed how Akaineko and Hat Vangart did with regards to these two metrics.

Negatively scored approvals:

The percentage of negatively scored posts is below 2% for Akaineko and below 4% for Hat Vangart

For Akaineko

Out of 3227 approvals within the last year:
53 posts were at score 0 approver:Akaineko score:0 age:<1y - that's only 1.6% posts
2 posts were negatively scored approver:Akaineko score:<0 age:<1y - that's about 0.06%

For Hat Vangart

Out of 671 approvals within the last year:
24 posts were at score 0 approver:Hat_Vangart score:0 age:<1y - that's 3.57% posts
6 posts were negatively scored approver:Hat_Vangart score:<0 age:<1y - only 0.89%

So in terms of negative scores, neither user is coming close to 10% negative scores across their approvals within the last year.

Median scores:

Both users do not deviate more than 2 points from the weekly average.
Nor they never come close the lowest median score of the week.

Median Score comparisons by week

Week	Average Median Score across all users	Max Median Score across all users	Min Median Score across all users	Median Score Akaineko	Median Score Hat Vangart
2018-04-01	7.45	21	2	7	N/A
2018-04-08	7.25	17	2	8	5
2018-04-15	7.23	16	2	8	N/A
2018-04-22	7.53	17	2	6	N/A
2018-04-29	7.18	17	1	7	N/A
2018-05-06	7.33	17	1	8	N/A
2018-05-13	6.88	15	1	7	N/A
2018-05-20	6.43	15	1	8	7
2018-05-27	6.14	15	1	6	6
2018-06-03	6.88	15	1	9	7
2018-06-10	7.25	22	1	9	7
2018-06-17	7.17	17	1	6	N/A
2018-06-24	6.85	18	1	8	N/A
2018-07-01	6.72	17	1	7	N/A
2018-07-08	7.11	16	2	7	N/A
2018-07-15	7.35	23	2	7	9
2018-07-22	7.64	26	1	5	7
2018-07-29	7.34	26	1	4	5
2018-08-05	7.44	24	1	5	6
2018-08-12	7.54	25	1	7	5
2018-08-19	6.88	17	1	7	7

As far as I can tell, both users stayed closed to the average baseline. They are certainly never outliers.

Unique downvotes:

I think the timing was rather unfortunate for both users.
They had a particularly bad week. In fact this week was their highest downvote percentage ever since April

Unique downvote comparisons by week

Week	Average downvote percentage across all users	Max downvote percentage across all users	Min downvote percentage across all users	Downvote percentage Akaineko	Downvote percentage Hat Vangart
2018-04-01	5.16%	30.77%	0.00%	13.79%	N/A
2018-04-08	4.59%	20.00%	0.00%	9.28%	8.00%
2018-04-15	5.30%	20.00%	0.00%	12.77%	N/A
2018-04-22	5.15%	22.22%	0.00%	22.22%	N/A
2018-04-29	5.74%	20.00%	0.89%	9.52%	N/A
2018-05-06	5.60%	21.62%	0.00%	9.52%	N/A
2018-05-13	5.58%	27.59%	0.00%	9.38%	N/A
2018-05-20	5.55%	60.00%	0.00%	8.11%	9.09%
2018-05-27	3.68%	15.38%	0.00%	15.38%	6.94%
2018-06-03	4.49%	23.08%	0.00%	16.22%	8.57%
2018-06-10	4.04%	17.65%	0.00%	17.65%	9.84%
2018-06-17	4.27%	15.38%	0.00%	14.63%	N/A
2018-06-24	3.64%	14.00%	0.00%	2.94%	N/A
2018-07-01	3.25%	12.50%	0.00%	3.57%	N/A
2018-07-08	3.96%	15.25%	0.00%	2.08%	N/A
2018-07-15	4.32%	27.27%	0.00%	3.70%	6.45%
2018-07-22	4.46%	28.57%	0.00%	7.59%	5.56%
2018-07-29	5.05%	21.43%	0.00%	11.63%	8.57%
2018-08-05	5.73%	21.74%	0.00%	21.74%	7.32%
2018-08-12	6.42%	25.00%	0.00%	17.86%	15.58%
2018-08-19	5.83%	33.33%	0.00%	33.33%	16.22%

NNT is correct that these two users are generally higher than the weekly average, and I think he's got it right by keeping the comparisons with context of approvers with similar approval numbers.

As NNT pointed out, these users have relatively few approvals and generally take breaks (Weeks for Akaineko, Months for Hat Vangart).
Given that reports operate on the last 30 days, if an approval gets downvoted, it takes four reports for it to fall off. This has particularly high impact on low volume approvers.

I decided to look closer into what kind of images they've been approving:

Akaineko's spike on the start of August occurs during these uploads: approver:Akaineko date:>2018-07-22...<2018-07-29
Since Akaineko didn't approve anything in the following week approver:Akaineko date:>2018-07-29...<2018-08-05, the 5 unique downvotes he got is going to keep hurting.
While the second spike is still going, somewhere in this batch approver:Akaineko date:>2018-08-12...<2018-08-19 he racked up another 3 unique downvotes, which breaks his previous percentage record by almost double. He's at 8 downvotes for 24 approvals in the last 30 days.

Hat Vangart had a pretty good start on July 15, being mostly within 2% of the average. But somewhere in the previous week approver:Hat_Vangart date:>2018-08-05...<2018-08-12 he approved a certain Miku post that earned him at least 5 or more downvotes. This sets a new high record by a fairly large margin. He's in a bad spot on August 12th and onward.

Right now, both of these users are having highest the downvote / approval ratios of their careers by almost a double amount.
Then this thread occurs and it just so happens that downvotes are analyzed.

I think it'd be fair to take into account additional performance indicators as well, such as deletion ratios, scores, etc before considering to write them off.
And since nobody has talked to these users, given them feedback or dmailed them (to my knowledge), they never got the chance to step up their game. Their demotion is quite sudden.

Finally, last thing to consider about using votes as a measure:
I've been told many times that scores and favourites are not a good indicator of what images belong on Danbooru.
The amount of highly scored and favourite imagesd that are deleted speaks to that score:>10 status:deleted
The opposite is true as well score:<0 status:active

So why are post votes now being used to screen approvers here?
Even unrestricted uploaders aren't subject to the amount of votes their uploads garner, nor is it a requirement to be promoted unrestricted and approver status.

Unless, that is changing too?