Rules
Terms of Use

Topic Options
#1144751 - Wed Sep 14 2016 05:46 PM Quiz Rankings
apathy100 Offline
Mainstay

Registered: Sun Jun 26 2005
Posts: 860
Loc: Ontario Canada
I am just curious to know how our quizzes are ranked. I have a quiz that has a Good and Excellent percentage of 86% for example, but is ranked lower than some of my quizzes with a percentage of 80%. As such, the quiz with a higher percentage has a lower ranking as well. I am just curious how it is determined whether one quiz is ranked higher than the other based on the percentages at hand?
_________________________
To deny people their human rights is to challenge their very humanity.

--Nelson Mandela--

Top
#1144753 - Wed Sep 14 2016 05:49 PM Re: Quiz Rankings
agony Online   content

Administrator

Registered: Sat Mar 29 2003
Posts: 16595
Loc: Western Canada
You don't see the other rankings, the Average and Poor. A quiz which really engages players strongly might get more Excellents but also more Poors, and that would account for it.

Top
#1145069 - Sun Sep 18 2016 07:58 AM Re: Quiz Rankings
lonely-lady Offline
Forum Champion

Registered: Thu Jun 19 2014
Posts: 6795
Loc: England UK
Will it ever be possible to see all rating % at some time? I think this may be a way to stop malicious ratings.
_________________________
I dreamed of swimming in an ocean of orange fizzy drink. It was a Fantasea

Top
#1145071 - Sun Sep 18 2016 08:38 AM Re: Quiz Rankings
kyleisalive Online   FT-cool
Forum Champion

Registered: Mon Mar 07 2005
Posts: 8760
Loc: Toronto, Canada, eh!
Approaching poorly rated quizzes thinking that they're cases of malicious intent is a poor way of thinking in my opinion. There's no way of knowing people's intents while rating. There are more viable reasons to rate poorly, and I would recommend taking the system as constructive criticism instead of as an attack. Alternatively I would recommend not reading the ratings at all. Failing that, I would recommend not writing quizzes as you would be creating your own stress. :s
_________________________
Senior FT Editor (Video Games, Television, and Entertainment)
Chat Board Moderator (Author's Lounge)
Amazing Trivia Race Taskmaster/Commission Hander-Outer/TRICster

Top
#1145072 - Sun Sep 18 2016 08:44 AM Re: Quiz Rankings
nautilator Offline
Mainstay

Registered: Mon Jan 09 2012
Posts: 719
Loc: Pennsylvania USA
A quiz with a lot of "good" ratings and few excellents will get a low ranking. You can probably see this if you've written enough. The excellents weight pretty heavily on the ranking, as do the poors. Most people rate things as good.

Top
#1145125 - Mon Sep 19 2016 04:53 AM Re: Quiz Rankings
lonely-lady Offline
Forum Champion

Registered: Thu Jun 19 2014
Posts: 6795
Loc: England UK
Denying that it (malicious rating) happens will not make it go away. Doing nothing about it allows it to continue.
_________________________
I dreamed of swimming in an ocean of orange fizzy drink. It was a Fantasea

Top
#1145130 - Mon Sep 19 2016 05:28 AM Re: Quiz Rankings
gtho4 Offline
Administrator

Registered: Sun Dec 26 1999
Posts: 54484
Loc: Sydney
oz downunder
There's a post by Terry, somewhere in here, advising us that he modified the routine for quiz rankings as follows:
the system ignores all quiz ratings by a member who rates (almost) every quiz played as #1.

[ Terry's post re this was four or five years ago ]

Top
#1145133 - Mon Sep 19 2016 06:31 AM Re: Quiz Rankings
namrewsna Offline
Mainstay

Registered: Wed Jul 16 2014
Posts: 777
Loc: Utah USA
Heh, so someone got banished from the ratings box for being too nice?

Not saying I have a problem with it, but it seems a bit funny next to some of the typical types of behavior which draw such exclusion.

Top
#1145136 - Mon Sep 19 2016 06:50 AM Re: Quiz Rankings
trident Offline
Multiloquent

Registered: Sun Feb 20 2005
Posts: 3332
Loc: Wisconsin USA
Originally Posted By: namrewsna
Heh, so someone got banished from the ratings box for being too nice?

Not saying I have a problem with it, but it seems a bit funny next to some of the typical types of behavior which draw such exclusion.


The exact opposite actually. Gtho is saying that nearly all quizzes were ranked Very Poor.
_________________________
Perception is everything.

Editor: World, History, and General

Top
#1145137 - Mon Sep 19 2016 07:12 AM Re: Quiz Rankings
kyleisalive Online   FT-cool
Forum Champion

Registered: Mon Mar 07 2005
Posts: 8760
Loc: Toronto, Canada, eh!
Quote:
Denying that it (malicious rating) happens will not make it go away. Doing nothing about it allows it to continue.


Please see my previous post. If you're just assuming people are going around spending money on Gold Membership just to maliciously rate your quizzes or anyone else's quizzes poorly out of spite without any real basis for your presumption, then I would recommend either not focusing on the ratings of your quizzes or not writing. At the point that you worry about it that much, it's just not worth it. And I say this as someone who's here to encourage people to pursue quiz-writing. I'm not here to draw the line for anyone, but if it honestly bothers you that much...I mean, if it were me, I'd just not do it. Life's too short.


Showing people their 'Poor'/'Very Poor' ratings opens up an entirely different can of worms.

My lowest rated quiz is 130808 of 128000. Should I be curious? Maybe, but I would only be curious as to how I can improve down the line. Should I be worried? Not at all. It's just a number. Does it reflect my writing skill? No; it reflects the biases of a small sample of players who took the quiz and decided to rate-- not the people who didn't rate, or the ones who couldn't rate.
_________________________
Senior FT Editor (Video Games, Television, and Entertainment)
Chat Board Moderator (Author's Lounge)
Amazing Trivia Race Taskmaster/Commission Hander-Outer/TRICster

Top
#1145143 - Mon Sep 19 2016 07:58 AM Re: Quiz Rankings
agony Online   content

Administrator

Registered: Sat Mar 29 2003
Posts: 16595
Loc: Western Canada
One of my jobs here (and one of Kyle's, too, for that matter) is to tell people things they don't want to hear, such as "No, I will not correct your spelling for you, you need to do that yourself".

If malicious rating was a real problem, no editor's quiz would have sunglasses - we are the most likely people on the site to have enemies. It's entirely possible that I have been the target of some malicious rating, but so what? It doesn't amount to much.

I believe there are algorithms in place to spot very suspicious patterns, so wholesale malice can't get much of a toehold. Beyond that, I advise a certain amount of detachment to ratings - high ratings are nice, but nothing really valuable depends on them.

Top
#1145149 - Mon Sep 19 2016 08:26 AM Re: Quiz Rankings
Shadowmyst2004 Offline
Enthusiast

Registered: Sat Jan 02 2016
Posts: 375
Loc: Ohio USA
Rankings and ratings are the least important part of the process, and somewhat random. A few quizzes I've thought were among my best ended up lowly rated, and one in particular that was done in a matter of an hour or so while stuck watching a stupid kids show with my nephew ended up being in the top 100 sitewide for a few weeks. (It's since fallen back but is still in the top 8000)

Just goes to show that it's random.


I've also noticed that the more plays a quiz gets, the more likely it is to level out with "average" rankings. No big deal.

I only remember this because it was my first quiz to get a pair of sunnies, but one I did on diseases of the brain, rated really good. Then it was used in a badgelet grouping, and the rating had dropped it back several thousand spots. That correlates to being played a lot more. The first few months up it had only a couple hundred plays, as that number jumped, the ratings fell down some. It happens.


Unless you are trying the Pangalactic challenge, the ratings mean very little. Enjoy the process, skip the worrying.

Top
#1145153 - Mon Sep 19 2016 09:18 AM Re: Quiz Rankings
dg_dave Offline
Champion Poster

Registered: Sun Oct 05 2003
Posts: 24575
Loc: near Stafford, Virginia USA
Originally Posted By: gtho4
There's a post by Terry, somewhere in here, advising us that he modified the routine for quiz rankings as follows:
the system ignores all quiz ratings by a member who rates (almost) every quiz played as #1.

[ Terry's post re this was four or five years ago ]


Quiz Rankings thread

This post was nearly ten years ago!


Edited by dg_dave (Mon Sep 19 2016 09:19 AM)
_________________________
The way to get things done is NOT to mind who gets the credit for doing them. --Benjamin Jowett
No one can make you feel inferior without your consent. --Eleanor Roosevelt
The day we lose our will to fight is the day we lose our freedom.

Top
#1145162 - Mon Sep 19 2016 11:16 AM Re: Quiz Rankings
namrewsna Offline
Mainstay

Registered: Wed Jul 16 2014
Posts: 777
Loc: Utah USA
Originally Posted By: trident
The exact opposite actually. Gtho is saying that nearly all quizzes were ranked Very Poor.


I see now. The numbers are no longer shown with the ratings for quizzes (or at least not on the mobile mode display) and for some reason instead of assuming a scale of 1 to 5, I thought of rating a quiz as #1 to mean the highest, ..as in "we're #1..woohoo!" After some competitive win. wink

Top
#1145165 - Mon Sep 19 2016 11:26 AM Re: Quiz Rankings
lonely-lady Offline
Forum Champion

Registered: Thu Jun 19 2014
Posts: 6795
Loc: England UK
Originally Posted By: dg_dave

Quiz Rankings thread

This post was nearly ten years ago!


That thread is fascinating, thank you for finding and posting it.
_________________________
I dreamed of swimming in an ocean of orange fizzy drink. It was a Fantasea

Top
#1145166 - Mon Sep 19 2016 11:33 AM Re: Quiz Rankings
andymuenz Offline
Enthusiast

Registered: Sun Sep 14 2014
Posts: 356
Loc: Pennsylvania USA
Originally Posted By: agony
One of my jobs here (and one of Kyle's, too, for that matter) is to tell people things they don't want to hear, such as "No, I will not correct your spelling for you, you need to do that yourself".

If malicious rating was a real problem, no editor's quiz would have sunglasses - we are the most likely people on the site to have enemies. It's entirely possible that I have been the target of some malicious rating, but so what? It doesn't amount to much.



Actually, Agony, you are one of my favorite people on the site. You helped me quite a bit with my first quiz a little over two years ago and it is still my highest rated quiz (and my next one will bring me to 30). So I appreciate the things that you and all of the editors do to help us submit better quizzes.

Top
#1145168 - Mon Sep 19 2016 02:27 PM Re: Quiz Rankings
kyleisalive Online   FT-cool
Forum Champion

Registered: Mon Mar 07 2005
Posts: 8760
Loc: Toronto, Canada, eh!
Quote:
Beyond that, I advise a certain amount of detachment to ratings - high ratings are nice, but nothing really valuable depends on them.


Exactly this.

Aim for high ratings; don't sweat the low ones.
_________________________
Senior FT Editor (Video Games, Television, and Entertainment)
Chat Board Moderator (Author's Lounge)
Amazing Trivia Race Taskmaster/Commission Hander-Outer/TRICster

Top
#1145182 - Mon Sep 19 2016 08:20 PM Re: Quiz Rankings
dg_dave Offline
Champion Poster

Registered: Sun Oct 05 2003
Posts: 24575
Loc: near Stafford, Virginia USA
Originally Posted By: lonely-lady
Originally Posted By: dg_dave

Quiz Rankings thread

This post was nearly ten years ago!


That thread is fascinating, thank you for finding and posting it.


I enjoyed going back and rereading that thread. I hadn't realized it was that long ago though!
_________________________
The way to get things done is NOT to mind who gets the credit for doing them. --Benjamin Jowett
No one can make you feel inferior without your consent. --Eleanor Roosevelt
The day we lose our will to fight is the day we lose our freedom.

Top
#1145184 - Mon Sep 19 2016 09:02 PM Re: Quiz Rankings
shuehorn Offline
Multiloquent

Registered: Tue Jul 04 2006
Posts: 3613
Loc: Lawrenceville Georgia�USA�...
I too enjoyed reading that thread. Fascinating, even though it is from 2003.
_________________________
Sue (shuehorn)

Top
#1145185 - Mon Sep 19 2016 09:23 PM Re: Quiz Rankings
gtho4 Offline
Administrator

Registered: Sun Dec 26 1999
Posts: 54484
Loc: Sydney
oz downunder

Originally Posted By: dg_dave
Quiz Rankings thread

This post was nearly ten years ago!

That's great, just great .. the brain cells are still disappearing as the years go by; and faster than I thought wink
This is the post I remember (scroll back a day and 20 posts):

Originally Posted By: Terry - #330858 - Sun Nov 12 2006 01:53 PM
I'll have to play with different input formats for ratings, because it's possible some people are accidentally rating quizzes incorrectly.

In checking for bugs in the system, I was astonished to find that some of our most active quiz takers rate almost EVERY quiz they take a 1 or a 2. In other words, they think that most quizzes on the site are crap yet they play thousands of them.

Take this active player for example:

+----------+--------+
| count(*) | rating |
+----------+--------+
| 33744 | 1 |
| 23427 | 2 |
| 6309 | 3 |
| 4384 | 4 |
| 188 | 5 |
+----------+--------+

He has rated 57000 quizzes a 1 or 2 and 4000 a 4 or 5.

Absolutely remarkable. I think I'm going to have to create an algorithm that will simply ignore the ratings from such players. If your average rating isn't between 2 and 5, you get ignored.

Terry

http://www.funtrivia.com/forums/ubbthreads.php/topics/330858#Post330858

Top
#1145187 - Mon Sep 19 2016 10:08 PM Re: Quiz Rankings
dg_dave Offline
Champion Poster

Registered: Sun Oct 05 2003
Posts: 24575
Loc: near Stafford, Virginia USA
Originally Posted By: gtho4
That's great, just great .. the brain cells are still disappearing as the years go by; and faster than I thought wink


I'm ten years older now than before too, George...lol. We're not getting any younger. wink

Makes me wonder how many quizzes said player has played now! With over 130,000 on the site, I would safely say he has played over 125,000!

I still feel the way today that I did on the final post of that thread...if not moreso now than then...and I wasn't even 30 years old then. http://www.funtrivia.com/forums/ubbthreads.php/topics/330900#Post330900.


Edited by dg_dave (Mon Sep 19 2016 10:12 PM)
_________________________
The way to get things done is NOT to mind who gets the credit for doing them. --Benjamin Jowett
No one can make you feel inferior without your consent. --Eleanor Roosevelt
The day we lose our will to fight is the day we lose our freedom.

Top
#1145191 - Tue Sep 20 2016 01:34 AM Re: Quiz Rankings
agony Online   content

Administrator

Registered: Sat Mar 29 2003
Posts: 16595
Loc: Western Canada
Ah, thank you, andymuenz!

We do get a lot of love from many quizmakers, but by no means all....

Top

Moderator:  agony