Rules: Read Me!
Admin: sue943
Legal / Conditions of Use

Page 1 of 4 1 2 3 4 >
Topic Options
#546398 - Sat Aug 21 2010 04:11 PM Quiz rankings
snediger Offline
Participant

Registered: Fri Jan 15 2010
Posts: 8
Loc: Amarillo Texas USA            
I don't understand the ranking of the quizzes.
For instance, I have authored one quiz that is ranked #8688...
with 39 ratings and a 79% "Ex. & Good."
However, another quiz I have authored is ranked #11,347...
yet, this was rated 89 times and has a 87% "Ex. & good."
Shouldn't the quiz that is rated HIGHER percentage-wise have the higher number rating?
Granted, we are talking about two quizzes authored this year within months of one another.
Please explain.

Top
#546399 - Sat Aug 21 2010 04:21 PM Re: Quiz rankings [Re: snediger]
jonnowales Offline
Prolific

Registered: Mon Oct 30 2006
Posts: 1518
Loc: Swansea,UK
You should have three percentages for each quiz:

Excellent & Good - x+g%

Excellent - x%
Good - g%

Could you give us a more specific breakdown, as it is likely that the statistics you omitted provide the answer. If it doesn't then it could be that your unseen percentages (average, poor and very poor) were a little less favourable for your lower ranked quiz than for the higher.

Top
#546400 - Sat Aug 21 2010 04:26 PM Re: Quiz rankings [Re: jonnowales]
snediger Offline
Participant

Registered: Fri Jan 15 2010
Posts: 8
Loc: Amarillo Texas USA            
Thank you, jonnowales. I never really thought about it that way.

Top
#546401 - Sat Aug 21 2010 04:43 PM Re: Quiz rankings [Re: snediger]
jonnowales Offline
Prolific

Registered: Mon Oct 30 2006
Posts: 1518
Loc: Swansea,UK
I think it is something that goes through most minds at some point. The rankings can seem wildly erratic even though there is a method to the madness! laugh

Top
#797964 - Tue May 29 2012 02:21 PM Quiz Rankings Question
triviaking162 Offline
Forum Adept

Registered: Sat Mar 10 2012
Posts: 138
Loc: New York USA
I'm just curious, but what are the criteria for a quiz getting sunglasses? I just want to know. One of my quizzes has 152 ratings (80% positive) and 753 plays and has no sunglasses (it's ranked at 36,968). What are the requirements to get sunglasses and get a high ranked quiz?
_________________________
TK

"That which does not kill us makes us stronger." - Friedrich Nietzsche

Top
#797971 - Tue May 29 2012 02:38 PM Re: Quiz Rankings Question [Re: triviaking162]
AdamM7 Offline
Mainstay

Registered: Sat Feb 18 2012
Posts: 528
Loc: Cheshire England UK           
The requirements to get a quiz with sunglasses is having it ranked in (roughly) the top 20% of quizzes. Terry updates the number every so often - I think it is currently at 22,000.

To get a high ranked quiz, you must not only get a lot of good ratings (80% good/excellent should be enough), but very few poor/very poor ratings. I don't think anyone apart from Terry knows the exact formula used to calculate it.

Since the author can only see the %age of goods and excellents, they can get mislead and think that the quiz should be ranked higher, but if there are a lot of bad ratings, the quiz might get ranked lower.

I would think that 80% good/excellent should be higher up than 37K (depending on the exact %age of good and excellent) but if most of the other 20% of the ratings were "very poor" then it would make sense.
_________________________
"No, please don't eat me. I have a wife and kids, eat them" - Homer Simpson

~(_8(I)

Top
#797973 - Tue May 29 2012 02:56 PM Re: Quiz Rankings Question [Re: AdamM7]
triviaking162 Offline
Forum Adept

Registered: Sat Mar 10 2012
Posts: 138
Loc: New York USA
OK, but I can't see why 80% would give positive and 20% very poor. Weird....
_________________________
TK

"That which does not kill us makes us stronger." - Friedrich Nietzsche

Top
#797974 - Tue May 29 2012 03:03 PM Re: Quiz Rankings Question [Re: triviaking162]
kyleisalive Offline
Forum Champion

Registered: Mon Mar 07 2005
Posts: 5558
Loc: Toronto, Canada, eh!
We do not know the ways of the ranking world. wink
It could be any number of factors.
_________________________
Senior FT Editor (Video Games, Television, Music, and Entertainment)
Chat Board Moderator (Author's Lounge)
Amazing Trivia Race Taskmaster/Commission Hander-Outer

Top
#797976 - Tue May 29 2012 03:08 PM Re: Quiz Rankings Question [Re: kyleisalive]
tiffanyram Offline
Mainstay

Registered: Tue Jun 13 2006
Posts: 537
Loc: Jackson Tennessee USA 
It does make you wonder about it when you have a quiz with a good percentage of Good/Excellent ratings that doesn't get sunnies. I usually look at how well the quiz rates compared to other quizzes in the subcategory it's in. If it's at the top of the list or near the top of the list, then I don't feel so bad. Maybe that subcategory just really isn't that popular.
_________________________
"You see a lot of smart guys with dumb women, but you hardly ever see a smart woman with a dumb guy." -Erica Jong

Top
#797978 - Tue May 29 2012 03:14 PM Re: Quiz Rankings Question [Re: tiffanyram]
triviaking162 Offline
Forum Adept

Registered: Sat Mar 10 2012
Posts: 138
Loc: New York USA
Ah, so there is no definitive answer. It was just confusing to me that the quiz in the initial post was ranked so low. I'll have to see if I can get an answer from Terry wink
_________________________
TK

"That which does not kill us makes us stronger." - Friedrich Nietzsche

Top
#797979 - Tue May 29 2012 03:15 PM Re: Quiz Rankings Question [Re: triviaking162]
dg_dave Offline
Star Poster

Registered: Sun Oct 05 2003
Posts: 19294
Loc: Dallas, TX USA              
Some things are better left unknown. Kyle is right about the rankings.
_________________________
The way to get things done is NOT to mind who gets the credit for doing them. --Benjamin Jowett
No one can make you feel inferior without your consent. --Eleanor Roosevelt
The day we lose our will to fight is the day we lose our freedom.

Top
#797995 - Tue May 29 2012 04:43 PM Re: Quiz Rankings Question [Re: dg_dave]
looney_tunes Online   content
Multiloquent

Registered: Tue Jan 20 2009
Posts: 2940
Loc: Briar Hill Victoria Australia 
Because there are so many quizzes being ranked, even a slight difference can have a big impact on rankings. The balance between Excellent and Good, for example, can make a big difference in the numerical result of analysing the results. 2 Excellent and 6 Good out of 10 ratings gives 80% Good and Excellent, but so does 6 Excellent and 2 Good, and the latter quiz will rank significantly higher than the former, if their unlisted ratings are the same as each other. Since we get no information about the other three scale values, it remains (intentionally, I presume) ambiguous.

In my early days here, I occasionally handed out a Very Poor rating by mistake, because I hit Page Down after making my original rating, to get to the tab to click in order to proceed, and only saw that I had changed the rating as it flashed off my screen. There is no taking it back. frown I have learned the lesson, but there are probably newbies still entering ratings they didn't mean. Then there are those who rate poorly if they did not do well - nothing to do about them except wait for more plays and let the majority of ratings make an overriding impact. That is one of the reasons for requiring a minimum of 20 ratings before a quiz is ranked. Even then, you may find that the rank fluctuates quite a bit from week to week until you get closer to 100 ratings, by which time it is usually pretty well settled down.
_________________________
(Editor in Humanities, Literature and Books For Children)
That's all, folks!

Top
#798010 - Tue May 29 2012 07:34 PM Re: Quiz Rankings Question [Re: looney_tunes]
triviaking162 Offline
Forum Adept

Registered: Sat Mar 10 2012
Posts: 138
Loc: New York USA
OK, L-T, thats probably why. It has 37 excellents and 84 goods. I was just wondering about the ranking process.
_________________________
TK

"That which does not kill us makes us stronger." - Friedrich Nietzsche

Top
#798155 - Wed May 30 2012 12:12 PM Re: Quiz Rankings Question [Re: looney_tunes]
habitsowner Offline
Forum Adept

Registered: Thu Jul 14 2011
Posts: 161
Loc: Arkansas USA
"Then there are those who rate poorly if they did not do well -"

I found that happened in the Great Quiz Race. I lost two sunnies there and the two I lost were questions in it and have 4 digits plays...they are also WAY up in the rankings, the wrong place to be. They weren't there before the Great Quiz Race so I have to blame it on that. Another, that didn't have sunnies, also have the same thing happen. Huge amounts shown in plays and also huge amounts now shown in rankings.

Terry changed things once, which got back one of the sunnies, but it was gone again the following week and the plays and rankings were even higher. AND...the Quiz Race was still going.

Top
#798159 - Wed May 30 2012 12:19 PM Re: Quiz Rankings Question [Re: habitsowner]
kyleisalive Offline
Forum Champion

Registered: Mon Mar 07 2005
Posts: 5558
Loc: Toronto, Canada, eh!
Quote:
"Then there are those who rate poorly if they did not do well -"

I found that happened in the Great Quiz Race.


That tends to be the way, even with the other scavenger quizzes. On the flipside, you do get wider exposure not only for those specific quizzes, but for yourself as an author as a whole. smile
_________________________
Senior FT Editor (Video Games, Television, Music, and Entertainment)
Chat Board Moderator (Author's Lounge)
Amazing Trivia Race Taskmaster/Commission Hander-Outer

Top
#798164 - Wed May 30 2012 01:31 PM Re: Quiz Rankings Question [Re: kyleisalive]
Terry Offline

Head Honcho

Registered: Wed Dec 31 1969
Posts: 17670
Loc: USA
I've always been hesitant to provide the precise ranking algorithm because it would make it easier for people to game the system. It's pretty straightforward though.

Even if you have a quiz with lots of high ratings (4s and 5s), you're not seeing what the OTHER ratings are (the 1s,2s,3s). It's possible that your quiz could have been dragged down by other low ratings.

Top
#798165 - Wed May 30 2012 01:47 PM Re: Quiz Rankings Question [Re: Terry]
habitsowner Offline
Forum Adept

Registered: Thu Jul 14 2011
Posts: 161
Loc: Arkansas USA
And, I think that's what happened with the Great Quiz Race, Terry.

I don't imagine there's anyway around it, and as Kyle says, it does have it's "good" side, too.

Since I've never rated a quiz as "poor" because first off I don't think the editors would allow a "poor" quiz to be online, I can't imagine doing it. Least of all because I didn't do well in the quiz. Heck, I don't do well in a lot of quizzes...but I usually rate them good, if not excellent.

Top
#798188 - Wed May 30 2012 03:38 PM Re: Quiz Rankings Question [Re: habitsowner]
triviaking162 Offline
Forum Adept

Registered: Sat Mar 10 2012
Posts: 138
Loc: New York USA
OK, thanks Terry. It was just confusing to me.
_________________________
TK

"That which does not kill us makes us stronger." - Friedrich Nietzsche

Top
#798290 - Thu May 31 2012 12:23 AM Re: Quiz Rankings Question [Re: habitsowner]
AdamM7 Offline
Mainstay

Registered: Sat Feb 18 2012
Posts: 528
Loc: Cheshire England UK           
Originally Posted By: habitsowner
Since I've never rated a quiz as "poor" because first off I don't think the editors would allow a "poor" quiz to be online, I can't imagine doing it. Least of all because I didn't do well in the quiz. Heck, I don't do well in a lot of quizzes...but I usually rate them good, if not excellent.


I have rated several quizzes as poor and a fair few as very poor. I don't think of myself as mean, but some quizzes that were written years ago were allowed through even though they are, to put it mildly, not very interesting. Editors usually reject quizzes on the basis that they are factually incorrect, contain typos or contain bad grammar and need to be worded. I don't think I have ever had a quiz rejected "because it was too boring".

There are 2 main things that make me rate a quiz "poor": no interesting info (except in brain teasers) or completely boring, useless (not meaning interesting but trivial facts - more on the lines of "What time of day was Marilyn Monroe born?" or "What was the score in the 3rd match of the 7th Superbowl after 27 minutes?") or downright wrong.

I rate a quiz very poor if it is riddled with typos/errors, has no interesting info and has 10 very similar, very boring questions (10 lots of "What is the capital of X?" would get at least a poor from me).


The options "poor" and "very poor" are there so people can use them. It might feel harsh, but if you are rating every single quiz "good" then Terry might as well get rid of the lower 3 options. I don't rate many quizzes below "average", but I do to some.


I agree with one thing you say - I can't imagine rating a quiz poor because I didn't do well in it. I have rated one quiz as "excellent" despite the fact that I got 2/10 correct (Q"uiz About This Quiz", I think it was called - I highly recommend it to anyone who has some time on their hands and doesn't get frustrated easily).
_________________________
"No, please don't eat me. I have a wife and kids, eat them" - Homer Simpson

~(_8(I)

Top
#798388 - Thu May 31 2012 09:04 AM Re: Quiz Rankings Question [Re: AdamM7]
agony Online   content

Administrator

Registered: Sat Mar 29 2003
Posts: 11496
Loc: Western Canada
Just because a quiz meets the standards well enough to go online, doesn't necessrily mean that it's good. If you disliked a quiz, feel free to give it a poor rating. That's what the range of ratings is for.

Top
#798402 - Thu May 31 2012 10:10 AM Re: Quiz Rankings Question [Re: agony]
LadyCaitriona Offline
Forum Champion

Registered: Thu Feb 08 2001
Posts: 5399
Loc: Ottawa Ontario Canada         
Originally Posted By: agony
Just because a quiz meets the standards well enough to go online, doesn't necessrily mean that it's good. If you disliked a quiz, feel free to give it a poor rating. That's what the range of ratings is for.


Agreed. There are many times where I have urged an author to consider expanding the information sections in a quiz, noting that it is the quality of the information sections of a quiz that can turn a good quiz into a great one. Some authors want to add more info, and others don't, but all quizzes will go online if they meet the basic minimum requirements set out by the Quiz Creation Guidelines and Category Guidelines.

The problem with making the QCGs too strict in terms of content is that A) who is to be the judge of what makes a quiz entertaining? Even within my own categories, there are very few topics that I could consider myself well-versed enough to judge accurately the content quality of a submission. And B) it discourages new authors from starting small and working their way up, as well as being an inhibitive factor for established quiz authors to branch out into new areas of interest.

With that being said, that doesn't mean that you have to rate every quiz as "A for effort". If you believe a quiz is poor, rate it as such. It's a good way to give authors feedback so that they can take a look at their quizzes that didn't do so well and see what they might do better for the next submission.
_________________________
Chan fhiach cuirm gun a comhradh.
A feast is no use without good talk.

Editor for Humanities and Movies

Top
#798480 - Thu May 31 2012 02:10 PM Re: Quiz Rankings Question [Re: LadyCaitriona]
habitsowner Offline
Forum Adept

Registered: Thu Jul 14 2011
Posts: 161
Loc: Arkansas USA
Adam, Agony and Lady C.,

Thank you for your comments. It's very hard for me to rate anything but good. I might try "average" if there's one allowed, which I think there is somewhere. But poor? Is it poor because I'm not interested in the subject matter or is it poor because I don't know the subject matter or is it poor because it is just poor. Very hard to tell the difference between the first and the third one. The second one is simply being mean.

Top
#798491 - Thu May 31 2012 02:35 PM Re: Quiz Rankings Question [Re: habitsowner]
kyleisalive Offline
Forum Champion

Registered: Mon Mar 07 2005
Posts: 5558
Loc: Toronto, Canada, eh!
I use the full scale of ratings myself-- seeing so many quizzes from this side I certainly value different types of quizzes. I'd never ask an author to overload the quiz on different things I like. Most newer quizzes I see get Excellent to Average. Poor and Very Poor I reserve for much, much older quizzes with no info which were written at a time when our standards were much lower.

How people rate shouldn't matter too much if they play a lot of quizzes; provided they use the same sort of criteria across the board, everything should even out quite nicely. Whichever approach you take is perfectly fine so long as every quiz you rate isn't 'Very Poor' or 'Excellent'.
_________________________
Senior FT Editor (Video Games, Television, Music, and Entertainment)
Chat Board Moderator (Author's Lounge)
Amazing Trivia Race Taskmaster/Commission Hander-Outer

Top
#798495 - Thu May 31 2012 02:50 PM Re: Quiz Rankings Question [Re: habitsowner]
shuehorn Offline
Multiloquent

Registered: Tue Jul 04 2006
Posts: 2993
Loc: Lawrenceville Georgia USA     
There are quizzes on topics I know nothing about but which have been wonderful, and I've rated them highly, even when I have just barely gotten enough questions right to have my rating count. There are others that are just badly written, with a bias or a tone that makes them unenjoyable, with information that is incorrect, or a style that is not interesting at all. I think I've only rated one or two quizzes very poor, but I do rate quizzes average when I don't think much effort went into them, and I have rated quizzes poor when they really are sub-standard. I guess we all have our way of evaluating them, and as long as we are consistent and give high marks for the things we appreciate, then the site and the authors benefit.
_________________________
Sue (shuehorn)

Top
#798509 - Thu May 31 2012 03:42 PM Re: Quiz Rankings Question [Re: shuehorn]
agony Online   content

Administrator

Registered: Sat Mar 29 2003
Posts: 11496
Loc: Western Canada
As Lady C said, sometimes I ask advise an author that a certain question type may be legal, but will probably result in poor ratings, as players don't tend to like it. If a fabulous quiz written in a lively entertaining manner gets the same ratings as a boring one, it's not really fair to the fabulous quiz. Not every quiz is above average, and it's all right for your ratings to reflect that.

Top
Page 1 of 4 1 2 3 4 >

Moderator:  agony, gtho4