Recently, I played a new quiz and scored 1/10. I submitted a rating (or tried to), and got a message that my score was too low for my rating to count. I do understand that a player who scores low might under-rate a quiz, but the average score for this quiz was only 3/10. Also, I'd like to think that someone who is a gold member and who has played and rated over 2000 quizzes wouldn't purposefully rate a quiz low out of spite. I don't recall that my rating had been rejected when I have scored low on other quizzes. Yes, I did rate this particular quiz "poor", but for a very specific, objective reason. As I recall, I don't think that I rated any of the other quizzes less than "average".

As I said, I know that there could be a concern that those scoring low might take out their frustration on the quiz author, but on the other hand, a player might score low because there is an issue with the quiz.

Regardless, if there is to be an objective, universal, player-driven rating system, I don't necessarily think it's appropriate to exclude a low rating just based on the fact that a player might not have scored very high. Also, if the system doesn't exclude every rating from a player who doesn't post a high score, it seems to me that such a system would skew the ratings of a quiz, resulting in a rating that is artificially higher than it should be. (I apologize in advance if my understanding of the system isn't accurate; it's just based on my recollection of what has happened when I have posted low scores and submitted a rating.)

I wish I had a suggestion of how to balance an open ratings system with the interest of protecting more difficult quizzes from artificially poor ratings, but I don't. However, I don't want to believe that there are that many registered users on the site who would rate a quiz as "poor" just based on their performance on the quiz, so I would advocate for allowing all ratings to count.

Terry is the only one that knows exactly what mechanisms he has in place re whihc ratings are ignored; this is one of them.

Quote:

---


Thanks I think this works a lot better too.

Number of ratings is usually considerably less than times played because:

- all guest plays don't rate

- ratings of some players with suspicious patterns of rating are discarded (cheat detection)

- players who have not been online for a certain period of time and who have not played a certain number of quizzes are also ignored until they do meet this criteria (cheat detection)

posted by Terry : Quiz ratings : Nov 2006



---

There are other posts by Terry over the years where he drops a hint or two re what goes on in the background; but they're so far back they're hard to remember/find. I don't think Terry will agree to allowing all ratings.

One thing I would like to know, is why rating quizzes is now restricted to gold members?

Apologies if this has been asked before, but as far as I know, it's a fairly recent development. I think it's unfair both to authors and users.

I have come across the occasional quiz that I can still rate. Not sure why this is.

Carol <><

Gold members can rate quizzes because the quizzes they play are tracked. Non-gold members can rate their first 100 quizzes but once their quiz plays are no longer tracked then the ratings they give are not tracked either.

This, as far as I am aware, is how it has always been. The only change is in the number of quizzes that a non-gold member can play before the tracking stops.

Thank you. I only found out recently I could no longer rate quizzes, so I guess I must have passed that limit.

However, I still think it's unfair to authors and non-gold members.

Carol <><

Carol, I just asked this question not two weeks ago. The original post is on this same page, Aug. 30, but the thread has been moved to Feedback.

http://www.funtrivia.com/ubbthreads/showflat.php?Cat=0&Number=1156948

It is perhaps unfair to authors, but not at all to non-gold members. The tracking bit is what makes it unavoidable.

I'd like to bring this discussion back to my original point. I understand and support Terry's point in the quoted post about the provisions to prevent cheating, but I am still curious why my rating in this specific instance was excluded, and whether only "poor" ratings are excluded from low scorers.

Again, I certainly agree that ratings from players that show a certain pattern indicative of attempting to skew ratings or be vindictive based on scores should be excluded, but I don't understand why a low rating from a player with a reasonable rating history should be excluded automatically just because that player scored low on the quiz, especially if a positive rating from the same player would have been counted.

I think the whole thing is automated - there is not a human person looking at your rating and deciding to disallow it. Terry will have put together a system that tends to do what he wants, and you seem to have fallen into it.

Could that system be fine tuned? Probably, and when someone like you brings up weaknesses in the system, it's helpful for the next time an overhaul happens.

We used to have a lot of people who would just take quizzes just to be counted as having taken a quiz then rate them all 'poor'. Or people who would take all the other quizzes in a category, rate them poor, so their own quiz would be a little higher in the ratings.

Sure, and I totally agree that those ratings should be disallowed. I assume that those sorts of actions are included in Terry's "suspicious patterns of rating".

And I wouldn't think there's some human volunteer screening each rating -- not only would that person deserve a "volunteer" badge, but a FunTrivia lifetime achievement award!

I am pointing out what seems to be a weakness in the system for the next overhaul, just as agony said. (As if Terry doesn't have enough other projects...) But it does appear to me to be a fairly significant failing in the rating system, all the same.

You cannot rate any quiz on which you score 0 or 1, even if you plan to rate it Good because (despite your lack of knowledge in the area) you can see that it is a well-written and carefully-researched quiz. This disappoints me sometimes, but that's life!

I have found out (mainly through working through the Mission Impossible challenge) that all scores aren't counted if you score too low, so there goes one of my original points. I don't know why this has to be life on FunTrivia; it's hard enough for quizzes on more obscure topics to get ratings, but to disallow ratings from players who might be trying to help out by rating a quiz just makes that problem worse.

I think it's similar to the Olympic judges in some competitions where the lowest and highest scores are discarded, just to ensure that ratings will be within a "reasonable" range and more reliable. In this case, I think the goal is to eliminate opinions that might judge a quiz overly harshly because the taker didn't get anything right.

Anyone who does take a quiz and do very badly on it is welcome to take it again later and then rate it.

So if I score 0 or 1, i can retake the quiz (having noted a couple of answers to get them right the next time) and regain the facility of rating it?

I seem to recall doing that a few times before. You just don't get a score for taking the quiz the second time.

You also don't get the 20 points for rating the quiz the second time or subsequent times after that (I suspect), but I am pretty sure your rating does count for the quiz maker.

I am not saying that you are wrong but I would be surprised if rating subsequent times would count or you could get someone taking a quiz over and over in the hopes of changing a rating - Terry is the only person who knows the answer for sure.

I am totally unconcerned about points, but when I take a good quiz and want to rate it well in appreciation, even though I didn't happen to know any of the answers, I would like to be able to do so. I understand why people scoring 0 or 1 are barred, and I had always assumed that if you retake a quiz you cannot rate it, as that would open the door for stacking things. But I thought I spotted a possible loophole - score 0 or 1, gather some facts to be able to score more the next time, take it again and rate it. Since there was no previous rating entered, maybe this one would?

It doesn't happen very often that I take quizzes about which I know so little, but my current effort to find and rate quizzes that have been online for a number of months without getting 20 ratings has led my to undertake some quizzes on topics which I would not usually consider, and my efforts are thwarted when one of these is a really good quiz about a tv show I haven't seen or a book I haven't read!

I'll be interested in seeing Terry's opinion on this too. I'd always thought that the very first rating of a quiz did count, though, even when it didn't correspond with the very first time the quiz was played, and each subsequent rating didn't count. Now I'm beginning to doubt my recollection of things, and I guess Terry is the only one who can put this one to rest.

Sue

I too have scored badly on quizzes which I still appreciated were well written. I think it's grossly unfair to quiz-writers that they are deprived of ratings by low-scoring players and non-paying members, and I agree with suggestions that the system produces skewed results, probably in favour of easy quizzes because they get the highest scores.

The purpose of this was to stop the angry trashing of quizzes. Some players would "play" a lot of another player's quizzes, not bothering to score properly and then rating the quizzes as poor or very poor. This was presumably done to lower the other player's ratings.