Rules
Terms of Use

Topic Options
#254398 - Sun Jan 16 2005 05:54 PM Order of Difficulty
IndieQueen Offline
Forum Champion

Registered: Tue Apr 17 2001
Posts: 7306
Loc: Pittsburgh
Pennsylvania USA
One of our CBer's asked me what the order of quiz difficulties are. I didn't know for sure so I figured I'd ask the experts.

To be more clear, I think they are ranked like this:
Simple
Very Easy
Easy
Average
Tough
Difficult
Very Difficult
Impossible

Is that all of the difficulty levels or are there more?
_________________________
[color:"purple"] "One of the best features of Forums is that they allow people to parade their monumental stupidity, their hang-ups, their little prejudices in public."
[/color]

Top
#254399 - Sun Jan 16 2005 05:59 PM Re: Order of Difficulty
Leau Offline
Forum Champion

Registered: Sun Jun 16 2002
Posts: 5337
Loc: Nijmegen/Brisbane
Those levels correspond with the ones on this page. Several quizzes have multiple difficulty rankings though, depending on what page you view them.
_________________________
The cost of living has not affected its popularity - Loesje

Top
#254400 - Sun Jan 16 2005 07:14 PM Re: Order of Difficulty
IndieQueen Offline
Forum Champion

Registered: Tue Apr 17 2001
Posts: 7306
Loc: Pittsburgh
Pennsylvania USA
Thanks, Leau. I'll pass that link on. I couldn't find the darn thing when I looked. Proving yet again that Leau can find anything anywhere online within a matter of seconds.
_________________________
[color:"purple"] "One of the best features of Forums is that they allow people to parade their monumental stupidity, their hang-ups, their little prejudices in public."
[/color]

Top
#254401 - Sun Jan 16 2005 08:44 PM Re: Order of Difficulty
d2407 Offline
Participant

Registered: Mon Jan 03 2005
Posts: 31
Loc: Rochester New York USA    
Not to ask a question that I've seen posed many other times in the forums, but how exactly does the algorithm for quiz difficulty work?

I have several quizzes with "Impossible" difficulties - kind of like the trivia equivalent of a movie getting a X or NC-17 rating: death at the box office. Yet those quizzes have several people scoring perfectly or missing one or two. If fifty people have taken a quiz with 10-15 questions, and 10% of them have gotten a perfect or near-perfect score, it's not impossible (my opinion!).

I have no idea about a better difficulty scale. But I do think a lot of quizzes get penalized unfairly because people start them, find themselves over their head, and bail before the end. Would it be possible to, if a quiz has, say, the final three or four or whatever questions left unanswered, count the score for the player's individual record, but omit that score in computing the difficulty?

Top
#254402 - Sun Jan 16 2005 09:55 PM Re: Order of Difficulty
MotherGoose Offline
Multiloquent

Registered: Mon Apr 22 2002
Posts: 4817
Loc: Western Australia
I can't comment on the algorithm for determining quiz difficulty - however, I am sure one of the other editors will be able to do that.

Bear in mind, however, that level of difficulty is a relative thing and very dependent upon the individual quiz-maker's areas of expertise.

For example, the easiest football quiz in the world would be very difficult for me because I have no interest in football at all and never watch it or take any notice of it. On the other hand, an "impossible" medical terminology quiz would probably be a piece of cake for me, because that is my area of professional expertise.

And, of course, some people get a perfect score because they look up the answers on search engines as they do the quiz. There have been threads debating the ethics of this, but some people do not regard this as cheating. Rather, they treat the quizzes as learning tools and they figure the scores are irrelevant as long as they've learned something as they've been researching the answers.
_________________________
Don't say "I can't" ... say " I haven't learned how, yet." (Reg Bolton)

Top
#254403 - Mon Jan 17 2005 12:14 AM Re: Order of Difficulty
Bruyere Offline
Star Poster

Registered: Sat Feb 10 2001
Posts: 18792
Loc: California USA
I personally feel that if your quiz has had an impossible rating that it may be aiming a bit too high over the average person taking it.
Or, there might be a technical reason it's getting the impossible rating. If you've put in an FITB that is impossible to type for example, you'll get poor results for the wrong reasons. This is why we strongly discourage some of them.
If someone does a few of your questions and finds them unfair or they don't give them a ghost of a chance despite their knowledge of the subject, they might give up on the quiz.

We try to avoid you this fate beforehand by advising you of which of your questions are working well, but, sometimes it's hard to foresee.

If several of your quizzes are rated impossible however, I'd think about how they look when someone takes them. Are they impossible for the right reasons?
_________________________
I was born under a wandering star.

Top
#254404 - Mon Jan 17 2005 06:48 AM Re: Order of Difficulty
damnsuicidalroos Offline
Multiloquent

Registered: Mon Feb 10 2003
Posts: 2167
Loc: Sydney
NSW Australia
From the link Leau posted.
Quote:

Quiz difficulties are dynamically created by the FunTrivia quiz system based on how well players perform on a quiz. If most people do really well on a quiz it may be rated as "Easy". If no one does well, probably "Impossible".



I have an idea that the difficulty rankings are based simply on the average score achieved. Is that correct?
If it is correct then I feel that the lowest scores, from 3 down, should be eliminated from the averaging formula. The percentage of people that are simply playing quizzes to get their quizzes played score up is not something I know but I suspect it plays an unfair role in difficulty rankings for quizzes.
If I`m not guessing correctly, well blame it on too much free time.


Edited by damnsuicidalroos (Mon Jan 17 2005 06:52 AM)
_________________________
Responds to stimuli, tries to communicate verbally, follows limited commands, laughs or cries in interaction with loved ones.

Top
#254405 - Mon Jan 17 2005 11:03 AM Re: Order of Difficulty
Bruyere Offline
Star Poster

Registered: Sat Feb 10 2001
Posts: 18792
Loc: California USA
I'm pretty sure Terry does some sort of calculation to do just that, (not my dept though nor my main concern here) however, once again, if your quiz has been ranked impossible, it's almost certain that there's a valid reason for it. Your questions are probably aiming too high, or the majority of your questions are causing people to zone out. One or two questions asking for highly specific material are the limit in my opinion.

Example: I take a quiz to think of something else one day, and I get a quiz that requires too deep a thought for my current mood. I don't want an easy one, I want something that digs into my knowledge but in a fair way. If I get a quiz that has about four questions that would require looking it up, I may stop and do something else.
If you're asking specific information from source material that few have seen, then you do risk people getting muddled on your quiz and giving up.
If you're asking how many bullet holes were in Tupac's body, then few would know that without looking it up.
If all of your questions on a quiz are like this, then, you risk having it rate impossible though, you don't realize why.

If you craft a quiz to meet a larger audience, then throw in a few clinchers, then you'll get a tough rating, or difficult which isn't bad at all.

The reason I've always admired Jazz's work is that even with the bare minimum of subject matter knowledge, I can always make an educated guess. Or if I haven't seen MASH in many years, it jogs my memory about the characters or surroundings.

One exception to the "impossible" rating scenario is when people do logic quizzes or puzzle type things. This means that people will do one or two then realize they're out of their depth and back out.

So that's my two cents on the impossible ratings. I'd take it as a sign that you needed to aim a little closer to the middle rather than having the entire quiz be too specific to a small group of people.
_________________________
I was born under a wandering star.

Top
#254406 - Mon Jan 17 2005 11:05 AM Re: Order of Difficulty
d2407 Offline
Participant

Registered: Mon Jan 03 2005
Posts: 31
Loc: Rochester New York USA    
Quote:

I personally feel that if your quiz has had an impossible rating that it may be aiming a bit too high over the average person taking it.




Yeah, I plead guilty.

Several of my quizzes, especially the earlier ones I did, were written about subjects that I had a pretty deep and hardcore knowledge about, and I was writing them for similar audiences.

That's not necessarily wrong. If I'm taking a quiz about something I care about, I'll want to be challenged. If, on the other hand, I wander into a quiz category about botany or 17th century literature, I'd expect to do horribly on even the "easy" or "simple" ones.

I've since tried to make my quizzes more general in appeal/difficulty, albeit I'll always throw in one or two questions for the hardcore people.

Here's a thread that I just started, where I take a closer look at one of my "impossible" quizzes (median score 4/15). My verdict on it is that one of my questions is unfair, but the rest of the quiz is fine for its intended audience: people who've read a book about the RFK assassination, or at least read extensively about RFK or the 1968 US presidential campaign.

Again, difficulty ratings for all quizzes are displayed, and obviously, people taking easy quizzes in subjects with which they're familiar will tend to score highly; people taking difficult quizzes in qubjects with which they're not familiar will tend to score low.

Top
#254407 - Tue Jan 18 2005 01:24 AM Re: Order of Difficulty
onlytrivial Offline
Explorer

Registered: Tue Jan 11 2005
Posts: 65
Loc: Melbourne, Australia
It seems to be implied by what you have said here that even when you back out of a quiz, your score is recorded and it affects the rating of the quiz. Is that what happens? I thought scores were only recorded when a quiz is submitted. Sorry, I just found that confusing.
_________________________
Why can I never think of something funny when I need to?

Top
#254408 - Tue Jan 18 2005 08:50 AM Re: Order of Difficulty
d2407 Offline
Participant

Registered: Mon Jan 03 2005
Posts: 31
Loc: Rochester New York USA    
Quote:

It seems to be implied by what you have said here that even when you back out of a quiz, your score is recorded and it affects the rating of the quiz. Is that what happens?



Oops. I wasn't clear.

I meant to say that perhaps sometimes people begin a quiz, get frustrated (especially if it's a long quiz) and just skip to the "submit" at the bottom to see their score and read the answers.

I am positive that only when a person submits a quiz for scoring does the result count for the quiz taker, and the score count for the difficulty rating.

The suggestion I was trying to make is that, if easily technically feasible, maybe if someone submits a quiz for scoring and it's obviously incomplete (for example, maybe three or more multiple choice questions left unanswered), the score not count toward the difficulty rating.

Disclosure: up the thread, I have a link to another thread about an "impossible" quiz I did about the assassination of Robert Kennedy. The median score on the quiz is 4/15; the most common score is 3/15. BUT... the last question on the quiz has been answered correctly about half the time, making it the third-easiest. So my theory about people giving up and submitting a quiz for scoring without answering every question is admittedly flawed!

Top

Moderator:  agony