Moving away from too easy questions?

It may just be me, bit it seems the newer authors are becoming increasingly whimsical and even a bit basic in their writing. It would be something like, which president was tall and wore a big hat?

Abraham Lincoln
Mother Theresa
Adolf Hitler
Sean Connery

Like, okay, I’m not sure what the point of that really is. I mean I guess it’s to point out that Lincoln was tall and wore a big hat? A lot of questions are getting like that now where there is only one obviously correct answer and the others are basically just filler.

I guess what I’m saying is a question should have at least semi-plausible selections (to the average reader, not a historian or expert), not three garbage choices wth the obvious answer thrown in.

Then again, what seems obvious to you may not seem that way to all players. Editors have to make a judgment call as to whether or not a question is viable. Your example is unlikely to make it through, but another that seems in your eyes to be equally a case of there being only one remotely plausible answer may not be so for players with a different knowledge base.

I agree in principle that there are many relativisms, including relative relativisms, and one of the biggest differentiators will be country. I couldn't tell you the first thing about most countries on earth and so something obvious to such a person wouldn't likely be obvious to me. But there are also items which are not absolute, but so far from being relative that they couldn't be compared to such cases, at least not directly. There is a sort of base of knowledge which all educated human persons have some, if only fleeting familiarity with. People are quite aware Hitler was a German madman, not a Siamese one. That beer is brewed and not distilled. That Pythagoras produced a timeless and eternally useful theorem. And there are gray areas extending out from that that are less clear, but you are correct in pointing out that it requires discretion and I certainly wouldn't claim to know the stresses of that position or lecture to those who do. I just was saying it's a trend I've noticed, and it's resulted in less enjoyable play.

Since all our games filter the questions by difficulty targets - both with a lower and upper bound - nearly all question difficulties are viable. The Sean Connery answer is semi-viable in this context; in fact, I would certainly allow this question with three male English language names as detractors, even if none of the others was American or a politician. Such a question is a good candidate for the Easy hourly - you have to know the nationalities and/or claim to fame for each name.

I understand how the example question would be allowable. Non-Americans may know Abraham Lincoln was an American president of some importance but not know his physical description or taste in hats.

Are you asking about quizzes that have recently gone online, or questions in the New Question game?

Because if they're not from those sources, any questions you get could be fifteen years old, or older, rather than recent.

Personally, since I dislike very easy questions that make a game simply a race, I don't play games like Piece of Cake, and I don't play quizzes rated "easy" or "very easy". A question such as your example would get a "poor" rating from me, unless it was in For Children.

Okay, agony, you raise a good point. I'd say probably mostly the New Question game. That's one I've been playing a lot in lead up to my Two Years of New Questions badge and that's probably where the really bad ones were that I saw.

Writers of single questions are often experimenting in those questions with adjusting the difficulty to get their percentage correct within the target range that counts for a badge. Sometimes, in trying to make sure it is not too hard, they may make it too easy for that target range, but the questions are still useful grist to the question mill for use in Piece of Cake, as Wesley pointed out.

In my eyes it is part of a tough balance in writing questions for New Q at least. There are so many competing desires of the overall audience. Difficulty level is just one facet..

Some are issues of regional based knowledge as has been mentioned..where what might be insanely common knowledge in one place is fairly obscure elsewhere.

but I have been frustrated often with questions I thought were fairly universally easy..but which came out below the target range of over 55 (or 60 or 65 seems like the New Q info screens vary a bit) correct.

That was after having numerous questions come out in the 15-25% correct range. Now it seems sometimes I cannot win that game..If i send in my unadulterated version..even with one junk answer which is obviously wrong...it fails the difficulty issue. However, if I go the other way and "dumb it down" I have had questions rejected for being "too easy"

I think part of the issue is lack of seasoned question writers contributing. The wait from submission to play held pretty steady at about 10-14 weeks for years and years. Lately it seems to have shrunk and now questions seem to barely be waiting even a month to be played.

So for both the long term health of the game and general quality of questions. This is going to sound like snarky mouthing off, but I am truly being earnest when I say, if you think you can do better please jump in and give it a shot.

It is definitely difficult to find the balance, even for seasoned writers. I usually know about where my questions will fall in terms of difficulty, but am often surprised.

In terms of submitting new questions, we have a new Author's Lounge challenge that focuses on single questions if anybody reading this wants to get some inspiration and start submitting their own questions. It is called 100 Nouns. Feel free to check it out if you want to write, but are having trouble figuring out what you might want to write about.

To namrewsna's point above, I think it is ridiculous for there to be a 10-14 week wait between the time a New Question is accepted by an editor and the time it finally appears in the New Question Game, *especially* as we keep getting Daily Challenges to submit more new questions (which I generally ignore, because of the cumbersome and lengthy manner of the process.) This is too long a backlog, in my opinion, and we don't need to be constantly encouraging players to make the backlog even worse by submitting new questions from daily challenges. I get these types of challenges about twice a week. Personally, I think they ought to be eliminated. But couldn't the frequency that they are given be cut back, at least?

Regarding the wait, it's a bit of a tricky situation because we get burned no matter which way we tilt things.

If we speed up the questions going into the game, we run out of questions.
If we add more pools per day/week we also run out of questions quicker
If we spread the questions out amongst larger pools just to get more out at once, it means less ratings.
This also means that people see the same questions repeatedly in the one game.

The questions are also pulled as per what Terry codes. If Terry has to rejig the code every time we have less questions ready to go just to stretch it out, then that's a struggle in and of itself.

The only real way to fix the problem is to build up the question cache to justify speeding them through even quicker, but to do this we need more submissions at a consistent rate.

It is not a backlog. That would suggest the questions have gone unedited. Instead the questions are put into a pool to be used for the New Question Game. Sometimes that takes a while since many questions are submitted. I don't understand the relation between being asked to submit a single question as a challenge and there being a backlog since players get the credit in any case. The wait afterwards is irrelevant.

Respectfully, I disagree. In order to improve one's approach to single question-writing, one needs to see the ratings on the questions that one has previously submitted. The wait for those ratings to start coming in is too long to be able to adjust one's approach. If you submit more questions before other one's you have already submitted appear in the New Question Game, you are just "shot-gunning" with no idea what their ratings might possibly be. Namrewsna's points about it being very difficult (especially for inexperienced writers) to "thread the needle" between being too difficult and too easy is correct. Only with feedback can you learn, and the feedback that is actually important is the rating. Different editors have vastly different opinions about questions.

Honestly speaking, I have seen that some of my worstly written questions (in my opinion) are very highly rated. And many of my best questions are badly rated. So, opinions can vary a lot.

Also, the fact that accepted questions are taking 10-14 weeks to get played is a good thing, right? It means many intelligent players on FunTrivia are writing many questions.

And the reason single question writing is encouraged in daily challenges is that it acts as a transition for quiz-writing. Newbie writers can dabble in single questions before taking the plunge into quizzes.

There are also people who may not have the time or motivation for writing full-length quizzes. Challenges for single questions are better suited for them.

I think what you're saying should be indicative of how the rating system is both a friend and a foe of less experienced authors. Of course there should be an aim for 'good-rated' questions, but I think that what makes a question 'good' is a very subjective thing-- at least subjective enough that I've stopped trying to aim for a 'type' and instead tried to scale things back.

A question like "What is the name of Super Mario's Brother?" is blisteringly easy to a video gamer and it's a pretty basic pop culture question, but it's not really too imaginative and it's quite overdone.

A question like "What green-clad video game character played the lead role in the game "Mario is Missing"?" is a bit tougher but has the same answer. It's ideal for a quiz and, to be honest, appears to me to be a better-written question, even if it's a bit tougher.

But because of the fact that they're both about video games, there's a high probability of the questions tanking in the ratings, regardless of how hard they are. Or maybe the right type of players for it get it in their set and give it a good boost to give it a respectable number. Maybe it becomes top 20 for the week. Maybe both of them do.


I think that, as an editor, the job for our staff in the queues is to give pointed suggestions as to what techniques can better-craft single questions. We will obviously send feedback for major issues and the like to help improve writing (for instance, if a question flips back and forth between past and present tense I'll usually say "there's something crazy going on with your grammar") and we do have a gauge for what is too obscure for a general audience to be able to say "this isn't really suitable for this game, but it's certainly great for a more specialized quiz".

And there's also the Author Lounge chat board. Sometimes it's a good idea to workshop a bit in there. Maybe there's some insight to gain from fellow authors in this regard.

Either way, I usually tell people 'don't worry too much about the ratings'. Good questions come in time, but the questions that don't get a good rating are still useful too.
I've certainly written a fair share of bad ones.

When I say ratings, I am referring not just to the strict positive/negative number. I am also referring more generally to another piece of very important information which does not come out until the question appears in the New Question Game, a piece of information that, in my opinion, is more important to determining whether a question was too easy or too hard or just right than any editor's opinion, and that is: the percentage correct. This is an extremely important number for determining the subsequent future of the question, it is a cold, hard number, and editors have no idea what it will be until the question hits the New Question Game. If an editor tells me the question was too easy but less than 50% get it right, I'll trust the number over the editor's opinion. From the few single questions I have submitted, I have found the best way to tweak the percentage correct on a question is not to tweak the question so much, it is to give very careful consideration to the incorrect answers you select. If your incorrect answers are not plausible, the percentage correct will be too high. If the incorrect answers are too obscure, the percentage correct will be too low. I like that number and I don't like waiting 10-14 weeks to start seeing it. That number is useful for designing future single questions and selecting the incorrect answers you want to include with them.

I agree that it can be frustrating to wait literal months before being able to see this information, and a new author could use it to tweak how their questions are written, but has to wait a long time to get that information. Unless a new system where questions can be rated independently of the New Question Game comes out, I am not sure that much can be done. We could be much more strict about submissions (which turns authors away) or we could open up more rounds (which means all questions get less data from being played).

Well we can't give an exact number, but I feel like I have a strong basis for being able to judge a ballpark. Our editors can tell if something is waaaaay too obscure. 'What colour tie the Malaysian President wore on July 7, 1992' is not a good question. Similarly, a question about the colour of a block in 'Minecraft' is not a good question for that game.

Difficulty is very much subjective, and I'm not professing to know everything or even how much the players will know about any given subject, but as an editor I'm here to give feedback, and I feel that personally, I've done a lot of work first-hand to track what works and what doesn't in the New Question game. Does that equal the perfect formula? No, but it's one of those situations where I feel I have a grasp for what our players are looking for based on my experiences and the feedback I've received from them and our prolific authors via the queues, the Author Lounge, and our editors.

I'm happy to assist anyone in crafting these questions and I'm sure that other editors would be happy to do the same (as would many of our high rated authors) and, like I said before, the Author Lounge may be a good place to workshop a little bit. Feedback from peers can be just as valuable in my opinion.

Unfortunately, I don't know of a viable faster wait time for the questions (Terry would be the only one with an answer for that, or maybe Wes) that allows the game to keep running without being periodically taken offline to replenish. Currently, our goal in the Lounge is to get people excited about writing in the New Question game to increase that pool, and as I noted in an earlier post, more questions could alleviate the wait time.

Trident is very right in that speeding the process means that less data can be attained. Less time in the game means you get a shakier number to base your findings on. It's the difference between 100 people scoring on the question and 200 people scoring on the question. I'd like to get the latter number.

The amount of times a question does get played in the NQG depends on the queue of approved and waiting questions, so theoretically the wait between your question getting edited and it being played should be roughly constant. There is an intentional buffer so we won't immediately have the game repeat questions for long times if people write less over some time (such as holidays).

I however don't know what wait time and buffer size the algorithm is aiming for.

Note that of course there is a minimum number of plays per question - we otherwise can't get decent data. If a lot of people write questions but no one plays the NQG (or people play but forget to rate), the queue will inevitably grow. Bottom line: Play the game and rate the questions every day, not only while you're trying to build your tiers!

As a matter of interest, a question that I submitted in the first week of March is being played in this week's round of the New Question Game. That would put the current lag time at only four weeks.

While it is useful to get feedback about whether your questions fall in the target success range, so you can try to see what works and use the knowledge to help you design future questions, I found it useful to just write them. Eventually you get feedback on a range of question subjects/styles/obscurity levels, and can have a look at the larger range of data than if you submit one and wait to see how it goes. And the ones that don't actually move you towards the badge still add to the pool for games. (And for every three you get an ingredient for the kitchen - you never know which one will be the one you have been waiting for.)

Thank you for that tip!