Imagine giving students a paper based assessment with all the correct answers highlighted. Nobody, I’m sure, would believe that this was a valid assessment. I'm pretty sure most would see this exercise as a waste of time.
Yet many online quizzes exist with equal or worse validity, and they are still passing scrutiny.
The following is a question out of a real quiz that I came across the other day. All of the other questions were of a similar format, although some had more distractors.
“From the statements listed below please identify the correct one:
The question itself is very verbose and could easily be presented in a way that is much simpler to understand, however that is not the issue being highlighted in this instance. The issue in this case is the methodology used. If a student selects the incorrect statement the program tells the student that they are “incorrect” and advises them to “have another go”. Nothing in the question changes, so the student knows that all they need to do to progress is to select the other option.
This format encourages the student to “click and see”. The worst case scenario is that the student will need to click each statement. Students quickly realise that they do not need to even read the question or the distractors, just click and click again until they complete the quiz.
The sad part for me is that no-one, I’m sure, sets out intentionally to create a quiz that is so totally lacking in validity. They just don’t know the questions to ask of the assessment system of the program that they are using. It is possible to have great and valid questions and methodologies, for example, one of our questions in the White Card course has 6160 unique versions of the question (720,000 when randomised) and the students are required to obtain 100%. This results is students themselves realising that it is easier to learn the content for the short learning activity than to try to cheat. They also realise and appreciate that they have earned their successful result. They didn't get it just by clicking.