Author |
Topic: Partial Marks on Multiple Choice Tests (Read 468 times) |
|
Amir Michail
Guest
|
 |
Partial Marks on Multiple Choice Tests
« on: Feb 29th, 2004, 5:54pm » |
 Quote  Modify
 Remove
|
I would like to give partial marks on multiple choice tests so that incorrect but more reasonable answers are not penalized as heavily as incorrect but way off ones.
One way to do this is to specify ahead of time what I think are reasonable answers. However, this is a real pain and I can't read the students' minds so my idea of reasonable may not match their idea of reasonable.
So I was thinking that I could simply use the actual answers given by students on the test to identify the top 20% of students based on the number of questions they got right.
To give partial marks for a particular question, I would see how many of these top students fell for incorrect answers. So if choice (D) say got more incorrect answers from the top students than any other incorrect choice, then choice (D) would have the smallest penalty say. However, if (D) was chosen by the most people but most of them were not top students, then the penalty can be larger.
Unfortunately, it is possible for good students to collude to defeat the system by say always answering (A) whenever they are not sure. As these good students are likely to do well on the test and as long as this strategy does not spread to the other students, then the good students can get an advantage.
One could randomize the order of answers on each test sheet perhaps, but good students could then collude to pick the answer that comes first numerically or alphabetically.
Is there some variation on this approach that would make collusion by good students difficult? I'm looking for an automated approach.
|
|
 IP Logged |
|
|
|
Speaker
Uberpuzzler
Gender: 
Posts: 1118
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #1 on: Feb 29th, 2004, 6:15pm » |
Quote Modify
|
This is maybe different from what you are thinking, but may attain the same results.
For instance you have a problem and it has one correct answer. But, in order to get that answer you need to do some intervening work or thought. So, you divide one problem into two. The first part is necessary but not sufficient for the complete answer which comes in part two.
For example:
Where is the tallest mountain on Earth is:
Continent:
Mountain Range:
Country:
This system is based on the fact that the smarter kids (kids that have learned about the subject matter) would be able to answer all parts, whereas less smart kids might only get some of the answers.
Now that I look at this it only seems to suggest that you put lots of detailed questions on the test... 
Maybe with a mathematics test:
Many proofs and problems need several steps to solve correctly. So, have two steps in the problem required. The first step should be easier than the final step.
Okay, different idea. You could have the students rank the possible answers.
First choice, second choice etc. Then, score them accordingly. This way if you had narrowed it down to two possibilities, then you would get some credit for almost getting the answer.
|
|
IP Logged |
They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. <Ben Franklin>
|
|
|
towr
wu::riddles Moderator
Uberpuzzler
Some people are average, some are just mean.
Gender:
Posts: 13730
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #2 on: Mar 1st, 2004, 12:51am » |
Quote Modify
|
on Feb 29th, 2004, 5:54pm, Amir Michail wrote:| However, this is a real pain and I can't read the students' minds so my idea of reasonable may not match their idea of reasonable. |
|
There are some problems with having students 'decide' what's reasonable. It depends a bit on what kind of questions you ask, but 'common knowledge' (or rather common misconception) might polute the answers.
For instance suppose you ask what might happen right after someone were to be dropped out of an airlock of a space station (without protection)
A) he puffs up and explodes within seconds
B) he freezes within minutes
C) he remains conscious for about 30 seconds then dies of asphixiation
A) would probably be a pretty popular answer, as that's what happens in most sci-fi movies. But it's a pretty ludicrous answer scientifically. C) of course is the right answer, and anyone who saw 2001 might guess that. And after death a body would eventually freeze (but not within minutes), so B) is at least somewhat reasonable, unlike A)
If they are to get any points for a reasonable, but incorrect, answer, it should be at least one that's 'scientifically' reasonable. And not reasonable by virtue of being a common misconception.
I suppose it's not a bad idea to have students rank the answer, as Sir Col proposes.
Or perhaps allow them to select two answers if they're not sure, and give them half marks if the correct one is among them.
Or have them assign probabilities to each answer (which should sum to 1), and give them the fraction (equal to the probability they attributed to the correct answer) of the maximum score
|
|
IP Logged |
Wikipedia, Google, Mathworld, Integer sequence DB
|
|
|
Amir Michail
Guest
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #3 on: Mar 1st, 2004, 1:59am » |
Quote Modify
Remove
|
I like those suggestions of allowing students to pick multiple answers or rank answers.
As for the reasonable issue, note that I would check to see what the top students think is reasonable -- not what all students think.
So their answers should be plausible.
Also, sometimes an answer is reasonable because I did not explain a concept as clearly as I could have. It is actually quite common for people to misunderstand in unexpected ways, so I would not be able to predict what would be reasonable given their misunderstandings.
|
|
IP Logged |
|
|
|
towr
wu::riddles Moderator
Uberpuzzler
Some people are average, some are just mean.
Gender:
Posts: 13730
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #4 on: Mar 1st, 2004, 2:36am » |
Quote Modify
|
You could just allow them to argue why their answer is reasonable after you've graded the test and returned it to them. At least, I assume you can still change the grade afterwards if you deem it necessary.
Or you could discuss the test after they made it, but before you graded it (f.i. the next day), and discuss the questions and answers then. This will again give them a chance to (collectively) argue why some answer should be considered reasonable, if not correct.
|
|
IP Logged |
Wikipedia, Google, Mathworld, Integer sequence DB
|
|
|
Amir Michail
Guest
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #5 on: Mar 1st, 2004, 2:52am » |
Quote Modify
Remove
|
Those suggestions are good but they would explicitly show that there is either something wrong with my lectures or with my test!
The approach I recommended would take into account similar factors but implicitly -- without admitting fault on my part in such an obvious way 
|
|
IP Logged |
|
|
|
towr
wu::riddles Moderator
Uberpuzzler
Some people are average, some are just mean.
Gender:
Posts: 13730
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #6 on: Mar 1st, 2004, 3:42am » |
Quote Modify
|
on Mar 1st, 2004, 2:52am, Amir Michail wrote:| Those suggestions are good but they would explicitly show that there is either something wrong with my lectures or with my test! |
|
At most it shows that you don't consider yourself infallible which, I assume, you aren't.
Besides, discussing a test is allways a good idea imo. After a test people will have a better idea of what they didn't understand, and this will give you a last chance to help them do so before you close the topic.
|
|
IP Logged |
Wikipedia, Google, Mathworld, Integer sequence DB
|
|
|
Amir Michail
Guest
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #7 on: Mar 1st, 2004, 4:32am » |
Quote Modify
Remove
|
I often put questions where a deep understanding gives you the answer but a more superficial understanding does not.
In this case, it would still be good to give partial marks for the superficial understanding.
The students of course with the superficial understanding have no idea that there is anything wrong with their answer.
|
|
IP Logged |
|
|
|
rmsgrey
Uberpuzzler
    
Gender:
Posts: 2874
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #8 on: Mar 1st, 2004, 8:43am » |
Quote Modify
|
I don't think there's any sensible way of awarding partial credit to "good" wrong answers automatically based solely on the pattern of answers given - how "good" a wrong answer is should depend on the reasoning that leads to the answer rather than how popular the answer is - as with the adrift in space situation, even students who generally do well in related questions are likely to fall for popular howlers rather than errors in working or minor misconceptions about the theory.
In the UK, there're a family of multiple choice maths papers used as a preliminary round in selecting candidates for the UK's International Maths Olympiad team. In looking at the question papers afterwards, there are often a number of wrong choices which are clearly derived from a simple arithmetical error or similar mistake. There are also questions where the choices are just a more or less uniform set of values around the right sort of range, with no obviously more forgivable wrong answer. On the former questions, I'd like to see partial marks for the plausible errors; for the latter, nothing for wrong answers.
On the other hand, I'd definitely endorse Towr's suggestion of letting students give multiple answers for reduced credit (which wouldn't substantially increase the workload if answers are being entered into a computer - rather than entering a single character per question, enter up to the number of choices per question). The expected number of marks per student would remain the same, but it would be much easier to spot those who just guess randomly because they would have a much higher percentage of 0 mark questions - a student who can narrow every question down to 2 possibilities would still score half marks, the same as the student who knows the right answer with probability p=(n-2)/2(n-1) and guesses a single answer otherwise, but you have a good estimator of how many questions the latter student guessed on by looking at the number he got wrong. What you then do with that information is another question entirely.
|
|
IP Logged |
|
|
|
rmsgrey
Uberpuzzler
Gender:
Posts: 2874
|
|
Re: Partial Marks on Multiple Choice Tests
« Reply #9 on: Mar 1st, 2004, 8:53am » |
Quote Modify
|
on Mar 1st, 2004, 4:32am, Amir Michail wrote:I often put questions where a deep understanding gives you the answer but a more superficial understanding does not.
In this case, it would still be good to give partial marks for the superficial understanding.
The students of course with the superficial understanding have no idea that there is anything wrong with their answer.
|
|
It sounds like you have some idea where the superficial understanding comes up - in which case, you should be able to assign partial credit to the "wrong" answers you put in to seem reasonable. I assume that the "reasonable" wrong answers are offered because they are plausible even though they're wrong - so you should be aware of that as you set them. Unless you're willing to get feedback afterwards on what your students thought were reasonable answers, which, as has been mentioned, is probably a good way to cover any gaps or misconceptions in their understanding, then assigning partial credit in advance is probably your fairest option.
It sounds like a good idea to look at some sort of statistical breakdown of the wrong answers you get anyway but I think you still need human judgement to decide which mistakes are reasonable ones to have made - particularly in situations where very few of your good students get questions wrong - which reduces your sample size rather.
|
|
IP Logged |
|
|
|
|
RIDDLES SITE
WRITE MATH!
Home 
Help 
Search 
Members 
Login 
Register
Reply
Notify of replies
Send Topic
Print