Geoff Tennant - Promoting access to mathematics for all
My Blog

20/12/15: is there a problem with the question?

As described in this blog post, the maths team at work hosted a large conference somewhat over a year ago now.  One of the results of this conference is a publication which is currently with the publishers, so one of the things I've been doing recently is checking the proofs, which of course means rereading our work, which feels a bit like meeting back up with old friends.

One of the chapters I co-wrote with a colleague is about achievement in mathematics in East Africa.  Finding various statistics is straightforward enough, the issue then is, on what are these statistics based?  If the statistics are about achievement on various examinations, then ideally the data is cross-referenced with the questions themselves.  Alas, this is not always possible, very often statistics on achievement are produced without giving the test items on which the statistics are based.  This is somewhat frustrating from an analysis point of view, as it feels very much as if we're working with one hand tied behind our back.  Saying that a youngster scored 60% on a test is meaningless, I would suggest, unless we know such things as what was on the test, how it was marked, and how the test items match with what has been taught.

But sometimes it is possible to cross-reference test items with the scores given.  There are instances from the UK, particularly in A level decision mathematics, of questions which are quite simply impossible to do.  To my mind, the solution here is quite straightforward - have an additional examiner involved whose job it is, at a late stage of draft, not to review the questions but actually to do them before they're finalised.  Also, I've had reason recently to watch a video of a boy aged about 8 answering a question with the starting point that the sine of the angle is 4/3.  As anybody who understands trigonometry will know, this is quite impossible, the sine of an angle can never be greater than 1.  So, a video which is supposed to be portraying a young mathematical genius is actually, apparently, showing a boy who has memorised various routines and is misapplying them with no underlying understanding as to what he is doing.  I'm happy to send the YouTube link privately on request but not happy to give it here!

But there's a problem here.  One of the very attractive features of East African culture is the respect in which older people are held - so being referred to as babu (man of grandfather age) should make me happy, the fact that it doesn't is a sign on my part of lack of acculturation.  But what this can mean is that in this case, clear mistakes in the question are swept under the carpet, what experienced setters of questions do cannot itself be called into question.  So in an examination situation, a youngster who states that the question is impossible is likely to score 0 marks.  An attempt to make it a possible question, eg. by starting instead from the sine of the angle being 3/4, is not likely to do much better.

Various other problems came up in our analysis.  We found an algebra question which, if taken literally, gave rise to 27 year old teenagers, with old age starting at 47.  47!  That's not old!  The examiners report stated that the question had been done badly, with advice on the teaching of algebra skills and interpreting problems stated in words.  There was no recognition at all that the question itself was flawed, with the clear possibility that youngsters able to solve that kind of problem assumed that they were making a mistake when in fact the problem was with the question.

And then there were more subtle issues.  One is the stating of a method to be used in solving a problem.  So, for example, in pre-calculator days, if finding the average weight of 10 sacks of wheat which are all just over 100 kg, a standard method would be to subtract 100, find the mean of the remainders, and then add 100 back on - ie. using an assumed mean.  But the use of calculators has made such methods redundant, and can make the question more difficult, particularly if the assumed mean gives rise to negative differences.  The use of the calculator in the mathematics classroom is an interesting one which I'll not go into now, but I would question the continuation of teaching methods which are helpful when not using a calculator but become redundant when there is one to hand.  I would include within that logarithms as a calculation tool - transforming multiplications into additions, etc. - if not related to the underlying calculus which is why these methods work.

'Old habits die hard' is a truism which reaches across many aspects of human activity, as we continue to do things which used to make sense but no longer do.  Certainly within mathematics there is the need to look again at what we are teaching, and how we are assessing what we are teaching, to ensure a match with what is going to be useful and applicable to youngsters entering adult life.  If we want our subject to be relevant, then we don't end up with 27 year old teenagers.  Continuing to teach things because we learnt them at school is not, in itself, a reason for teaching them in this generation.

One of the reasons I'm thinking about questioning the question is because I've been following the current series of the 'Apprentice', albeit one day late each episode, with the finals due this evening.  Have to say, the premise of the series is absolutely absurd, conducting a knock out competition over 12 weeks doing a wide range of tasks to find a person to receive an investment in their business (and previously to be offered a job) is massively inefficient and based on the assumption that we should all be good at absolutely everything, whereas in a team we can bring a range of enthusiasms and skills to bear.  But it does make amusing television, with yet another group of people making inflated claims about themselves before being made to look like idiots.  Surely, apart from series 1, candidates in one series have watched the programme themselves?  Or do they not apply what they see in other people to themselves?

But this year, after the most astonishing success of the candidates in selling millions of pounds worth of property in just over a day (I assume that this was genuine), both teams failed to get any sales when they were required to come up with a new healthy snack to pitch to major retailers.  That the remaining candidates are highly competent had been demonstrated the previous week.  As with the maths questions considered above, nobody seems to have asked the question - was there a problem with the task?  Is it reasonable to imagine that a small group of people can, within a very short period of time and no background in food design, come up with an idea to rival huge great companies?  Again, power relations are such that it is the candidates who are made to feel like failures, but it would have been so brilliant had one of them had the gumption to say, "Look, Lord Sugar, this task was a nonsense, you're fired."

Showing respect to elders, and valuing the contribution that older people can make in society, is very valuable.  But showing respect to somebody does not always mean agreeing with what they say or do, any more than experience and qualifications automatically mean that what we say and do are correct.  Humility, openness and preparedness to concede that we may be wrong are needed from all of us, irrespective of age and experience, I would want to suggest.

2 Comments to 20/12/15: is there a problem with the question?:

Comments RSS
carolyn on 20 December 2015 06:20
Hi Geoff, a few things spring to mind! I remember helping a child with maths homework and the question stated, If a girl drinks 2.5litres of lemonade each day, how much does she drink in a week? Never mind the maths, my mind was occupied with her health! Having spent recent years working very closely with individual children, I conclude that nothing can replace observing and listening to a child's reasoning when they are completing assessments. Points are scored and lost in a random fashion! As for the apprentice, I don't think that particular task was any'harder' than the others. They are all challenges set to see how people respond, not necessarily to check for an end result. And finally...I thought of you the other day when I returned some faulty boots to a shop. They could not replace the boots exactly but offered a similar pair costing £120. The original pair had cost £80. Wait for it! The assistant got her calculator to work out the difference! I remained calm and pleasant! I decided I was not prepared to pay £40 so, imagining myself on the apprentice, said, 'We are talking about replacing my boots here, so I am prepared to pay £20 difference. Can we make a deal? ' The girl then took hold of her calculator again and declared she could take 20% off. I waited with baited breath! 'So, that's £16 to pay' , she said. 'Deal! 'I replied. I paid my money and took my nice new boots. At my leisure, I calculated that she had given me a discount of 20% from £120 of course and had been quite right. She had missed the opportunity of making an extra £4 for the company! Immediately thought of you and calculators and real life mathematics!
Reply to comment
Geoff on 20 December 2015 23:03
Great to hear from you, Carolyn, agree that 2.5 litres of lemonade per day is rather a lot, a good example of what appears to be 'real life' mathematics not standing up to examination. And your percentages example reminds me of the story, which I've heard twice totally independently, of the jumper in a sale which was advertised at 20% off. In going to buy 2 jumpers, the customer was given 40% off. The customer then asked to buy 5 jumpres, presumably at 100% off....

Add a Comment

Your Name:
Email Address: (Required)
Make your text bigger, bold, italic and more with HTML tags. We'll show you how.
Post Comment
RSS Follow Become a Fan

Delivered by FeedBurner

Recent Posts

15/7/18: when to use smart phones?
8/7/18: is there some football event on at the moment?
1/7/18: how (not) to give praise
24/6/18: FINALLY got through that pile of fish
25/3/18: an incident on a bus


Jamaica trip 2012
Return to the UK 2017
Tanzania 2013
Tanzania 2014
Tanzania 2015
Tanzania 2016
Tanzania 2017
Transition to Tanzania
UK 2018
powered by

Website Builder provided by  Vistaprint