Friday, December 18, 2009

How long a test do you need?

During my first few years teaching ESL at university, my oral exams were scheduled to be five minutes long but always took longer.  When a student had trouble with one question, I would give a different one to offer every possible chance for the student to show that they knew some English.  A coworker, Tom, (and I am talking about events occurring six or seven years ago so, if Tom is reading this, I approved of and admired your methods, even if events were not quite as I remember them) would offer a few questions and if a student had trouble, Tom would offer one more.  If the student was unable to answer it, Tom thanked the student and sent him/her on their way.  He was always done testing faster than I was.

How long do you need to judge a person's English ability?  In my case, I have shortened my tests somewhat but still keep them longer than I really think necessary to let the student feel it was a real test; long enough to be taken seriously.

So, I now think a short test is sufficient.  But is there a minimum length?  Is there a length beyond which you are clearly wasting your time.  And counter-intuitively, if a test goes beyond a certain length does it actually hinder the evaluation process?

Cognitive Daily has had a series of posts on similar subjects lately.
[T]hin-slicing studies ... the idea that a few brief exposures to an individual can give just as accurate an impression of key traits as much more extended interactions. For judging sexual preference in men, a 10-second exposure to pictures of faces isn't any better than a 50-millisecond exposure. For evaluating teaching ability, a few 10-second movie clips are nearly as good as an entire semester in class.

The posts linked above suggest that tests can be much shorter than what feels seemly.

I am live-blogging my research, I guess.  At this point, I have not found any definitive research but here are some possibly relevant links.

This test seems well-thought out and is 3 minutes long with the teacher judging up to four students during those 3 minutes.

Wigglesworth wrote about An investigation of planning time and proficiency level on oral test discourse for Language Testing in 1997.
The inclusion of planning time in semi-direct oral interaction tests adds consider ably to the overall length of the test, and it is important to be clear that the increase in length is justified by the language outcome. Previous research has shown that the effect of planning time in second language can differentially influence the resultant discourse with planned discourse eliciting more complex language on a range of measures.
Wigglesworth seems (I only read the free abstract) to be working to define how long a test needs to be and is focusing on how much time should be allotted to a student between giving a question and demanding an answer.  This 'planning time' appears to be not very important for low-proficiency students.  Again, this means that giving the low-level student time to think about the question does not affect much the quality of the answer.

Other articles found didn't seem to apply and I am too lazy on my Winter break to play with search terms to find more.

I am concluding by saying that a properly prepared test for low-proficiency students can be quite short - which is exactly what I had hoped and already thought.  I should stop now before further research throws my conclusion into doubt!

Of possible interest regarding test-takers and test length. Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions  How long a test is too long and is the Soo-Neung (Korean University Entrance Exam) too long?

No comments: