How long a test do you need?

During my first few years teaching ESL at university, my oral exams were scheduled to be five minutes long but always took longer.  When a student had trouble with one question, I would give a different one to offer every possible chance for the student to show that they knew some English.  A coworker, Tom, (and I am talking about events occurring six or seven years ago so, if Tom is reading this, I approved of and admired your methods, even if events were not quite as I remember them) would offer a few questions and if a student had trouble, Tom would offer one more.  If the student was unable to answer it, Tom thanked the student and sent him/her on their way.  He was always done testing faster than I was.

How long do you need to judge a person's English ability?  In my case, I have shortened my tests somewhat but still keep them longer than I really think necessary to let the student feel it was a real test; long enough to be taken seriously.

So, I now think a short test is sufficient.  But is there a minimum length?  Is there a length beyond which you are clearly wasting your time.  And counter-intuitively, if a test goes beyond a certain length does it actually hinder the evaluation process?

Cognitive Daily has had a series of posts on similar subjects lately.

[T]hin-slicing studies ... the idea that a few brief exposures to an individual can give just as accurate an impression of key traits as much more extended interactions. For judging sexual preference in men, a 10-second exposure to pictures of faces isn't any better than a 50-millisecond exposure. For evaluating teaching ability, a few 10-second movie clips are nearly as good as an entire semester in class.

The posts linked above suggest that tests can be much shorter than what feels seemly.

I am live-blogging my research, I guess.  At this point, I have not found any definitive research but here are some possibly relevant links.

This test seems well-thought out and is 3 minutes long with the teacher judging up to four students during those 3 minutes.

Wigglesworth wrote about An investigation of planning time and proficiency level on oral test discourse for Language Testing in 1997.

The inclusion of planning time in semi-direct oral interaction tests adds consider ably to the overall length of the test, and it is important to be clear that the increase in length is justified by the language outcome. Previous research has shown that the effect of planning time in second language can differentially influence the resultant discourse with planned discourse eliciting more complex language on a range of measures.

Wigglesworth seems (I only read the free abstract) to be working to define how long a test needs to be and is focusing on how much time should be allotted to a student between giving a question and demanding an answer.  This 'planning time' appears to be not very important for low-proficiency students.  Again, this means that giving the low-level student time to think about the question does not affect much the quality of the answer.

Other articles found didn't seem to apply and I am too lazy on my Winter break to play with search terms to find more.

I am concluding by saying that a properly prepared test for low-proficiency students can be quite short - which is exactly what I had hoped and already thought.  I should stop now before further research throws my conclusion into doubt!

----
Of possible interest regarding test-takers and test length. Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions  How long a test is too long and is the Soo-Neung (Korean University Entrance Exam) too long?

Meta comments on exams

I've just finished marking midterm exams. I'm not satisfied with my exam making skills, but student's marks seemed well spread out which is the main thing. I think I'm actually checking the things I think I am but it's not always clear.

In the first year book, there is a conversation with the lines:

A3: What's your student number?

B3: It is 20070000.

Oh, the threes mean these are the third line each character speaks.

On the exam, I had the same question and many times got the exact same answer (20070000). As almost all the students started at university in 2009, the above number just has to be wrong. Also, no one gets a null number (....0000).

Did the students think the conversations, rather than models to be adjusted to fit, were authoritative, historic quotes to be memorized? Mister Lee and Doctor Choi (the authors of the first year text) have done their best, but they are no Shakespeares.

Possibly more interesting to my readers are the extra comments written on a few papers and addressed to me, rather than focussed on the exam itself (click to embiggen).

This student was not a student in the Tuesday class. I don't think he/she was a math student but I am not sure how else to explain the 'pi' signs on the face (yes, I know, the Korean letter for "YU" looks both like a Greek 'pi' and tears falling.

The big red loop, some of which can be seen means that not one question was answered on this page and the total for this page is zero.

I think the 'T's at the end of the second line are also representative of tears.

I'm not sure how I specifically helped this student but the comment is pleasant. With that score, it seems more apple-polishing than anything else, though.

This student loves me -and was paying more attention to me than other commenting students, as evidenced by the above-50% mark.

Another student loves me, although she (I blanked out the name, but it was a feminine name) was unable to look at the upper left, where I wrote my own name, for spelling purposes. On the bottom-right is the beginning of another long loop: this student did not answer the first few questions or the page at all.

----------

Oh, I want to tell you about a great prank I played on my students. On Monday and Tuesday, I had second year exams and I had some blank exams left over so on Thursday and Friday, I would go into class early, write some notes on the board and left, absent-mindedly leaving the (wrong) exams on my desk. Then, from the corner of a window I would watch a few brave or desperate students crowd around the desk, checking out the exams. In two or three classes the students didn't notice or were too honest or apathetic and didn't move from their desks. Still, good times.