Computer-Aided Assessment in first year Microeconomics

Over 400 students take the introductory Microeconomics course at the University of Aberdeen. That makes the marking load quite substantial, so the Department was looking at the ways of addressing this issue in order to free the resources and reduce the marking cost. Computer-aided assessment (CAA) seemed to offer a solution to it so, together with the Learning Technology Unit, two years ago I became involved in a trial of its use.

It involved the creation of two Question Mark (QM) tests - one for an initial session designed to familiarise students with the technology and the other for the actual assessment, which could be only attempted once. The students were expected to use Microsoft Excel while answering the questions. In addition the students were also offered two workshops to help them complete the initial session and some additional assistance for those that encountering problems. Evaluation of the project was carried out through semi-structured interviews with some students during the workshops and through a Web-based questionnaire after the assessment. To encourage student participation in the evaluation scheme, they were required to fill in the questionnaire in order to receive the results of their CAA test.

Several problems were encountered with both the QM tests and questionnaire. Some students lacked basic ICT skills and had difficulties using Excel or switching between applications. More disturbingly when some students tried to submit their tests unidentifiable error messages were displayed and we had to ask them to re-submit. The biggest problem we have come across was multiple test submission. There were clear instructions forbidding this but it still occurred, making it difficult to identify the marks that would be recorded for continuous assessment purposes. It has been suggested that in the future it would be easier to put all the test results into QM Reporter and sort them out that way. Another possible deterrent would be to limit the availability of the test to no longer than a week. So, although there were problems, with the help of Computer Centre personnel, we were able to overcome them.

The results of the evaluation showed that the majority of students were happy to use CAA, and 38% of them thought that more CAA should be used in course work. Although the majority of students did not have problems understanding the instructions (only 16% of them found QM instructions difficult to understand and 11% had problems with Excel), their answers in the interviews suggest a need for clearer instructions. Workshops were found to be essential for the success of the CAA.

The lessons learnt from this experience allowed us to further improve the experience for students in the following year. Clearer instructions were provided, and later mounting of the real case study dramatically reduced multiple submissions. The project was then modified and extended and introduced into second year macroeconomics.

Glossary

Question Mark - an off-the-shelf software package that allows you to design and implement computer-based tests and questionnaires.

QM Reporter - Report Managing software included as part of the Question Mark package.

ICT - Information and Communication Technology