2.4 Assessing alternative types of activity in economics
Many student activities that are traditionally examined through written reports or essays may alternatively be examined orally in the form of a viva. Potentially, this approach can give a much clearer idea of the depth of students’ understanding. There is no scope for plagiarism, and little scope for regurgitation of material, at least in carefully managed interviews. There are also benefits in terms of development of interpersonal skills and interview technique.
The time costs should not be severe – there is no marking, although the assessor must see each student individually and this is a logistical problem, especially for large groups. One has to think carefully about the questions asked (with different questions to prevent student collusion). Oral examination can be a risky approach, since validation by external review may be complex and there is likely to be some student resistance. Certainly, an assessor will have to write reports on each student’s performance, detailing the questions asked and the basis for assessment.
The approach is definitely worth thinking about however, and should perhaps be tried out on a small scale as a complement to a more traditional assessment such as an exam – say, allocate 30 per cent of the final mark to the oral examination, reducing the requirements of the exam accordingly.
As another example, suppose students are told to attend a 15-minute viva in which they will be examined on the economics of the East Asian financial crisis of 1997. The lecturer has prepared a bank of questions to ask students that relate not only to students’ knowledge, but also to the process they undertook in preparing for the examination. Some questions can be narrow, to test their basic knowledge, whilst others can be broader and more searching, viz.:
- ‘What countries were involved and which were most affected?’
- ‘Do you have data to illustrate your answer?’
- ‘What model would you use to show how exchange rate depreciation, for example, affects the macroeconomy? Talk us through the dynamics.’
- ‘Could such a crisis happen again?’
If well prepared, the oral examination allows the instructor to investigate students’ knowledge, skills and commitment in a way that is often impossible in written unseen examinations. It also requires them to read and research much more extensively. On the negative side, there is the ‘stress factor’ of undergoing live interrogation.
With regard to resource costs, this approach need not involve much additional staff time, since it saves on the laborious marking of written scripts. It is of course, time-consuming to initiate, although future years should benefit as both experience and the assessment bank are built up.
Top Tips 5 Oral examinations
- As economics makes heavy use of graphic analysis, in many cases it is a good idea to give students access to a whiteboard or PC during the oral examination.
- Oral examinations can be an intimidating experience for everyone. One might divide it into two parts, and, in the first part, test students on questions that they have already seen and prepared for – this makes the process more reassuring, less arbitrary and easier to verify. In the second part, students respond to unseen questions.
Testing skills instead of knowledge
One of the problems with unseen exams is that questions are so closely related to the material covered in the course and in the textbook that students tend to memorise and regurgitate without any deep understanding. An alternative approach involves testing students with questions relating to issues or material that is not familiar, but which does require the kind of approach to problem solving that is developed in the module. In this way, the assessor is testing the learning process developed in the course rather than the knowledge provided.
As an example, students are asked to answer ten questions in 40 minutes. Each question is worth 3 marks. The right is reserved to give negative scores for logically wrong answers and bonus points for excellent answers.
Box 7 An example of skills testing
Imagine you are an economics adviser starting an assignment in an unfamiliar society. Please summarise your approach to the following issues in up to three points (the points do not have to be in any order of significance).
Here are four of the ten exemplar questions.
- What reasons would you stress as to why changes in the money wage rate in a remote rural village with a large subsistence sector might not be a good indicator of changes in the real standard of living?
- What variables would you use to assess the degree of segmentation in the urban labour market?
- Why might small borrowers have problems obtaining credit for physical capital investment?
- What determines how much of a household’s savings are held in the form of money?
Presentations are a well-established method of assessment. They help to develop skills required in the workplace, as well as student confidence, oral skills and the use of relevant software. However, it is true that use of presentations in HE is often felt to be inadequate. Students may regurgitate material without properly engaging the audience, and may invest a lot of time in their own presentation at the expense of other work. For example, students seldom prepare for the topics covered by their colleagues’ presentations. Top Tips 6 contains a couple of useful tips to improve the efficacy of presentations.
Top Tips 6 Presentations
- Make the presentation part of the module mark – this creates a real incentive.
- As with all methods of assessment, ensure the student knows the broad criteria on which they will be assessed.
With projects, students are often free to choose the topic, title and methodology to be studied. Projects are useful in developing independence, organisational skills, resourcefulness and a sense of ownership over work, and may induce a deeper level of learning.
On the other hand, they may be unpopular and where the project is an option, take-up may be low. Students may believe that it involves a greater amount of work than a standard module and/or that there will be insufficient supervision. Some tips for improving the take-up rate and usefulness of project work are listed in Top Tips 7.
Top Tips 6 Projects
- Make the project compulsory.
- Allow project work to be done as a group work.
- Require students to submit work to supervisors in stages – this should include a plan of work, a brief literature review with references, and subsequently a first draft.
- Projects may be examined by oral examination as well as written report (the benefits of this were discussed in the section ‘Oral examination’ above).
Consider asking students to prepare a literature review on a given topic. This develops a number of research-type skills, encouraging students to source material, use search engines and be able to assimilate large amounts of material and select the most important.
While students are often expected to review literature as a matter of course (for example, as part of an essay submission), they normally underperform, being overly reliant on key ideas presented to them in core textbooks. This approach encourages them to do it, makes plagiarism more difficult and can be quite popular, since the process of searching and understanding a wider literature makes students feel more involved.
Article reviews involve students presenting in written or oral format a critique of one or more articles. However, this approach can be somewhat demanding for undergraduates.
As an example, we could pose the question whether the UK should join the common European currency or not. Students should submit a comprehensive literature review on this controversial topic. They are expected to source a range of material and arguments relating to this debate, and prepare a report referring to the original sources.
Literature reviews are probably easier to read than other types of written work, which therefore eases the burden of marking.
Objective tests and multiple-choice questions
Box 8 Advantages of multiple-choice questioning
- Quick and easy to mark.
- Useful for large groups.
- High inter-tutor reliability.
- Remedial opportunities – can find out the correct answer.
- Very useful for self-assessment and learning.
- Can test a wide range of knowledge quickly.
- Web-based texts may include a bank of such questions.
- Marking may be done by computers
Table 3 Types of multiple-choice question
Box 9 Disadvantages of multiple-choice questioning
- It is difficult to design questions.
- It is difficult to test the depth of learning in economics.
- Multiple-choice questioning does not develop or test presentational skills.
- Text-based answers can be difficult to mark by computer.
Given the disadvantages listed in Box 9, these types of question should not be used as the sole means of assessing student performance in a given module.
Computer-aided assessment is discussed in another chapter in this handbook. But for now, consider that computer software such as Question Mark can be used to format multiple-choice questions and mark and analyse the results. It may be time-consuming to set, but marking is very fast and accurate. However, as with other objective tests, it is difficult to test the depth of learning.
Portfolios in economics
A portfolio is a collection of work commonly used in the assessment of vocational training (such as industrial placement).
Box 10 Advantages of including portfolio assessment
- Portfolios can be useful for students with work experience to claim credit for tasks done at the workplace and to tailor work tasks in a way that promotes learning and development. They can also be useful as a basis for interviews and promotion.
- A portfolio is usually a collection of work developed over time and may help the student think about what is being achieved in an ongoing way.
- It is felt that students have a degree of control over what goes into the portfolio.
- As evidence of a student’s achievements, a portfolio can foster confidence and a sense of achievement.
- The process can foster dialogue with tutors and assessors.
Portfolios may not be an ideal way of testing knowledge and the analytical, conceptual and problem-solving skills required in economics. Also it does not fit readily within a modular structure. However, there is clear potential for portfolios in the development of transferable skills – a requirement of degree programmes can be the demonstration of various activities and skills, such as leadership, co-ordination and research. This could easily be embodied within a portfolio submitted at the final level.
As an ongoing means of assessment, portfolios would be a useful device for indicating the importance of transferable skills, and requiring students, rather than lecturers, to find ways of developing these skills. Traditionally, students have used extra-curricular activities to demonstrate their range of personal and interpersonal skills, which are summarised and publicised in curriculum vitae.
Box 11: Issues concerning portfolio assessment
- Portfolios do not lend themselves easily to being graded – the student is usually required to demonstrate that they have completed a range of tasks and activities to a minimum standard.
- Portfolios can involve tedious paperwork – students must give a large amount of paper evidence of their achievements.
For much more detail on portfolios and a case study, see Baume (2001) and Coates (2000).
Other aspects of assessment
There are many other methods of assessment besides those discussed above – poster sessions, open-book examinations, seen-examinations, profiles, single-essay examinations and various combinations of the above. For more detail, consult section 4.