1.2 Evaluating learning objectives
You are here
‘…assessors will wish to judge the appropriateness and adequacy of provision (of assessment strategies) against the stated course aims and objectives’. (HEFCE Circular 3/93)
An exercise in identifying learning objectives and assessing ‘your’ assessment procedures
Take a sheet of paper and write down what students would be expected to obtain from the module. What are the anticipated learning outcomes, and what is the relative importance of each? Having done that, consider the extent to which existing modes of assessment promote the desired learning outcomes. Think carefully about what types of skill (and knowledge) are being testing in each assessment, and the areas of student activity that are promoted by its nature and design. On the sheet, place a tick (cross) next to those learning outcomes that it is believed are (not) adequately addressed. Finally, take a look at the list of learning outcomes in Table 1, and consider whether the methods of assessment used are consistent with these.
Table 1 Learning objectives
|Skill area||Do your assessment methods promote this skill?
|Knowledge of economic principles|
Problem solving under pressure, breadth and depth of understanding of complex problems
|Written communication skills
Writing well-presented and structured reports and essays
Ability to work with others, demonstrating management and leadership skills
Skill in using basic computer packages – word processing, spreadsheets, PowerPoint; using the web to research information
Autonomy, self-reliance, self-motivation
|Flexibility and resourcefulness
Ability to respond to unusual and unpredictable circumstances
|Strategic thinking skills
Ability to determine own strategy and direction, self-knowledge and self-monitoring of effectiveness
Finding out, using libraries, finding sources of information
Managing time and deadlines – organising material
Hopefully, this has been a useful exercise.3 It gives instructors an opportunity to reflect on learning methods in terms of goals and effectiveness. The purpose is to show any discrepancies between what is and should be achieved in learning delivery, and the kinds of activity and learning process promoted by the assessment procedures employed. No doubt, individual lists of goals look very different from Table 2, and this should raise some interesting questions. The same table emphasises various transferable skills and this point is taken up later in section 1.
Evaluating a traditional approach to assessment
Brown and Glasner (1999) have found that 90 per cent of a typical British degree depends upon unseen time-constrained written examinations, and tutor-marked essays and reports. The typical approach to assessment in economics may well look a little like this.
Students are expected to prepare answers to a series of ‘shortish’ conceptual questions that are subsequently discussed in tutorials in an informal way under the leadership of the tutor, with the implicit expectation that the tutor provides model answers.4 Midway through the module, students submit an essay from a list of broad questions. The majority of the final mark comes from an unseen examination, usually taken at the end of the module. Students are normally asked to answer three or four questions of a fairly broad nature but closely related to the material of the lecture course and the principal textbook. Typically, answers are in essay form, each of three to four pages in length.
How successful is this approach in promoting learning outcomes?
The traditional approach promotes a number of learning outcomes. The unseen examination requires students to respond to pressure and time constraints. They develop strategic capacity in respect of the topics studied and the questions answered, and selectivity in the material presented.
In other respects, the traditional approach fares less well. Box 1 lists the broad learning goals that are not adequately promoted by this means of assessment – readers may well disagree.
The list is long and reflects a growing and widespread criticism of assessment methods in HE. A broad criticism of assessment practice is that it is too narrow in its goals. Other authors argue that current assessment practice, rather than promoting learning, is in fact injurious to it (see, for example, Boud, 1992; Atkins et al., 1993; Erwin and Knight, 1995).
A core objective of HE is the development of analytical or ‘thinking’ skills. It is expected that graduates should be able to deal with complex problems in a logical manner, and be able to communicate and present solutions in a variety of ways. However, there is increasing and disturbing evidence that students do not engage in the deep learning process that promotes these kinds of skill, engaging in surface learning and regurgitation of memorised material in a disorderly way (Entwistle, 1981; Gibbs, 1992; Boud, 1992).
Whilst students are encouraged to be self-reliant and self-motivating, assessments rely primarily on appreciation of core material available in key textbooks. However, there are inadequate incentives for initiative in identifying alternative sources of material and developing research-type skills. The more entrepreneurial skills, whilst very difficult to promote, are not addressed. These require students to have much greater control over goals than is usually the case, and a greater flexibility in the method of assessment.
The final area of concern relates to students’ motivation (although this is not a learning outcome). It is clear that the motivation of many students is very narrowly defined in terms of exam performance, whilst there is often little evidence of an appreciation or interest in what they are learning or why they are learning it. Educators must consider the relationship between assessment, the issue of motivation and interest in learning. Any approach that emphasises a wider range of skills and actively engages students in different kinds of activity is likely to generate greater motivation.
Box 1 The consistency of the traditional approach to assessment and identified learning goals
Strengths of a traditional approach to assessment
- Strategic thinking.
- Responding to pressure and time constraints.
- Encouraging a broad knowledge base.
Weaknesses of a traditional approach to assessment
- Thinking skills – identifying and solving complex problems.
- Presentation and oral skills – presenting complex problems and solutions orally in a comprehensible way, confidence building, use of PowerPoint, responding to unknown questions orally.
- Interpersonal skills – communicating with colleagues, negotiating, developing leadership skills and managing interpersonal problems.
- Research skills – finding unknown sources of information, research on the web and using libraries.
- Entrepreneurial skills – identifying personal goals and the means to achieve them.
- IT – basic skills, such as familiarity with core software and use of the internet.
- Self-motivation and assessment – understanding personal motivation and objectives, and assessing progress achieved.
Transferable skills and government policy
Underlying the debate on the effectiveness of assessment strategies is a more fundamental debate on the purpose of HE. Brown et al. (1994) make the following observation:
There is increasing acceptance that it [assessment] is at least in part to do with preparation for later life and work beyond academia. This recognition has brought with it a gathering momentum for a shift in emphasis from the acquisition of knowledge to the acquisition of skills, from product to process, from grading to competence.
This shift in emphasis towards transferable skills is strongly endorsed by the government and HEFCE. It is motivated in part by surveys of employer dissatisfaction with graduates’ skills, particularly regarding negotiation, decision making and leadership.
Pressure on instructors to diversify learning aims and assessment procedures also comes from students themselves in a competitive labour market. The increasing number of students entering HE implies a greater heterogeneity of backgrounds, student objectives and modes of participation (official and unofficial). To demonstrate students’ abilities, and develop their interests, educators are now obliged to offer richer and increasingly diverse modules.
These issues are relevant to the teaching and learning of economics. It is clear that the large majority of students taking economics modules do not go on to become practising economists – future careers are in finance-related professions and, in particular, accountancy. Arguably, these students have less demand for knowledge of economics and its methodology than for work-related skills.
The issues here raise radical questions about the respective roles of secondary and tertiary education and the structure of educational institutions, departments and degree programmes. What is clear, however, is that instructors must diversify their assessment procedures, so this chapter is designed to aid instructors with some ideas and tips on ways to diversify assessment practice.
Readers who are only interested in practical ideas may wish to skip section 1.3 and go directly to the main content (section 2).