This chapter will outline some assessment and feedback activities that could be used to help overcome some of the problems identified in Chapter 2. It will include two detailed case studies of innovations that were introduced onto an economics degree programme.
Some possible methods that could be used to provide prompt feedback include:
Instead of reading all the assignments before providing any feedback, the lecturer could read through a sample of the assignments and report back on some of the common weaknesses as soon as possible. The weaknesses could be communicated to the students in a number of ways:
As previously discussed, imperfect feedback received quickly can often be more effective than detailed feedback provided much later.
Ideally the hand-in date and time for the coursework should be at the start of a lecture. If the work has to be handed in to a central point then the hand-in date could be set on the same day as the lecture. If possible, the hand-in time should be set an hour before the lecture begins. Without the time constraint there is a danger that a number of students might miss the lecture because they are busy completing the coursework! At the start of class the students should be provided with a coloured sheet with some numbered points that have been previously prepared. These numbered points should include a brief explanation of the likely mistakes you anticipate the students will have made on their assignments and/or some of the key features or characteristics of a good answer. With only limited teaching experience, lecturers can soon predict the sort of mistakes the students will make. You could provide examples of useful diagrams or mathematics you would expect to see in an excellent answer. Typically about 15 numbered points have been included when this approach has been used on a level one microeconomics module. Some examples are shown below:
Do not simply start the lecture by explaining each point on the sheet. Instead, let the students read through the list for five minutes and anticipate that the room will go very quiet as they realise the sort of issues they should have included and the mistakes they have made! After five minutes of quiet reading time, explain some of the points on the handout to the students. The amount of time and number of points you want to explain is up to the individual but it is probably best not to spend more than 10–15 minutes on this activity. When you take the work away and mark it you can include statements such as ‘please see the following points on the handout: 3, 7, and 8’ rather than having to write out the same comment over and over again on the assignment. The written comments that you do make can be more personalised to each piece of work.
One potential issue with this method of feedback is how you deal with students who hand in work late with an agreed extension. For this reason, when used in the microeconomics module the session had to be delayed for one week as a couple of students had an agreed extension. However when surveyed at the end of the academic year, 85% of the economics students still found this to be the most effective piece of feedback that they had received.
One possible method is to not release marks on the VLE until the students have collected their coursework from the seminar/workshop. The lecturer would take a marks list to the workshop and return the coursework without a mark. The students are then given some time to try and work out their mark from the feedback. They could also be encouraged to compare the feedback on their work with that of their peers in order to help them with the process. A grade incentive could also be used to encourage the students to take the activity seriously. For example they could be awarded the higher of the two marks if they self-assessed within 5% of the lecturer’s assessment.
What methods could be used to encourage students to engage with assessment criteria both before they start studying for the coursework and while they are researching/writing the coursework? Two possible activities will be outlined in some detail.
The first activity is a marking exercise which could help students to engage with the assessment criteria before they start their assignments. A detailed case study is included which outlines and evaluates how the exercise was implemented onto an economics degree programme. The second activity is the use of self-assessment sheets. These could be used as a way of encouraging students to engage with the assessment criteria while actually writing their assignments. Some discussion of the different ways of introducing these sheets is included as well as some discussion of the potential drawbacks and how these might be avoided.
As previously discussed in Chapter 2, students may not respond to feedback because they do not understand the assessment criteria. One approach to this problem is to provide more detailed written guidance. However given the difficulty that many students find with the precise meaning of much of the language, this response on its own will probably meet with limited success. In order to complement the written guidance, sessions could be arranged that actively engage the students with the assessment criteria. One such approach is a marking exercise. The rationale behind this type of initiative is that students will gain a better understanding of terms such as ‘analysis’ and ‘evaluation’ when they see examples in the context of a specific economics assessment. Research by Rust et al. (2003) found that these sessions had a statistically significant impact on the achievement of business students in subsequent coursework. The following case outlines how this type of activity was introduced into a mandatory first year skills module entitled ‘Developing Academic and Professional Skills’. This is a 10 credit module on the economics single honours degree programme at Coventry University which runs in the autumn semester/term with approximately 150 students. It operates with a one-hour lecture per week plus a one-hour workshop per week. There are approximately 25 students in each workshop. The module aims to provide students with a range of skills which will enable them to become effective learners in higher education and enhance their chances of obtaining graduate employment.
It is difficult to assess the benefits quantitatively as this requires a control group which is prevented because of ethical concerns. In order to get a more intuitive sense of the effectiveness of the activity, a series of student focus groups were carried out. Comments about assessment criteria included:
‘We are often given a marks scheme but it is written in lecturer’s language.’
‘It is difficult to apply what was said in the assessment criteria as the language is very general and difficult to apply to a particular assignment.’
‘I was unsure of the precise meaning of comments such as – you need to go into more detail with your economics analysis.’
Comments about the marking exercise included:
‘I found reading examples of other students work on similar topics far more useful than being given any assessment criteria or marking sheets.’
‘Different lecturers weighted different aspects of the criteria they provided differently. Without the examples we can’t tell exactly what they want.’
Overall the feedback was very positive with students requesting that similar activities should be introduced into other economics modules on the course.
The marking exercise could be implemented in a number of different ways.
The decision to get the students to mark the work outside of class was taken in this case because the timetabled slot for the workshop was only 50 minutes long. It was thought that this would not provide enough time for the students both to mark a 1500 word piece of work and discuss the comments with their peers. However this does create a problem as there are always some students who attend but have not completed the work. One possibility is to group them together on the basis of those who have and those who have not completed the task. If a longer slot (minimum of 90 minutes) can be timetabled the activity might work more effectively if the marking is completed in class. The students could be given 25 minutes to mark each assignment before discussing their thoughts and reporting back their agreed grades and feedback. A drawback with carrying out the marking in class is that students will have different reading speeds and so will finish at different times making it difficult to start the group discussions. Also some students may struggle to understand the piece of writing in the time allocated in class.
The lecture on academic writing skills may not be an effective way of communicating assessment requirements. Student feedback from the focus groups suggested that they had not found it very useful. The comments tended to support the previously discussed point that students often find it difficult to understand assessment standards if they are communicated in a passive environment and outside of the context of a particular piece of writing. A more effective alternative may be to omit the lecture altogether and instead provide the students with copies of the marking sheets and assessment criteria that were used to mark the essays originally. They could then use these to mark the work themselves either before class or in-class. Another option would be to ask the students to mark the work without any guidance. They would have to decide what makes a good assignment and in the process develop their own assessment criteria. This could then be compared with the actual assessment criteria used by the tutor with some discussion around the similarities and differences. This may prove a useful way of overcoming any misconceptions students have about academic writing.
Some of the following concerns have been frequently expressed by tutors:
Some colleagues have argued that providing exemplars is an example of spoon feeding and may contribute towards learned dependence i.e. over reliance on the tutor. There does appear to be some tension between the guidance demanded/expected by students and the onus placed on independent/self-directed learning by tutors. However the transition of students from school to university is probably more difficult than it has ever been. The learning environment they will have experienced at school is increasingly likely to have been one of detailed guidance and high dependency. League tables have generated strong incentives for teachers to ensure their pupils achieve the highest grades possible and a rational response has been to increase the level of coaching. Given this prior experience, departments may need to introduce activities such as the marking exercise in order to help their students become more independent learners. The focus should be on using these methods with the first years with the aim of developing their self-directed learning skills as the course progresses. The ultimate aim is to make the students independent learners by the end of the course and they may require some assistance towards this goal in the early stages.
There may be worries that students will try to copy the exemplars when completing their own work. This makes the choice of examples very important. They need to be similar enough to current assignments for students to see the relevance, while not so similar that they believe they can simply copy them. It is also important to stress that the purpose of exemplars is to provide a guide and not a model answer. A related concern is that the exemplars may be interpreted in a very prescriptive manner. Rather than an example, they may be read as a blueprint for success. This may encourage the student to stick rigidly to the structure of the good exemplars and stifle originality. Perhaps to overcome this problem it would be helpful to use two very different essays which both received a very high mark.
It might be tempting to think that simply providing the students with the samples of work without carrying out the workshop could capture most of the learning gains but at a far lower cost. However evidence from Rust et al. (2003) suggests that being actively involved in the marking activity is an important determinant of the improved student performance. Simply providing examples of work may only capture a small fraction of the potential learning benefits as it remains a fairly passive way of transmitting the assessment information.
Experience from running the workshops has shown that students are far more confident in judging some aspects of the work than others. They found it relatively straightforward to comment on items such as ‘referencing’, ‘labelling on diagrams’, ‘quality of English’ and ‘presentation’. They will be far less confident applying criteria such as analysis, evaluation and synthesis. It is very important that the discussion in the workshop focuses on these more difficult issues and the tutor is able to highlight sections in the exemplars to help illustrate what is meant by these terms in the context of an actual economics assignment.
Experience from running this workshop has shown that the economics students prefer the sample assignments to be as recent as possible. Ideally they would be from the previous academic year. This does mean having to change the examples every year.
It may be optimistic to expect many of the students to appreciate fully the meaning of terms such as ‘analysis’ and ‘evaluation’ after just one marking exercise. The activity will probably need to be repeated across a number of modules before they develop an understanding similar to that of the academic staff. Ideally it would be used in both Microeconomics and Macroeconomics modules in the first year of an undergraduate degree programme.
Make sure you focus on examples of work which demonstrate and illustrate the parts of the assessment criteria that students find the most difficult to understand, e.g. analysis, evaluation and synthesis. Try to make the exemplars as recent as possible.
Self-assessment sheets could be used to develop the students’ ability to evaluate their work during the research and write-up stage. The forms can provide a structured framework to help them reflect and critically evaluate their own assignments. Generic versions are available from various websites (for example) and could be easily adapted by tutors for use with specific coursework. One option is to include some simple yes or no questions, for example ‘Please indicate whether you have checked the following:
The form could ask the students to assess their own performance against a number of criteria. For example, they might be asked to rate their own work as excellent, good, adequate or poor against the following:
Some questions could be much more open ended with space left for the student to make some comments, for example ‘Please list what you feel are the strengths of the work’; ‘Which aspects of the work did you find the most difficult or are most dissatisfied with?’ Another alternative would be simply to ask the students to assess their work using the same marking sheet as the tutor. Two ranking lists could be included against each of the criteria, i.e. one for the student to complete and one for the lecturer to complete. An example is shown below.
Please rate your own work using the following scale:
1 = poor 2 = adequate 3 = good 4 = excellent
Understanding and application of relevant economic theory
1 2 3 4
1 2 3 4
Whatever version of the form is adopted, once completed, the students should attach a copy to their coursework when it is submitted.
Rather than honest self-evaluation students may behave strategically if they believe that their rankings will have an influence on the tutor. For example, they may think that ranking their own work as poor will increase the likelihood of receiving a low mark from the tutor. It may be more effective to have two separate forms – one for the feedback provided by the tutor and one for the self-assessment sheet. It might also be useful to ask the students to attach the self-assessment sheet at the end of the assignment and the marking feedback sheet at the front of the assignment. The lecturer could explain that they will only look at the self-assessment sheet after they have filled in the marking sheet and awarded a final mark.
Evidence from their use in three Geography modules found that completion rates varied between 28% and 75%. You could make submission of the self-assessment sheets part of the assessment criteria and deduct marks if they fail to attach them to the coursework. Alternatively you might explain that you will only release marks once the forms are submitted. However this may create some extra administrative costs for the tutor.
Studies have found that a majority of students admit to filling out the form at the last minute. They also acknowledged that they tended to simply tick the boxes without giving much thought to the self-evaluation process. Feedback from their use on Geography modules suggested that when filling out the forms a number of students realised the potential benefit but printed them out so late that there was no time to change their coursework. Those that had used them more than a few days before the deadline had found it useful in helping them to focus on the precise requirements for the coursework. They had adapted their coursework in response to this structured reflection process. This suggests that it is very important that the potential benefits of self-assessment forms are explained before they are used. However students may simply have to experience the forms and the regret of missing the potential benefits before they will engage with the process on subsequent assignments. It is strongly advised therefore that they are used on a number of modules and especially in the first year of the course. The form could be adapted to the assessment requirements of each particular module.
If possible use the self-assessment sheets across a range of modules as students do not seem to fully appreciate the potential benefits until they have failed to take advantage of the sheet on at least one occasion.
The use of exemplars and self-assessment sheets are activities which are designed to help the students engage with and understand the assessment criteria. In the process they should become better prepared to receive feedback on their work. However there is still no guarantee that they will engage with and use the feedback. As previously discussed they may not fully appreciate the potential benefits to their learning. Perhaps there is an element of hyperbolic discounting: they keep meaning to read the feedback the next day. Unfortunately when the next day arrives they delay it again! This process continues with the student never looking at the feedback. Perhaps some assessment activities could be introduced which magnify the potential benefits and incentives of using the feedback. This could be within and across assignments. Also in some instances the feedback may have been read and understood and its full value appreciated. However the student still fails to improve because they simply do not know how to respond to the feedback. The following section will outline and assess some potential methods of dealing with some of these issues. They include:
The student submits a draft version of the assignment a few weeks before the deadline for the final copy. The draft could simply be an outline/plan or a draft of the final copy. The lecturer provides feedback on the draft which can then be used by the student to improve the final copy before submission. The rationale behind this type of assessment design is to magnify the incentives to the students of using the feedback provided by the tutor. The immediate usefulness and ‘feedforward’ of the comments should be more apparent. Hopefully the experience will have a lasting effect. Having seen the usefulness of feedback through the draft assignment it should help them to appreciate the potential usefulness and value of feedback provided on the final copy of subsequent assignments.
Students may game the system as they believe that they can effectively get the tutor to write the work for them. This problem could be made worse if multiple drafts were permitted. This fear was expressed very powerfully by an economics student at a focus group:
‘A student could put a small amount of effort into an initial draft that was worth about 40%. They would then receive guidance from the lecturer about how to make this piece of work worth 70%. I could work really hard on my draft copy and produce a piece of work worth 65% and get feedback on how to make it worth 70%. This is not fair because the lecturer is giving the lazy student a short cut to the high mark.’
The comment helps to illustrate an important point about the type of feedback provided on draft assignments. It should focus on how the student can improve their own work rather than feedback that directly improves the work. There is often a fine line between these two objectives.
Tutors may resist this type of assessment because of the potential increase in marking loads. However this could be overcome to some extent by only providing a mark on the final piece of work or very brief feedback. Also staff may worry that they will get more cases of students complaining on the grounds that they have carried out all the suggested improvements and have still got a relatively low mark. Another innovative way of dealing with the workload issue is to get the students to peer assess and provide feedback on the draft copies.
Care needs to be taken with the way feedback is provided on drafts. It needs to focus on the way the student can improve the work rather than being given feedback that directly improves the work.
This activity is used in the skills module outlined in the first detailed case study. Having completed the exemplar marking activity, the students have to write a 1000-word essay on an economist of their choice as part of the summative assessment. They are told that the essay must include an introduction, conclusion and some personal information about the economist. However it is stressed that the main part of the essay should include a detailed discussion and assessment of the economist’s contribution to the development of the subject. The innovative part of the assessment is the peer review process which is outlined below.
Concerns about the use of peer review have been expressed by both tutors and students.
A major issue with this type of activity is getting all the students to engage with the process in a meaningful way. Some initial attempts were made in the skills module to assess the quality of the feedback provided, i.e. by grading it and making it worth 10% of the final mark. However this proved difficult for a number of reasons. Deciding on criteria by which the feedback would be marked proved to be very problematic. The students were asked to write a minimum of three comments against each point. This resulted in them focusing on the quantity rather than the quality of the feedback. As one participant in the student focus group stated:
‘I wrote things just for the sake of it because it was being assessed.’
The marking added to the lecturers’ workload and delayed the time it took for the students to receive the feedback. Tutors had to collect the sheets at the end of the class and mark them before they could be returned to the student. It was judged by the tutors involved with the delivery of the module that the costs of assessing the feedback had outweighed the benefits.
There were a number of comments made in the student focus groups about the quality of the feedback provided by their peers and the worry that their work would be copied. Some students admitted that they would even try and game the system. Comments included:
‘I am worried that other students are not qualified enough to mark my work.’
‘I expect to receive feedback from an expert – not a novice.’
‘Because I would not be confident in the quality of the feedback received from other students, I would be less likely to act upon it.’
‘It would just give the lazy students a chance to copy my work.’
‘I would seriously consider leaving out some of the better bits from the version of my coursework that I submitted for peer review so that it could not be copied.’
One potential solution to the problem suggested by the students themselves would be to get the students from the year above to carry out the peer reviewing. It was felt that they would have more experience and expertise and also would not be able to plagiarise or steal good ideas. In addition when asked about their expectations or experience of this type of activity the students tended to focus on the benefits/weaknesses of receiving the feedback. They tended not to think about the potential benefits from the act of reviewing somebody else’s work. Perhaps some of these benefits need to be made clearer. For example the following advantages of peer reviewing could be stressed:
Expect students to have strong reservations about peer review. It is very important that the potential benefits are clearly explained before they take part in the activity. In particular spend some time focusing on the potential benefits from reviewing somebody else’s work.
Some students may find the reviewing process both difficult and stressful. They may feel uneasy about making critical comments if their identity is known by the reviewee. The whole process could be made less daunting by making it anonymous. Allocating the papers by student ID does create a certain level of secrecy in the activity previously outlined. However there is still a possibility that each student may become aware of exactly who else in the room is peer reviewing their work. The use of technology could be used to increase the level of anonymity by allowing the activity to take place outside the classroom. The Centre for Academic Writing at Coventry tried to develop software to do this. More recently a ‘Learning and Teaching Development Project’ has been funded by the Economics Network which evaluated the use of an online peer review system on an economics module. The software used in the project was the peer review function embedded within Turnitin which is used by many departments. Setting up peer review within Turnitin is a fairly straightforward three-step process.
For more details and evaluation see the final project report on the following link http://www.economicsnetwork.ac.uk/projects/mini/chew_international
The drawback of creating anonymity is that it reduces the opportunities for students to compare and discuss the peer review with colleagues. This is something that a number of them said they would find very useful. They also argued that they would have more confidence in the feedback and were more likely to respond to the comments if they were the agreed outcome of joint discussions. An alternative way of running peer review which encourages dialogue is outlined in the following section.
This activity would require a two-hour workshop and may get logistically complicated as the tutor would have to make sure that two students are looking at the same piece of work. This is more involved than in the exemplar activity where all the students are looking at the same pieces of work.
The draft submission clearly increases the incentives for students to engage with feedback as it can be immediately used to improve the final submission of that piece of work. The process happens within an assignment. Ultimately, the aim is to get students to appreciate how feedback can be used across different assignments. We want them to become fully aware of how they can use feedback in one assignment to help them improve their subsequent assignments. The following methods/activities magnify the incentives for the student to use feedback across different assignments.
As part of the submission for a piece of coursework students are asked to complete a form that asks them what feedback they have received in previous assignments. They are also asked to explain how this feedback has been used to help improve the current assignment. The process could be taken one stage further by setting a whole assignment on the use of feedback. One method that has been used on an economics module is outlined below.
The students are asked to write a 1000 word reflective essay with the following title ‘Identify your strengths, weaknesses, priorities and actions taken/actions required to close deficiencies with regard to your academic skills’. When answering the question students are asked to focus on the marks and feedback received. Each strength and weakness identified has to be evidenced with reference to a particular feedback comment. They are instructed to attach the feedback sheets in an appendix to the essay. They are also asked to complete a learning style questionnaire e.g. Honey and Mumford. Having identified their dominant learning style the student also has to assess the associated strengths and weaknesses of that learning style.
The timing of this type of activity/assessment can make it difficult for the students to collect enough evidence. If the assessment has to be completed by the end of the first term/semester then many students will have only received limited feedback on their work.
Perhaps the ideal time for this type of activity would be the beginning of the second year. Students could be asked to collect all of their first-year work and identify any key strengths or weaknesses. A really innovative approach would be to ask them to include feedback from their examination papers if that is possible.
This final assessment activity uses peer discussion to help students think about the most effective ways of responding to feedback. The activity works as follows:
A key part of the discussion could be based around how the student intends to work on improving any weaknesses. The student could be asked to develop an action plan which outlines how they will respond. The appropriate action will depend on the nature of the weakness identified. For example one prompt would be to ask if the feedback refers to any weaknesses or gaps in their understanding and application of economics. If this is the case they could be asked what they think is the most appropriate action. Are they simply going to look at the same textbook/material again or are they going to try and find alternative sources that explain the same principles but maybe in a way they can better understand? Have they thought about making an appointment with the lecturer to discuss the issue or have they considered discussing it with friends on the course. Another prompt would be to ask if some of the weaknesses are more generic in nature and relate to study skills. For example if the comments specify that academic writing skills are poor what could the student do about it? Where could they go for help? Are there any student support centres in the university where they can get help?