Ipsative #assessment for keeping track of learner progress

This morning I was cognitively mesmerized by the seminar on 'capturing an online student feedback history to enable ipsative assessment and sustained motivation' presented by Gwyneth Hughes. Liveblogging from CALRG seminar, so typos and incomplete sentences possible.  A WONDERFUL seminar, really worth it, so buy the book Ipsative assessment: motivation through marking progress! Or read Hughes papers.

Ipsative assessment: compare it to your own personal best (e.g. in athletics, personal sport…). And why this is of interest: because it is much more related to self-determined, self-directed learning, as the learner looks at their own journey, not the final product comparing it to others. As such this personal growth is an interest for higher education, and more specifically it would be good if a more personalised (personal best) assessment might become part of formal education.  (remark to myself: use this for the CLIL MOOC project).

Outline

Competitative assessment in HE and the problems

Most of the time is a measurement of learning and marks. So assessment is the pinnacle of meritocracy (broadfoot, 1996).

This approach risks to be de-motivating to others, and it does not necessarily result in selecting the best students.

Does criterion referencing remove competition? 

No, as there is evidence that by marking teachers do rank their students, this makes marking – in part – subjective. So, classic assessment is not necessarily objective, nor normless.

Problems arising from assessment competition

Focus on grades and marks detracts from the actual learning process

Competitive assessment can damage self-esteem and reduca learning (Dweck, 1999)

Problem of how to maintain self-respect where there is unequal talent and success (Sennett, 2003).

Most of the time, most people will not achieve perfection or excellence, but most people will make improvements most of the time (Hughes, 2014).

3 benefits of ipsative assessment

Distinguishes between learning and attainment

Enhances student motivation and self-esteem

Provides a longer-term overview of learning (it is a continuum, also connected lifelong learning and learning skills)

Decoupling learning and attainment

Assessment pass threshold is only the starting point (a diversified starting point), and looking at the ipsative view of assessment you look at the learning progress from the initial starting position.

Ipsative assessment builds motivation and self-esteem

Measures and reivews learning not outcomes (Sennett, 2003)

All learners can succeed – like doing craft work for itself

Setting one’s own goals and standards so self-esteem is not compromised

i.e: doing one’s best in the circumstances, and looking after oneselves

An overview of learning

Longer term overview of learning

What to do about students who are not progressing?

If they do not move, ipsative assessment can be used to identify students who are not progressing, which would be a student of real concern.

Ipsative assessment in practice

Differing approaches to ipsative assessment Ways of marking progress include:

Ipsative feedback: feedback on how they progressed since previous work

Ipsative marks or grades: marks that show progress

Ipsative assessment criteria: the criteria is to demonstrate progress

Assessment careers

Enhancing learning pathways through assessment

5 programmes piloted assessment careers using assignment cover sheets to encourage studentes to respond to feedback and ipsative feedback

Feedback analysis tool for staff development

Feedback history reporting tool in Moodle piloted

Student feedback response coversheets

Thinking about the feedback on your draft (or previous) assingmnts  please indicate what you took out of that feedback, and whether they wanted some particular feedback.

Challenges and solutions

Students approach learning at the short-term module level but find it helpful to review past feedback

Assessors reported that it is difficult to compare two different pieces of work- feedback is not centrally stored

Development of Moodle assignment reporting tool (feedback history report)

Piloting the reporting tool with EdD taught programme

Piloted Moodle reporting tool in 2014 through demos to EdD supervisors and interviews

Implemented reporting tool access for all

Staff views on pilot demos of feedback reporting tool

Convencience of students work and feedback

Potential use in all scenarios, but … the EdD programm is very large, so 100 of supervisors that would be very useful of those people’s feedback, but some supervisors did not all have access to the tool.

Concerns about the reporting tool

Students

Previous grades available as markers

“You know how juries are not supposed to have knowledge of the past of defendant, so should tutors know previous comments?”

Staff

Part of an extreme accountability culture, permitting surveillance coparison and judgement of tutor feedback

Concerns about practicaliites and effective implementation

“All this assumes that tutors put the feedback on Moodle – is this the case?”

Concerns over anything that might increase staff workloads

How the tool could be used

Automatic access to past feedback in VLE so students don’t need to collate this manually for a portfolio (as in the past)

Annual reviews

Change of supervisor, allows new supervisor to access previous student work and feedback

When students are experiencing difficulties, their record can be reviewed

Staff development through sharing feedback practice and consistency across their programme.

Noted questions and answers from audience

In a perverse way the top educational students, are keeping the assessment approaches in HE, so shifting from the existing assessment approach, demands a mind shift. On another level, in MOOCs, self-regulate learners choose what they want to learn, and self-assess their learning to what they wanted to learn (Allison Littlejohn and all). New term: ‘learning gain’ to numerically capture learning progress

Learning locker: the learning path overlap?

What about language in which the feedback is given? Different use of concepts

Abstract provided by CALRG

Ipsative assessment is a powerful and under-used approach that provokes a radical rethink of the purposes and methods of assessment (Hughes, 2014). In higher education a strong focus on grades, marks and the implicit ranking of students in a competitive system means that it is not possible for the majority to excel and for many learners it is difficult to maintain self-esteem and the motivation to learn (Dweck, 1999; Sennett, 2003). By contrast, ipsative assessment means an assessor makes comparisons with a learner’s previous work to record progress and enables learners from all backgrounds to achieve an academic ‘personal best’.

However, ipsative assessment requires that learning can be easily tracked over time and that students can demonstrate how they have built on feedback to meet self-identified goals. The Assessment Careers project at the UCL institute of Education explored how technology could support feedback over a whole programme and not only at the module level. Feedback was stored in many different places, sometimes in the VLE but not always easily accessible to a tutor and it was difficult to follow whether or not students had engaged with feedback.  A Moodle plug in was developed to enable reporting of a student feedback history over a programme and the system was piloted with an EdD programme. In this presentation, the feedback history report will be demonstrated along with some of the findings of the pilot. Some of the challenges of introducing technology enhanced assessment processes will be discussed in particular in this case when the underlying principle – ipsative assessment and feedback – is not fully discussed by participating staff and students.