Paper to Digital and Beyond: the future of digital assessment

Paper to Digital and Beyond: the future of digital assessment

RM Assessment was pleased to sponsor the e-Assessment international conference and awards last week (10-12 June). We took a novel approach to the event and invited some of our clients along to join us to discuss their experiences of digital assessment.

For this talk about the future of digital assessment and some of the emerging themes, Ian Castledine (Head of Proposition at RM) was joined by Dr. Matthew Glanville, Director of Assessment at International Baccalaureate (IB) and Dr. Gráinne Watson, Chief Digital Officer at RM Assessment.

RM Assessment has successfully partnered with IB, an educational programme for 3 to 18 year olds, since 2008 to deliver their digital marking, and is continuing with them on their digital assessment journey into the future following the announcement of a new 7 year contract.

What Matt was focusing on during panel discussion was IB’s diploma programme, for 16–18-year-olds, equivalent to A-Levels, and serving a similar purpose around the world, preparing learners to move on primarily to universities and the world of work. At its heart, the IB is trying to make the world a better place by helping students understand alternative viewpoints.

“That is a really important element of the mission. It’s not about knowing things, it's about understanding. Understanding different ways of thinking and that's one of the aspects that we really try to emphasise in our exams. It's the why and it's the how and it's the compare and contrast different viewpoint elements… and that’s why I’m so excited about the opportunities that digital assessment offers us,” explained Matt.

Digital assessment for an interactive exam experience

Rather than an uninspiring blank sheet of paper, the shift to digital assessment provides a platform for a highly interactive experience for the learner. IB will start relatively small, moving paper on screen, so interaction will be limited to video as a source material, rather than just pictures or text, and allowing learners a bit more space into some of the graphical design areas. IB is looking at tools that will really make full use of the opportunities digital assessment provides. The aim is to eventually encourage schools and learners to become more confident in what digital assessment can look like and then start to build it up into a truly interactive model in the future.

Exams developing work-ready learners

In response to Ian’s question about the importance of having work-ready students, Gráinne’s opinion was that it is really important and said the assessment industry is now coming round to that way of thinking as well - “There's not a single job I've ever been in where somebody's handed me a blank piece of paper and for two hours I wasn't allowed to look at anything else… and yet that is still the way we are assessing people and those 2 hours become the most important 2 hours for a lot of students, whether it’s PQ (professional qualifications) or whether it’s GQ (general qualifications) that we’re working on. We have to find a way of using technology to try and teach people what is actually going to happen to them in the workplace.”

Gráinne also made the point that it is important to try and teach people to use work relevant skills the whole way through their educational journey, rather than just telling them on the way out of school or university that the world of work is going to be massively different to anything they’ve done before.

A key part of the IB philosophy is that individuals coming out of education are attuned to work and the realities of 21^st century living. It means they are ready to succeed, not as experts on only passing schools exams, but as experts able to learn, develop and adapt. That requires more sophisticated assessment tools than a piece of paper we currently use.

Emerging themes around digital assessment

Ian asked Matt if IB are starting to see emerging themes that show what the shape of the future may look like. Matt explained that it is exactly as you’d expect:

• Problem solving

• Modelling and 3D models

• Looking at the skillsets that are in the business world and teaching them in the classroom

Using the example of RM Assessment’s PQ customers in financial services, Gráinne said that because the majority of accountants still work on spreadsheets, they have the ability to go into a digital exam and use spreadsheets on computer in their assessment.

“You can now use augmented reality to train people. you can put on a headset if somebody's doing underwater pipe repair, and you don't want the first time they do that to be underwater because you're asking for trouble there so the ability to use augmented reality to do things like that is useful,” Gráinne added. She also used the example of digital adding authenticity to language exams, with the use of audio and a set of earphones so the learner can move the volume up or down or change the pitch. That also helps learners who require additional accommodations in exams and allows you to assess where somebody is in their language ability, rather than how they cope in a big room full of people with lots of other stimuli.

Change management

Digital adoption is a massive programme of change. Ian asked Matt if there were particular challenges he was starting to see emerge around keeping IB’s various stakeholders on the digital journey with them.

“The first bit is just to recognise “Change for whom?” because for a lot of our young people, the change is not being able to write their history essay on a word processor and having to spend 3 hours handwriting it,” said Matt. “For a lot of people, this is the normal way of working. This is why I’m so keen to get there within our school community.”

Proof that digital assessment works

IB has experience of delivering its middle years programmes (GCSE equivalent) digitally for 5-10 years, so it can show it does work. It has prepared for any logistical or technical issues, like computer failure where you swap replace the laptop in the exam and the learner can carry on from where they left off, so it is trying to give schools the confidence the logistics can be managed. Also for consideration is how IB encourages schools and teachers to start to embed the technology into their classrooms. The COVID pandemic gave the use of technology a boost. Matt feels that technology can really help and support learning.

IB is working closely with stakeholders to build on their experience to develop trust and confidence in digital assessment. It runs a dual system, rather than forcing its schools to go all digital. It gives them the opportunity to choose to either do English literature in digital format or in paper format, to run both parallel, phasing up subjects and then phasing out paper as schools become accustomed to the move on-screen.

“What we are hoping and expecting to see is the reassurance from our other IB schools that it worked, it worked well and it benefited their students. It will then spread to the rest of the community. I’m not worried so much about the technological barriers,” said Matt.

Reliability of the test system and security are paramount

In response to Ian’s question about how RM deals with the risk of denial-of-service attacks on countrywide testing, Gráinne explained that RM has very robust cyber protocols and good frameworks and layers of contingency plans - “We go right down in terms of what could happen, what our response to that would be but I think it's also the understanding that the same level of problem can happen with paper."

Matt agreed “At the moment, it’s a different set of risks… somebody with a pair of scissors can break into a paper-based exam. Those papers have to sit in the school in a locked cabinet for a week or so before the exams happen. In the IB’s case, we have to send those around the world when they are potentially vulnerable. Digital gives us much more control over that. We aren’t sending large portions of paper around.”

Matt also explained that building on IB’s existing knowledge of what makes a good safe exam in the paper world - good invigilation and good preparation in addition to all the technical firewalls and elements and the ability of students to have that recovery of their work if something goes wrong – makes him believe that the security of these systems is actually going to be more secure than the paper based exam because the papers are not physically sitting somewhere for a long for time.

Digital assessment creates a more level playing field

Ian stated that the perception is that actually the opportunity to support all learners regardless of their access arrangements is a key part of digital now. He asked Matt if that was his experience through what he had taken from their middle years programme.

“Absolutely. I mean one of our principles is sort of a “universal access by design”, allowing students to have the setup that they want and it's so much more powerful to say to a student you can control the font size, you can control the colour. We will preset it if you've got access arrangements declared but it doesn't matter, whereas currently they've got to ask us for a particular font size,” said Matt. “How much more straightforward to be able to put [accessibility] technology in the hands of every single student? Being able to control that themselves there's a huge advantage”. Matt explains that one of the most regularly used arrangements for accessibility is currently offering students a laptop to take their exam on. There’s huge potential and opportunity in that we can now we can offer laptops to everyone and they can take control of their access needs, plus all of the other advantages digital has over paper to offer content in different formats

Gráinne agreed, especially in relation to handwriting “I think that’s a leveller and it makes whatever you’ve written much more easily understood.”

The future of digital assessment

For his last question, Ian asked Matt if digital was perceived in IB as dumbing down or was it recognised as being the future.

“Well, I think that's the really important message to get out. We are taking our current exams and moving them on screen. There's no dumbing down. We're not moving them all to the multiple choice or short answer or anything like that. It's the same questions being moved on screen…then it’s the adding the interaction. So not only do they have to evaluate a table of data to see what the trends are, they actually get to run the experiment in the virtual world and control the variables themselves, so they make the selection of what they control and what they don't and then see the results, which is, of course, a big step forward… It’s that authenticity. All of us use digital tools in our day-to-day lives now. Why are we not letting students do that for some of the high-stakes assessments that they are currently doing?” concluded Matt.

A big thank you to Matt, Gráinne and Ian for sharing their thoughts on the future of digital assessment.