Do students want to switch to digital exams? Do teachers? What are the barriers to change, and when and how should e-assessment be introduced?
These are just some of the questions explored by assessment specialists at an RM event hosted by the IB in October.
One exam board which delivers 90 per cent of the exams to the school leavers in their country, has recently switched the marking of its pre-university Information Technology (IT) and physics assessments to onscreen marking (e-marking).
They report that teachers are generally divided between those who embrace and celebrate the change; and those who feel that there is no benefit or rationale to switching to e-marking, particularly in arts and humanities subjects.
While teachers discuss the merits of change, the outcomes are proving very persuasive. Double marking and comparison of the results for this exam board proved that e-marking has the same quality standards as paper marking, but that the time involved has been significantly reduced.
The International Baccalaureate (IB) delivers exams across the world and has introduced an eAssessment option for the Middle Years Programme (MYP) exam which almost 120,000 students have now successfully completed.
In developing this exam, the IB had to overcome questions and challenges from schools. ‘How will all its students have equal access to a computer?’ was one obvious question, along with other concerns such as ‘What happens if a computer fails or freezes?’, ‘Will the battery last?’, and, ‘What if a student is slower on a keyboard than writing by hand?’ ‘What is the impact of the cognitive effort required using a key board versus writing by hand?”
These have all been overcome through testing and demonstrating results. For example, Bring Your Own Device (BYOD) is allowed, but each machine must run a compatibility test to ensure the exam packages can be used on the device. When the exam is loaded the computer is locked down just for that exam. The computer recognises each individual with their unique exam ID and can adjust screen settings, timing etc. to that student’s exact needs.
Once teachers are aware of the basic procedures in place, their fears are allayed. For example, the MYP exam is delivered through web based or local area networks or can be loaded from an IB memory stick. If a machine fails, the student can reopen their exam portal from another machine, and in worse case scenarios work can be retrieved by a rollback mechanism.
Professor Stuart Kime emphasised the importance of good teacher training and support in helping to reduce teacher concerns. Results from the Department for Education (DfE) Workload Challenge Surveys have shown that without a strong understanding of the theory of why certain things are needed, then any change or perceived extra work just adds to workload and stress for teachers.
Clearly no one wants to add any stress or workload for teachers, and the discussion showed how e-assessment, delivered well, can be a real help.
In vocational and professional assessment for example, moving to digital testing with multiple choice answers has allowed for automated marking, reducing marking time and cost, and giving much faster feedback to the student.
It can also give much greater opportunity for support and collaboration between examiners and team managers where automarking isn’t possible. Recruitment and retainment of examiners is a major undertaking for all awarding organisations, and by using the data obtained through e-marking, much quicker intervention and support can be offered which could help with both individual and team work
For example, analysis has shown that the more times an examiner looks at a response, for a short answer question, the less accurate the marking is. Using this insight while e-marking is taking place could flag up a training or management intervention.
The importance of training was highlighted in other ways relating to e-assessment. Professional and vocational awarding organisations, who are early adopters of online assessment, have discovered that where a test takes place in a hall situation, with multiple devices available for students to use, the invigilator must be trained in ‘on-the-spot problem solving’, in a way that is rarely or never required in a pen and paper situation.
The invigilator must be prepared to deal with technical IT support questions, often in moments of high stress. Some awarding organisations have found that the invigilator in this situation is considered to be a representative of the awarding organisation itself, in a way that a traditional pen and paper invigilator is not. This calls for a higher level of training and support.
In conclusion, it is important when moving to an e-assessment delivery system that teachers, as well as students and parents, are on-board early on in the process to ensure benefits are fully realised, and any concerns alleviated.
Watch “2020 and Beyond: Opportunities and Challenges for Assessment in the Digital Age” for an overview of the day.
