How Adaptive Comparative Judgement enabled finance candidates to be assessed through real-world simulation

Amplify Trading was founded in 2009 to develop traders that are equipped to perform in the new era of global markets. Using technology to bridge the gap between theory and practice, their innovative approach to capital markets training is now used by many of the world’s largest financial firms to find and develop their next market leaders.  

Amplify Trading wanted to develop a new corporate finance simulation product to serve the growing mergers and acquisitions (M&A) market. Traditional assessment techniques lacked what was needed to properly identify underlying skills and competencies in this highly competitive and demanding domain. Amplify Trading partnered with Assessment from RM to explore alternative methods. RM Compare offered a unique approach, that uses Adaptive Comparative Judgement (ACJ) to encourage diverse responses and allows for holistic consideration.   

This ground-breaking collaboration produced an innovative solution to a unique problem, namely the ability to assess candidate pitch decks rigorously and authentically. In doing so, it has been used successfully by several world-leading financial institutions and premier university business schools to successfully identify and nurture early career talent.  

The challenge 

The skills and competencies needed to be successful in corporate finance are complex and wide ranging - it is a multidisciplinary domain. While no two projects are the same, most tasks for new entrants to the profession can be broken into two parts. First, they are required to analyse a merger or acquisition opportunity on behalf of a simulation client using Microsoft Excel to create and refine a valuation model. Secondly, they need to report back recommendations from the analysis in a way that meets the client’s requirements. Typically, this would involve presenting complex conclusions in a way that is easily understandable and actionable – usually in the form of a high quality, professional 'discussion materials' pitch deck.  

While confidence in assessing competency of the first task is genuinely high, the second task posed a fundamental challenge:

• Accurately and objectively assessing the qualities needed to carry out the task successfully: creativity, collaboration and communication, professionalism, and high attention to detail. 

The solution

The goal for Amplify Trading was to produce an authentic, innovative Corporate Finance talent identification solution for its target market. The key skills listed do not typically lend themselves to an absolute approach, but instead require a more holistic appreciation, and Adaptive Comparative Judgement is uniquely placed to do so. RM Compare was used to assess the presentation task with the candidates acting as judges in a peer assessment format.   


The implementation process 

Early engagement and iterative learning
Amplify Trading initially engaged with RM Compare by taking advantage of the on-demand free trial. Key members of both teams met for a series of in-person planning sessions to discuss an extended 8 month learning project. This would deliver up to 15 assessment sessions to a range of corporate and university clients, the aim being to test the feasibility, viability, and desirability of the solution. The pilot started in December 2022 and concludes in September 2023.      

At the time of writing, 7 sessions have been completed:  

• 5th September / Amplify Trading Summer Analyst Training / Corporate / 85 candidates  

• 30th September / Credit Suisse Talent Identification / Corporate / 65 candidates  

• 14th December / Birmingham University Talent Assessment / University / 20 candidates  

• 16th January 2023 / Evercore Diversity Talent Identification (1) / Corporate / 90 candidates  

• 17th January Evercore Diversity Talent Identification (2) / Corporate / 90 candidates  

• 1st February / Bahrain Future Leaders Talent Assessment / Corporate / 35 candidates

• 21 February / UBS Talent Identification / Corporate / 120 candidates   

The Amplify Trading solution combines two tasks into a single 2-hour simulation. The examination cohort, typically around 80 - 120 candidates, complete the tasks at the same time and usually online. Members of the RM Compare team attend all sessions in an observational and support capacity. At the end of each session, it is evaluated carefully by both sides to uncover and prioritise learnings. Several RM Compare feature and functionality iterations have been made along the way to better meet the emerging requirements. A good example of this is the introduction of white label functionality to give a more coherent user experience. This flexibility enables Amplify Trading to iterate their value proposition and go-to-market strategy as the project progresses.  

Innovation behind the screens 
Whilst the principles of comparative judgement have been understood for a while, its use at scale in assessment is relatively recent and still very much in its early stages. This is especially true for ACJ where the introduction of an adaptive algorithm adds additional complexity and opportunity.  ACJ has been used with some success in several formative and summative contexts. Its use as a talent assessment solution in a workplace like global finance is truly distinctive and innovative because of the unique challenges presented.  

An agile and user-led approach 
Being agile has enabled Amplify Trading and RM to complete very short feedback loops, making and testing adjustments based on the learning acquired. There is also a shared recognition that value is only properly understood when the solution is in the hands of the users and our shared priority is to satisfy end users through the early and continuous delivery of working software.    

Close collaboration 
The partnership has enabled Amplify to confidently develop their new product proposition and respond quickly to learnings. This has allowed them to move much faster to a position of product market fit. As a result, they have been able to gain early traction in their target market through the delivery of several successful simulation sessions at scales to high value clients.   

Who wins?

Key benefits to Amplify:

• ACJ has enabled Amplify to provide a unique solution in a new market, unlocking a new commercial opportunity. 

• ACJ and the relationship with RM Compare has enabled the streamlining of Amplify's assessment simulations, whilst also providing high quality analytics for the Amplify team.

Key benefits for the Amplify clients:

• High quality performance and assessment analytics, used by clients as they assess candidate suitability,

• A high quality, interactive experience used as an attraction event for high potential candidates.

Key benefits for the simulation participants:

• A realistic, assessed representation of working in M&A.

• The opportunity to showcase talent not only through numerical ability, but also through qualitative presentation submission.

Key benefits for the RM Compare Team:

• Ability to quickly test value hypotheses directly with Amplify, Amplify clients and simulation participants.   

Accessibility and support  

At its core, RM Compare has been built as an online tool and carries several design principles including accessibility. It has a responsive design that can be used on any device and is white labelled to provide a coherent experience for candidates. The simulation was designed and built by industry experts to allow all candidates to demonstrate their proficiency across a range of key skills and competencies demanded by the industry. RM Compare has worked closely with Amplify to ensure that all members of the testing team are able to successfully administer the simulation.  

Delivery of the Amplify solution takes place entirely online, enabling candidates to attend from offices, university, and home locations around the world. They are guided through the simulation by the expert online host and other members of the Amplify Team are available for support through the chat functionality. In advance of the session, candidates are supported by access to Amplify’s learning management system where a wide range of videos and documents are available, ensuring that all candidates arrive at the simulation with the same baseline knowledge.   

The future  

Currently, the simulation model brings together up to 100 candidates at the same time for an online experience hosted by an industry expert from Amplify Trading. In this regard, it is a ‘live’ experience with tasks completed at the same time.    

The RM Compare element takes place toward the end. Under the direction of the simulation host, candidates are required to submit a completed presentation into the system for peer assessment. They are free to use any format however, most choose PowerPoint. The upload process takes place in the candidate interface with the administrator taking a global view of proceeding. This means that in under two minutes, the system allows the entire cohort to simultaneously upload their items into a single session.    

Once all items are in the system, the judging session is initiated by the host. Candidates are now invited to peer assess items in a comparative judgement session. Both the upload and judging process has proved highly reliable and robust. RM Compare is hosted entirely on Amazon Web Services (AWS) and is certificated through the AWS Foundation Technical Review process to meet the highest standards of security, reliability and operational excellence.     

The intention is to progressively scale the cohort numbers in the current simulation. At the same time, work is ongoing to move to asynchronous ‘when ready’ models, with open windows for submissions and rolling judgements. This will be accomplished through continuous collaboration to develop a new RM Compare On-Demand system.