Shortlisting

Shortlisting is undertaken by way of a paper sift or qualifying test.

Paper sifts
The JAC generally uses paper sifts for small selection exercises and for some senior posts. If a paper sift is used, the self assessment and references will be assessed by a panel against the qualities and abilities in order to determine the candidates to invite to a selection day.  Each sift panel will usually comprise of an independent Panel Chair, a Judicial Member and one other.

Qualifying tests
If a qualifying test is used, the test results alone will determine the candidates to invite to a selection day, at which stage their self assessments and references will be considered by the panel, along with their performance on the day. Rather than setting a pass mark for qualifying tests, the JAC determines the number of candidates to invite to a selection day, usually at a ratio of between two and three per post. Based on this ratio, the highest scoring candidates are shortlisted.

Qualifying tests provide an objective and efficient means of shortlisting candidates and are generally used to sift larger number of candidates. The development of  these tests becomes less efficient where there are fewer candidates in which case a paper sift based on self assessment and references is likely to be used.

Qualifying tests are designed to test some of the key qualities and abilities, in particular a candidate's ability to absorb and analyse information quickly, identify issues and to apply the law. Other qualities and abilities that are frequently assessed include an ability to work at speed and under pressure, sound judgement, objectivity, and the ability to explain decisions reached clearly and succinctly.

Qualifying tests are normally jurisdiction specific. Candidates will always be told in advance what type of test they will be sitting and whether any special preparation is required, they will also be provided with any materials and background needed to sit the test. Tests might consist of case studies, multiple choice, or technical questions. For examples of previous tests, click here to see past papers and feedback reports.

Assurance
Tests are piloted in advance with a number of people representative of those likely to apply including people from outside the jurisdiction. Pilots are designed to check - among other things - whether any group of individuals are disadvantaged and whether the time allowed on the day is sufficient for all candidates. While tests are designed to be fair to all eligible candidates, regardless of their specialism or legal background, experience of the jurisdiction concerned will no doubt make preparation for the test less onerous.

Qualifying tests are set by judges from the relevant jurisdiction, and are subject to a robust quality and equality assurance process to ensure they are fit for purpose. Qualifying tests are trialed in a mock environment and agreed by the JAC Advisory Group, a committee composed of members drawn from the Judiciary and professional bodies and chaired by a JAC Commissioner.

Depending on the type of test it will either be marked on an automated basis by computer or by judges from the jurisdiction, in which case all answer papers are moderated to ensure consistency in the marking. Moderation involves the marking judges and the JAC Assigned Commissioner sample checking 10 per cent, and 20 per cent are subject to sample checking by JAC Directors. If any marking discrepancies are identified the 'sample check' may be extended up to and including a 100 per cent check of all test papers if necessary.

As a result of robust moderation and quality assurance process the JAC will not revisit qualifying test papers for individual candidates.

Tests online

We are now delivering tests online. Candidates have told us they want this in order to provide improved anonymity when applying for judicial roles and to enable them to sit a test at a convenient time and location. Online tests should also improve the speed and cost effectiveness of selection processes. The expectation that all candidates will take tests online is in accordance with the Government's strategy of 'digital by default'.

Qualifying test and paper sift feedback reports

We recognize that candidates wish to understand why their application was unsuccessful and to consider that in light of future applications. Due to the high numbers of applications received, it is not possible to provide individual feedback. However, a feedback report is provided, for each exercise, for qualifying tests and paper sifts to help candidates understand what characterised a successful application or test.