Qualifying Test and Paper Sift FAQ

What are qualifying tests?

Qualifying tests are written papers, used to short-list candidates for the selection day stage of most selection exercises. They are designed to assess candidates' ability to perform in a judicial role, by analysing case studies, identifying issues and applying the law.

When are tests used for shortlisting?

The JAC uses qualifying tests for most selection exercises but we tailor our processes to each post. That means we might not use qualifying tests for exercises where there is a small number of vacancies or in other limited circumstances.  We will always specify in the information pack to candidates if a qualifying test is to be used.

Why are tests used?

Qualifying tests are considered to provide more objective evidence of candidates' abilities, irrespective of their specialism within the legal profession, compared to the alternative shortlisting method of a written application and references.

When there is a qualifying test, are references and self assessments also taken into account?

The qualifying test results alone are used to determine which candidates to invite to a selection day. The self assessment and references of those shortlisted will then be considered by the selection panel, along with all of the other evidence gathered at the selection day.

What will the test paper be like?

Each test is different, to fit the particular nature of the posts being advertised. Tests are designed to be fair to all eligible candidates, regardless of their specialism or legal background. You will always be told in advance what type of test you will be sitting, told if any special preparation is required, and be provided with any materials and background as necessary.
Tests might consist of case studies and/or technical questions. For instance, one way of testing the ability to perform in a judicial role is to use a hypothetical jurisdiction and test some common principles. 

An alternative approach is to supply - as part of the test papers on the day - the specific rules or parts of legislation which are relevant to the test, and ask candidates to apply those to the scenarios in the test. 

Another approach sometimes used is to provide detailed jurisdictional material a few weeks in advance to ensure that candidates from outside the jurisdiction can compete fairly with those already working in the jurisdiction. 

Will a test set in a specific jurisdiction not disadvantage candidates who are not familiar with that jurisdiction, particularly when time pressure is so great?

Tests are piloted in advance with a number of people representative of those likely to apply including people from outside the jurisdiction. Pilots are designed to check - among other things - whether any group of individuals are disadvantaged and whether the time allowed on the day is sufficient for all candidates.

What do the papers test?

Candidates are tested on the qualities and abilities required for judicial office, such as intellectual capacity, their ability to absorb and analyse information quickly, their independence of mind and sound judgement, and their ability to explain procedure and the decisions reached succinctly and clearly.

Shortlisting is a competitive process, so the tests are designed to be challenging, in terms of both time pressure and efficiency. 

Can I see a test like the one which will be used in the specific exercise I have applied for?

As a guide to what to expect, we have published examples and descriptions of tests used in previous exercises. When taken together with the detailed guidance in the application pack, you should have an idea of what to expect.

How are the papers set and marked?

The tests are developed and marked by experienced judges from the relevant jurisdiction, and all answer papers are moderated to ensure consistency in the marking. Marking and moderation is done in such a way that the marker cannot identify the candidate from the answer script.

How is the pass mark determined?

Rather than setting a pass mark for a qualifying test, the JAC predetermines the number of candidates that are to be invited to an interview, at a ratio of two to three per vacancy.

Why does the Commission not provide individuals with their qualifying test scores?

JAC officials would be placed under significant pressure to provide further information to a large number of unsuccessful candidates, which would detract from their ability to deliver current and future exercises.

If I am shortlisted, will my test score be considered by the selection panel when assessing my suitability for appointment?
The qualifying test score may be made available to the panel at the selection day because it provides evidence of a candidate's qualities and abilities.

Where are the tests held?

Most tests take place in London, but for some of the larger or location specific exercises they may be held at venues around the country. The application form for the exercise will usually list the likely test locations and dates. They are held under examination conditions with up to 50 candidates sitting the test together in a room.

Will my expenses be paid?

Unfortunately we are unable to reimburse any expenses incurred attending the test. We will make limited exceptions for those with disabilities and in limited exceptional circumstances. Please discuss with our Candidate Services Team who can be contacted on 020 3334 0123.

Can candidates use a computer?

Yes - the application form for the exercise will usually ask you to state whether you wish to answer in manuscript or with a laptop. If you choose the latter, you will use a secure laptop provided by the JAC.

What are you doing to protect the anonymity of candidates who sit the qualifying test?

We have limited the number of candidates to a maximum of 50 in each sitting. We also ask that candidates respect the confidentiality of others. We continue to work with our partners and others to identify ways in which we can improve candidate confidentiality. In the longer term we hope to be able to use technology to reduce our reliance on candidates attending a test centre. 

Will you make reasonable adjustments for any disability?

Applications for reasonable adjustments must be made on your application form. The JAC is committed to considering any reasonable adjustments needed to ensure that you can participate in the selection process fairly. The application form asks you to identify any arrangements and adjustments you may require. Requests will be considered on a case-by-case basis and the information given will not be used in selection decisions.

What should I do if I need reasonable adjustments and didn't request them on my application form?

You should telephone the JAC Candidate Services Team as soon as possible to give them as long as possible to consider and respond to your request. If you make the request on the day the JAC staff will do their best to make the adjustments you need, but it will not always be possible if - for example - you need extra equipment. 

What written explanation will I receive following the qualifying test?

While the sheer number of applications we receive means that we cannot provide individual feedback following a qualifying test, we have, since March 2010, published Qualifying Test Feedback Reports for each selection exercise. The Feedback Report is designed to help candidates understand what characterised a successful test, and to consider that in the light of their experience. 

When are paper sifts used?

The JAC has adopted qualifying tests as an objective method for taking shortlisting decisions.  However, we tailor our processes appropriately and might not always use qualifying tests.  For example, when there are a small number of vacancies or in other limited circumstances we may instead choose to use a paper sift. 

What information is considered by the sift panel when determining which candidates to invite to a selection day?

Decisions are based solely on the information contained in a candidate's self assessment and the information supplied by their nominated referees.

At every stage of the selection process candidates are assessed against the qualities and abilities used to define merit, and considered by the Commission to be inherent in all good judges. In order to prove the strength of their candidacy, candidates are required to provide specific examples in their self assessment to demonstrate how their skills and knowledge match the qualities and abilities sought. Similarly, referees are required to provide specific examples against the same qualities which demonstrate the candidate's strengths and/or weaknesses. The more evidence provided by you and your referees in the examples provided, the more information the panel will have by which to assess your application.

Who does the sift panel comprise of?

Each Sift Panel usually comprises an independent Panel Chair, a Judicial Member and another Independent Member.  They all receive selection exercise specific briefing and training before starting the selection exercise.  In very exceptional circumstances, a senior JAC staff member may replace a panel member who has not been able to attend the sift at short notice, for instance, due to illness.  The senior JAC staff member would not chair the panel, but would only replace the independent member, either because the independent member is absent or because the independent member is sitting in place of the Panel Chair. The JAC staff member would not be a member of the team managing the selection.

What is a paper sift feedback report?

While the sheer number of applications we receive means that we cannot provide individual feedback following a paper sift, we recognize that candidates wish to understand why their application was unsuccessful and to consider that in light of future applications. In July 2011 we, therefore, introduced the  publication of sift feedback reports. The paper sift feedback report is designed to help candidates understand what characterised a successful application.  Candidates may also find it useful to refer to the guidance on references and completion of the self assessment, alongside the sift feedback report.