The latest viasto study investigated the reliability and objectivity of decisions based on asynchronous video interviews. The result: a candidate’s aptitude is assessed more reliably with asynchronous video interviews than it is with CV-screening. Hence, the interview suite offers crucial quality advantages in the early stages of the recruiting process.

In today’s “war for talent”, it is necessary to use selection methods and instruments that produce reliable and objective results. The results have to enable an objective assessment of a candidate’s aptitude that is independent of the evaluators (HR and hiring departments). It should make no difference whether Ms. Smith or Mr. Roberts evaluates the candidate – the result has to be the same. Only a selection method that produces objective results, leads to correct hiring decisions. This is the only way in which the best candidates can be identified and selected. With increasing pressure on HR departments to make the quality of their work measurable within the organization, Key Performance Indicators (KPIs) are increasingly in focus. This leads to the question: How reliable are the selection instruments currently in use?

The study: comparison of asynchronous video interviews and CV-screening

The objectivity and reliability of CV-screenings and asynchronous video interviews in personnel selection was assessed in a study with more than 130 participants. Participants were asked to judge whether or not the candiate was suitable for an advertised job on the basis of a candidate’s CV and a candidate’s answers in an 8-minute asynchronous video interview. The participants were shown the CV or the video interview, and asked to evaluate whether they would invite the candidate to next round of recruiting steps (e.g. “I would invite the candidate to a face-to-face interview.”). Furthermore, the participants evaluated the candidate’s performance on competencies like “motivation to perform”, “team spirit” and “communication skills”. The quality of those decisions was assessed in the study. In professional aptitude testing, quality of decisions is described by calculating the inter-rater agreement. The higher the agreement between the evaluators concerning the candidate’s aptitude, the more objective and reliable the considered selection method is (see infobox). In analyzing the results, CV-screening and asynchronous video interviews were directly compared: Which method has the higher inter-rater- agreement? Which method is therefore more suitable to assess a candidate’s aptitude and competencies reliably?

Overview of the results

1. Candidates’ competencies are reliably assessed based on asynchronous video interviews
2. Asynchronous video interviews are more reliable and objective than CV-screening
3. Hiring decisions based on asynchronous video interviews are fair
4. A set of different candidate’s competencies can be assessed clearly and selectively

Result 1: Candidate’s competencies are reliably assessed based on asynchronous video interviews

Candidate’s competencies can be assessed more clearly, unambiguously and objectively with an asynchronous video interview. Based on the competency definitions provided below the video answers, the participants assessed the candidates faultlessly and made the correct decisions for their organization. With an inter-rater agreement of 0.91 (as a reminder: 0 = arbitrary assessment, 1 = perfect agreement), an outstanding quality of decisions was achieved. According to scientific guidelines inter-rater agreements of 0.60 are already regarded as satisfactory (Obermann, 2009). In comparison: structured face-to-face selection interviews achieve an inter-rater agreement of 0.70 on average (Conway, Jako & Goodman, 1995; Wiesner & Cronshaw, 2011).

Result 2: Asynchronous video interviews are more reliable and objective than CV-screening

The results showed that the same study participants, who reached a very high agreement concerning the video interviews, assessed the CV (every participant assessed the same CV in relation to the same job advertisement) with a very low agreement. Statistical testing showed that the assessment of a candidate’s aptitude was significantly more precise in the video interview than in the CV screening. This means: If decisions in personnel selection are based on asynchronous video interviews, they are reliable. Suitable candidates are invited and unsuitable candidates are filtered out early in the recruiting process. As a result, candidate’s aptitude is always assessed in exactly the same manner, no matter who evaluates the video sequences. The method itself guarantees good quality decisions, irregardless of who carries out the assessment.

Result 3: Hiring decisions based on asynchronous video interviews are fair

One attribute that often endangers diagnostically valuable decisions is the candidate’s physical attractiveness. When evaluating a candidate’s aptitude based on video interviews, the study participants were not biased by the attractiveness of the participant being evaluated. Participants who described the candidate as attractive evaluated the candidate’s aptitude just as reliably as participants who described the candidate as unattractive. This was a further indicator of the high objectivity of asynchronous video interviews. Since the General Anti-Discrimination Act was published in 2006, personnel selection in German organizations has to be proven to be free of discrimination whatsoever. The decision of whether an applicant is invited to further recruiting steps or hired, cannot at any point in the process, be biased by his or her physical appearance, age or gender. Evaluators are allowed to take into account is the candidate’s objective aptitude for the job. This is exactly what happens in asynchronous video interviews.When evaluating the CV, however, the study participant’s evaluation were biased by the candidate’s attractiveness, visible in the photo on top of the CV. The aptitude of exactly the same candidate was rated higher by participants who perceived the candidate as highly attractive, compared to participants who perceived the candidate to be unattractive. “The more attractive, the more suitable” was the implicit but wrong attitude of the participants when it came to evaluating the CV.

Result 4: Different candidate competencies can be assessed clearly and selectively

Study participants were able to distinguish clearly between the different competencies they were asked to evaluate in the video sequences. Competencies like “team spirit”, “analytical thinking” or “competence in a foreign language” could be directly and unambiguously observed in the video interview. The competency-based evaluation process of the video interview makes it easy to display complex competency profiles of candidates – a prerequisite for making sound hiring decisions. In order to evaluate a candidate’s fit to the job and the organization, hiring managers want to increasingly have a more valid impression of a candidate’s abilities and skills at an early stage of the recruitment process. With asynchronous video interviews, this assessment of distinct competencies is already available during the first screening processes.

Summary

The results show that asynchronous video interviews, allow those involved in the recruiting process to make more reliable and objective decisions concerning a candidate’s aptitude. The distinct evaluation of various competencies enables sound assessment of the candidate’s fit to both the job requirements and organization’s culture. By integrating the interview suite, organizations can ensure the crucial quality advantages at an early stage of the selection process.