SURVEY QUALITY CONTROL

We developed a comprehensive quality control system that runs throughout the entire field stage to assure high-quality data on time

SURVEY QUALITY CONTROL

We developed a comprehensive quality control system that runs throughout the entire field stage to assure high-quality data on time

Valid evidence-based decisions require reliable data. However, data can be subject to many different types of error generated during the data collection process. Such errors must be detected and addressed in the field since most of them cannot be effectively solved later in the office.

At Sistemas Integrales we have specialized in collecting high-quality data. Our Comprehensive Fieldwork Quality Control (CFQC) system brings together established methods, new technologies and extensive field experience to secure the validity and consistency of the collected data. The CFQC system runs throughout the whole fieldwork process to detect errors immediately in the field and take corrective actions on time. Thus, final data are clean and ready for analysis very shortly after the end of fieldwork.

Valid evidence-based decisions require reliable data. However, data can be subject to many different types of error generated during the data collection process. Such errors must be detected and addressed in the field since most of them cannot be effectively solved later in the office.

At Sistemas Integrales we have specialized in collecting high-quality data. Our Comprehensive Fieldwork Quality Control (CFQC) system brings together established methods, new technologies and extensive field experience to secure the validity and consistency of the collected data. The CFQC system runs throughout the whole fieldwork process to detect errors immediately in the field and take corrective actions on time. Thus, final data are clean and ready for analysis very shortly after the end of fieldwork.

The CFQC system integrates a wide
range of quality control tools

01

Computerized checks

We build an extensive, customized list of checks and validations into electronic questionnaires to provide instant feedback to interviewers. Range and consistency checks are also run over all submitted interviews to detect univariate and multivariate outliers and other anomalies.

02

Monitoring of data quality indicators

Paradata and interview data are used to build survey-specific quality indicators by interviewer, which are visualized in the quality control dashboard and analyzed in real time on an ongoing basis. A composite Interviewer Risk Index synthesizes the quality indicators and swiftly identifies underperforming interviewers, who are retrained if necessary.

03

Monitoring of nonresponse and missing data

We closely monitor unit nonresponse over time, by team and interviewer, to prevent potential nonresponse bias. Furthermore, we oversee the number of missing values in key questions by interviewer to identify those who are not sufficiently probing.

04

RANDOMIZED AUDIO SUPERVISION

The randomized audio supervision is conducted through RAS, a self-developed application for auditing. Random parts of 100% of interviews are recorded and assessed by trained auditors to assure the accuracy of the data registered by the interviewers and compliance to protocols.

05

Double-blind back-checks

A random subsample of respondents is reinterviewed and reasked a set of randomly selected questions.

This verifies the accuracy of the data and detects proxy reporting and fabrication. It can be done in person or by phone.

06

Interview observation

We observe entire interviews to assess the level of understanding and the performance of all interviewers. Observation reports are completed during the interview and summarized centrally.

07

Mystery respondents

We use trained mystery respondents who are unknown to field workers. This permits assessing interviewing behavior, attitude, level of effort and compliance to protocols. It is particularly useful when randomized recording is not feasible.