Common data quality problems in field surveys
Missingness, duplicates, straight lining, and timestamps that do not match the protocol.
Most weak evidence is made in the field, not in the statistics package. Daily monitoring exists to catch that early.
Missingness is not one thing
Items skipped because of poor routing, fatigue, or an interviewer who will not wait for an answer are not the same as a true not applicable. Record why a value is missing. A blank and a refusal are different facts.
Duplicates appear when an interview is submitted twice, when a household is visited by two teams, or when identifiers are reused. Unique IDs assigned before the field, and a daily check against those IDs, stop a lot of this.
Speed and location are signals
Straight lining is when a respondent, or an interviewer, ticks the same option down a block of items. It can be haste, confusion, or fabrication. Timing helps: an interview that finishes in a fraction of the expected length for a long instrument deserves a second look.
Timestamps that do not match the protocol are a signal. Night interviews for a daytime facility study, bunched completions at the end of a quota, or GPS points that sit far from the assigned cluster, all warrant a supervisor query the same day.
Fix the instrument while teams are out
Values outside a plausible range, and skip pattern breaks, are often instrument faults. If a pregnancy item appears for a male respondent, the form is wrong. Fix the form while teams are still out. Do not wait for the analysis file to discover it.
Inconsistent related items (age versus date of birth, household size versus the roster) should be resolved with a rule written down, not with silent edits. The original record stays. The correction lives in a working file with a note.
Look every day, while interviewers can still go back. A ruined file found after demobilisation is usually a monitoring failure.
What this note is not
None of these problems is rare. The method is to look while the field is live rather than to discover damage after the teams have gone.
This note describes patterns that appear in field surveys. It does not report rates from AcadStat projects.
