Data cleaning should be performed by the system manager in collaboration with the data collector. This stage can occur at the same time as data compilation. Any missing values or variables should be queried to ensure all possible information can be included. This may require revisiting case files to check missing information. As a result, data cleaning should not be considered to be complete until the end of the year when all data have been processed for the annual report.
The help of a statistician can be useful in this process, to ensure that all necessary data are present to perform statistical analysis.
It is important that:
- All discussions and decisions at this stage are recorded for future reference (e.g. document all cases removed). While time-consuming, it is critical for ensuring the overall reliability of the data
- Key lessons from this process are fed into the standard operating procedures and back to data collectors as part of continued personal development
- Original data files are retained in case a re-check is required