Assessing community readiness online: a concurrent validation study

Iordan Kostadinov, Mark Daniel, Linda Stanley, Margaret Cargo

Community readiness for facilitation and uptake of interventions can impact the success of community-based prevention efforts. As currently practiced, measuring community readiness can be a resource intensive process, compromising its use in evaluating multisite community-based prevention efforts. The purpose of this study was to develop, test and validate a more efficient online version of an existing community readiness tool and identify potential problems in completing assessments. This study was conducted in the context of a complex community-based childhood obesity prevention program in South Australia. Following pre-testing, an online version of the community readiness tool was created, wherein respondents, with detailed knowledge of their community and prevention efforts, rated their communities on five anchored rating scales (Knowledge of Efforts, Leadership, Knowledge of the Issue, Community Climate, and Resources). Respondents completed the standard, over-the-phone community readiness interview ("gold standard") and the new online survey. Paired t-test, St. Laurent's correlation coefficient and intra-class correlation (ICC) were used to determine the validity of the online tool. Contact summary forms were completed after each interview to capture interview quality. Twenty-five respondents completed both assessments. There was a statistically significant difference in the overall community readiness scores between the two methods (paired t-test p = 0.03); online scores were consistently higher than interview scores. St. Laurent's correlation of 0.58 (95 % CI 0.42-0.73) was moderate; the ICC of 0.65 (95 % CI 0.35-0.83) was good. Only for the leadership and resources dimensions was there no statistically significant difference between the scores from the two methods (p = 0.61, p = 0.08 respectively). St Laurent's correlation (r = 0.83, 95 % CI 0.71-0.92) and the ICC (0.78, 95 % CI 0.57-0.90) were excellent for leadership. Qualitative results from the standard interview method suggest that some respondents felt reluctant to answer questions on behalf of other community members. This may have impacted their self-selected ratings and/or responses to questions during the interview. Concurrent validity for the online method was supported for the Leadership dimension only. However, the online method holds promise as it reduces time and resource burden, allowing for a quicker return of results to the community to inform program planning, implementation and evaluations to improve community health.