Healthy Kids Community Challenge (HKCC)

Objectives

The objectives of the outcomes evaluation were to determine the impact of the HKCC at the provincial level on:

• child healthy weights
• child health behaviours
• parental support behaviours
• perceived barriers to supporting child health behaviours
• awareness of and participation in the HKCC program

The outcomes evaluation included primary data collection, as well as analysis of existing datasets. A separate evaluation was developed for the six HKCC Aboriginal Stream communities.

Activities

1. Parent computer-assisted telephone interviews

This telephone survey targeted parents with at least one child living in the household. Its purpose was to determine the impact of HKCC on:

• parents’ support for child health behaviours
• perceived barriers to parental support
• parental awareness of the HKCC program

Baseline data was collected in 2015 prior to the start of the HKCC. Several peer-reviewed manuscripts based on the baseline data have been published :

• The impact of different types of parental support behaviours on child physical activity, healthy eating, and screen time: a cross-sectional study
• Do parents’ support behaviours predict whether or not their children get sufficient sleep? A cross-sectional study
• Parents’ Perceived Barriers to Accessing Sports and Recreation Facilities in Ontario, Canada: Exploring the Relationships between Income, Neighbourhood Deprivation, and Community
• Exploring parent-reported barriers to supporting their child’s health behaviors: a cross-sectional study

Follow-up data were collected in 2018–19 after the conclusion of the HKCC.

2. School-based data collection with direct measures — Pilot

The purpose of this planned evaluation activity was to assess the impact of HKCC on: 

• child healthy weights
• child health behaviours
• the relationship between parental support behaviours and child health behaviours
• child awareness and participation in HKCC 

A pilot of data collection tools and activities was completed, however, the full data collection was suspended. To request the full Health ChAMPS and Health BOSS Survey tools, please contact hkccevaluation@oahpp.ca.

3. Evaluating Health Behaviours Using the Ontario Student Drug Use and Health Survey (OSDUHS)

OSDUHS is a cross-sectional, provincially-representative survey of Ontario students in Grades 7-12 collected every two years. There are 20 cycles of OSDUHS available for analysis of trend data over time at the provincial level. 

OSDUHS data were used to evaluate the impact of HKCC on children in the database, including: 

• self-reported Body Mass Index (BMI)
• physical activity 
• sedentary behaviours
• healthy eating 

Multiple cycles of OSDUHS data were analysed to evaluate differences between HKCC and non-HKCC communities.

4. Evaluating BMI using Electronic Medical Record Administrative data Linked Database (EMRALD)

EMRALD consists of clinically relevant information that comes from electronic medical records maintained by select family physicians practicing in Ontario. EMRALD data, stored at the Institute for Clinical Evaluative Sciences, were planned to be used to evaluate the impact of HKCC on directly measured BMI of children aged 1-12 years. A comparative analysis was conducted to evaluate differences between HKCC and non-HKCC communities, at baseline, published in PLoS One.

Objectives

The objectives of the process implementation evaluation were:

• To assess the extent to which the HKCC :
  
  • reached its target audience
  • was adopted as planned 
  • was implemented as planned 
  • was maintained over the course of the intervention, including plans to sustain these initiatives in the long-term
• To understand multi-level factors that contributed to HKCC implementation at the program, community and provincial levels.

A separate evaluation was developed for the six Aboriginal Stream HKCC communities. 

Process Implementation Evaluation Activities

1. Theme-based action plans and project activity reports

Local Project Managers (LPMs) from each community completed theme-based action plans (TBAPs) and project activity reports (PARs) for every HKCC theme. TBAPs collected detailed information about proposed interventions prior to implementation for approval from the Ministry of Health and Long-Term Care (MOHLTC). PARs were used to track process measures and actual implemented interventions and weresubmitted to MOHLTC upon completion of each theme. We compared TBAPs to PARs to determine whether the interventions were reaching their target audience, were being adopted by community partners, and were being implemented as planned.

2. Local Project Manager training survey

A paper-and-pen survey was administered to LPMs before and after an in-person training sessions held by MOHLTC. The purpose of this survey was to understand the usefulness of the training session provided to LPMs and the level of knowledge or skills acquired during training.

3. Local Steering Committee survey

An online survey was distributed to Local Steering Committee (LSC) members through the LPMs at two time points. This survey aimed to explore key stakeholders’ knowledge, attitudes and practices as they related to the intervention and perceptions of the implementation of the program. A manuscript based on the first iteration of the survey is available in BMC Public Health.

4. Local Steering Committee interviews

Some LSC members were interviewed over the telephone. The purpose of these interviews was to provide a deeper understanding of the results from the LSC survey regarding implementation of HKCC. A manuscript based on the second iteration of the LSC interviews and survey is available in the International Journal of Environmental Research and Public Health.

Six of 45 communities participating in the HKCC were funded through Aboriginal Health Access Centres/Aboriginal Community Health Centres (AHACs/ACHCs).

A participatory evaluation was co-designed in collaboration with AHACs/ACHCs, Public Health Ontario, and a team of researchers and evaluators. Data collection activities included:

• Concept mapping, which identified community-relevant program outcomes for the evaluation to measure and understand;
• Digital storytelling, which engaged participating children and families to describe the impact of the program on themselves and their community;
• Key informant interviews, which asked program implementers about how the program worked and its successes and challenges. 

A Canadian Institutes of Health Research Operating Grant (No. GI1-145124) has provided funding to enhance the evaluation of the HKCC in these communities. To inform this evaluation, a scoping review on principles, methods, approaches and tools for program evaluation in Indigenous contexts was completed.