Healthy Kids Community Challenge – Evaluation

The Healthy Kids Community Challenge program (HKCC) is a Ministry of Health and Long-Term Care funded program that supports 45 Ontario communities to improve children’s health by promoting healthy behaviours. This page provides information and resources related to our provincial evaluation of the HKCC program.

 

The HKCC has a focus on promoting healthy behaviours in children (0 to 12 years) in 45 communities across Ontario. Key stakeholders in each community implement locally-relevant programs and policies that focus on important health behaviours. These stakeholders include a Local Project Manager and a Local Steering Committee who work to coordinate planning and implementation of the program.

PHO contributes to the HKCC in three ways:

  1. Evaluating the implementation and impacts of the HKCC at child, parent and community levels.
  2. Hosting the Scientific Reference Committee and its Aboriginal Stream Scientific Subcommittee.
  3. Providing evidence-based advice and support to our clients by producing resources such as infographics, technical reports and scientific publications, or through consultations.

Objectives

The objectives of the outcomes evaluation are to determine the impact of the HKCC at the provincial level on:

  • child healthy weights
  • child health behaviours
  • parental support behaviours
  • perceived barriers to supporting child health behaviours
  • awareness of and participation in the HKCC program

The outcomes evaluation includes primary data collection, as well as analysis of existing datasets. A separate evaluation is being planned for the six HKCC Aboriginal Stream communities.

Activities

1. Parent computer-assisted telephone interviews

This telephone survey targets parents with at least one child living in the household. Its purpose is to determine the impact of HKCC on:

  • parents’ support for child health behaviours
  • perceived barriers to parental support
  • parental awareness of the HKCC program

Baseline data was collected in 2015 prior to the start of the HKCC. Follow-up data will be collected in 2018 at the end of HKCC.

2. School-based data collection with direct measures — Pilot

The purpose of this planned evaluation activity was to assess the impact of HKCC on: 

  • child healthy weights
  • child health behaviours
  • the relationship between parental support behaviours and child health behaviours
  • child awareness and participation in HKCC 

A pilot of data collection tools and activities has been completed. To request the full Health ChAMPS and Health BOSS Survey tools, please contact hkccevaluation@oahpp.ca.

3. Evaluating Health Behaviours Using the Ontario Student Drug Use and Health Survey (OSDUHS)

OSDUHS is a cross-sectional, provincially-representative survey of Ontario students in Grades 7-12 collected every two years. There are 20 cycles of OSDUHS available for analysis of trend data over time at the provincial level. 

OSDUHS data will be used to evaluate the impact of HKCC on children in the database, including: 

  • self-reported Body Mass Index (BMI)
  • physical activity 
  • sedentary behaviours
  • healthy eating 

Multiple cycles of OSDUHS data will also be analysed to evaluate differences between HKCC and non-HKCC communities.

4. Evaluating BMI using Electronic Medical Record Administrative data Linked Database (EMRALD)

EMRALD consists of clinically relevant information that comes from electronic medical records maintained by select family physicians practicing in Ontario. EMRALD data, which is stored at the Institute for Clinical Evaluative Sciences, will be used to evaluate the impact of HKCC on directly measured BMI of children aged 1-12 years. A comparative analysis of multiple years of EMRALD data will also be conducted to evaluate differences between HKCC and non-HKCC communities.

 

Objectives

The objectives of the process implementation evaluation are:

  • To assess the extent to which the HKCC is:
    • reaching its target audience
    • adopted as planned 
    • implemented as planned 
    • maintained over the course of the intervention, including plans to sustain these initiatives in the long-term
  • To understand multi-level factors contributing to HKCC implementation at the program, community and provincial levels.

A separate evaluation is being planned for the six Aboriginal Stream HKCC communities. 

Process Implementation Evaluation Activities

1. Theme-based action plans and project activity reports

Local Project Managers (LPMs) from each community complete theme-based action plans (TBAPs) and project activity reports (PARs) for every HKCC theme. TBAPs collect detailed information about proposed interventions prior to implementation for approval from the Ministry of Health and Long-Term Care (MOHLTC). PARs are used to track process measures and actual implemented interventions and are submitted to MOHLTC upon completion of each theme. We compare TBAPs to PARs to determine whether the interventions are reaching their target audience, being adopted by community partners, and being implemented as planned.

2. Local Project Manager training survey

A paper-and-pen survey will be administered to LPMs before and after in-person training sessions held by MOHLTC. The purpose of this survey is to understand the usefulness of training sessions provided to LPMs and the level of knowledge or skills acquired during training.

3. Local Steering Committee survey

An online survey will be distributed to Local Steering Committee (LSC) members through the LPMs at multiple times throughout the program. This survey aims to explore key stakeholders’ knowledge, attitudes and practices as they relate to the intervention and perceptions of the implementation of the program.

4. Local Steering Committee interviews

Some LSC members will also be interviewed by either face-to-face or telephone interviews. The purpose of these interviews is to provide a deeper understanding of the results from the LSC survey regarding implementation of HKCC.

Six of 45 communities participating in the HKCC are funded through Aboriginal Health Access Centres/Aboriginal Community Health Centres.

A subcommittee of the Scientific Review Committee, the Aboriginal Stream Scientific Sub-Committee (ASSSC), has been established to provide Indigenous and scientific research, evidence and guidance to support the implementation and evaluation of HKCC in participating Aboriginal Stream communities.

Participatory evaluation planning is being undertaken in partnership with the ASSSC and representatives from HKCC Aboriginal Stream communities.

A Canadian Institutes of Health Research grant has been funded to enhance the evaluation of the HKCC in these Aboriginal Stream communities.

The HKCC Scientific Reference Committee provides scientific research, evidence and advice to support the implementation of the Healthy Kids Community Challenge. Specifically, the committee focuses on HKCC social marketing theme development, interventions and identifying evaluation indicators.

Committee members include:

Dr. Mark Tremblay (Chair)
Director of the Healthy Active Living and Obesity Research Group, Children’s Hospital of Eastern Ontario Research Institute
Professor of Pediatrics, Faculty of Medicine, University of Ottawa

Dr. Catherine Birken
Staff Paediatrician and Scientist, The Hospital for Sick Children
Associate Professor, Department of Paediatrics, University of Toronto

Dr. Erin Hobin
Scientist, Health Promotion, Chronic Disease and Injury Prevention, Public Health Ontario
Adjunct Assistant Professor, School of Public Health and Health Systems, University of Waterloo

Dr. Ian Janssen
Professor, School of Kinesiology and Health Studies, Queen’s University
Canadian Research Chair in Physical Activity and Obesity

Dr. Janet Smylie 
Scientist and Staff Physician, St. Michael’s Hospital
Associate Professor, Dalla Lana School of Public Health, University of Toronto

Dr. Jean-Philippe Chaput
Research Scientist, Children’s Hospital of Eastern Ontario Research Institute
Assistant Professor (Department of Pediatrics) and Adjunct Professor (School of Human Kinetics), University of Ottawa

Dr. Jess Haines 
Associate Professor, Department of Family Relations and Applied Nutrition, University of Guelph

Dr. Jessica Hopkins
Medical Officer of Health, Regional Municipality of Peel
Assistant Professor, Department of Health Research Methods, Evidence and Impact, McMaster University

Dr. John Cairney 
Professor, Faculty of Kinesiology and Physical Education, University of Toronto
Director of Infant Child Health Research Lab in Family Medicine, McMaster University

Dr. Kim Raine
Professor and Associate Dean of Research, School of Public Health, University of Alberta

Dr. Margaret MacNeill  
Associate Professor, Faculty of Kinesiology and Physical Education, University of Toronto

Dr. Michael Hall
Vice President, Program Research and Development, YMCA of Greater Toronto

Dr. Sharon Kirkpatrick 
Assistant Professor, School of Public Health and Health Systems, University Waterloo

Chat icon

Contact the Department

hkccevaluation@oahpp.ca

Updated 6 May 2019