2. Link the Goals of the Evaluation to Answerable Driving Questions
The next step in the roadmap is to develop and refine driving questions that effectively guide a policy or program evaluation. It emphasizes the importance of crafting a limited number of clear, answerable questions that align with the evaluation’s goals, specify key variables, and identify the target population and research site. The questions should be empirically testable and value-neutral. Additionally, it encourages ongoing revision of these questions to ensure they remain aligned with the broader strategic goals of the leadership team.
2.1. Develop a limited number of driving questions to guide the evaluation.
| Example | The Virginia team wrote a series of questions to drive a more ambitious set of analyses around mentoring and induction in the state. This began with a descriptive question about how school divisions supplemented state-provided resources before asking a follow-up question about the effect of state resources for mentoring and induction on new teacher outcomes. |
|---|
2.1.1. Do the driving questions point to the program/policy the team plans to study?
| Example | The Ohio team asked specifically about the policy they were studying (i.e., challenges principals encounter in their efforts to use practices aligned to the sixth leadership standard in the state). |
|---|
2.1.2. Do the driving questions indicate the outcome variable, when relevant?
2.1.3. Do the driving questions indicate the level of aggregation for study variables?
2.1.4. For a quantitative study, is the the implied relationship clearly specified (e.g., cause, associated with)
2.1.5. Do the driving questions identify the group of interest/participants for the study?
| Example | The Tennessee team asked a research question that focused specifically on teachers who sought out a special education or English language learners endorsement. This focus clarified exactly who they surveyed. |
|---|
2.1.6. Do the driving questions indicate the research site?
2.1.7. Can the question be answered through empirical data? Is it feasible to answer the driving question? Is the driving question too specific (e.g., Yes/No)?
2.1.8. Is the question worded in a value-neutral fashion?
| Example | All teams asked value-neutral questions poised to identify clear facts that could inform ongoing decision-making in their state. |
|---|
2.1.9. Is there an explicit question pertaining to disparate relationships for nondominant communities?
2.1.10. Revise driving questions throughout the evaluation to align to broader goals of the SLT.
