April 02, 2024
Agencies Publish Their FY25 Annual Evaluation Plans
By The Evidence Team, OMB
Agencies recently published their latest Annual Evaluation Plans, describing the significant evaluation activities they will undertake during fiscal year 2025. The plans demonstrate the continued advancement of agency evaluation capacity and the shift toward a culture of evidence across the Federal Government. Agency evaluation plans include studies covering a wide range of programs as well as operational practices that will generate evidence on processes, outcomes, and impacts, and reflect the full suite of evaluation methodologies. We encourage you to explore the documents linked here or from the agency pages.
A few highlights from the FY 2025 plans are below:
The Department of Education is conducting an impact evaluation to understand how training in multi-tiered systems of support for reading in early elementary schools (MTSS-R) helps to improve the quality of reading instruction and the ways struggling students are identified and provided with extra help. This evaluation contributes evidence about the effectiveness of these approaches on a large scale. The study evaluates two strategies that differ in the way they help teachers with instruction of the core curriculum and in how closely that curriculum is linked to the supplemental support.
The National Highway Traffic Safety Administration at the Department of Transportation will conduct an outcome evaluation of its recent high visibility enforcement (HVE) “Click It or Ticket” campaign. Prior evaluations of seat belt HVE campaigns have shown an increase in seat belt use. These evaluations are from the early 2000s and may not reflect the current outcomes of these campaigns. This evaluation will inform the design and implementation of evaluations of similar paid media campaigns and HVE efforts in the future.
The Economic Development Administration (EDA) at the Department of Commerce will assess how EDA-sponsored activities funded through the American Rescue Plan Act (ARPA) improved the capacities of the communities they served. Areas for exploration may include measuring how EDA investments in a regional cluster helped organizations conduct technical assistance and how that assistance supported the creation of new businesses. This evaluation builds on previous years’ work in measuring the outputs that have resulted from ARPA investments and the findings can inform future program design.
The US Department of Agriculture's (USDA) Food and Nutrition Service plans to assess how summer food programs work together to improve nutrition and reduce food insecurity. The USDA implemented a new and permanent Summer Electronic Benefit Transfer (EBT) program that goes into effect in 2024. It also implemented a new policy to allow summer sites in rural areas to provide meals in a non-congregate setting. The findings from this mixed-methods evaluation will be used to inform how summer food options work together to provide nutrition support to children during summer months when school is closed and to identify service gaps or areas for improvement.
The Department of Labor’s Chief Evaluation Office will evaluate the impact that capacity-building grants to community colleges may have on student employment and educational outcomes. This will be a continuation of the Department’s evaluation of the Strengthening Community College (SCC) Grant administered by the Employment and Training Administration in order to evaluate grants awarded in Round 4. The evaluation will examine the extent that aggregate participant impacts vary across subpopulations and identify key program elements that are likely associated with successful outcomes. The evaluation will identify which capacity-building and/or systems change strategies show promise in reducing equity gaps in workforce outcomes for community college students.
The Small Business Administration (SBA) will study how effective the Small Business Innovation Research and Small Business Technology Transfer programs are in stimulating technological innovation in the US. These programs provide opportunities for small businesses to engage in Federal research and development with the potential for commercialization. The evaluation will assess the outcomes of these programs, including the number of jobs created or retained, business survival, sales, exits, and spin-offs, revenue, patents, and business obtainment of additional R&D funding. This evaluation presents an opportunity to understand the program outcomes as they are considered for reauthorization in 2025.
The Cybersecurity and Infrastructure Security Agency (CISA) at the Department of Homeland Security (DHS) is conducting an outcome evaluation to assess program and other factors associated with intended outcomes of the known exploited vulnerability (KEV) catalog such as awareness, understanding of information provided, mitigation, timeliness of mitigation, and other organizational uses of the information. Understanding how to increase the visibility of the KEV catalog and responsiveness of stakeholders will enable DHS to improve efforts to promote the KEV catalog, produce more actionable information that enables user responsiveness, and address barriers to mitigation.
The U.S. Agency for International Development (USAID) will evaluate the impact of the Promoting Education for Early Learners Activity which aims to improve learning outcomes of marginalized children. Through the teacher training program, teachers will be given the tools and knowledge to create a supportive and interactive classroom environment. The impact evaluation will assess how effective the program is in promoting diversity, equity, and inclusion and improving students’ learning outcomes. It will also help the agency understand how USAID programs and operations mitigate harm to underrepresented and marginalized populations, while promoting equity and inclusion.
These examples illustrate a small selection of the evaluation efforts planned across the Federal Government to ensure that programs and policies use and reflect the best available evidence. Check out the full agency FY 2025 Annual Evaluation Plans to learn more.
Tags: