This evaluation incorporates qualitative and quantitative data from three primary sources: (1) site visits and personal interviews with local Weed and Seed Program Coordinators and program staff; (2) reviews of grant applications, supporting materials, and grant reporting documents; and (3) target area reporting documentation. Each type of data is discussed in further detail below. Because the type and sophistication of data collection systems varies across localities, the measures available to support the evaluation differed in each local Weed and Seed site.

Data Sources

Site Visits and Program Staff Interviews

Evaluators conducted two site visits to each Weed and Seed program reviewed in this report. During each site visit, lengthy interviews were conducted with Weed and Seed staff using interview instruments constructed by the evaluation team. Generally, the individuals interviewed at each site included the program coordinators from both the Weed side and the Seed side, a supervising law enforcement officer from the target area, a crime analyst and / or information systems personnel, and Seed staff.

The preliminary evaluation interview was conducted at each program site in June 1998. In general, interview questions prompted staff to specify the program components, describe the activities funded under the Weed and Seed program, and review the existing data collection and reporting protocols. Second site visits were conducted during Spring 1999. At this time, interviewers collected more detailed information on each program and its operations. Interview questions covered a number of different topics, including target area descriptions, reviews of needs assessment strategies used in program development, obstacles to program implementation, program administration, staffing, grant reporting, and staff perceptions of program impact.

A number of brief, follow-up interviews were conducted as needed by phone during the last year of the evaluation to obtain updated information about local activities and program perceptions.

Administrative Document Review

Program evaluators also reviewed numerous administrative documents throughout the course of the project. These documents included the grant solicitation for each funding cycle, grant applications for each program site, and all grant progress and evaluation reporting forms. The documents were used to further understand the grant requirements, program design, implementation of each local program, and crime trends in the target areas.

Target Area Data

Although grant progress reports from each Weed and Seed site were required by DCJS, no specific evaluation data reporting requirements for grantees were established at the onset of the Weed and Seed program. Consequently, a review of documentation provided by each locality revealed significant differences in the type, amount and specificity of program performance information available in each Weed and Seed site. The lack of standardized and consistent reporting by the sites led evaluators to develop a three-section standardized reporting form implemented in December 1998, to capture the following information:

Section I: Monthly submission of target area crime statistics (offenses and arrests reported) derived from local Uniform Crime Report (UCR) or Incident-Based Reporting (IBR) systems.

Section II: Monthly data for other activities within the target area (e.g., hours of police presence, quantities of drugs seized, other special activities, etc.).

Section III A and B: A quarterly narrative intended to collect specific information not captured in other existing reports (e.g., numbers of participants in Seed programs, program implementation obstacles, etc.).

These reports were used to update program descriptions and assess the impacts of Weed and Seed in each target area. While the breadth of available information varied significantly across localities, most sites were able to provide basic information and indicators on Weed and Seed-funded activities in the target areas. (See Appendix B for a copy of the Evaluation Reporting Forms.)

Limitations on Use of Target Area Crime Statistics

Although monthly crime statistics were collected on the standardized evaluation reports, data issues emerged as the reports were compiled for analysis purposes. While evaluators intended to use these data to help assess the impact of Weed and Seed programs on crime levels in the target areas, limitations precluded their use for this purpose.

Most importantly, the transition from UCR to IBR data reporting systems presented significant data interpretation problems. During the time span of the Weed and Seed evaluation, Virginia localities were transitioning their offense / arrest reporting systems from the UCR to the IBR system. In effect, these transition experiences resulted in reporting / formatting variations for some portion of the study period in each site. This problem essentially nullified the utility of crime data comparisons across multiple years within one program site, as well as comparisons across program sites. Because variations in numbers of reported offenses and arrests may be due to UCR - IBR translation issues rather than changes in actual numbers of offenses and arrests in the target areas, these types of comparisons are not presented in this report. In addition, the availability of data for selected offense / arrest categories also varied across sites during the transitional period. Therefore, longitudinal effects of the program could not be measured as was requested by the General Assembly in the 1998 Budget Bill.

Complete IBR data were available for only the 1999 – 2000 grant year; therefore, offense and arrest figures are examined during the third year only. Data reporting the most common types of arrests are presented for each Weed and Seed locality. Additionally, supplementary data (e.g., calls-for-service, etc.) are provided as very general indicators of crime activity in the target areas. However, these data are subject to local reporting variations and also cannot be compared across localities.

Continue on to Chapter 6 or
Return to the Table of Contents.