AEP Online
Featured Columns
Blaschke on Fed. Funding
A+ Advice for Parents
    Archives
Archives
    Education 
    Legislation
    Technology
    Market Trends
    Misc. Topics
About

Blaschke on Federal Funding

Preliminary Design of National Evaluation of Technology Use Effectiveness Mandated by Congress Will Focus Narrowly on Reading and Math Improvement

Over the last year, the USED Office of Education Technology, in conjunction with the Institute of Education Sciences, has been planning the Congressionally-mandated evaluation of the effectiveness and use of educational technology. This evaluation will culminate in a report submitted to Congress in April 2006. During a recent conference call between USED officials and a number of education software publishers who are members of the Software & Information Industry Association (SIIA), many questions and concerns about potential unintended consequences and negative findings were addressed. The plan for the evaluation has a number of design features that software publishers should seriously consider, including risks and benefits, when deciding whether to participate.

Some of the important design features, which currently exist in the preliminary mode, are as follows:

• the primary evaluation question is whether the use of a "class" of technology, which will include a number of very similar interventions (e.g., phonics or whole language approaches to teaching early reading), will increase student achievement in reading and math through the use of the technology and under what conditions (e.g., the nature and level of teacher training required to use the intervention effectively);
• the "period of performance" will begin with pretesting at the beginning of the school 2004-2005 year and finish at the end of that school year;
• participating firms would provide a free license for selected districts to use their technology intervention in between 40 and 60 classrooms, and the firm would provide necessary staff development and support;
• USED would select one of several national norm-referenced tests to be used for both pre-test and post-test of students randomly assigned to treatment and control classes;
• to ensure some degree of buy-in, teachers would be provided some flexibility in deciding the specific interventions within the overall class of interventions they would like to use, and then students would be randomly assigned to the teacher and control teacher;
• in return for participation, firms would be provided findings on student performance where the intervention was used and those of students in control classes; however, in the final report, findings will be presented in terms of the class of interventions, not the relative effectiveness of any specific individual firm’s intervention.

The two intended benefits for participating firms include: (a) evaluative information which can be used to improve the effectiveness of the intervention; and/or (b) evidence that can be used for submitting reports to the What Works Clearinghouse. There is, however, a risk that, for whatever reason, the firm’s intervention may not produce adequate results.

Several questions must be addressed by firms as they decide whether to apply for participation. The primary question is whether the firm’s intervention, over a period of eight or nine months, will significantly improve student test scores in math and/or reading. A related question is whether the firm’s intervention is properly aligned with the domains of the national norm-referenced test selected for use in the evaluation.

The firm should also negotiate an agreement that would allow its intervention only to be used in states where state standards are aligned with the national norm-referenced test. USED officials, during the conference call, emphasized that, if a firm is not able to do so, USED would provide the necessary resources to ensure that adequate teacher training is provided prior to and during the implementation of the intervention. Another negotiating point would be whether USED would ensure that other important implementation variables recommended by the firm are actually in place or will be used (i.e., students will receive at least 150 minutes of instruction per week using the intervention)

Questions, ideas, or in need of more information? Please contact Stacey Pusey at 856-241-7772.

Click here for more information about the National Education Technology Evaluation.

© 2008 The Association of Educational Publishers
510 Heron Drive, Suite 201 • Logan Township, NJ 08085 • P:856-241-7772 • F:856-241-0709 • Email: mail@AEPweb.org
 
Satellite Offices:
Two Bala Plaza, Suite 300 • Bala Cynwyd, PA 19004
C/O Knowledge Alliance • 815 Connecticut Avenue NW, Suite 220 • Washington, DC 20006