FOSS Program Development Process

The approach used to develop the original FOSS elementary materials included a three-phase process. 1) Development Phase: FOSS staff work directly with students in local classrooms to test out activity ideas. 2) Local Trial Phase: Local trial teachers (20–30 per module) were provided with drafts of teacher guides, prototype student equipment kits designed at the Lawrence Hall of Science, and training. About 240 teachers from 60 schools in the San Francisco Bay Area contributed to the development of the FOSS modules (Mount Diablo, Vallejo, and Oakland Unified School Districts and San Mateo County Schools) by using the materials in their classrooms and providing feedback to the staff. Staff and graduate student observations of classrooms and interviews with teachers provided formative data for program improvement. 3) National Trial Phase: Ten national trial center coordinators participated on the FOSS advisory board and provided training for over 400 teachers from 125 schools in 10 geographically different national locations. 

Formative Research 

For each of its modules, the FOSS development team at the University of California at Berkeley (teachers, scientists, science educators) blends research with instructional activity in several ways. First an extensive analysis of appropriate academic research precedes the development of a module. Second, the instructional activities are thoroughly tested in the hands of both expert and novice science teachers. Feedback is gathered and analyzed. This formative research provides an assessment of each instructional component in such detail that the component can be improved, redesigned, or eliminated. Third, the conceptual continuity of the FOSS program through all grade levels provides a “classroom laboratory setting” for researchers to examine the effectiveness of theories over time in real-world settings. In this partnership, research is used to inform and influence FOSS, and FOSS is now used to influence and contribute to theoretical research studies.

Data Collection Instruments. Because FOSS is a program that initiates engagement with scientific ideas using materials (rather than books), it is important for the program developers to know how well the materials work in the hands of students and teachers in typical classrooms. During the local and national field trials we collected formative data that helped us answer these questions.

1. Written materials for teachers.

• Are they clear and easily understood?

• How much background knowledge must be written?

• Is the time frame for each activity and the whole module reasonable?

• Are any burdens placed upon the teacher that make the instruction more difficult?

• Will the materials work in the hands of teachers who receive introductions to them from secondhand sources (i.e., not members of the developmental staff)?

2. Instructional materials for students.

• Are they durable?

• Can youngsters manipulate them as needed?

• Are they storable without placing hardships on the classroom?

• Are the student sheets and reading materials clear and easy to use?

Information was derived from several sources: 

FOSS Feedback Forms These forms are filled out by each teacher teaching a module. These forms provide specific and descriptive data concerning:

• Use of the equipment - additional sources of equipment, assistance in preparing materials, condition of the equipment when received, problems with management or adequacy of materials

• The time it took to teach an activity - time to prepare, time to carry out activity

• The adequacy of the written materials - clarity and completeness of background information for the teacher, ease or difficulty for students using student sheets.

• The students - reactions to the activity, appropriateness of content, quality of learning, differences among students, e.g., boys/girls, disabled/non-disabled, ESL/non-ESL.

FOSS Teacher Meetings Following one-month of instruction using a module, teachers attend a district-wide meeting in which additional information is gathered. This question-and-answer session provides the writers of modules and other staff members the opportunity to probe more deeply the comments made by teachers. Care is taken to make these meetings useful to teachers and constructive for the program. It is also an opportunity for teachers to socialize and exchange stories about their experiences in teaching a module. 

FOSS Observations While a module is being taught, a trained group of graduate students visit the classrooms to take notes on the structures in the room, the use of the materials, the interactions among students, and other aspects related to a particular activity. The written observations are reviewed by the writers of the module and other FOSS staff in conjunction with other assessment data. The analysis of the data leads to rewriting, restructuring, or redesigning aspects of a module. 

FOSS National Trial Sites After revisions, the modules are used by leadership people at 10 national trial sites. The leadership people have the responsibility to gather data in ways that are similar to that gathered at the local home site: Teacher Feedback Forms, Classroom Observations, Teacher Meetings. This information is analyzed by the FOSS staff and further revisions are made as needed. The testing of the FOSS program and the collection of data is once removed from the FOSS staff at this phase to minimize researcher bias and to have multiple investigators in a number of locations.     

Impact on Students and Teachers
Other tools were used to gather information about effect of the FOSS program upon student learning, teacher learning, student attitudes, and teacher attitudes. We asked questions about:

1. The student's knowledge of science—before and after the FOSS experience.

• Has the student learned the particular concepts presented in a module?

• Has the student become more competent in the content domain?

Information was gathered during trials with pre and post-end of module tests and some student interviews.

2. The student's attitude toward learning science—before and after experiencing FOSS.

• What are the student's attitudes toward science?

• What is the student's sense of accomplishment, level of aspiration, and confidence?

Information was gathered during trials through student interviews (verbal and written)

3. The teacher's knowledge about science—before and after the FOSS experience.

• Is the teacher more knowledgeable and more confident in his/her ability to teach science?

• Has his/her understanding about the nature of science changed?

Information was gathered during trials through teacher questionnaires and surveys.

4. The teacher's attitude toward teaching science—before and after teaching the FOSS Program.

• Is the teacher comfortable teaching a materials-centered science program?

• Will the teacher continue to teach science in this way independently of the program?

Information was gathered during trials through teacher questionnaires and surveys and discussions during meetings facilitated by Center Coordinators.       

Current Research on FOSS Assessment System

A first generation formative and summative assessment system for FOSS is published. The developers of FOSS have been improving the FOSS assessment system since 1998 when work was begun on the 2000 Edition of the program. At that time, several elements of Wilson's constructive approach (2001) were used to guide the development of items and scoring guides that accompany each item. With new funding from the National Science Foundation for a project called Assessing Science Knowledge (ASK, 2003-2007), we will be collecting validity and reliability data on the FOSS summative assessments. In addition, we will develop domain frameworks and module progress maps detailing the multiple paths of conceptual development that are common among students as they construct and transform intuitive ideas into more complex scientific understandings. The frameworks will describe a continuum from less to more sophisticated understandings within a domain. Teachers will benefit in at least three ways from having access to frameworks and progress maps. They can be used to anticipate typical misunderstandings and levels of conceptual development in the student population. They can help teachers reflect critically about classroom instructional interactions. And they can help teachers become more sophisticated in their own science content knowledge.