This study examines formative research as an alternative research methodology for improving instructional-design theories. The research methodology was proposed by Reigeluth
(1989). He argued that instructional-design theory is prescriptive, and synthesis and optimality are two major directions for instructional-design theory to strive for. Thus, the issue in instructional-design theory does not lie in 'proving' but in 'improving.'
Experimental research approach is not relevant for 'improving' instructional-design theories.
The process of formative evaluation in systematic design of instruction (Dick & Carey, 1985) provided an idea to construct a research methodology to improve instructional-design theories. The formative evaluation process is utilized to improve a draft of instructional products. The formative research methodology was originated from the formative evaluationm
process, and the methodology consists of two steps. First, design and develop an instructional product solely on the basis of an instructional model. Second, conduct a series of one-on-one formative evaluation of that product (Reigeluth, 1989).
Some empirical studies using formative research methodology were conducted (Roma,l990; Simmons, 1992; Lim, 1994). As the methodology was proposed in terms of its basic steps, those studies identified methodological problems and implemented new solutions. They were as follows: ① The first problem was about the amount of the instructional unit to be examined at once. More than one and half hour for an instructional activity was too long to continue. When an instructional activity was finished, both researcher and participant felt tired. They could not begin debriefing session fresh and fair. A solution for this problem was to divide the whole instruction into segments (around 50-60 minutes) and have a debriefing session (around 20-30 minutes) after each segment. ② The type of open-ended questions during the debriefing session needed to change. The question type was effective to gather information regarding the theory improvement. However, there were low frequency for each category of participants' comments, and it made difficult to generalize their comments. Thus, it was suggested that a specific set of questions should be developed before the study begins. It can be used after open-ended questions are asked.
③ The way of interaction between researcher and participant was pointed out to change. It was found that active interaction between participants and researcher during the instructional activity came to inhibiting the learning process by the participant. The active interaction should be restricted within debriefing session. ④ It was found that the establishment of secured research site and technological supports such as video recorder was crucial to the successful implementation of the study. A closed room for observing instruction and having a debriefing session is preferable. ⑤ Finally, the use of 'think-aloud' technique was found to be less effective because it was used only by highly-talented participants and inhibited their learning process.
Based upon the findings and suggestions from the previous studies, this study proposes methodological changes in terms of three steps of the research procedure: the design and development of instructional program, the implementation of the program and data collection, and the analysis of participants' comments. Regarding the design and development of instructional program, the study should clarify what aspects of the instructional program are not based upon the theory to be examined. It will make the discussion of the study focusing on the theory not on other theories. In the implementation of the program and data collection, four aspects of the methodology can be suggested : ① the implementation of the instructional program and debriefing session should be limited less than 1 hour and 30 minutes respectively; ② the interaction between researcher and participants should be restricted to during debriefing session; ③ the debriefing session should be divided into two steps -- first step with open-ended questions and second step with specific questions; ④ finally, the data can be collected from instructors and instructional designers as well as from learners. The analysis of participants' procedure can be divided into threes steps: data reduction, data categorization, and data display.
These new methodological suggestions should be constantly examined and improved to secure its methodological reliability and validity for improving instructional-design theories.