A Long-Term Study of Programmed Instruction in Mechanics

Helen L. Plants, Professor and Wallace S. Venable, Assist. Professor

West Virginia University

From: Moderne Methoden und Hilfsmittel der Ingenieurausbildung

Technische Hochschule Darmstadt, 1978, page 337

For more than ten years the faculty at West Virginia University have been involved in writing validating, and testing programmed instruction in mechanics. During that period of time, the tests and validation have involved more than twenty teachers and several thousand students, and the production and revision of the programs has required almost the entire effort of two faculty members. Three full length programmed texts have been produced and revised after student trials. The text validation and revision process has been reiterated until the texts are now considered essentially finished. In addition, a system for using these programs in class has been gradually perfected. Since 1973 an ongoing series of tests has shown the students studying the programs to have scored significantly higher on the final examination than have those students studying the same courses taught in the traditional manner.

This paper will deal first with a description of the programmed materials and their use, together with the reasons that they are written and used as they are. The final portion of the paper will describe the testing program which has compared the examination scores of students studying programs to those studying in conventional classes.

The Programmed Materials - Their Format and Rationale

The programmed materials used at West Virginia University consist of three programmed textbooks and three notebooks. There is a text and a notebook for Statics, for Dynamics, and for Mechanics of Materials. Together these cover the elementary mechanics course required of all second year engineering students.

The programs are written in a question and answer format. The basic element of a program is a frame. Each frame consists of some information. stated or implied, a question or questions about that information and the correct answers to the questions. The frame may be quite simple or extremely complex. Thirty to sixty frames are put together to form a unit, or chapter, which constitutes one day's lesson.

The programmed books are so organized that each page contains the information and question part of the frame. The answer appears at the top of the following page above the beginning of the next frame.

The use of one frame per page means that there is space for students to do nearly all of their work in the book itself. The student needs only a pencil and a pocket slide rule or a calculator in order to complete his assignment. He can work with the book while sitting at a desk, lounging in an easy chair, or sprawling on the floor without losing the benefits to be gained from writing out complete solutions and answers.

The use of one frame per page also means that there is little need to spend time hunting through the book for answers, references, or the next question. The programs make it easy for the student to give his answers by providing him with quick answer "check-a-box" questions wherever appropriate. The purpose in designing the book for ease of use was to allow the student to spend most of his study time thinking about the material and working with problems rather than grinding through routine busy work. To this end he is often asked to do only the crucial part of a problem rather than solving it completely.

Programmed books have features which obviously set them off from conventional texts. First. they consist of a series of questions and blanks. This is a feature that they share with the workbooks which have been popular at the primer school level for at least a quarter of a century. This form is chosen because it is well known that people learn easily by doing things. In the words of the behavioral scientist, they learn through active responses. A programmed book requires an active response to each item to be learned.

In addition, programs have a feature which sets them apart from most workbooks at an obvious level. A program contains the answers to its questions in a readily accessible location. The student is able to find out whether he is correct or incorrect as soon as he answers a question. He gets immediate confirmation of his response and can use this feedback to correct his work and to guide him in formulating answers to following questions. Immediate confirmation has been found to facilitate learning.

Good programs also have features which arc less easily observed. In an effective program, the student progresses through a series of steps of increasing difficulty. A program dealing with verbal material may begin by asking a student to complete sentences with a single word. and progress through questions to be answered with a phrase. a sentence, and finally a paragraph. In engineering, an effective program may begin with questions which require the student to interpret only one term in an expression or to evaluate a single piece of data. The sequence may then lead him through calculations involving multiple terms and solutions of ever more complicated equations until. at the end of the program, he is working complete problems from beginning to end. In short. good programs consist of a series of steps which are graduated in size so that the student progresses from the simple to the complex.

If a program is effective, the observer will generally find that most students answer most of the questions correctly. This is a part of the designer's plan since people tend to repeat behaviors they have already performed. If a person is encouraged to make a mistake once, it is quite likely that he will repeat his mistake when he is asked to perform under pressure. If, on the other hand. he has always done an operation correctly in the past, he will probably continue to do so in the future. Therefore, most programs have their questions organized and stated in such a manner that students _ill rarely make mistakes during study.

To insure these characteristics, programs are tested for effectiveness during development. While he is preparing a program. the programmer evaluates his materials through the use of quizzes and by examining the student's responses to individual questions. When students fail to perform correctly. the programmer considers himself at fault and accepts the responsibility for the student's error. He must correct his program to prevent recurrence of the error.

These characteristics are all reflections of the basic difference between text books and programmed instruction. In the development of a text, the author concentrates his attention on organizing the content of his subject matter. In writing programmed instruction, the author focuses on organizing the behavior of his students. Because of the unusual organization of the subject matter a program is not likely to be particularly remarkable as a reference work, however the focus on the development of behavior makes programs outstanding tools for instruction.

In an attempt to overcome the admitted shortcomings of programs as reference materials the notebooks are provided. Each notebook has been designed to provide the student with a concise recapitulation of the major points in the units at the same time that it provides an annotated index for ready reference and review. It is an integral part of the programs so that the student is directed to work first in the programs, then to summarize his work in the notebook and to return to the programs for the next topic.

Every item called for in the notebook is referred to a frame in the program where the item is discussed. This enables the student to check his reply and to build up a completely correct notebook for future reference, at the same time that it provides him immediate access to the programmed treatment of the topic should he wish more detail.

Teaching with Programs

Since programs do differ rather radically from textbooks it is not surprising that a different approach must be used by the teacher. There are apparently many ways to use programs effectively in a class depending on the teacher, the students, and the course material. The system in use at West Virginia University has grown up over a period of eleven years in a sort of trial and error manner. The authors have tried many teaching systems and are convinced that the system currently in use works best for the WVU students and the materials now in use. It has been called Elastically Structured Teaching (EST) since it provides a great deal of elasticity for the individual within a rather rigidly structured course.

The success of the EST system may be attributed to several causes. First, it provides for the student's need to interact with the teacher and provides the teacher with a chance to model proper problem-solving behavior. Second, it provides a strong feeling of structure to the class so that the student knows exactly what is expected of him and when he will be measured. Third, it provides almost continuous feedback so that a student can accurately assess his progress. Fourth, by providing considerable flexibility in the schedule it allows the student to make necessary deviations at the same time that it encourages him to get back; to the class schedule as quickly as possible. (In a sense, he is thus encouraged to substitute self-discipline for teacher-imposed discipline.) Last, it reinforces persistence. self-reliance, and daily study.

The system is based upon scheduled. required quizzes over each unit of programmed material. Classes are met on a regular schedule. The teacher begins class by distributing tests which cover the previous night's assignments. These tests are designed to be completed in about fifteen to twenty minutes. The instructor collects the tests and works them out for the class. He then discusses any questions which arise from the day's assignment, and may use any remaining time to show the class his personal tricks and applications for the concept.

Class days on which no tests are scheduled may be used for problem sessions, discussions, demonstrations or lectures.

If a student is dissatisfied with his performance on any of the daily quizzes he may try again on an alternate version. All attempts are averaged to arrive at his score for the unit.

While classes and quizzes are held on a regularly scheduled basis. attendance is not required. Those students who fall behind may take a make-up quiz in a Self Study Center. Students are expected to complete the material before the end of the course, and nearly all of them do. Those who need extra help are encouraged to visit the instructor in his office.

The instructors accept any excuse for a temporary delay in completing a study unit, but no one is excused from completing all work The individual student seems to respond to this by setting up his own standard of permissible excuses for postponing an assignment.

Grades are based in large part on traditional style examinations, but the frequent graded quizzes and discussions keep each student informed of his mastery of each concept so that he may review knowledgeably for the more critical examinations.

Students have been reasonably happy with the system so that the programmed sections usually fill first at registration. A large proportion of those who have had one EST programmed mechanics course take a second one. and on questionnaires 59 to 90 percent indicate a willingness to take a second. were it available. There are, however, those who dislike the EST approach rather intensely. This seems to run at about 10 percent. About 25 percent are very vocal fans of the system. The remainder of the students are neutral in their feelings about the course.

Testing the of Programmed Instruction.

In the Fall term of 1973 an extensive test of the effectiveness of the programmed instruction was begun. It is still in progress. but at this writing has involved 14 faculty, 54 classes and well over a thousand students. On the basis of the data currently available, students taught with programmed instruction have scored significantly better than those taught by conventional methods (figures 1 and 2).

Figure 1. Mean examination score in Mechanics I.

Figure 2. Mean examination score in Mechanics I.

The data were collected from two courses. Mechanics I and Mechanics II. The two courses comprise what is referred to as the Mechanics Year and are taken by all engineering students - usually as sophomores. The course content includes statics, dynamics and mechanics of materials, with each course including some elements of all three subject areas. Both courses had been taught for several semesters before the beginning of the study. Fully programmed courses in Mechanics have been taught at West Virginia since 1967 and some programs had been in use since 1962. It should therefore be noted that the 11-year local history of programmed instruction prior to the start of this study should be sufficient to dissipate any effects of novelty.

Students are assigned to programmed or conventional sections on a quasi-volunteer basis. At registration certain sections are specified as programmed, and students may enroll in any section that is not filled. Demand for programmed sections has tended to exceed availability, so that many students in programmed sections may be assumed to have chosen them. In recent semesters an attempt has been made to provide both types of instruction at the most popular hours. Overflow sections added after pre-registration have been arbitrarily assigned as programmed or conventional for administrative convenience.

During its long history the use of programmed instruction had polarized the mechanics faculty. Programmed instruction had both its supporters and detractors, all convinced that a fair comparison of methods would bear out their contentions. Consequently, almost all faculty were initially quite willing to participate in this study.

The basis of comparison consists of scores on a final examination administered simultaneously to all students enrolled in sophomore mechanics courses. Two examinations are given each term, one in Mechanics I, the other in Mechanics II. The experiment was started one semester sooner in Mechanics I than in Mechanics II.

The first examination was drawn up by a committee of those teaching the course and approved by all course teachers. This was quite cumbersome, however, so that subsequent examinations have been written by faculty not currently teaching the course and are kept secret from the teachers until the beginning of the examination.

In order to assure uniformity in grading. one person is given the responsibility of grading one problem for all students. By using a written grading scale and allowing a very short time for grading- less than 24 hours - a high degree of uniformity in grading has been attained. Grades are obtained by summing the individual problem grades and are based on 100.

To date, approximately 350 students have been tested for each method in each course. This sample size is sufficiently large to make it possible to identify important differences between the methods. The investigators draw added confidence from the fact that both courses give similar results. and that similar results have been obtained each semester. There is still insufficient data to provide firm evidence for conclusions on differences between instructors and differences between class sections, but several interesting trends are becoming apparent in these areas.

Summaries of the statistical analysis of the differences between methods are shown in figures 1 and 2 and arc given in tables 1 and 2. Each table lists the number of students (n), mean score (x) and standard deviation (s) for each method on each separate trial, as well as the total results from all tests. In addition, values of Student's to, degrees of freedom. and the resultant probability that the results are due to chance are tabulated for each comparison.

Table 1. Means, standard deviations and t-tests of significance of difference between methods for Mechanics I.

*estimated

Table 2. Means, standard deviations and t-tests of significance of difference between methods for Mechanics II.

In every case the mean performance of the programmed sections has exceeded that of the lecture sections. On the separate trials. that difference is statistically significant on eight of the eleven trials. The overall results are significant for both courses.

It is apparent that programmed instruction as administered in the mechanics sequence at West Virginia University results in significantly better performance on the final examination than does conventional instruction. This is a major conclusion in view of widely prevalent findings of no significant difference in instructional methods, and warrants further discussion.

The first point to be made is that the test was not merely of programmed instruction but of programmed instruction used in a carefully designed system - Elastically Structured Teaching. It is impossible to say whether the results would hold up if the programs were used in another setting. One piece of evidence points to the conclusion that the,, might. During the earliest trial several topics in Mechanics I were not programmed. In subsequent trials more topics were programmed and student scores were improved.

A second important point to be noted is that this study does not involve self-paced instruction. A self-paced option is available at West Virginia University using the programmed materials, but the number of students who have chosen it is small. (All have been successful, but their success is probably inherent in their high ability.) Elastically Structured Teaching is instructor-paced although quite highly individualized. Consequently, these findings are entirely based on instructor-paced, rather than self-paced, instruction.

A third point to note is that lectures are not shown to be bad teaching, nor is programmed instruction necessarily good. Individual students from lecture sections have made very high scores and some lecture sections have done as well as some programmed sections. Conversely, some individuals in programmed sections have failed dramatically. Scores for both methods are normally distributed, and while means and variances vary, the extreme range for both methods is from 24 to 100. It is on the average that the programmed sections are significantly better. The score distributions are shown in figures 3 and 4.

Figure 3. Distribution of examination scores in Mechanics I.

Figure 4. Distribution of examination scores in Mechanics II.

A last point to be considered in connection with the major conclusions is that the finals given may not have accurately reflected the objectives of the lecturers. This is quite possible. Since many of their objectives have never been stated behaviorally, it is difficult to measure performance in these areas. The instructors in the programmed sections feel that the examinations provide a reasonable assessment of the problem solving skills and strategies they attempt to teach.

In addition to the major conclusion, several minor inferences may be drawn.

First, variation among semesters and among sections in a single semester is less with programmed instruction and elastically structured teaching than with conventional instruction. In a service course which is a prerequisite for many others, this sort of quality control may be important. While the lecture sections have means varying from 51 to 71 in Mechanics I and from 40 to 60 in Mechanics II, the programmed sections have varied from 57 to 77 in Mechanics I and from 60 to 67 in Mechanics II (tables 1 and 2). The number of sections for which data are available is too small to permit a statistical argument at this time.

Second. along the same lines, programmed instruction and elastically structured teaching minimized differences between teachers. Lecture teachers in Mechanics I have posted class averages ranging from 43 to 73. Teachers using programs in the same courses have ranged from 56 to 77. Lecturers in Mechanics II have produced section averages from 40 to 63. Teachers using programs in the same course have taught sections whose averages ranged from 60 to 70 (table 3). It should be noted in this connection that both programmed and lecture sections used both veteran and novice instructors. Again, sufficient data are not available to make a conclusive statistical argument.

Table 3. Section mean scores arranged by instructor

On the other hand, the fact that the programmed sections do differ from one another raises another interesting point. It implies that there is still room in programmed instruction for the teacher to affect the performance of his class. This is further borne out by the fact that Teacher A has managed to have the top section in Mechanics I or in Mechanics II in five out of six semesters. There is room for personal excellence in teaching programmed instruction as well as in lecturing.

A final observation for which no explanation is offered is that the average performance of the programmed sections has proved to be the upper bound on the performance of individual lecture sections (figures 5 and 6). While individual lecture sections have frequently approached the average performance of the programmed sections, only two have exceeded it. Similarly the average of the lecture sections has tended to provide a lower bound on the performance of individual programmed sections.

Figure 5. Mean examination scores in Mechanics I by lecture instructor expressed as a percentage of mean score for programmed sections.

Figure 6. Mean examination scores in Mechanics II by lecture instructor expressed as a percentage of mean score for programmed sections.

In summary, analysis of the data taken at West Virginia University in Mechanics I and II supports the conclusion that programmed instruction coupled with elastically structured teaching results in higher and more uniform student achievement than does conventional instruction.

*** Authors Note: An earlier report of this project appeared in Vol. 1. n. 1 of the Annals of Engineering Education published by the American Society for Engineering Education.