Calculus reform: the case of Roger Williams University

Title: Does calculus reform work?
Author(s): Joel Silverberg
Publication type: article
Online: link

The author describes the results of an experiment at Roger Williams University where some sections of the first course in the calculus sequence were taught using a 'reform' approach and the other sections were taught using what the author calls a 'traditional' approach.

The author mentions the textbook used in the reform section (Dubinsky-Schwingendorf) and describes the teaching in these sections (which it seems was all done by himself), but does not mention the textbook(s) used in the so-called traditional sections or what kind of teaching actually went on in these sections. He does mention that there was a weekly computer lab associated with the so-called traditional sections. This alone would qualify them as reform and not as traditional in many eyes. So we might actually be look at reform versus reform here.

The test used to assess the experiment was a common final. The following is what the author has to say about this.

The instructors of the traditional sections prepared a final examination to be taken by all calculus sections. The exam was designed to cover the skills emphasized in the traditional sections rather that the types of problems emphasized in the reform sections. In an attempt to minimize any variance in the ways individual instructors graded their examinations, each faculty member involved in teaching calculus graded certain questions for all students in all sections.

The experiment lasted three semesters, each semester starting with a new population of students. The following information on the grades on the final exams is given.

Final Exam Grades/ Traditional Sections
Percentile Range:0-2425-4950-7475-100
semester 1CCBA
semester 2FCC+A-
semester 3DCBB

Final Exam Grades /Reform Sections
Percentile Range:0-2425-4950-7475-100
semester 1FDCB-
semester 2FC-CB
semester 3CBBB+

We are not given the number of students or the precise number of sections involved.

The above indicates that the performance of the students in the reform sections was worse in the first two semesters of the experiment and better in the third semester of the experiment. The author credits changes that he made in implementation for this. There is of course another possible explanation: it could be that the students who took the third semester version of the reform section were better. It seems that the students were not randomly assigned to sections, so this is something to take into serious consideration. From the article it can be deduced that Roger Williams University, like most universities, knows the SAT scores of its students and has a mathematics placement examination. This data can be used to correct for initial differences between the students in the reform and the traditional sections. This is however not done. Due to this basically no conclusions can be drawn from this study.

As we have seen, this study has several significant flaws. Summarized:

  • It isn't clear whether the sections that are labeled traditional are really traditional.
  • The reform sections seem to all have been taught by the same professor. So it could be that it is just this professor's teaching ability versus that of his colleagues that is measured.
  • The number of students involved is not indicated.
  • No effort is made to adjust for initial student differences. Since the assignment of students wasn't random, this is detrimental to the study.

So the question 'does calculus reform work?' can unfortunately not be credibly answered by this study.