Summary: It's hard to find a path to educational reform. Using "deficiency scores" on norm-referenced tests is and ineffective approach, even if it's sincerely intended only as a startup catalyst. They just aren't family-, school- or teacher- friendly enough to translate into effective reform. The heat they generate consumes resources while enhancing nothing but the level of distrust of Federal change efforts.
Effective reform must translate directly into classroom change, one child, one objective and one lesson at a time. (And yes, NCLB doesn't work because, while it may generate just criticism, it does not simultaneously generate effective change.) What could fit these "good change effort " specifications is curriculum based testing in multiple developmental areas. The pay off for such a monumental test development and administration effort is that, even if the results do not flatter our present instructional efforts or systems, we will have created a precise "what to teach next" estimate for each individual in each of multiple developmental areas.
One motivator: A local demonstration the reality that individual growth isn't yet touched by what we do now. One approach to such a sketch, cousin to the present dysfunctional consumer of state funds, could take flesh as a statewide assessment of a representative sample of schools. This assessment would actually help has to in two ways that our present mandated system does not: a) it would be curriculum referenced and directly translatable into retargeted, individually tailored instruction, and (b) we would then have a sense of what is possible (from trial teaching and summer teaching) and what is real, right now. We could use these discrepancies to focus our reform efforts.
A piece of such an analysis is sketched and explained below.
I'll illustrate by filling in (hypothetical) results for the sixth grade in the state of Floriana. For this illustration I'm discussing results in 3 of 14 areas of development* from the table of my last entry and interpreting. Pieces of the table with interpretation follow below:
| Statewide Measures of Developmental
Knowledge: Sixth Grade |
||||||||
|
Data Timing or Analysis--------> Type of Skill |
A Test
|
B
|
C
|
D
|
E Summer Growth/ Week |
F
|
G |
H National Annual Growth/ Week
|
| General Cultural Information | 660 | 770 | 830 | 2.8 | 5.0 | 3.3 | 1092 | 2.4 |
| Receptive Communication (Listening, Recognizing Signed Communication, Reading, etc) | 902 | 1092 | 1110 | 4.8 | 1.5 | 4.0 | 1010 | 3.0 |
Social Interaction (Including Self Control) |
1242 | 1382 | 1424 | 3.5 | 3.5 | 2.0 | 800 | 1.5 |
| Ethical/Moral Understanding and Behavior | 1010 | 1154 | 1221 | 3.6 | 5.6 | 4.1 | 600 | 1.5 |
In this example I am assuming that 3900 objectives have been carefully and rigorously sequenced as stairways to growth for for any learner. The phrase "carefully and rigorously sequenced" means that if one is tested and found to be ready for objective 29 then s/he has mastered/doesn't need all objectives of lower number and would fail at tests of mastery of objectives of higher number. There are more than enough possibilities in each area for even bright 25 year olds.
Columns A-F report state averages whereas columns G and H report national averages.
Columns A-C and G report report average positions within learning sequence at state and national levels, respectively.
Columns D-F and H report average growth per week at the state and national levels, respectively.
Now that the stage has been set for our mental experiment, what might Floridians note about this data? How might they interpret it. How would these intrepretations affect future actions?
First, let's note column D . It's values are the computed average weekly growth rate during the school months (roughly 36 weeks of school distributed over 40 weeks of the year). We find that in the state of Floriana students master an average of 2.8 objectives of general cultural material, 4.8 objectives having to do with receiving and understanding information that made accessible in books, lecture, television, conversation,etc,3.5 objectives per week having to do with social interaction skills (manners, reading body language, negotiations, games, are examples) and 3.6 objectives per week having to do with moral/ethical behavior (for example, behaving in such a way so as to benefit others and to support, say, social order and what society considers "a good life"). In short, progress is made in the sixth grade; there seems to be markedly less learning in the area of general cultural understanding, however.
Comparing what we've just learned from column D with the information from column E allows us to compare the impact of school with the impact on non-school life as they each affect the growth of sixth graders. This comparison makes it obvious that, in the State of Floriana, non-school has a decidedly stronger impact on growth in all areas but receptive communication. What this means will require further research by Florianan policy makers.
Finally, Florianans will probably want to compare Florianan to National levels of achievement and growth rates.Levels of achievement: Florianan students are higher in all but general cultural information.
Growth rates: Sixth grade Florianan students have a higher annual growth rate in all areas: for the first three areas growth rate is 25-33% higher. Perhaps most interesting is the growth rate of Florianan students in ethical/moral behavior. It is just short of three times greater than the national average. With a difference this big I suspect that the only surprise will be for an out of state analyst. This will come as no surprise to Florianans once noted. It may well be an indicator of a strong and unique divergence of the Floridian life-style and belief-system from that of the nation as a whole.
However Florinian analysts end up calling the divergence from national norms, the differences between Summer growth rates and School Year growth rates needs understanding. What is it that accelerates the summers (or depresses growth in the school year)? Are there factors in classrooms , curriculum choices, preprofessional or inservice training or supervision that could be altered. However this analysis turns we can check growth rates again next year and determine if our chosen solution was effective. That is, we can check if we adopt and use the criterion-referenced, curriculum-referenced test system that was initially developed to assess the system as a whole.
Don't worry, in the criterion-referenced view, teaching to the objectives is fair and appropriate. Just don't teach memorized answers from a purloined or otherwise copied master test.
