Friday, November 12, 2010

The "evidence" in evidence-based decision making with High Stakes Testing

High Stakes Testing provides the opportunity for data-based decision making. The question is: will doing so be an opportunity for a "garbage in, garbage out" process, or for true "evidence-based practice?"

Advantages of High Stakes Testing:
At the individual student level, one little-discussed advantage to high-stakes testing is that appropriately calibrated and normed tests can give data that validate grades earned. This would end the dilemma faced by students with high grades from poorly-achieving schools. Students presently enrolled in schools like New York City’s John Dewey High School face challenges in applying to prestigious colleges: admissions counselors can say: “Sure, you have a high GPA, but it’s from a school under registration review. What does it mean?” If that grade is accompanied by a set of correspondingly high scores on achievement tests, it’s clear that the student’s grades are reflective of earned accomplishment.

At the school building level, the use of data from high stakes testing (and other assessments) can clear the path for educational leaders to change students’ experiences to alternate modes of education (for example to an in-school tutoring model, to different materials, or to Direct Instruction curricula) so that they no longer have to repeat an experience that was already measurably ineffective for them. This use of test data can help solve a “culture problem” in education today: the rift between philosophy and outcomes. Doing what is right for children isn’t always easy in education because we are in a field with many stakeholders, and each group tends to hold disparate philosophies of education. It seems that many sub-groups, by dint of previous training and orthodoxy, are equally assured that their way of providing education is the “one true way.”

Disadvantages of High Stakes Testing:
At all levels, just because a decision is “data-driven” doesn’t mean that it is correct. Knowledge may be power, but power corrupts; and the absolute power that corrupts absolutely probably had its case argued by mis-analyzed or frankly manipulated data. In some localities, administrators and school boards use aggregated performance data to make wrongheaded decisions; take for just one example the article’s report that in Texas low-performing schools are bolstered with additional financial support and rarely closed. In some fields, such an arrangement would be called “anti-merit pay.”

Finally, from a personal level, an observation that data from a poorly constructed or badly normed test may be worse than no data at all. I was a member of the cohort of students that was in the fourth year of Ohio Ninth-Grade Proficiency Tests (now being phased out) and the first year of Ohio Twelfth-Grade Proficiency Tests. My Ninth Grade proficiency tests were not a problem at all. As a daughter of one of the high school faculty members, it was expected that I performed well in high school. In fact, I represented the school in statewide comparisons by taking “Scholarship Tests” each year. When my results came in for the Twelfth Grade tests, imagine my surprise to learn that I had “pegged” all the other subtests, yet failed Reading Comprehension. To this day, I have no explanation for this spurious result. Was the test faulty? Was the wrong key used to score my test? Did anybody in the school pass? Was I “one bubble off” in my response sheet? Don’t know, don’t care, glad it didn’t count, except for the fact that my high school diploma did not have an extra sticker indicating that I passed those Twelfth Grade tests. Who knows what became of other “star students” who came in the next cohort for whom that subtest was a requirement, and not a pilot project. It could have been a disaster, and in that place at that time, I’m personally very glad the State of Ohio was NOT “playing for keeps.”

Monday, November 1, 2010

Multiculturalism - an "ism" of Education?

Racism. Sexism. Elitism. Generally, it’s not good to have an “ism.” I know when and how I caught my case of Multiculturalism, and it has given me discomfort ever since. In the late 1990s, when I was an undergraduate student, the college I attended overhauled its core curriculum. Suddenly, we had Themes around which to choose core and elective classes. The Themes were intended to be somehow universal for all majors and minors, transcendent; perhaps we accepted them as beyond reproach or debate. I don’t really remember. In fact, the reform effort was such a smashing success that I not only can I not name all the Themes that the Educational Policies Committee worked so hard to craft. Whether the effort was futile or not, we will never know; the entire college shuttered its campus within the decade.

What I do remember, however, was that one Theme among the others was the hardest around which to create consensus and the hardest to defend. That Theme, I recall with a shake of the head, was “Globalism & Diversity.” Discussions about globalism and diversity, infused throughout the curriculum, were meant to create a multicultural appreciation that would aid in creating honorable behavior from all students, from those with Peace Studies minors to business students to educators. The problem with using Globalism & Diversity as a curricular theme was that talking about diversity is similar to talking about the weather: most people agree on the facts; many people disagree about the interpretation; and just about nobody does anything about it.

And talking about Globalism? Forget about it. Nobody could determine then (and few can now) whether increasing globalization is a net good or a net ill for our own society, let alone the world’s. The recommendations were endless and usually contradictory. Embrace free trade and a liberal import/export policy; no, wait – demand that your goods are made by people making a living wage and using ecologically sustainable practices. Assist developing countries to have a Green Revolution; no, wait – enjoy only local, organic foods. Who knew? Who knows?

Perhaps the problem is that the College was ahead of its time. The World Trade Center tragedy of 2001 demonstrated far too realistically how very real and present the impacts of globalism were. And although we did not know it then, and we still may not yet fully appreciate what to do about it, we are coming to understand events such as those terrorist attacks as having come from under-appreciation of diversity.
Slightly uneasy with “multicultural” efforts for years, I was not quite sure what bothered me about the Globalism & Diversity Theme until quite a bit later. After graduate school and after having taught in and left the NYC public schools, I picked up Thomas Friedman’s The World Is Flat, which by 2005 presented a detailed if not succinct account of the enormity of the changes in the world and the aftermath of globalization.

Finally, I realized that my discomfort with most discussions of diversity came from education efforts that were well-intentioned but involved a severe push-pull between cultural appropriation and cultural misappropriation.

"Cultural appropriation is the adoption of some specific elements of one culture by a different cultural group. It describes acculturation or assimilation, but can imply a negative view towards acculturation from a minority culture by a dominant culture. It can include the introduction of forms of dress or personal adornment, music and art, religion, language, or social behavior. These elements, once removed from their indigenous cultural contexts, can take on meanings that are significantly divergent from, or merely less nuanced than, those they originally held (Wikipedia).

UUA Task Force on Cultural Misappropriation:
“Cultural misappropriation is the term given to the set of injuries marked by:
- using music, reading, symbols, ritual, or iconography of a group without a willingness to engage in their struggle and/or story and connecting their struggle and/or story with our own
- the use of cultural practices as bait rather than an as organic part of our cultural experience
- an unwillingness to respect the community of origin or dishonoring the refusal of a community to share
- disrespect or casual engagement with a practice, or
- unwillingness to share the pain caused by intentional or unintentional misuse.”

It seems to me that teaching tolerance, inquisitiveness, and respect for one another is the true ultimate goal of multicultural education. Unfortunately, however, it has been my uncomfortable experience that many attempts at “multicultural education” inadvertently injure the parties that they were meant to include. Particularly when there is no representative from the “diverse” culture available to participate in the planning of an event, clumsy attempts at “celebration” more closely resemble cultural misappropriation than integration. In this situation, the notion that symbols and practices “take on meanings that are significantly divergent from, or merely less nuanced than, those they originally held” is especially true. If the United States is still a “melting pot,” then certainly it is also a crucible; let us hope that we continue to refine our skills at multicultural education, even as we praise those uncomfortable efforts.