+1 (208) 254-6996 essayswallet@gmail.com

In another case, the quality of a hospital could be assessed by looking at its spe- cial programs, its operating facilities, its emergency room operations, its in-patient operations, its pharmacy, and so on, by experts in medicine, health services, and hospital administration. They could examine facilities and equipment/supplies of the hospital, its operational procedures on paper and in action, data on the fre- quency and outcomes of different procedures, the qualifications of its personnel, patient records, and other aspects of the hospital to determine whether it is meeting appropriate professional standards.

Although professional judgments are involved to some degree in all evalua- tion approaches, this one is decidedly different from others because of its direct, open reliance on professional expertise as the primary evaluation strategy. Such expertise may be provided by an evaluator or by subject-matter experts, depend- ing on who might offer most in the substance or procedures being evaluated. Usually one person will not own all of the requisite knowledge needed to adequately evaluate the program, institution, or agency. A team of experts who complement each other are much more likely to produce a sound evaluation.

Don't use plagiarized sources. Get Your Custom Essay on
In another case, the quality of a hospital could be assessed by looking at its spe- cial programs, its operating facilities,
Just from $13/Page
Order Essay

Several specific evaluation processes are variants of this approach, including doctoral examinations administered by a committee, proposal review panels, site visits and conclusions drawn by professional accreditation associations, reviews of institutions or individuals by state licensing agencies, reviews of staff performance for decisions concerning promotion or tenure, peer reviews of articles submitted to professional journals, site visits of educational programs conducted at the behest of the program’s sponsor, reviews and recommendations by prestigious blue-ribbon



128 Part II • Alternative Approaches to Program Evaluation

TABLE 5.1 Some Features of Four Types of Expertise-Oriented Evaluation Approaches

Type of Expertise-Oriented Evaluation Approach

Existing Structure

Published Standards

Specified Schedule

Opinions of Multiple Experts

Status Affected by Results

Formal review system Yes Yes Yes Yes Usually

Informal review system Yes Rarely Sometimes Yes Usually

Ad hoc panel review No No No Yes Sometimes

Ad hoc individual review No No No No Sometimes

panels, and even the critique offered by the ubiquitous expert who serves in a watchdog role.

To impose some order on the variety of expertise-oriented evaluation activ- ities, we have organized and will discuss these manifestations in four categories: (1) formal professional review systems, (2) informal professional review systems, (3) ad hoc panel reviews, and (4) ad hoc individual reviews. Differences in these categories are shown in Table 5.1, along the following dimensions:

1. Is there an existing structure for conducting the review? 2. Are published or explicit standards used as part of the review? 3. Are reviews scheduled at specified intervals? 4. Does the review include opinions of multiple experts? 5. Do results of the review have an impact on the status of whatever is being


Developers of the Expertise-Oriented Evaluation Approach and Their Contributions

It is difficult to pinpoint the origins of this approach, since it has been with us for a very long time. It was formally used in education in the 1800s, when schools be- gan to standardize college entrance requirements. Informally, it has been in use since the first time an individual to whom expertise was publicly accorded ren- dered a judgment about the quality of some endeavor—and history is mute on when that occurred. Several movements and individuals have given impetus to the various types of expertise-oriented evaluations.

Elliot Eisner, an early evaluator discussed later in this chapter, stressed the role of connoisseurship and criticism in evaluation, roles that required exper- tise in the subject matter to be evaluated. James Madison and Alexander Hamilton took on the role of “expert evaluators” in discussing and elaborating on the meaning and merits of the newly proposed Constitution in The Federalist Papers. (They were experts because they were both present and active at the Constitutional Convention that drafted the document. As such, they were also



Chapter 5 • First Approaches: Expertise and Consumer-Oriented Approaches 129

internal evaluators!) Their writings were influential at the time and are still used by jurists in the U.S. courts to interpret the meanings of the Constitution, illustrating the important actions that can come from reasoned judgments by experts about a product. Accreditation of institutions of higher education is the primary present-day application of expertise-oriented evaluations. The New England Association of Schools and Colleges, which granted the first accredita- tion and continues accreditations for colleges and universities in New England today, began in 1885 when a group of headmasters of preparatory secondary schools began meeting with presidents of colleges in New England to discuss what graduates should know to be prepared for college. Thus, more than 100 years ago, school and college leaders were talking about ways to align their curricula!

Formal Professional Review Systems: Accreditation

Historical Foundations. To many, the most familiar formal professional review system is that of accreditation, the process whereby an organization grants approval of institutions such as schools, universities, and hospitals. Beginning in the late 1800s, regional accreditation agencies in the United States gradually supplanted the borrowed western European system of school inspections. These agencies became a potent force in accrediting institutions of higher education during the 1930s. Education was not alone in institutionalizing accreditation processes to determine and regulate the quality of its institutions. Parallel efforts were under way in other professions, including medicine and law, as concern over quality led to wide-scale acceptance of professionals judging the efforts of those educating fellow professionals. Perhaps the most memorable example is Flexner’s (1910) examination of medical schools in the United States and Canada in the early 1900s, which led to the closing of numerous schools he cited as inferior. As Floden (1983) has noted, Flexner’s study was not accreditation in the strict sense, because medical schools did not participate voluntarily, but it certainly qualified as accreditation in the broader sense: a classic example of pri- vate judgment evaluating educational institutions.

Flexner’s approach differed from most contemporary accreditation efforts in two other significant ways. First, Flexner was not a member of the profession whose efforts he presumed to judge. An educator with no pretense of medical ex- pertise, Flexner nonetheless ventured to judge the quality of medical training in two nations. He argued that common sense was perhaps the most relevant form of expertise:

Time and time again it has been shown that an unfettered lay mind, is . . . best suited to undertake a general survey. . . . The expert has his place, to be sure; but if I were asked to suggest the most promising way to study legal education, I should seek a layman, not a professor of law; or for the sound way to investigate teacher training, the last person I should think of employing would be a professor of education. (Flexner, 1960, p. 71)



130 Part II • Alternative Approaches to Program Evaluation

It should be noted that Flexner’s point was only partially supported by his own study. Although he was a layman in terms of medicine, he was an educator, and his judgments were directed at medical education rather than the practice of medicine, so even here appropriate expertise seemed to be applied.

Second, Flexner made no attempt to claim empirical support for the criteria or process he employed, because he insisted that the standards he used were the “obvious” indicators of school quality and needed no such support. His methods of collecting information and reaching judgments were simple and straightforward: “A stroll through the laboratories disclosed the presence or absence of apparatus, museum specimens, library, and students; and a whiff told the inside story regarding the manner in which anatomy was cultivated” (p. 79).

Third, Flexner dispensed with the professional niceties and courteous criti- cisms that often occur in even the negative findings of today’s accreditation processes. Excerpts of his report of one school included scathing indictments such as this: “Its so-called equipment is dirty and disorderly beyond description. Its outfit in anatomy consists of a small box of bones and the dried-up, filthy frag- ments of a single cadaver. A cold and rusty incubator, a single microscope, . . . and no access to the County Hospital. The school is a disgrace to the state whose laws permit its existence” (Flexner, 1910, p. 190).

Order your essay today and save 10% with the discount code ESSAYHELP