Response Dear Editor

N

References

Achen, C. H. 1986. Interpreting and Using . Beverly Hills, Calif.: Sage PubliRegression cations.

Blalock, H. M., Jr. 1979. Social Statistics. 2d ed. New York: McGraw-Hill.

Response

what

Dear Editor,

particular

c.

thought. Our minds thus changed, even if someone else’s mind might not be changed. one

were

Dr. Briassoulis, in her critique of our study, raises two separate categories of issues. The first is why do quantitative analyses of relationships among context, process, and outcome variables in planned change efforts, when the nature, importance, and effects of those relationships are already &dquo;self-evident?&dquo; Given that such studies are likely to be flawed

methodologically, they are unlikely to convince skeptics of the importance of planning and planners to producing successful projects, even if significant impacts of planning and planners on success are found. The second category of issues

project success notea on p. 1 ’j.j or our article was not &dquo;self-evident&dquo; to us in advance, and, in the absence of empirical research, never would have become &dquo;known&dquo; to us. We therefore think it is a mistake not to put &dquo;received wisdom&dquo; to the test, because it may be wrong, and the &dquo;reality&dquo; may be quite different from

concerns

specific alleged methodological inadequacies of our article. In many ways, the hypotheses with which we began the study represent &dquo;organized common sense: ’ They emerged from our reading of the literature and from our substantial collective experience as practicing planners and consultants. They represented statements of what we believed to be true - in the absence of empirical evidence to the contrary. We then tested these hypotheses on a sample of 58 planned change efforts to find out whether our beliefs would be supported by the data. In many cases they were, but in other cases they were not. In fact, in several instances a relationship exactly the opposite of what we hypothesized emerged. What we thought to be &dquo;true&dquo; thus was not, and &dquo;organized common sense&dquo; was wrong. We believe quantitative empirical research of some sort is the only way to make such discoveries. For example, the complicated set of interrelationships and impacts on

Two additional reasons should be noted for undertaking quantitative, empirical studies of the relationships among context, process, and outcome variables. First, if effective

planning practices vary across planning contexts, then scholars should try to understand how they vary and why. Few qualitative studies examine a sufficient variety of contexts to make sound judgments about what works in which circumstances, and why. Second, to the extent that different scholars and practitioners derive different lessons from their experiences, careful empirical research is essential to distinguish the correctness and generality of the various positions. Our response to the methodological issues raised by Dr. Briassoulis are as

follows:

1. The

While

freely acknowledge

tion is desirable.

d.

Statistically significant relationships identified on the basis of a single data set are sufficient not to reject a theoretical statement. One can never &dquo;prove&dquo; that any theoretical statement is &dquo;true&dquo; one only obtains evidence consistent or not consistent with the -

statement. e.

We are interested both in theoretical significance and statistical significance. Our study uses the

contingency theory approach to research on planning, management, and organization, in which the appropriate choices for any planning process are contingent

the di-

of the cases in our sample, we also believe that they provide a useful and workable sample of &dquo;cases of planned change.&dquo; That is the universe from which the sample comes. A number of our concerns about the sample were covered in the article; here we respond to Dr. Briassoulis’s specific comments.

versity

We don’t say the various organizations’ interests are equivalent, but rather try to understand how they effectively work toward their interests. b. The organization theory and decisionmaking literatures provide

organization.

generic project planning training. We do not think that either rejec-

Study Sample

we

Kinct or

While there may be a need for such models, they do not yet exist in well-articulated form, at least to our knowledge. We do not say that success means the same thing in each case. Instead, we argue that it is meaningful to ask, &dquo;Did the agency get what it wanted?&dquo; If one a priori rejects the hypothesis that an underlying set of factors may influence success across a set of kinds of planning &dquo;projects&dquo; (which includes the assumption that success is meaningful across projects), it seems to us that one also a priori rejects the idea that generalizations can be made about effective planning processes across diverse domains, and the idea of

any number of factors. We have tried to advance that theoretical stream with our work, and have used model building and hypothesis testing, in which statistical significance plays an important role, to do so. on

a.

general

models for

not models which

organizations,

apply only

to

a

2.

Questions Concerning Ordinal

Variables and the Use of sion Analysis

Regres-

Dr. Briassoulis follows what psychometricians call a fundamentalist position in asserting that our data are ordinal because the variables 177

Downloaded from jpe.sagepub.com at UNIV CALIFORNIA IRVINE on June 13, 2016

measured initially using Likert scales, and so cannot be used with parametric procedures such as regression. The original data are ordinal, but the actual numbers used in regression analyses are the averages over multiple raters and differing numbers of raters. We thus do not have five categories in the estimations, as Dr. Briassoulis apparently assumes, but rather many more. The outcome variable takes on 47 different values. We should note that the use of parametric techniques with Likert scale data constitutes absolutely standard practice in organizational and psychological research were

Comparing Regression

Table i

and

Logit Analyses

(Nunnally 1967). checked whether estimating the model with logit, which is more appropriate for discrete data, gives different results than regression. We thus reestimated the outcome equation using multinomial logit (see Judge, Griffiths, Hill, Lutkepohl, and Lee 1985).’ We did it two different ways. First, we made every different outcome value a separate category (i.e., we had 47 categories and 58 observations). Sec-

Nonetheless,

ond,

we

set up

we

categories

Notes As is conventional, both logit estimations included differing intercepts for each category. These are omitted here since the intercepts are not being discussed. All intercepts were statistically significant. b. Actual probability p = .0518 c. Actual probability p = .0508 .

a.

every .5 4.5 to

(i.e., 1 to