General Disclaimer

One or more of the Following Statements may affect this Document

This document has been reproduced from the best copy furnished by the

organizational source. It is being released in the interest of making available as

much information as possible.

This document may contain data, which exceeds the sheet parameters. It was

furnished in this condition by the organizational source and is the best copy

available.

This document may contain tone-on-tone or color graphs, charts and/or pictures,

which have been reproduced in black and white.

This document is paginated as submitted by the original source.

Portions of this document are not fully legible due to the historical nature of some

of the material. However, it is the best reproduction available from the original

submission.

Produced by the NASA Center for Aerospace Information (CASI)

https://ntrs.nasa.gov/search.jsp?R=19700023367 2018-06-29T07:19:56+00:00Z

A

f

t

y^

R

F.

EE

1

I

iTOWARD AN ORGANIZATIONAL

ACCOUN"!' x NG SYSTEM FOR

GOVERNMENT LABORATORIES

° byROSS CLAYTON Ph. D.

UNIVERSITY OF SOUTHERN CALIFORNIA

February, 1970

Supported in part by theNational Aeronautics and Space Administration

Agreement No, NGL 05 -018-098

.. k I A k ,I.^ 1(..

ACCOUI LING SYSTEM FORCOVERNM-.'N7 LABORATORIES

by

ROSS CLAY'ro,% I'h . I).

UNIVERS l'11' O1' SOUTHERN CALII'ORNIA

N•o

:Ecc

0

v4

--V --3 2 6

(ACCESSION NUMBER)

1P G€S

L1f R OR TMX OR A UMBER)

78

(IH )

(CODE)w^

(CATEG Y)

s

PREFACE

This report is addressed to the need for a more pragmatic basis for

evaluating the performance and heal-th of government laboratories. The

report proposes the development of an "organizational accounting system"

for this purpose. Included in the report are some specific recommendations

for developing an organizational accounting system and illustrative "unobtru-

sive" and "reactive" measures encompassed within an integrative systems frame-^ g y

work.

s

This report is based upon a dissertation on the subject, Organizational

Evaluation and the Government Laboratory. The author is indebted to the

Naval Weapons Center at China Lake, California and the National Aeronautics

and Space Administration for financial support during different phases of

the dissertation research effort.

ii

x

.C TABLE OF CONTENTS

Page

PREFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

LIST OF TABLES AND ILLUSTRATIONS o . . .. . . . . . . . .iv

PARTS

I. INTRODUCTION o . . . . . . . . . . . . . . . . 1

OrientationBackgroundOrganization of Remainder of Paper

II. THE NEED FOR MORE FUNCTIONAL METHODS OF EVALUATION . . . . 4

Central ApproachesProblems with Existing LiteratureEvaluative Practices

III. DESIGN PARAMETERS FOR AN ORGANIZATIONAL ACCOUNTING SYSTEM. ..9

AcceptabilitySubjectivityMultiple UtilityAdministrative SimplicityPrecisionComprehensivenessContinuitySpecificity

IV. AN ORGANIZATIONAL ACCOUNTING SYSTEM . o . . .. . . . . 15

Unobtrusive MeasuresSystems FrameworkReactive Measures

V. CONCLUSION , .. . . . . . . . . . . . . 31

iii.

p

*^ g

r

3

t

LIST OF TABLES AND ILLUSTRATIONS

Table Page

1. Illustrative [Unobtrusive and Reactive Measures Relatedto Laboratory Health and Performance . . . . . .22 - 25

Figure

1. Central Evaluative Tendencies . . . . . . . . . . . . . . . . . 4

2. General Systems Model . . . 0 0 . 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 . 0 0 21

r

4-

PART I: INTRODUCTION

It is easy to establish organizations; it is not too hard toget some work out of them; but it is exceedingly difficult to assurethat organizations so established operate at high levels of efficiencyand, above all, effectively carry out the purposes which called theminto be:ing.l

Orientationfi

The subject addressed in this paper is the assessment of a particular

type of complex organization, the government laboratory. In recent years

a considerable growth has taken place in the amount of funds expended fori

research and development both in the private sector and in government.

Concomitantly, increased concern has been expressed about the health and

effectiveness of the governments in-house laboratories. This paper is one

response to the growing interest in developing new methods for evaluating

these laboratories; it contains a discussion of a rationale and methodo-

logy for an "organizational accounting system" containing a set of "measures"

which should serve to lower the degree of subjectivity required in the

assessment process.

Background

This paper is based upon the author's doctoral dissertation, Organiza-

tional Evaluation and the Government Laboratory. The need for more functional

methods of organizational evaluation provided the impetus for that dissertation.

1. Philip Selznick, at. al. The Adaptation of Complex Organizations toChanging Demands, A Report to the Office of Science, Director of Defense

Research and Engineering (Washington, D.C.: Smithsonian Institution,1959), P.J.

r

2

In the process of carrying out the research for the dissertation an attempt

was made to explore and describe how managers concerned with research and

development organizations are handling their responsibilities for assessing

organizational effectiveness. An empirical effort was undertaken because

the literature concerned with organizational evaluation is deficient in

descriptive materials. As a second facet of the dissertation, an effort

was made to capitalize upon the knowledge acquired from the empirical effort

by developing an approach to organizational evaluation consistent with the

research findings. This report draws principally from the latter facet of

the study.

The Naval Weapons Center (NWC) at China Lake, California served as the

(1, focal organization for the dissertation research. A conceptual scheme

derived from the literature on role theory was employed, and data were

collected from thirty NWC project managers and thirty-eight managers from

project sponsoring organizations within the Naval Systems Commands in

Washington D.C. No systematic discussion.of the dissertation research

findings is attempted in this report; however, they underlie the "organiza-

tional accounting system" which will be described below.

Organization of Remainder of Paper

This paper is divided into four additional parts. Part II indicates

the need underlying the recommendation that organizational accounting

systems be developed and maintained for purposes of evaluating government

laboratories. Part III identifies a set of "design parameters." These

design parameters are principal considerations to be weighed in establish-

ing the nature and content of an organizational accounting system. Part V

.4

3

sets forth a "model" organizational accounting system. it provides a

discussion of "unobtrusive" and "reactive" measures that would be included

and suggests an integrative "systems framework". Substantial illustrative

materials are provided to concretize the discussion in part IV. Finally,

concluding observations are offered in Part V.

0

4.

PART Il. THE NEED FOR MORE FUNCTIONAL METHODS OF EVALUATION

This section of the report attempts to explicate the need for improved

means of evaluating government laboratories. Initially, three central

approaches to organizational evaluation which one encounters in the liter-

ature are noted and br2,efly described. Then, some of the problems encount-

ered when one attempts to apply existing approaches are discussed. Finally,

some of the implications of evaluative practives presently employed in the

milieu of government laboratories are considered.

^entral Approaches)

There is a wealth of material which pertains with varying degrees of2

directness to the "criterion problem." Much of this literature clusterst

around what might be described as goal oriented, employee oriented and

systems oriented evaluative approaches.

Goal oriented approaches might be characterized as focusing upon organi-

zations perse and their performance. These approaches give particular

attention to organizational outputs within a time space from the near past

to the present and employ effect,veness as a criterion; i.e., the degree

1. A reasonably thorough bibliographic review of literature on organizationalevaluation is contained in Chapter II of my dissertation which was citedabove

2. The search for a satisfactory basis for evaluating organizations is attimes referred to as the "criterion problem" in the literature on organi-zation theory.

r

s

of accomplishment of organizational goals.

Emlloyee oriented evaluative approaches differ from the above in

that they focus upon the satisfaction of employees as individuals and are

particularly concerned with the status of employee morale, loyalty etc.

within a present-near-future time space.

System oriented approaches differ from those mentioned above in

degree rather than kind. Organizational "health" is the criterion of this

approach and its advocates typically conceive organizations as open systems

and emphasize such organizational processes as problem solving, adaptation

and growth.

These three central approaches are summarized in the figure below.

Approaches

Pivotal Elements

Goal Oriented

Employee Oriented System Oriented

^j

s

EvaluativeDimension

Unit of Analysis

Systemic Functirn

Time Space

EvaluativeCriteria

Performance Satisfaction Health

Organization Individual System

Output Maintenance Process

Present -Near Past

Present -Near Future

Present -Distant Future

Effectivenessetc.

Morale, Loyalty,etc.

AdaptabilityGrowth, etc.

Fig. 1.---Central Evaluative Tendencies

H

t

Problems with Existing W terature

'T'here is no shortage of literature suggesting h w organizations ought

to be assessed; however, there is reason to doubt that managers in complex

organizations are employing th. s "conventional wisdom". Additionally # organi-

zation theorists themselves are dissatisfied with existing criteria and

methods for ,assessing organizations. Some of the reasons why this is so

are discussed in this section.

Many of the significant contributors to the literature on the "criterion

problem" are also its principal. critics. Frequently, prior to offering

their own views as to what is necessary to resolve the problem of developing

satisfactory means for evaluating organizations, these individuals take note.

of the deficiencies in the existing literature. For example, Katz and Kahn

have pointed out,

There is no lack of material on organizational success. The literatureis studded with references to efficiency, productivity, absence,turnover and profitability -- a1l of these offered implicitly or ex-plicitly, separately or in combination, as definitions of organizational.effectiveness. Most of what has been written on the meaning of thesecriteria and on their interrelatedness, however, is judgmental andopen to question. What is worse, it is filled with advice that seemssagacious but is tautological and contradictory.)

other authors such as Seashore feel that present models of organizational

effectiveness are too simplistic. Seashore states:

Historically, much of the work so Ear. has rested upon a simple dichotomousconception of organizational effectiveness: assessment variables areoften viewed as belonging to two classes, some representing work perfor-mance and others representing satisfaction or morale. The difficulty

1. Daniel Katz and Robert Kahn, The Social. Psychology of Organizations (NestYork: John Wiley and Sons, 1 57), p. 149.

i

r

with this view, aside from its being too simple to be helpful inresolving many issues, arises from uncertainty about the relation-ship between the two classes. Some hold that there is an inevitableconflict between the requirements of formal organizations and thebasic human needs such that the two classes of evaluative variablesare incompatible; others hold that high satisfaction is ordinarilya necessary condition for achievement of high work performance. Theweight of evidence suggests that neither of these views is adequate 1and that some more complex image of organization effectiveness is needed.

Some additional shortcomings that are identifiable in the literature on

organizational evaluation are presented in summary form below.

---Frequently, erroneous "simplifying" assumptions are made concerniagorganizational goals; for example:

1). That organizations have a single paramount ,goal or a smallnumber of compatible ones.

2). That organizations rationally pursue their ends.3). That agreement exists as to what the goals of the organization

are.4). That organizational goals can be specified in operational

terms so data can be obtained on the degree of organizationalgoal attainment which has been experienced.

---The tendency to treat organizations as though they are all the same.Much of the literature on organizational effectiveness is based onthe model of the factory; there is a real question as to whether theresulting analyses are even modestly transferrable.

---There is a shortage of literature which treats effectiveness orhealth explicitly, in operational terms.

---There is a virtual lack of descriptive materials on the evaluativepractices actually being employed by managers of complex organizations.

These deficiencies suggest that organization theorists must make a

more meaningful contribution in the area of organizational evaluation. To

date, the majority of the contributions have been quite abstract% Conceptual

1. Stanley, E. Seashore, Assessing Organizational Performance With BehavioralMeasurem_nts, (Ann Arbor, M i chigan: The Foundation For Research on HumanBehavior, 1964), p. 72

I

t

a

frameworkswhich "ideally" or "in principle" are applicable to a wide

range of organizations are basic to the building of generalizable theories

for organizations. Nonetheless, theorists also could make a meaningful

contribution to organizational practice by formulating conceptual frameworks

in operational terms and then testing them through application in particular

E

organizations. A more pragmatic approach to this problem seems necessary.

evaluative Practices

Realities of organizational life in the milieu of government laboratories

are such that systematic organizational evaluations are conducted infrequently

if at all. Management personnel, in the laboratories and at the headquarters

level have numerous demands on their time emanating from other facets of

their wor}; and these preclude their devoting much time to orderly analyses

of laboratory health and performance. In some instances an annual laboratory

review is made by a panel of experts who take time out from their normal

duties to spend a brief time reading or listening to reports and then offer

their experienced judgements as to the laboratory's status. In addition,

there are occasional ad hoc evaluations; however, both of these approaches

produce an evaluation at a particular point in time rather than a multi-purpose

evaluative data base available for use by managers throughout the organization.

Despite the fact that they can devote little if any time to evaluation

activities, managers, of necessity, are ,Judging the performance and health

of laboratories and are willing to and do differentiate between them in terms

of their capacity to perform. To the extent research focused upon a single

government laboratory is generalizable, it appears that managers "simplify"

their responsibilities for evaluations by relying upon the limited data that

r

`x

9

C

became available to them through normal work processes and direct personal

experiences with laboratory personnel,-reports, etc. It also appears that

substantial differences exist in the laboratory "role definitions a held

by managers at various levels in agencies; that is, in the expectations

that are held for laboratory attributes and behaviors.

If one combines the above (differing expectations and exposure through

direct personal experience) with the inevitable differences in personal and

organizational self interests of managers at varying levels in an agency,one

can readily surmise that there is considerable "relativity" of both "facts"

and "values" in the normal processes of laboratory evaluation.

Differences in "values" and the concomitant subjective quality of

organizational evaluations .seem inescapable. However, the limited nature

of the "factual" data base available to managers for their evaluative

judgments appears to be a fundamental shortcoming which can be addressed.

This constitutes the "need" toward which tt^e remainder of this paper directed.

c

r

4

10

PART III. DESIGN PARAMETERS MR AN ORGII.NIZATIONAL ACCOUNTING SYSTE14

It is apparent that in developing an evaluative ;approach considerable

care should be given to ensuring that it ;iill be consistent with manager-

ial needs and organizational realities. Our objective in this section is

to identify and provide a discussion of some of the significant parameters

which should be taken into account in the design of an approach to organi-

zational evaluation. Design parameters which will be discussed below

include the following: acceptability, subjectivity, multiple utility,

administrative simplicity, precision, comprehensiveness, continuity and

specificity.

Acceptability

One of the critical parameters for any workable approach to organi-

zational evaluation is a pragnatic one, its acceptability to evaluators.

It is desirable to design into an approach the basis for its achieving a

substantial degree of face validity in the eyes of the potential user. An

awareness of some of the factors which are perceived to be important to

effective laboratory performance by these evaluators coupled with a working

knowledge, of terms and concepts germane to the evaluation process which

have meaning for them should prove helpful in this respect. Data of this

nature can be obtained from persons who make laboratory evaluations by

probing Their thinking on such matters as perceived determinants of success-

ful laboratory performance, the degree of importance attached to various

laboratory roles, behaviors and attributes, and evaluative criteria and

data presently in use in the laboratory's milieu.

J,

3

e

11

Subjectivity

As noted in part 11 of this report, subjectivity is present in the

evaluation practices of managers for a number of reasons. It is apparent

that development of a data base on laboratory performance and health will

not by itself produce an organizational evaluation. Availability of an

organizational data base may produce an improvement in the quality of evalu-

ative judgments, but it will not supplant the necessity for subjective

judgment. Evaluators ultimately will be required to synthesize the contents

of an organizational data base with additional information inputs and

value considerations. To be of value the data base must be interpreted by

knowledgeable persons as to its meaning in relation to organizational goals,

previous experience, etc.

This lack of automaticity in the organizational evaluation process

has important implications for the design of an evaluative =iita base. Care-

ful consideration must be given to the set of evaluators for whom the in-

formation base is being developed to ensure that their informational needs

are being addressed. As noted below, this is particularly true if the

data are to be made available to both headquarters and laboratory level

managers.

Multiple Utility

The multi-purpose character of an organizational data base cannot

be overlooked in its design. Too narrow a conception of the potential

utility of an organizational data base should be avoided. Organizational

evaluation is an important managerial function in and of itself, but it

is not isolable from other managerial needs for data related to organiza-

tional conditions. Awareness of those managerial needs provides additional

as

r

12

guidance as to the form and content of the data base. These needs include:data which permit managers to exert control and take corrective actions;

data for use in the organizatiofts planning and policy setting processes;

and data for use in the political processes of the organization.

A related design consideration concerns who is to make use of the

organizational data base. Are the data being assembled and analyzed for

use by laboratory managers, headquarters personnel, or both? If, the data

are to be made available to persons at the headquarters level, the manner

in which they are to be employed becomes crucial. If the data are to be

used as a basis for criticism, further inquiries and managerial inter-ventions, one would expect little enthusiasm at the laboratory level for

developing and maintaining a quality data base. Conversely, if laboratory

personnel are convinced that the data are for labortory level managers

only or that they will be used in a positive manner by headquarters person-nel, they are more likely to cooperate in the development and updating of

an accurate set of data. These perceptions of personnel at the laboratory

level are likely to determine whether the organizational data base is

viewed as just another management report or as a significant management

tool worthy of use and improvement.

Administrative Simplicity

Another basic parameter to be considered in designing an evaluative

approach is its administrative simplicity as gauged by the amount of effort

required of the evaluators and those who must, administer the data collection

systems and assemble and analyze the evdLluativo data. Factors such as

amounts of time required and costs involved in both the initial and subsequent

a

13

applications of the approach are important design dimensions. The designer

must confront the inevitable trade-offs between what would be ideal and

what seems feasible. Approaches which presume the establishment of new

record keeping systems or rely upon the administration of lengthy question-

naires which require considerable time to complete and are time-consuming

to analyze are unlikely to succeed.'

Precision

Related to administrative simplicity as a design parameter is the

degree of precision f4 be sought in the data base. Data which are pre-

cise and elegant are clearly desirable, but organizational realities may

dictate that one settle for something less in corder that timely inputs,

` can be made to management. At times it may be necessary to use data from

existing rather than ideal sources, proximate rather than direct measures,

and qualitative rather than quantitative data. This is particularly so

while an organizational data base is being established; opportunities

for gradually making the data base more precise will exist during each

successive update iteration.

Comprehensiveness

Another design parameter is the degree of comprehensiveness to be

sought in constructing the evaluative data base. In reviewing the liter-

ature on organizational evaluation one of the things which is striking is

that many of the theorists who contribute to this literature largely em-

phasize those variables which their particular conceptual approach best

explains. In constructing the evaluative approach being suggested here it

seems appropriate to seek information pertinent to each of three evaluative

dimensions; namely, performance, satisfaction and health. In short, the

{

r

14

approach should seek to be as inclusive as possible consistent with the

parameter of administrative simplicity.

Continuityity

Many of the evaluations of government laboratories with which this

writer has become familiar have been of an ad hoc nature and tend to

emphasize differing facets of laboratory performance and health from one

occasion to the next. Doubtlessly, there has been and will be considerable

value in such inquiries; however, the approach being suggested in this

paper calls for development of a cumulative data base.

A design parameter is that the evaluative approach be consistent with

a conception of organizational evaluation as a continuous iterative process.

underlying this parameter is an assumption that much can be gained from

instilling continuity into the evaluation process through the use of period-

ically updated data or indices which permit trend analysis. It seems

unlikely that an optimal evaluation approach can be conceived and developed

in a single, discrete effort. While attempting to design in continuity,

one would expect that it also will be necessary to modify and improve

upon the evaluative data base at each iteration.

Specificity

Finally in designing the evaluative data base potential methods of

analysis must be considered. one obvious basis for analysis is to employ

periodically collected data to compare a laboratory with itself over time.lel-

z For this purpose descriptive data which are unique to the laboratory

are quite useful. However, the importance of data which are more general-

izable should not be ignored. Most laboratories are sufficiently similar

A,

I

in structure and function to at least a few other governmental and contract

laboratories that some value would be obtained from measures and data which

could be used in comparative analyses between organizations.

15

r

16

PART IV • AN ORGANIZATIONAL ACCOUNTING SYSTEM

Our purpose in this section is to outline an evaluative approach which

is consistent with the above mentioned design parameters. The essence of

the approach which is elaborated below is a methodology for developing and

arraying a set of data which can be used by managers who are reasonably

familiar with the organization to be assessed; that is, an "organizational

accounting system." The approach does not seek to produce an effectiveness

score or grade for an organization; however, it is intended that the descrip-

tive data included in the evaluative base enable managers to make informed

judgments of an organization's performance and health.

Methodologically this approach involves the use of data derived from s

-` combination of unobtrusive and reactive measures and arrayed within a systems

framework. 1 The methodological aspects of the approach are discussed.in some

detail below; illustrative tables are included.

Unobtrusive Measures

Webb et^.alr have written a stimulating book on this topic which illus-

trates the potential of informational resources which exist in one's environ-

ment for the construction of unobtrusive or nonreactive measures. These

1. Unobtrusive or nonreactive measures are based on research data which arenot obtained by interview or questionnaire ands as a result, are notsubject to the "reactive effect" of measurements which directly involverespondents. Reactive or obtrusive measures are those which do rely uponinterview and questionnaire data. For a thorough discussion of theseconcepts see: Eugene J. Webb, et_ 1_ Unobtrusive Measures: NonreactiveResearch in the Social Sciences (Chicago: Rand McNally & Co., 1966).

}

F

J,

1.

L

17

authors begin their book with the following observation on the state of

social science research.

Today the dominant mass of social science research is based upon inter-views and questionnaires. We lament this overdependence upon a single,fallible method. Interviews and questionnaires intrude as a foreignelement into the social setting they would describe, they create as wellas measure attitudes, they elicit atypical roles and responses, they arelimited to those who are accessible and will cooperate, and the responsesobtained are produced in part by dimensions of individual differencesirrelevant to the topic at hand.

B_ he principal objection is that they are used alone. 1

In setting forth their argument in favor of the use of unobtrusive measures

to supplement reactive research techniques these authors provide numerous

examples of nonreactive measures which have been reported in the literature.

Two such examples are cited below to illustrate the nature of these measures.

The floor tiles around the hatching-chick exhibit at Chicago's Museum ofScience and Industry must be replaced every six weeks. Tiles in otherparts of the museum need not be replaced for years. The selective erosionof tiles, indexed by the replacement rate, is a measure of the relativepopularity of exhibits.

Library withdrawals were used to demonstrate the effect of the introductionof television into a community. Fiction titles dropped, nonfiction titleswere unaffected. Z

A number of examples of research which has made use of nonreactive measures

constructed from government records also are cited in this book. The following

quote is illustrative:

A particularly interesting and novel use of data comes from the studyof city water pressure as it relates to television viewing. For sometime after the advent of television, there were anecdotal remarks abouta new periodicity in water pressure levels - a periodicity linked tothe beginning and end of programs. As the television show ended, so thereports ran, a city's water pressure dropped, as drinks were obtained

1. Webb, 12. Ibid., p• 2.

i

X

I

C'

185

and toilets flushed. A 3raphic display of this hypothesis was providedby Mobley (1963), who published a chart showing the water pressure forthe city of Chicago on January 1, 1963. This was the day of'a particu-larly tense Rose Bowl football game, and the chart shows a vacillatingplateau until the time the game en4ad, when the pressure level plummetsdownward.l

Unobtrusive measures described in the book which are most relevant in

terms of this effort to develop a data base for organizational evaluation

purposes are those based on archival data. Of particular interest are organi-

zational records and files from which data can be abstracted and normalized

for purposes of trend analysis.

In their discussion of the hidden potential of existing records these

authors note the value of constructing index numbers as an alternative to

the employment of experimental designs in acquiring data for comparison

purposes. They point out that transformation of raw data into index numbers

provides an improved basis for comparisons even though such numbers may fall

short of some ideal in terms of their interpretability.2

A number of the features of unobtrusive measures make them very attractive

for purposes of laboratory evaluation. The fact that they seem to be based

on "harder" data than can be acgijired from reactive techniques should make

them more acceptable to managers who have been trained in the sciences and

engineering. Clearly, tionreact,ve measures are consistent with the design

parameter of administrative simplicity, and they are a source of diagnostic

information which managers can use to identify and assess changes in their

organization's performance, character, complexion and methods of operation.

Finally, these measures make it possible to go back several years in an organi-

1. Ibid., pp. 73-74.2. Ibid. , p. 7.

l

k

r

19

zation's history and extract data which enable comparisons to be drawn with

prior years' experience. it is for these reasons that the evaluative approach

being suggested in this Chapter calls for developing a data base with a com-

bination of reactive and unobtrusive measures.

Nunes sous unobtrusive measures pertinent to the health and performance of

the laboratory at China Labe were developed as a byproduct of this author's

dissertation research. Several are described in the discussion which follows,

and others are included for illustration purposes in Table 8 which appears

later in this Chapter. Our intent here is to illuminate this particular

method of acquiring evaluative data, not the results.

One of the concerns of managers in government laboratories is that their

organizations may evolve into contract management rather than science and

engineering institutions. Management's need for data which will enable them

to detect and monitor any movement in this direction can be employed as an

example to demonstrate the value of unobtrusive measures in organizational

evaluation processes. Several measures relating to this managerial concern

were developed for the laboratory at China Lake.

One measure was developed from records maintained by that laboratory's

Supply Department. Direct Support Stock Units (DSSU'S) are serviced by Supply

Department personnel. These DSSU's basically are mini-shop stores which

contain small electronic components and hardware items; they are widely3

scattered throughout the organization being located at sites which are con-

venient to in-house technical personnel. Supply Department records indicate

the number of DSSU'S which are "active" and those which are receiving"curtai.led

14

L:

20

service" at particular points in time. Active units are serviced twice a

week; curtailed service units are those which are serviced irregularly in

response to customer requests. Trend data on the number of active and

curtailed service DSSU'S were used as one unobtrusive measure of in-house

technical activity.

A related measure was developed from records maintained by the China

Lake's Engineering Department. These records provided data on the quantity

of "sketch" numbers (SK's) which had been issued to the organization's

Technical Departments. The data had been kept for a period of years and could

be exploited with a minimum of effort. SK's are the initial engineering

drawings which are produced as a result of in-house design and development

efforts; they constitute a fairly direct reflection of the

amount of development effort which is taking place in the organization. Use

of these data in portraying trends over an eight year time span provided a

second "barometer" of in-house activity.

Examples of other unobtrusive measures developed for the NWC organization

which were partinent to the managerial concern under discussion include:

1) Trend data on the number of contract aftinistrators employed bythe organization, and the number of man-hours devoted to trainingprograms on contract administration for in-house personnel;-

2) Normalized data on the number of patents and publications per onehundred professional employees; and

3) A chart depicting the yearly ratios between dollar commitments forin-house labor and project and total funds expended by the organi-zation.

In the section which follows a framework for a data base which includes

these unobtrusive measures is described.

y

i

r

• 21

Systems Framework_

The utility of some conceptual framework for arraying information acquired

for an evaluative data base was self-evident. It seemed likely that a macro

level systems framework would be suitable. A'variety of systems models are

available in the literature; but one which seemed tailor-made for our purposes

1was found in the book, Social Indicators. This book articulates a rationale

for the development of a new social accounting structure to provide the gov-

ernment with a data base for its decision-making responsibilities for the

health and performance of the social system. Many of the concepts contained

in Social Indicators are transferable from the societal level to the level of

organizational systems. In fact, examples of the application of his ideas

to the organizational level are provided in the contribution by Gross.2

In his essay Gross addressed himself to the question of how one can best

appraise the state of a nation and recommends use of a social system model in

developing a framework for social system accounting. He identifies system

structure andp erformance as two multidimensional elements which can be used

. a as a framework for a set of social indicators. In describing his model he

states,

.'

The elements of system structure deal with the internal relations amongthe system's parts, the elements of system performance withthe acquiringof inputs and their transformation into outputs. Both involve relationswith the external environment. This model, or any part thereof, may beflexibly applied to describe the unique characteristics of any countrywhatsoever, no matter what the level of industrial development or thetype of political regime.3

1. Raymond A. Bauer (ed.), Social Indicators (Cambridge, Mass.: The MIT Press, 1966).

2. Bertram M. Gross, "The State of the Nation: Social System Accounting,"Social Indicators, ed. by Raymond A. Bauer (Cambridge, Mass.: The MIT Press,1966)s pp. 154-271.

3. Ibid., 155.

if

E INPUT OUTPUT

22

Gross portrays his system model graphically as indicated below in Figure 8.1

lie uses t-he term, transformations, to refer to the changing of inputs into

outputs.

i

Figure 2 -- General System Model

Gross goes on to elaborate on the elements involved in system structure

and performance. For system structure his model includes the following:

people, nonhuman resources, sub-systems, internal relations, external relations,

values, guidance system, and from preindustrialism to postindustrialism.

Far system performam:e his model includes: satisfying interests, producing

output, investing in the system, using inputs efficiently, acquiring resources,

observing codes, behaving rationally and industrializing and postindustrializing.

In Table 1 which follows a flexible application of the above elements is

made in ordering an illustrative set of unobtrusive and reactive measures.

1. Ibid., 182.

r

23

:1

•

t^

O

-

1 •

• aH -^

0 G' AM v O 0^3 >wr1 1 O 0 t 1 O cZ r, o $4 n4 to to MCd 0 t + •ri • U O 0 -r1 $4 41 yt Cd •r4 •rl 0-1

V •ri 0 -P p 1 trl $4 '► , O C1 O O rl .•. M 4S Cdo P, c3 0 134 O p 4l W -r4 a, >-i Cd b M P M I~

M W f=4 W t 1 4.1 0 ti4) +a A c)OQ ai QOS+ pi U :~ i`i O AN!- t{Z, U 0 •r

J^ ^ •rrq N u O u -H

C) ^,i O V 1 I'7 •+ Q4 V ra b }y •r1 ry r^+CZH

C^F:

Of

• .-IA

C) O O:.+) r •ri ^'

V)'`li

r-1 4:cs d

Mtd

G1O .=: :5-- 0 0 :3 C) m

4-3 ta+ O 4i ti C7 r7+ ^0 ti

E-H .0 E-1

Io o 01a,la^

O40io0

-r'w d to:s Or-lMrlti• Po 9 W 0 g 0V)CdPL40 •ri o V O N •,A O .. V 0 U

•'.l 'CS b fJ 4-1 4-) •r1 Pl. C4 •-. -r-1 0 -r1 Q

o , ` 0 C.; 0 OO -r1

C-14-1 C) •r1 rC7

Cr"1 r"1 ovcS O E:•-1

V;rJ

rcl p^c3

O t^t+ .^

Or-1t 1 .O P, 0 0

rdE•1 o E:-1

r^EC-1

r •1 --t, 1rI cr2 v ^: r cs

4-)G ^,

r-1 a, , 1 U 4) O O H 04 E1 a^43 P 4-3 0 43 d h o w t4 0 G,4 0 04 P-1 •rl ^0 0 0 V. r-1 r: ; ^ '•> 2S r•! 'J V O ,ii — 4i O 0 i 1 r0

+r-1tl0

U :> C) o'd

O ti M 9 1 • .3 •-P •N tt

O 1v'3,0 O

^d

PU E-i

0 p 1 -M •P CJ 0? 0 0 43 r, $ i r P 0 c3

r-1 ci r!U

0O EA

Url rti ..r--1 E=1

-.ti0 t1

p ,;0

C' V vt•1,.)

P4 C) O O o C) 0►4 t} A a '>'4P4 f'4 p., Vj C3 o .O O W r^S O

0

C$cs

U - - 0 O tp 4i ;itt ^+ ti

..col a ^

N

C7 V M N N

4a csti ^ t) t1 C1 : i

•r {t

O•ri

o

tJ 41 r-i ci1"

v c!t^ V .c2 ci •U1

v, '- Q E;-1

n

•

i• ii

i

04

ti

hcsOg v

Cs

0 Oco CH

doP4

b

^43

C1 r-'1cl

4-34

O

O0 cu

of.>

Cd Iwo

•3

1

GE-+

! t71.t ^

^T

3

i

U i+

O t"

V}

U

coCd0

;ice

1;-I10

4)^ O

t3 r •11 ^-a

24

•rl . i 43O y O Gi

SP4 Sri

•-)O

4 +^ -P

O14

0

a. a wti0

1 p.

S O r.., •,.{ VJ Fa 0) m0 0•ter+s s a, Fi o .rn,

N M 4-) 0 F4 r4 ► O 00 +11

$4O

+^ oa4.1 CO OH too r•-{ O rx, 0 1-11 F, w vi

cr) cC r :s .N.^ci

O ^ 4^ ti a UN F♦$4 10 p 0 4a Al

F^ O U t+ p, A, 43 O rclA +^ O •rl •rl •rj --1 O O O •d O•ri O H p4 O •P ^c 0 to •n W tS F^ti-1 A V n .d p, •r4 V. Q $4 c,4 O N ci 10 c^ O0 4:1tj 43 00, 0 0 -PN 0 O ro:a m $4 4 A V. 0 4--1 w r A4 C) Fi Id .^t; 0C)0a cd¢,•H0 0 00) 0-90 4-I •H 4-1 P PLi 0 43 P co 0 ru Pi i'4 0

•rl > cs r-1 C) A-, 4,-) PI P, O I,j p, rC l •r1• 3 O F•+ Q^ N • r-I >O Cti 4-I O cl 0 N 9 C) i I F'' •V N 4--1 a ) 4^ v) a •H F4 ,o O t> 4^ F4 a) p, c^ c^ ,-I ra c0 CJ Cd co CS -P •r4 .>I O co O J^ ^ ,Q -F^ 0 04-I v M • q 0 Q co Ca •rl " -P cS +d ^;-, c^ ^^ 0 FaQ •r1 O O 0 F♦ 0 45 •rl (1),,Q 0 co 0

•p CJ a h001~0 S;c3 0 to0 010 r-10t+ 0 cM O •,-I 0 •t + -i 4.5 '1,i 4-I V z ^-a •rl rr•• 0 c3 4-I Pi O H ci N 4,C!1 UJ •u . ^ ci 0 O •rt V) O O 4.3 ,.) V Fi r-1 W O 000 W 0 .H

cS tJ cS 'd m 1"4 > •-) (-11 a •rt ;:

1•O cd ^I ♦ 0 ` % Q 43 ,> { ZS

Fi to t 2 ^I C•7 On • 1 ;-4t-q •ri O T r l , ,-0 r1 O 7-1 rQ -P I''4 / \O l+ S - M(D© f~.' •r4 co •rl N v O 0 0 O rP, Fi d cS O `O Pi N r-i 0 r-2, 0 r'^P •r1 r•{ W, r-1 •rl r: 4 0 ^^ c3 = 3 r-1 -F) U ri ► .O .n Lt c•3• Cd G .a 9-4 •rl

4-3 S C) ',•-) f^^ r-•1 43 N O

a-OA

C's O

0M

to

op O•rl

03O

"a)cs0IN,

443

O•rl O 0V 43: ; •rl

44 V 4-3O

-P 0O ^, t^ +) P ^t G^ t:P

r^ .,1 . Q) r-1 Ql .4 S 0

G 3 t ' ,^•,,:^ C,'a^

;It _

f?1

s-1•rl

O Fa

QO ^

H (U

aH

i

0

•ri

OUi1

Cn

H

oa

O

CD i;F; c)

C5I^1W

r

w

43 m•r1 ^i

10 4-5

•0 7 O Sri0 +-1 ...4 43

•r4 r+q T10

{ 9Y•1 9 J7 ^ 0 V)0 0 4 ^ , ^e. , •r1 t-.- U i 0 •^{ 4.2 •r•1 d.a? ^1J

t)Pt :, d w

.43 c) pc^ ^ U 43 0 d 0 0 SIP

r

r1 v L> P, U1

43IP$ .1^ +

4z•rl

Pt

C►-i G I i 1 t^ i ci

d O ^ ^ C) r

r

W

co

^S

jJ

f

. 25

Q) m

Sri

•O 0 O 44

0A w 0 .0

s

O SIP

A

o 11 Sp 43 O

.N b0 r-1 Pct

0

>l1 ccscd H P4 $^i

oCH N o

oo

0V .

ri430 0 o 0 N XLD wd as o ff a ca

9 Q P p1>v•,4 o] 40 4 O V. ^] trl ON H P c0S> x+14 fH<4 ^E1 N 0 0 Cd«-^ r+ co to d o P,

P, o d Cl•{' p a ai

^ 5. a) A o0 O P:n r •0 o 0

A

Q 0 ^0

i

Oy O 0 O-P 05 p^••>1 r-1 ti >l A •r4 ^^.W ^i43 d '; 'd rl ?4 o) 043 o 04 'd •13 asC) 0^ •r•1

d >1 :; rD •r-i rd O c3 O a^ d rl 0 r-1 4.3 H W U H rd Nf 1

4.3r^ s^ 0

•r1r •t`{ ^^."

4-1 •r-1

]r +^.^ CG ::y •r i Cd $1 d •r-i O

9 .G C: 4i O 4-1 P, r-1, i'^

r:C)

•-4{

,, M OtrlA

4)043

S•iO

G' .^+bD c') ,Q

N F4a) :3

0 'J

O•'• O•r♦0 to VJH O O O -H CJ 04) d •^+ 4.) r-1 1~ l;•1 r-1 p4

0cd O d 0 ti ^' O

1,M N

to ri rd d

41 ^'a ^ d (^ C^ 4-1 • r4 O d ^' 4-5 S4 •4-3 C) +3 0 tO CH O CH 0 4-1 P • .3j •rl i'-400 d 0 0 p 0 .43 ^ r O c3 ^ddE-+cl 14d) 0 " 0 OON to

H A >I j~ t; C) o V 0 A =a o -P o O OO, rd ^+ •r-1 ti O ;:'^ 0 • -1 ^.' b 'd Si 'd ,O 1d 0 U m p, O a O Gr' ..i $4 •r-i.o r8

A.2

a 9 U v.0 .c? .o

d 0)

Mt^H

o P"k

v to $4 N 9 5 V) tiV S-1 E•+ E♦ A 43

m Q1^i

V C) p

^430

0

-0

- iT•r1

cs1 9) H

0 t

VP4 d

M

V)O

NCdO

as

Oc 1t

Cf•1

O 4-:1O

^ F+^ 0d r-1s-4 t4

^a

W

/N7

W

C

26

43 43cs is

z mw

1 O O43 cl

g U

4-3 •ri O^^ A. •^ ^ toN U v O N @1

•ri C1 Cl t, C) ^Z

4J ^'f •rA Ito 43 rl CS

•ri :!^ V ^; Ci

O •^C4 Pi ti

t-IO 0 ,5C © •ri

cS cZ 1 cS CS •,a ti 0 ,o Pi t) V

o o 'Cl (1) t, •NO ,o ;^ v ,Q J ^v a^

c74 w

a.

R

x

C

• OU

11

r.i 0cr•a

g^ 4'i U

J .1

E1 -? a 0v rl

l 1 :.4

^1^N^

1

e

p1'

i

- -P 4)^

_.. Y

O ad O0 04

43 0

s-.$1 r♦ O •N `^ +^ O

•a± ^' N y a o

0 O 0E-1

to •ri 43 b C1H :0 U

O 0-P cs .A ^

F1 cs 4.1 ccd 4-t H ;4 ^-sO

U I N cj :_^4 cd c3 ,QO 'd

ari •ri In 4.3 Q

0 rl 4-3 V 'd CS dc b GA

CAS

c, d O OAb 4-1Q) t:0 :0 •r♦ N cJ

,G •- O N rd to •i^ •r 1 0 0 AC? •ri r1 U .;J ;-4 v r-1 'd A4

v) P rl «S •N •-I G4 CS P E-i OO C) vi ,3 •r1 C) F CS .N

+^ ^J O+

r; 0 C) 04 t,-) $• :: 'd dCl 0 P t+ " t•i &1 to O d 0allx •r! c3 O -: to 44 CS ; ; N

43 ;-A v 0 0 H -4 0ti ctf ^ 4 .4 P4 Ova O r ♦ O

O CS ca O ^; ^, 4.3

O 'd 'C1 C) t) F; .a •r1 RIO OC t;

fa CS W ,'^ ;1 =1 v ,O U) =i to `ri O.-A

N H x a a

aVoS

Y

t♦

P-1

G;

'na J G

^ ^i•N;O 0

l C3 t P •rtri t i :5 t1 •1.

co N

r,t : ; • J c; 4-3

r t

V1 07

O v

cS ^~ Cr^ UJ 40 P: f.M -4

r^i ^•i ^•tct c} C)

U .3 CI

f{^^F^

X

27

Reactive Measures

The third major element of the evaluative ap proach being ou tlined3 pp e gin this

Chapter involves employment of reactive measures to complement the unobtrusive

data collection methods discussed above. Since organizations are abstractions

and are dependent upon the behavior of their participants in order to function,

it seems reasonable to include data on participant perceptions in an evaluative

data base. Admittedly, such perceptions may be erroneous and there are

methodological limitations to reactive research; nonetheless, they are impor-

tant data because of their behavioral consequences. The statement of Harold J.

Leavitt that, ", The world as ie is perceived is the world that isr

behaviorally important," is suggestive of their managerial significance.l

Our purpose in this final section is to suggest how a reactive instrument

for use in organizational evaluations might be constructed.

The general system model depicted in Figure 8 and the structural and

1 performance elements of Gross's social system model are suggestive of reactive

` data that should be included in an evaluative data base. People represent the

Fundamental input of an organizational system; how well the internal organi-

zational environment satisfies their interests is an important consideration.

' Similarly, it is desirable to maintain data on the extent to which non-membersY ^

of the organization who are a part of its external environment are satisfied

with its performance and outputs. Finally, perceptions of organizational

participants concerning the transformation processes which the organization

employs in task accomplishment should prove useful.

1. Harold J. Leavitt, Managerial Psychology, (2nd ed. revised.; Chicago:fi

University of Chicago Press, 1964). p. 35.

s

r

4

28

V

f.

A questionnaire format employed by Stephenson and Gantz to inventory the

perceptions of organizational members concerning the climate of their organi-

zations seems well suited for purposes of acquiring the desired reactive

data. l For each item on their questionnaire they asked respondents to in-

dicate the ideal circumstance and the actual. This permits comparisons

between the two and should be helpful in problem identification. It also

makes it possible to compare the "actual" responses at differing points in

time to discern the nature and direction to changes in perceptions. This

questionnaire format is illustrated in some detail below in conjunction with

a listing of potential questionnaire items which were compiled from data

acquired during the research effort for the author's dissertation.

F

ii

XI

K

8

Ytd

1. Robert W. Stephenson and Benjamin S. Gantz, "A Problem - Oriented InventoryFor Use in Research and Development Organizations," 1967 (Unpublishedreport available in the files of the Naval Weapons Center).

r

1

r29

Ques tionnaire For:.at

INSTRUCTl tv. u: Put "A" for actual. and "I" for ideal inthe appropriate colural. How characteristic is this of yourorganization?

' Not at all A little Modorately Extremely Don't KnowCharacter- Character- Character- Character-

istic isti c istic istic

. EXAMPLES:

Employees are committed to goals.

Employees have substantial Freedom in their work.

.^ Or.—

Questionnaire Items

Y . Mcinbar SatisfactionAction

Rmployoes are adequately recognized and respected bymanage:rient.

Employees enjoy their work.Employees have a high level of ©nthusiacm for their work.&iplo° eos exhibit a sense of urgency and i mportance with.^ t^ y p

respect to their work.Employees have adequate opportunities for professional

gro vy th .Wor ; assi.gnod to employees is technically challonging.Employees participate adequately in the organization's

policy proces Os.Emplo ree accorplisk=,enis axe suitably re, yarded.

s1 Empl.oyeos feel they are orap).cy ed in a succes:t.ful and viableor"^niz t^ti on.

E.mn].oyees hGf.* substantial fr. ► eodorl in their work.

II. .4on-M, enber Satisfaction

lha o a •ni? atiun is responsive 'Co the of its spc: scars.The cl` nen;: and ccrmet—nco of the or,' T a2izat cn t s per.,(. net

a.r.o reco Mi zed and rospecte i by headquarto r ; levea por-

The 2 , "; 'M ira^ on lhi as tihr.' repuv .vi.on of b''ing .an ei ect .ve1 al) 6 ,atc:.-y

The o. ^, ^ z ,. ^. u ^. o n is re ...p or.s i v e to th .. :. . ., cl.., of the us .. ,.aC.

30

r

The techni eel wor lic of the laboratory is regarded aswell conceived, plannod and co:ducted.

The products of the organization ere regarded to bereliable and effective.

III. Transformation Processes

A. Resources

The organization has highly qualified personnel.The organization's facilities are adequate.The support services provided to technical personnel are

adequate.The organization has sufficient personnel to accomplish

its workload.Funds available to the organization are adequate in

t quantity and sufficiently firm to ensure continuity ofeffort.

Unfetteredfunds are available in the organization tosupport investigations of new ideas,

B. Managerial Climate

The organization evinces a high degree of interest in newideas.

The organization requires its members to adhere strictlyto the chain of co in,snd in communications.

Engineers arc permitted much latitude in their.selectionof technical approaches.

Ropercussi.oris are Minimal if an engineering approach whichwas well conceived fails.

Employees are encouraged to attend professional i,:eetings,present papers, etc.

Decision making authority is widely dispersed in the organ-ization.

The organization's operating philosophy emphasizes anorientation to the needs of the users of its products.

Project managers have considerable discrot.ion in terms oftheir authority to make decisions as represontativesof the orga.ni ation.

Interpersonal vol&tions of orrn.ni.zational e.npl.oyeas exhibitmutual respect, slnd good co!^,:.I:unieati nn .

The organiza.ti ^o.^ responsiveness to its sponsors.The organizat; -k n,' Aampl oyoos consider their laboratory to

be a coi.:p et,: 3• to other gotr ez'n.::nt laboratories and/orprivate , ndu^ t!t'.

C. Pcr► sonrxel Polic i es and Pis a , tices

^{1h^j organi zation, p ion ai.1.c^ • ► s its ^1 cons4deraale, latitude in' tor;as of their interrla:. to aid filerz-:n inac..raving ca C

Syoc^cl f z U b

fe L .,:^o ^..,..s._ .i^..EC.., ^.x2ci ins,.,

Imo, ».. .1 ^l.w ^5 4,=:^. .i^ nii %.t.y4

^

. .s, c Vhe; i z ons a

31 t

01

n

The organization' s recruit-mont practice:: adequatelyinfuse "new blood" into the oro-a::ization.

The organi-7ation' s promotion prac -_-' cas p >oduca wellqualified supervisory porsonnol.

The organization's promotion p„acti,ces r ,:,, .iard personswhose contributions aro primarily technical.

D. Work Practices

Operating practices permit laboratory e:.:ployoes to eepth©ir fingers dirty” through direct involve ,4ent intechnical work,. •

The organization's work practices result in its profes-sional employees assuraing considerable responsibilityearly in their careers..

Operating practices keep employoes a:•rare of the environ-ment in which their products rust operate; e.g.,"knowing how sailors think It

Personnel of the organizavion are adequately involvedin the overall weapons planning process of the Davy.

In conducting their work employees actively seek inputsfrom the potential users of their products.

The organization promotes diverse projects and interests.The laboratory has imposed upon itself a number of rules

and regulations which hamper the effectiveness of itsemployees.

Considerable duplication exists among pro jects boi.ng con-ducted at the 1^iboratory.

The laboratory has sufficient administrative flexibilityto respond to'.urgent problems by establishing newpro jects.

The majority of the technical projects of the organiza-tion have been internally generated.

The organization is able to exercise considerable discretion,in teiv-is of whether it will take on or re ject taskas signiments .

The organization's employees are known for their opemriind_edness to ideas which have been conceived by otherokganinations.

Considerable emphasis is placed by the organization onthe avoidance of technical mistakes.

E. Managel.ient Practices

lnspi-national leadership is f orthconing from the topmany ^^:,ient, of the or-ani.zation.

• The c; 'g.n_ni z a tion' s ma.na -f.,,, ^.-tnt .zas the abil it try to b ackand promote its projects.

The O^;c`llliz -i on ^'i^' s an xli ► ` :SIi d andt1I1C1es'.°.i:?:YICGiClfytop 2:ianag ement .

The mana—g ria:l, ,14-y '! c o ' the or, a^i ,zation' .s top r-.anage-rae it IS to provide gone ral not detailed ga d^ nc^^

r

32

4

J

PART V. CONCLUSION

In this paper an attempt has been made to set forth a functional method

for the evaluation of government laboratories. The organizational accounting

system which has been described represents an approach which would produce a

periodically updated data base which would be continuously available to

managers for purposes of evaluation. Such a data base clearly would serve

to lessen the subjective nature of `present laboratory evaluations even though

it Vbuld not eliminate the necessity for subjective judgment.

The specific "measures" included to illustrate the nature of unobtrusiveG

' and reactive measures: and the integrative systems framework doubtlessly wouldw

not be directly transferable to most government- laboratories; nonetheless,

they should be suggestive of measures that would be useful. The underlying

rationale of the proposed organizational accounting system including the

discussion of relevant design parameters should provide the necessary

conceptual base for developing a set of measures that would be suitable for

'. a particular laboratory or set of laboratories relatively comparable in terms

of scope and function.

Assuming that those concerned with the management of government laboratories

agree with the premise that there is a need for improved methods of evalua•-

tion, it is submitted that the development and gradual refinement of an organi-

zational accounting system along the lines suggested in this report would

produce a pragmatic basis for making evaluative judgments.