Observation of an organization to identify its practices and to expose mechanisms that control change.
This is in principle used for the same purposes as other interview methods. In practice, the method is most relevant during the early Explorative P h a s e – for instance, where the attitudes or problems of social groups need elucidation or when a model solution is being established.
Page no
73
88
91
96
102
109
111
116
66
Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-04 00:20:40.
C o p yr
ig h t ©
2 0 0 6 . E
ls e vi
e r
S ci
e n ce
& T
e ch
n o lo
g y.
A ll
ri g h ts
r e
se rv
e d .
I-IANDBOOI< OF– ~VALUATION METHODS
Functionality Assessment
Grounded Theory
Heuristic Evaluation
Interview (nonstandardized)
Prospective Time Series
Questionnaire (nonstandardized)
RCT, Randomized Controlled Trial
Risk Assessment
Root Causes Analysis
1. Validation of fulfillment of objectives (realization of objectives) – that is, the degree of compliance between the desired effect and the actual solution
2. Impact Assessment (also called effect assessment) 3. Identification of problems in the relationship
between work procedures and the IT system’s functional solution
The method will expose severe ergonomic and cognitive problems, but it is not dedicated to capture details of this type.
Supportive analytical method for data acquisition methods that generate textual data, such as some open questionnaire methods and interviews (individual and group interviews).
This is used when no other realizable possibilities e x i s t – for instance, when:
�9 The organization does not have the necessary time or expertise
�9 There are no formalized methods �9 There is not something tangible to assess yet.
Is in particular suited for the elucidation of individual opinions, attitudes, and perceptions regarding phenomena and observations.
Measurement of development trends, including the effect of an intervention.
Questionnaires are used to answer a wide range of questions, but its main area of application is (qualitative) investigations of subjective aspects requiting a high level of accuracy.
Verification of efficacy – that is, that the IT system – under ideal conditions – makes a difference to patient care. Particularly used in studies of decision-support systems and expert systems.
Identification and subsequent monitoring of risk factors, making it possible to take preemptive action.
Exploration of what, how, and why a given incident occurred to identify the root cause of undesirable events.
120
128
132
142
159
163
172
185
188
67
Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-04 00:20:40.
C o p yr
ig h t ©
2 0 0 6 . E
ls e vi
e r
S ci
e n ce
& T
e ch
n o lo
g y.
A ll
ri g h ts
r e
se rv
e d .
HANDBOOI( OiZ EVALUATION METHODS
Social Network Analysis
SWOT
Technical Verification
Think Aloud
Usability
User Acceptance and Satisfaction
Videorecording
Assessment of relations between elements within an organization (such as individuals, professions, departments, or other organizations), which influence the acceptance and use of an IT-based solution.
Situation analysis: establishing a holistic view of a situation or a model solution.
Verification that the agreed functions are present, and work correctly and in compliance with the agreement. This may take place, for instance, in connection with delivery of an IT system or prior to daily operations and at any subsequent change of the IT system (releases, versions, and patches).
An instrument for gaining insight into the cognitive processes as feed-back to the implementation and adaptation of IT-based systems.
Assessment of user friendliness in terms of ergonomic and cognitive aspects of the interaction (dialogue) between an IT system and its users.
Assessment of user opinion, attitudes, and perception of an IT system during daily operation.
Monitoring and documenting as a means of analyzing how work procedures and user activities, respectively, are actually carried out or for investigation of complex patterns of interaction.
190
196
199
204
207
215
219
6.4 Overview of Assessment Methods” Evolution Phase
The starting point in time o f this phase is usually considered to be when the entire IT-based solution has reached a state o f sufficient stability with respect to bugs and corrections and w h e n evolutionary activities are started. Consequently, the shi~ be twe e n this and the previous phase m a y be fluid.
66
Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-04 00:20:40.
C o p yr
ig h t ©
2 0 0 6 . E
ls e vi
e r
S ci
e n ce
& T
e ch
n o lo
g y.
A ll
ri g h ts
r e
se rv
e d .
I-IANDE~OOI< (3[= EVALUATION METHODS
Method
Analysis of Work Procedures
Balanced Scorecard
BIKVA
Clin&al/Diagnostic Performance
Cognitive Assessment
Cognitive Walkthrough
Delphi
Equity Implementation Model
FieM Study
Areas of application
Elucidation of how things are actually carried out, in comparison with the expected. This includes its use in relation to measures of effect.
Ongoing optimization of the outcome of a development project by balancing focus areas by means of a set of indicators for a set of strategic objectives.
Critical, subjective assessment of an existing practice.
Measurement of diagnostic ‘ correctness’ (for instance, measures of accuracy and precision) of IT-based expert systems and decision-support systems.
Assessment of the cognitive aspects of the interaction between an IT system and its u s e r s – for instance:
�9 Identification of where and why operational errors occur
�9 Identification of areas to be focused on for improvement in user friendliness.
Assessment of the user ‘friendliness’ on the basis of system design, from specifications, muck-ups, or prototypes of the system, aimed at judging how well the system complies with the users’ way of t h i n k i n g – for instance:
�9 Identification of where and why operational errors occur
�9 Identification of causes behind problems with respect to user friendliness and consequently identification of areas for improvement.
1. (Qualitative) assessment of an e f f e c t – for instance, where the solution space is otherwise too big to handle
2. Exploration of development trends 3. Elucidation of a problem a r e a – for instance, prior
to strategic planning.
Examine users’ reaction to the implementation of a new system, focusing on the impact of the changes such a system brings about for the users.
Observation of an organization to identify its practices and to expose mechanisms that control change.
Page no
73
