+1 (208) 254-6996 [email protected]
  

Mettler PH, Baumgartner T. Large-scale participatory co-shaping of technological developments, first experiments with Pardizipp. Futures 1998;30(6):535-54.

i s 6

Don't use plagiarized sources. Get Your Custom Essay on
Mettler PH, Baumgartner T. Large-scale participatory co-shaping of technological developments, first experiments with Pardizipp. Futures 1998;30(6):535-54.
Just from $13/Page
Order Essay

Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-06 01:57:27.

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

 

 

HANDBOOK OF EVALUATION P1ET~qODS

Prospective Time Series

Areas of Application M e a s u r e m e n t o f a development trend, including, for example, the effect o f an intervention:

�9 Time series �9 Before-and-after studies, where the work involved could be the

introduction o f an IT-based system, for instance

Description M e a s u r e m e n t o f a n u m b e r o f measures over time shows h o w an activity or an outcome changes as a function o f either time alone or as a function o f different initiatives. Such studies m a y be either simple or controlled, depending on the level o f control o f experimental factors. Simple before-and-after studies address a single case before and after an intervention, while the controlled approach matches a n u m b e r o f cases to be studied in parallel.

Assumptions for Application The use o f time series as o n e ‘ s assessment design requires control over what changes take p l a c e – intentional and u n i n t e n t i o n a l – within the organization during the study.

Perspectives Time series are in principle carried out as a matched-pair design, just like traditional, controlled studies including RCTs. Therefore, they contain the inbuilt assumption that there is no interaction between case-specific matching characteristics and a potential intervention (Suissa 1998). The p r o b l e m occurs in particular w h e n there is just one case in the s t u d y – the simple before-and-after s t u d y – which often happens in assessment o f IT-based solutions. Implementation o f IT-based solutions often involves radical changes in the organization, its structure, and w o r k processes. Consequently, when there is only one case and the intervention is the introduction o f an

7 The same comment is valid regarding the economy as described for Clinical~Diagnostic Performance.

159

Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-06 01:57:27.

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

 

 

HANDE~OOI< O~ z {]VALUATION M~TIdODS

IT-based solution, an interaction between the case and the intervention will be present. Thus, the control group (the first measurement(s) in the time series) and the intervention group are no longer identical in all aspects other than that o f the intervention applied. See also Sparrow and Thompson (1999).

Frame of Reference for Interpretation The frame o f reference is generally included in the overall d e s i g n – for instance, as the first point within the time series.

Perils and Pitfalls One question to keep in mind is: Do the measures keep representing exactly the same characteristics o f the system or the organization? This needs to be verified for the data collected, and it is particularly important when a time series spans the implementation o f an IT s y s t e m – often over several years. Evans et al. (1998) apparently elegantly handle this error source. However, they do fall into the trap o f giving the decision-support system the credit for the improvements without acknowledging that, simultaneously, drastic changes to work processes take place (over and above what is needed by the system functionality).

As always, one has to ascertain that the conditions for the use o f ones metrics (calculation techniques) are fulfilled, and if one does performs statistical calculations on successive steps in a series o f measures, it is a requirement for the use o f concrete techniques that the measures are independent (‘carryover effects’ and ‘conditional independence on the subject’). See details in (Suissa 1998).

In cohort studies with the follow-up o f cases over time, biases may occur as a consequence o f cases that drop out along the way (see Pennefather et al. 1999). In principle this corresponds to cases with missing data for the later points o f the time series. This bias may be caused by lack o f representativeness for the drop-out group compared to the study group as a whole. An important example is successive questionnaire studies. Another example is the representativeness o f the users involved around the IT system. Therefore, in the final conclusion it is important to explain the causality in connection with these missing data.

Advice and Comments

16o

Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-06 01:57:27.

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

 

 

I-IANE~OOI< OF:: EVALUATION METHODS

References

Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Lloyd JF, Burke JP. A computer-assisted management program for antibiotics and other antiinfective agents. New Engl J Med 1998;338(4):232-8.

Pennefather PM, Tin W, Clarke MP, Fritz S, Hey EN. Bias due to incomplete follow-up in a cohort study. Br J Ophthalmol 1999;83:643- 5.

Sparrow JM, Thompson JR. Bias: adding to the uncertainty, editorial. Br J Ophthalmol 1999;83:637-8.

Suissa S. The case-time-control-design: further assumptions and conditions. Epidemiol 1998;9(4):441-5.

Supplementary Reading

Ammenwerth E, Kutscha A, Eichst/idter R, Haux R. Systematic evaluation of computer-based nursing documentation. In: Patel V, Roger R, Haux R, editors. Proceedings of the 10th World Congress on Medical Informatics; 2001 Sep; London, UK. Amsterdam: lOS Press; 2001. p. 1102-6.

A good before-and-after case study o f the quality o f nursing documentation records and user satisfaction based on a combination o f many methods (a multimethod design).

Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’LufN, Boyle D, Leape L. The impact of computerized physician order entry on medication error prevention. JAMIA 1999;6(4):313-21.

A case study using prospective time studies to assess the impact o f the introduction o f an IT-based solution on a concrete activity.

Brown SH, Coney RD. Changes in physicians’ computer anxiety and attitudes related to clinical information system use. JAMIA 1994;1(5):381-94.

In a case study the authors investigate physicians ‘fear o f new I T technology in a before-and-after study.

Kelly JR, McGrath JE. On time and method. Newbury Park: Sage Publications. Applied Social Research Methods Series 1988. vol. 13.

The book thoroughly and stringently (and with a philosophical background) discusses a number o f aspects concerning time and the influence o f time on experimental studies. Further, beyond the

161

Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-06 01:57:27.

C o p yr

Order your essay today and save 10% with the discount code ESSAYHELP