Part I-Dirty Data
Activity 1: Reflection questions:
1. Answer the following questions:
a. What factors played a role in this decision?
b. Did you talk to anyone? If so, who?
c. Did you do some research or look up some information? If so, where?
d. Who or what do you trust when it comes to your health?
e. Write down a few notes describing your decision making process.
Activity 2: Definition of Terms
1. Define the following terms:
2. Explain how anecdotal evidence about alternative medicine can lead you astray:
|Data Table 1. Evaluate the Scenarios|
|Scenario||Example(s) of this type of dirty data:||Explanation|
Part II – Cleaner Data
Activity 4: Describe the Methods
|Data Table 2. Summary of Study Designs|
|General Name||Specific Name||Description of Method||Advantages||Disadvantages|
|Review & Meta-Analysis|
|Experimental/Intervention||Clinical Trial (Randomized Controlled Trial)|
|Animal & In Vitro Studies|
|Cross-Sectional Survey / Population Studies|
Activity 5: Determine the Study Type
|Data Table 3. Types of Studies|
|Study #||Type of Study:|
Activity 6: Create a Hierarchy
|Data Table 4. Hierarchy of study designs|
a. For studies that are likely to produce scientific evidence for health recommendations, what are the caveats to putting all studies of this design near the “best” evidence side of the hierarchy?
b. For studies that are least likely to produce scientific evidence for health recommendations, what are the caveats to putting all studies of this design near the “worst” evidence side of the hierarchy?
c. Why are all these study designs utilized if they aren’t all used for health recommendations?
d. Could some of these study designs be used and still result in pseudoscience?
e. What are some aspects of specific experimental design that would change the order of the hierarchy?
2 | Page
Part I – Dirty Data
Activity 1: Reflect on Experience
1. Reflect back to the last time you made a decision about your health. Maybe it was a decision about whether or not to go to the doctor, take an over-the-counter pain reliever, or try a new diet.
2. Answer the Activity 1 Reflection questions in the Case-Study review.
Activity 2: Watch the Videos
1. Watch the video titled “What is Pseudoscience?” at
(This video introduces pseudoscience as a science mimic that lacks empirical facts and procedures. The birth of Spiritualism from the Fox sisters’ hoax is briefly described. Running time: 3:53 min. Produced by Objective: Change the World, 2015.)
3. Now watch the video titled “Anecdotes” at
https://www.youtube.com/watch?v=QDlPoSSVPuA&feature=youtu.be (This video gives an overview of the distinction between “clean” and “dirty” data and argues that the plural of “anecdote” is not “data.” The narrator examines various scenarios involving Reiki healing to highlight the risks associated with anecdotal evidence as it relates to health decisions. Running time: 7:42 min. Produced by Body of Evidence, 2016.)
Activity 3: Evaluate the Scenarios
1. As you read the following three scenarios, look for examples of “dirty data” including:
a. anecdotes or testimonials
b. potential regression to the mean
c. the use of multiple treatments at once
d. confirmation bias
e. the placebo effect
2. In Data Table 1 of the Case Study Review, indicate which type or types of ‘dirty data’ is present, and why you think that.
Scenario1 – A Pain in the Knee
Last year, Michelle tore her ACL during soccer practice. She had surgery to repair it and is back playing this season. After a particularly rough game, her knee started to swell up more than normal and she was limping. Th e athletic trainer gave her some ice and suggested she rest for tomorrow’s practice to see if the swelling would go down. As she was icing her knee in the training room, her friend Diego, who ran cross country, came in.
“Hey, Michelle. how’s the knee?” asked Diego.
“Meh,” replied Michelle. “It just started swelling and hurting randomly after our game. I really hope I didn’t re-injure it. There’s a big tournament this weekend, and I gotta play.”
“Hey, I had some knee trouble a few years ago and acupuncture cleared it right up. You should totally try it. I’m afraid of needles, but if I didn’t watch, it was fine – it didn’t hurt. I would walk out of the appointment and my knee would immediately feel better.”
“Maybe, thanks,” responded Michelle.
As she sat in the training room with ice on her knee, she was feeling desperate. “I can’t just sit around and rest,” she thought to herself. “I need to do something to help my knee. I’m gonna search for a way to get an acupuncture appointment quickly.” She pulled out her laptop and started searching online for an acupuncture clinic nearby.
Scenario 2 – The Power of Touch
Maria’s mom was diagnosed with breast cancer a few months ago. Luckily, it was caught early, but the tumor was aggressive and, with the doctor’s advice, her mom was undergoing chemotherapy. Last week, her mom came down with a high fever. Because her immune system was compromised from chemotherapy, it is standard procedure to admit chemotherapy patients with fevers to the hospital to administer antibiotics if needed and monitor their health. Maria’s mom was admitted for treatment.
Maria stopped by to see her mom in her hospital room after class. She was looking better, sitting up in bed, and reading a book.
“Hi, Mom, how are you feeling?” asked Maria, as she washed her hands and put on a mask to be sure not to infect her mom further.
“Oh, pretty good, sweetie-pie. I’ve been pretty tired and run down, but yesterday I had some therapeutic touch and it really energized me. Th e therapist is going to stop by this afternoon, too,” remarked her mother, Isamar.
“Therapeutic touch?” asked Maria.
“It sounds kind of crazy, but they just kind of wave their hands around me. Th e therapist said that she could sense the negative energy from the infection and the cancer and was moving it around. After she left, I felt so light and I wasn’t tired anymore.”
“Okay. I have some studying to do. Do you mind if I hang out in your room and keep you company, too?”
“Sounds great, sweetie-pie.”
After about an hour and a few study breaks later, a nurse walked into the room. “Good afternoon, Isamar. You are looking in good spirits today! Is this your daughter?”
“Oh, hi, Nurse Hathaway. Maria, this is Nurse Hathaway, my therapeutic touch practitioner.”
“Hi, Nurse Hathaway. My mom said you helped her feel a lot better. Thanks so much for all you’re doing!”
“Of course, I’m back today to do a little more therapeutic touch and help your mom fight off this bug.”
Nurse Hathaway started rearranging some things in the room and Maria went back to studying, glancing up every once in a while to see what was going on. After a few minutes the nurse started holding her hands a few inches above her mom’s body and closed her eyes. She seemed to glide her hands over the surface of the hospital gown, but never quite touched her mother. Maria watched intrigued. How on earth could this help her mother? Nurse Hathaway doesn’t even touch her. Maria pulled out her laptop and starting searching for information on Therapeutic Touch.
Scenario 3 – It’s Magnetic
The basketball team has been volunteering with Elder Helpers, a food delivery service for homebound seniors for a couple years. Blake, a player on the team, had developed a friendship with one of the homebound seniors, Robert. Robert was a big basketball fan and Blake and he would chat about the sport while Blake made sure Robert had the food set up within reach. Robert suffered from diabetic neuropathy in his right foot and couldn’t walk easily. Th is past Friday when Blake delivered Robert’s food, he found Robert shuffling around in the living room.
“Hey, Robert, what are you doing on your feet?” Blake rushed over to give a hand.
“Oh, I have these new magnetic insoles that my granddaughter bought for me and my foot doesn’t hurt as much when I walk ever since I started wearing them. I was just going to get my reading glasses—I left them on the kitchen table.”
Blake walked over to offer support nonetheless. “How about last night’s game?” he casually switched the conversation to basketball, but made note to look into these magnetic insoles. He didn’t want Robert to fall. He really shouldn’t be walking too much when he’s alone. Blake put it on his mental to-do list for later. He would look up magnetic therapy tonight.
Evidence-based medicine aims to provide recommendations for prevention and treatment of various conditions based on scientific evidence or, using a term from the first video that you watched, “clean data.” Scientific evidence is collected through various types of studies that involve humans, animals, or cells. Scientific studies generate objective evidence by using proper controls for comparison, selecting a nonbiased population, collecting evidence as objectively as possible, and utilizing peer review. Even when scientific studies are done as scientifically as possible, they still differ in the quality, consistency, reliability, or credibility of evidence produced.
Generally, the gold standard of evidence lies in double-blind, placebo-controlled clinical trials that can provide evidence on how an intervention causes changes in health. However, evidence related to human health and disease comes from other types of studies, too. Th ese other types of studies provide evidence that is sometimes based on correlation or may not be directly translated into health interventions but instead serve as preliminary evidence to pursue with additional research. Often, multiple studies contribute evidence that then leads to health recommendations. In order to differentiate between scientific evidence and pseudoscientific information, let’s learn more about the different scientific approaches to studying human health.
1. Watch this video to learn about the types of study designs that collect evidence about human health at:
https://www.youtube.com/watch?v=GZ84DHwmSr8&feature=youtu.be Not all evidence is equal; there are differences in validity, credibility, and the ability to make direct applications to human health. Some of these differences come from the way data is collected or experimental design. This video identifies the main categories of experimental design that apply to human health. Running time: 6:32 min. Created by Chantilly A. Apollon for the National Center for Case Study Teaching in Science, 2017.
2. After watching the video, complete Data Table 2 in the Case Study Review. Start with the column titled Description of Methods”. Add information in the description column about what would make the study design scientific and not “pseudoscientific.”
3. Next, complete the “advantages” and “disadvantages” columns of the chart to identify what situations or types of evidence each of the study designs would be useful for and what situations or types of evidence the various study designs would not be useful for.
1. The following dichotomous key (Figure 1) can be used to identify the type of study described. Read the excerpts of the four study designs below and then follow the dichotomous key (see Figure 1) to identify which type of study is being utilized. Answer in Data Table 3 in the Case Study Review.
Figure 1. Dichotomous Key for Types of Studies Related to Human Health
70 patients with active arthritis symptoms each wore four different bracelet devices over a five-month period, reporting on their pain, disability, and medication use throughout the study. One bracelet was demagnetized and the other three were made of different magnetized metal components. Participants also provided blood samples at the beginning of the study and after wearing each device for five weeks, in order to monitor changes in inflammation. Th e sequence of the four devices was random in order. The individuals analyzing the blood samples were blind as to which device corresponded to the blood samples.
Researchers asked 35,016 postmenopausal women who did not have a history of breast cancer to complete a 24-page questionnaire about their use of non-vitamin, non-mineral “specialty” supplements. After six years of follow-up, 880 cases of breast cancer were identified.
Researchers tracked antibiotic prescriptions for 170,504 people who had Type 2 diabetes and for 1.3 million people who did not have diabetes. Th e researchers identified the subjects using records from three national health registries in Denmark.
The Danish study includes 52,061 subjects, aged 50–65 years, from the two main cities of Aarhus and Copenhagen, who participated in the study called Diet, Cancer and Health. From 1993–97, they reported on their physical leisure activities, including sports, cycling to/from work and, in their leisure time, gardening and walking each year. Th e researchers then estimated air pollution levels from traffic at their residential addresses.
1. Using the information presented so far, create a hierarchy of study designs in relation to evidence for health recommendations. Order the designs from most likely (1) to least likely (8) of producing scientific evidence for health recommendations. Record your answer in Data Table 4 of the Case Study Review.
1. After creating a hierarchy, answer the discussion questions in the Case Study Review.
American Association for Cancer Research. 8 July 2010. Fish oil may reduce risk of breast cancer. ScienceDaily. Retrieved from <http://www.sciencedaily.com/releases/2010/07/100708071349.htm>.
Andersen, Z.J., A. de Nazelle, M.A. Mendez, J. Garcia-Aymerich, O. Hertel, A. Tjønneland, K. Overvad, O. Raaschou-Nielsen, and M.J. Nieuwenhuijsen. 2015. A study of the combined effects of physical activity and air pollution on mortality in elderly urban residents: The Danish diet, cancer, and health cohort. Environmental Health Perspectives 23(6):557–563. doi: 10.1289/ehp.1408698
Corker, R. 30 May 2001. Distinguishing science and pseudoscience. Quackwatch. Retrieved from <https://www. quackwatch.com/01QuackeryRelatedTopics/pseudo.html>.
Duke University Medical Center. 1 March 2017. Introduction to evidence-based practice tutorial. Retrieved from <http://guides.mclibrary.duke.edu/c.php?g=158201&p=1036002>.
Objective: Change the World. 7 Dec 2015. What is pseudoscience? [Video fi le]. Retrieved from <https://youtu.be/sifrwJ8xQE>.
Richmond, S.J., S. Gunadasa, M. Bland, and H. MacPherson. 2013. Copper bracelets and magnetic wrist straps for rheumatoid arthritis – Analgesic and anti-inflammatory effects: A randomized double-blind placebo controlled crossover trial. PLoS ONE 8(9): e71529. Study Designs in Epidemiology. n.d. Study Designs in Epidemiology. University of Ottawa. Retrieved from <https:// www.med.uottawa.ca/sim/data/Study_Designs_e.htm>.
Tanner, K.D. 2010. Order matters: Using the 5E model to align teaching with how people learn. CBE Life Sciences Education 9(3): 159–164.
The Body of Evidence. 18 July 2016. Anecdotes. [Video file]. Retrieved from <https://youtu.be/QDlPoSSVPuA>.
The Endocrine Society. 27 August 2015. Antibiotic use linked to type 2 diabetes diagnosis: Antibiotics may contribute to or serve as early signal of developing condition. ScienceDaily. Retrieved from <http://www.sciencedaily.com/ releases/2015/08/150827141859.htm>.
6 | Page