bRESEARCH
DESIGN
Spring 2015
|
Class Information |
Instructor Information |
|
PSYCH 6002 |
Helen C. Harton, Ph.D. |
|
Feb 12-Bartlett 1060 |
Baker 357 |
|
W 12-2:50pm |
273-2235; harton@uni.edu |
Office Hours: WF 11:30-11:50 and F 1-3;
pretty much any time I’m around
Readings:
Trochim, W., & Donnelly, J. P. (2007). The research methods knowledge base (3rd
ed.). Cincinnati, OH: Atomic Dog Publishing.
Kuhn, T. S. (1996). The structure
of scientific revolutions (3rd
ed). Chicago: University of Chicago Press.
Course Description and Objectives: Research is at the heart of psychology; psychology’s
focus on research and empirical evidence is one of the things that
differentiate the field from other similar disciplines. In this course, you’ll
learn about the basics of research, explore several “hot topics” or new methods
in psychology, and become a better designer, describer, and critical reader of
research. The course is designed to give you a breadth of knowledge on research
design in psychology and to facilitate your future research career and thesis
progress.
By the end of this course, you
should
1) know the advantages and
disadvantages of the commonly used methods in psychology and when to best apply
them;
2) know the advantages and
disadvantages of several new methodologies in psychology;
3) be able to critique and
integrate previous research;
4) be more aware of ethical
concerns in research and how to deal with them;
5) be a better scientific
writer;
6) be able to design and carry
out your own research project; and
7) have made significant
progress on your thesis.
Course Format: Each week you will have several readings that you should have processed
and thought about before class. In class, we will discuss the readings and the
issues they bring up. You should contribute thoughtfully to the discussion and
build on and gently challenge the comments of other students.
Course Requirements:
Class discussion (20%). In graduate school especially, you learn not only from books and professors, but from interactions and discussions with peers. Discussing information also helps you to think about it more deeply and learn it more quickly. You are expected to contribute meaningfully to class discussions. While mere attendance is not enough to get a good grade for this component, it is imperative in that you can’t participate if you’re not here. Obviously, it also requires that you read and think about the readings. Both frequency and quality count. You will get graded on discussion each week; you can drop your lowest score.
Thought papers (15%). Each week you’ll also be asked to email me a short (1-2 pages) thought paper addressing the readings for the week. These emails should have “thought paper” in the subject line and be sent by midnight Monday nights (just paste it in the message instead of attaching it). The thought paper should address all the readings for the week at some level. Think about how these readings relate to your thesis; bring up questions you have; relate the readings to other things you know. You want to show me that you’ve read, understood, and carefully processed the readings for the week. The purpose of these assignments is to get you thinking more deeply about the method and its application to your research and to facilitate in-class discussion. You can drop your lowest thought paper score. If you turn in one the first week of classes, you can drop your lowest two.
Discussion and thought papers will be graded on the following scale:
0 = not there
2 = attended but didn’t participate, or turned in, but not very relevant (below average)
3 = comments or questions relevant, but didn’t involve much insight (average)
4 = comments or questions relevant and insightful (good)
5 = more than one comment or question showed a significant contribution (excellent)
Research proposal (45%).
One of the best ways to show that you understand research design is by applying
it. You will submit a complete research proposal
(7-8 page introduction, complete method and appendices, section on ethics and
validity, plan of analysis, discussion of expected results/what it would mean
if you didn’t find them, in APA style). This paper doesn’t have to be your
thesis proposal, but if you can write about something that does develop into it
(or another research project), all the better. There will be assignments due
during the semester to provide you with feedback on this paper (e.g., outlines,
rough drafts) that will count toward the “assignment” portion of your grade. Obviously, this should not be a paper that
you have written or are writing for another class. If there is any question
about whether it might be too collaborative or too close to something you’ve
already done, talk to me before you
get started to see if it is acceptable. The paper should be in good APA
style. Those that aren’t, will be returned
ungraded and lose at least 1 letter grade (1 if turned in that same day, -1
more for each day after).
Proposal presentation (10%). During one of the last class sessions, you will present your proposal to the class (background, method, expected results, what they would mean, etc.). Your presentation, which should include some audio-visual effects (e.g., PowerPoint), should last no more than 12 minutes, followed by a discussion of the proposal by the class (we will play the role of thesis committee). You can (and should) integrate any helpful comments from the class into your proposal before you turn it in. You’ll be graded on presentation style as well as knowledge about the area. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdFI0TXU0T3BweEljS1M1Wndmdi1hMWc&usp=drive_web#gid=0 to sign up for a time. NOTE: You have to do this from your uni google account.
Assignments (10%). In addition to the weekly thought papers, you will turn in several assignments directly related to your research proposal (i.e., topic, articles, outlines, rough draft) and provide feedback on a peer’s draft. You will also review articles demonstrating different methods. You will also present on a particular method or “hot topic.” For your topic presentation, you should have at least five references. These should include one or more articles or chapters on the method itself (or evaluations of it) and often, depending on topic, one or more articles demonstrating good or bad use of that method. If I listed an article with the presentation topic in the class schedule, that should be one of the ones you read. In the presentation, you’ll tell us about what the method is, what its advantages and disadvantages are, when it should/shouldn’t be used, and depending on the topic, either best practices in using the method and/or how (in general) you would analyze data collected by the method. You should also provide examples (actual research examples) of good and not-so-good uses of that method. Sometimes there are additional questions you should address that I put on the class schedule. You’re giving a brief overview so that everyone learns the issues involved, but you don’t need to describe it in enough Feb 4Fedetail that we could actually go out and do it. Presentations should be 10-15 minutes and include your references in your final slide. Send me a copy of your Powerpoints before the presentation. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdHp6WDI2QkltMGl1LWdSTXQ4cEx1THc&usp=drive_web to sign up for a topic and time. NOTE: You have to do this from your UNI google account.
Makeup and Late Paper Policies: Class discussion grades and presentations cannot be
made up. Proposals will be accepted up to three days past the due date, but one
letter grade will be deducted for each day until they are turned in. Plan ahead
and don’t wait until the last minute to finish (or start) the paper, in case
something unexpected arises.
Academic Honesty Policy: Cheating and plagiarism of any kind will not be
tolerated and will result in a 0 on the assignment in question. This includes
using a paper from another class or that you have worked on with another faculty
member to fulfill a requirement in this class as well as using quotes from
materials without attribution even in short assignments or thought papers. For more information on UNI’s academic
honesty policies, see the University Catalog as well as the information in the
Department of Psychology Graduate Student Handbook. If you have any questions
about what is acceptable, ask.
Disability Services: The Americans with Disabilities Act of 1990 (ADA)
provides protection from illegal discrimination for qualified individuals with
disabilities. Students requesting instructional accommodations due to
disabilities must arrange for such accommodation through the Office of
Disability Services. The ODS is located at 213 Student Services Center, and the
phone number is 273-2676.
CLASS SCHEDULE
Jan. 14—Foundations and theory
T&D, Chapter 1
Kerr, N. L.
(1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2,
196-217. doi:10.1207/s15327957pspr0203_4
Simmons, J.
P., Nelson, L. D., & Simonsohn, U. (2011). False-positive
psychology: Undisclosed flexibility in data collection and analysis allows
presenting anything as significant. Psychological
Science, 22, 1359-1366. doi:10.1177/0956797611417632
Bakker, M.,
van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called
psychological science. Perspectives on
Psychological Science, 7, 543-554. doi:10.1177/1745691612459060
Issues for
discussion: What is the theoretical basis for your thesis? What is good
and bad about theories? What examples have you seen of HARKing? Is it really a
bad thing? What are the effects of treating science as a “game”? What do you
think of the suggestions of the authors? Are they good? Are they practical?
What should we do to avoid misleading reports of research in psychology?
Jan. 21--Philosophy of science
Kuhn book
Issues
for discussion: What is Kuhn’s view of science? Do you agree? How does science
change? How should it change? What is and should be the goal of science? What
are examples of “revolutions” in psychology?
Turn in
topic description for your research proposal for class; put it in my mailbox by
Friday at noon.
Hertwig, R.,
& Ortmann, A. (2008). Deception
in experiments: Revisiting the arguments in its defense. Ethics & Behavior, 18, 59-92. doi:10.1080/10508420701712990
Becker-Blease,
K. A., & Freyd, J. J. (2006). Research
participants telling the truth about their lives: The ethics of asking and
not asking about abuse. American
Psychologist, 61, 218-226. doi:10.1037/0003-066X.61.3.218
Moreno, M.
A., Goniu, N., Moreno, P. S., & Diekema, D. (2013). Ethics of social media
research: Common concerns and practical considerations. Cyberpsychology, Behavior, and Social Networking, 16, 708-713. doi:10.1089/cyber.2012.0334
Emery, K.
(2014). So you want to do an online study: Ethics considerations and lessons
learned. Ethics and Behavior, 24,
293-303. doi:10.1080/10508422.2013.860031
Issues for discussion: Are there any ethical issues
that would need to be addressed with your thesis? How could you deal with them?
Is deception okay? When? How should people be debriefed? Are our current
techniques adequate? Should “vulnerable populations” receive more protection?
How or why? Should we ask about things like abuse?
Debriefing
presentation—Is debriefing effective?
When or why? Briefly review research on debriefing and give us some “best practice”
suggestions.
T&D Chapter 2
Henrich,
J. Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?
Behavioral and Brain Sciences, 33, 61-83.
doi:10.1017/S0140525X0999152X
Maxwell, S.
E., Kelley, K., Rausch, J. R. (2008). Sample size planning for statistical
power and accuracy in parameter estimation. Annual
Review of Psychology, 59, 537-563. doi:
10.1146/annurev.psych.59.103006093735
Mervis, J.
(2014). Building a cheaper, better census. Science,
345(6197), 608. doi:10.1126/science.345.6197.608.
Issues for
discussion: Is sampling an issue for your thesis topic? What type of sampling
will you use? Should you use? What populations have been used in your area? Is
that a problem or not and why? If there are WEIRD samples used, how is this
likely to interact with your thesis topic area? When are WEIRD samples more or
less of a problem? How large should your sample be for your thesis? How will
you determine this?
Assignment. Find
10 articles for your research proposal for this class and submit an APA style
reference section. If they are not in near-perfect APA style, they will be
returned ungraded, and you will get a 0 on the assignment.
Cross-cultural
research presentation—When is cross-cultural research most useful? What
particular things do researchers need to keep in mind? What are best practices?
Feb. 11 --Reliability and validity
and scaling
T&D Chapters 3 and 5
Issues for
discussion: What are the various types of reliability and validity that we
should be concerned with? How can those best be tested? What factors does one
need to consider when choosing or developing a scale? What is the problem of
method bias? How can it be decreased?
Podsakoff,
P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias
in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539-569.
doi:10.1146/annurev-psych-120710-100452
Assignment. Find
an article that describe the development of a scale related to your thesis
topic (should include multiple studies, assessing reliability and validity).
Evaluate the article and scale (and include a brief description of the scale,
including the APA style citation). How did they show the various types of
reliability and validity? Do you agree? What type of scaling is the measure?
What type of scale? Are there similar concepts that still need a scale
developed? How would you go about creating a scale? What would you correlate it
with to test reliability and validity?
Outline of
intro for proposal due
T&D Chapter
4
Stern, M.
J., Bilgen, I., Dillman, D. A. (2014). The state of survey methodology:
Challenges, dilemmas, and new frontiers in the era of the tailored design. Field Methods, 26, 284-301. doi:10.1177/1525822X13519561
Tourangeau,
R. (2004). Survey research and societal change. Annual Review of Psychology, 55, 775-801. doi:10.1146/annurev.psych.55.090902.142040
Graham, J. W.
(2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60,
549-576. doi:10.1146/annurev.psych.58.110405.085530
Issues for
discussion: Bring in some examples of “bad” survey questions. Think about any
issues that you might need to deal with in terms of how you ask questions on
your thesis. What types of issues (e.g., population, sampling) might come up
and how would those affect what method you choose? What are the advantages and
disadvantages of survey methodology for your topic?
Diary
presentation—Include information on
the variety of diary methods and how they are analyzed (sometimes called
“event-sampling” as well)
Social network analysis presentation—What is social network analysis? When and why might
you want to use it? How can it be analyzed?
Burt, R. S., Kilduff, M., & Tasselli, S. (2013). Social network analysis: Foundations and frontiers on advantage. Annual Review of Psychology, 64, 527-547. doi:10.1146/annurev-psych-113011-143828
Feb. 24-- Experimental design ****Note that this is a TUESDAY
T&D Chapters 7 and 9
Baron, R.
M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in
social psychological research: Conceptual, strategic, and statistical
considerations. Journal of Personality
and Social Psychology, 51, 1173-1182. doi:10.1037/0022-3514.51.6.1173
MacKinnon,
D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58,
593-614. doi:10.1146/annurev.psych.58.110405.085542
Issues for
discussion: Which threats to internal validity are most likely to be a problem
in your thesis? How can you diminish them? What potential mediators and
moderators might be involved in your thesis topic area? How can we
differentiate between mediators and moderators? Should you include covariates
in your thesis? Why or why not?
Outline of
proposal paper due
Placebo
presentation—What are placebo effects
and what causes them? What are best practices in placebo research?
Price, D.
D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the
placebo effect: Recent advances and current thought. Annual Review of Psychology, 59, 565-590. doi:10.1146/annurev.psych.59.113006.095941
T&D Chapters 10 and 11
Shadish, W.
R., & Cook, T. D. (2009). The Renaissance of field experimentation in
evaluating interventions. Annual Review
of Psychology, 60, 607-629. doi:10.1146/annurev.psych.60.110707.163544
Issues for
discussion: Is your thesis a quasi-experiment? If so, what kind? What threats
to internal validity might be a problem? Are the elements of “good design”
present in your thesis? Explain. What are the advantages and disadvantages of
field research? How could it be (or has been) used in the area of your thesis
topic?
March 11—Evaluation research and
meta-analysis
T&D Chapter 16
Rosenthal,
R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in
quantitative methods for literature reviews. Annual Review of Psychology, 52, 59-82. doi:10.1146/annurev.psych.52.1.59
Field, A.
P., & Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63, 665-694.
doi:10.1348/000711010X502733
Fritz, C.
O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current
use, calculations, and interpretation. Journal
of Experimental Psychology: General, 141, 2-18. doi:10.1037/a0024338
Issues for
discussion: What types of evaluation are relevant for your thesis topic? Why?
Which is more important: clinical,
practical, or statistical significance? Have there been meta-analyses on your
thesis topic? If not, is the area ready for one? What key words would you use?
What mediators or moderators would you test? Find and evaluate a meta-analysis
that interests you/is related to your topic (include citation and brief
description of method/findings). What is good and bad about what they did?
Address things like inclusion criteria as well as fixed vs. random effects,
etc.
March
18—Spring break
March 25—Quantitative analysis
T&D Chapters 12 and 14
APA
Publications and Communications Board Working Group on Journal Article
Reporting Standards. (2008). Reporting standards for research in psychology:
Why do we need them? What might they be? American
Psychologist, 63, 839-951. doi:10.1037/0003-066X.63.9.839
Kashy, D.
A., & Kenny, D. A. (2000). The analysis of data from dyads and groups. In
H. T. Reis & C. M. Judd (Eds.), Handbook
of research methods in social and personality psychology. New York:
Cambridge University Press. On reserve in the library (2 hour hold).
Cumming, G.
(2014). The new statistics: Why and how. Psychological
Science, 25, 7-29. doi:10.1177/0956797613504966
Issues for
discussion: How will you analyze your data for your thesis? What tests will be
involved? What assumptions do those tests have? Make sure you include enough
description of your method that I can understand whether these analyses are
appropriate. How could some of the analyses that you were less familiar with be
useful in studying your topic?
Quantitative
paper critique due (see questions below).
April 1--Writing it up, peer review
T&D Chapter 15
Strunk, W.,
Jr. (1999). The elements of style.
New York: Bartleby.com
Bem, D. J.
(2002). Writing the empirical
journal article. In J. M. Darley, M. P. Zanna, & H. L. Roediger III (Eds.), (2002). The compleat academic: A
career guide. Washington, DC: American Psychological Association.
Sternberg,
R. J. (1993). How to
win acceptances by psychology journals: 21 tips for better writing. APA
Observer.
Issues for
discussion: What things do you need to be more careful with in your own
writing? How can you become a better scientific writer?
Bring in
rough draft of thesis proposal for peer review and comment. (class may be tba
because of Graduate Student Symposium 12-4).
April 8--Other methods: Implicit
measures and physiological measures
De Houwer,
J., & Moors, A. (2010). Implicit measures: Similarities and differences. In
B. Gawronski & B. K. Keith (Eds.), Handbook
of implicit social cognition: Measurement, theory, and applications (pp.
176-196). Available in resource room.
Sowden, P.,
& Barrett, P. (2006). Psychophysiological methods. In G. M. Breakwell, S.
Hammond, C. Fife-Schaw, & J. A. Smith (Eds.), Research methods in psychology (3rd ed.) (pp. 146-159).
Thousand Oaks, CA: Sage. Available in resource room and possibly on
googlebooks.
Issues for
discussion: What are the advantages and disadvantages of implicit measures?
What are they really measuring? When should they be used? What do physiological
measures tell us and what can they not tell us? How could either of these types
of measures be used in research in your thesis area? What are questions or
comments do you have about these readings?
fMRI
presentation—What is fMRI? What does
it tell us? How can it be used in psychology?
Vul, E.,
Harris, C., Winkielman, P., & Pashler, H. (2009). Puzzingly high
correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4, 274-290.
doi:10.1111/j.1745-6924.2009.01125.x
ERP/EKG presentation—What are ERP and EKGs? What do they tell us? How can
they be used in psychology?
Eye tracking presentation—What is eye tracking? How can it be used in
psychology?
Duchowski,
A. T. (2002). A breadth-first survey of eye tracking applications. Behavior Research Methods, Instruments, and
Computers, 34, 455-470. doi:10.3758/BF03195475
Hormone
sampling presentation—What hormones
have typically been tracked in psychology? How are these collected? What do
they tell us? What are best practices in collecting them (e.g., times of day,
people for whom the results aren’t accurate)?
April 15—Other methods: Spatial
analyses and internet-based research
Goodchild,
M. F., Anselin, L., Appelbaum, R, P., & Herr Harthorn, B. (2000). Toward
spatially integrated social science. International
Regional Science Review, 23, 139-159.
doi:10.1177/016001760002300201
Sui, D.,
& Goodchild, M. (2011). The convergence of GIS and social media: Challenges
for GIScience. International Journal of
Geographical Information Science, 25, 1727-1748. doi:10.1080/13658816.2011.604636
Skitka, L.
J., & Sargis, E. G. (2006). The internet as psychological laboratory. Annual Review of Psychology, 57,
529-555. doi:10.1146/annurev.psych.57.102904.190048
Lewis, K.,
Kaufman, J., Gonzalez, M., Wimmer, A., & Christakis, N. (2008). Tastes,
ties, and time: A new social network dataset using Facebook.com. Social Networks, 30, 330-342. doi:10.1016/j.socnet.2008.07.002
Issues for
discussion: How does space affect your thesis topic, if at all? Are there
potential spatial effects either that may be interesting in their own right or
that should be controlled for? Research using the internet brings exciting
advantages, but also significant disadvantages (e.g., ethical concerns). What
do you think about research on the internet—are there certain methods or populations
or topics that shouldn’t be studied there? What other questions or comments do
you have about these methods?
Computer
simulation presentation—How can
computer simulations add to our knowledge of psychology and human behavior?
When are they best used? What advantages and disadvantages do they offer?
mTurk presentation—What is mTurk? When is it more/less useful?
Buhrmester,
M., Kwang, T., & Gosling, S. D. (2011). Amazon’s mechanical turk: A new
course of inexpensive, yet high-quality data? Perspectives on Psychological Science, 6, 3-5. doi:10.1177/1745691610393980
Buhrmester,
M. (2010). Amazon
mechanical turk guide for social scientists.
Smart phone
presentation—What types of
applications can smartphones be used for? Try one out if possible (e.g., iEar).
What are its advantages and challenges and how can the challenges be addressed
(think ethics as well)?
Raento, J.,
Oulasvirta, A., & Eagle, N. (2009). Smartphones: An emerging tool for
social scientists. Sociological Methods
and Research, 37, 426-454. doi:10.11177/0049124108330005
Virtual
reality presentation—How has virtual
reality been used in psychology? What types of virtual reality have been used?
What are the advantages and disadvantages of these approaches? (Jim Blascovich
is a good researcher to start with)
Second life
presentation—How can second life be
used, either to study topics or to recruit participants? Are there ethical or
methodological concerns?
Dean, E.,
Cook, S., Murphy, J., & Keating, M. (2012). The effectiveness of survey
recruitment methods in Second Life. Social
Science Computer Review, 30, 324-338. doi:10.1177/0894439311410024
April 22--Qualitative research
T&D Chapter 6 and 8 and 13
Smith, C. P.
(2000). Content analysis and narrative analysis. In H. T. Reis & C. M. Judd
(Eds.), Handbook of research methods in
social and personality psychology (pp. 313-335). New York: Cambridge University Press. On reserve in the library.
Issues for
discussion: What are the advantages and disadvantages of qualitative research?
What new or useful information could qualitative studies provide on your thesis
topic?
Qualitative
article critique due (see questions below).
Semi-structured
interviews presentation—What are the
advantages/disadvantages of this method? What are some best practices? How can
the data be analyzed?
Bartholomew,
K., Henderson, A. J. Z., & Marcia, J. E. (2000). Coded semistructured
interviews in social psychological research. . In H. T. Reis & C. M. Judd
(Eds.), Handbook of research methods in
social and personality psychology (pp. 286-312). New York: Cambridge University Press. On reserve in the library.
April
28—Proposal presentations ****Note that this is a TUESDAY
Up to six
proposals
Monday,
May 4 (1:00-2:50)—Proposal presentations
Up to four
proposals
Papers
due Wednesday, May 6 at 5pm. Turn in rough draft and peer review comments as
well.
1. What was the purpose of the article? How does
it fit into previous literature?
2. What
was/were the research question(s) or hypotheses?
3. What is the
research design (be specific)?
4. How were the
variables operationalized or measured (mention for IVs and DVs)?
5. Are the
measures reliable and valid? What evidence do you have of that?
6. Was the
sample representative of the population of interest? How much does that matter
for this study?
7. Was there
sufficient power to obtain significant results? Include details on your power analysis.
8. What types
of statistical analyses were done? Do
they match the research questions/hypotheses? Are there any concerns related to
these analyses?
9. What were
the results? (in statistical terms—e.g., main effects and interactions—and in words)
10. Is the
discussion of the results in line with the results? (comment on this; don’t
just say yes or no)
11. Give at least 2 strengths of the article and at
least 2 weaknesses or concerns you would have as a reviewer. For the concerns,
are there ways that the authors could or did fix or address them?
Critique of Qualitative
Research Article
1.
What type of qualitative study is it? (ethnography, case study, focus
group, etc)
2.
What is the purpose of the study? What are the research questions?
3. How does the study build on previous
literature?
4. What sources of data over what period of time
were used?
5.
What does this study tell you that a quantitative one would not be able
to?
6. Is the
discussion of the results in line with the results? (comment on this; don’t
just say yes or no)
7. Give at least 2 strengths of the article and at
least 2 weaknesses or concerns you would have as a reviewer. For the concerns,
are there ways that the authors could or did fix or address them?