RESEARCH
DESIGN
Spring 2017
|
Class Information |
Instructor Information |
|
PSYCH 6002 |
Helen C. Harton, Ph.D. |
|
Bartlett 34 |
Bartlett 2080 |
|
W 12-2:45 pm |
273-2235; harton@uni.edu |
Office Hours: see homepage
Readings:
You’ll have 4-5 articles and/or
chapters to read each week. Some may be a review, whereas others may require
three readings to fully understand them. Readings will come from your textbook
as well as other online sources. I’ve linked to the articles for your
convenience. You may have to be logged in to the UNI system in order for some
links to work. It’s also possible that some links may no longer be live in a
couple of months. In any case, you should be able to easily locate the
articles. References with an * will appear in your dropbox
or be emailed to you.
Trochim, W., & Donnelly, J. P. (2007). The research methods knowledge base (3rd ed.). Cincinnati, OH: Atomic Dog Publishing. (most of this book should be a review)
Course Description and Objectives: Research is at the heart of psychology; psychology’s
focus on research and empirical evidence is one of the things that
differentiate the field from other similar disciplines. In this course, you’ll
learn about the basics of research and current trends in the field, explore
several “hot topics” or newer methods in psychology, and become a better
designer, describer, and critical reader of research. The course is designed to
give you a breadth of knowledge on research design in psychology and to
facilitate your future research career and thesis progress. This is an exciting
time to be learning about methods in psychology! There are lots of current
changes and controversies in methods and statistics, and we’ll discuss many of
those in class.
By the end of this course, you
should:
1) know
the advantages and disadvantages of the commonly used methods in psychology and
when to best apply them;
2) know the advantages and
disadvantages of several newer methodologies in psychology;
3) be able to critique and
integrate previous research;
4) be
more aware of ethical concerns in research and how to deal with them;
5) be a
better scientific writer;
6) be a
better consumer of research;
7) be able to design and carry
out your own research project; and
8) have
made significant progress on your thesis.
Suggestion to help you learn more about methods: As I noted before, right now is
a very exciting (and sometimes scary) time for those interested in
psychological methods. To learn more about current issues, I suggest that you
subscribe to Facebook groups and/or blogs that deal with some of these issues.
Here are some options:
Psychological Methods
Discussion Group in Facebook
Personality psychology
meta-blog
Psychological
Methods Blog Feed
There are also psychologists
interested in these issues who are really active on Twitter and who have active
blogs. Let me know what good ones you find!
Course Format: Each week you will have several readings that you should have processed
and thought about before class. In class, we will discuss the readings and the
issues they bring up. You should contribute thoughtfully to the discussion and
build on and gently challenge the comments of other students. I will sometimes
provide background information for a topic, but this is not a lecture class.
Course Requirements: There are several types of assignments in this class. Each are designed
to help you understand research methods in psychology better, apply and
critique methods better, and/or to progress on your thesis. I don’t assign
things as busywork. It takes me a long time to read and provide feedback on
your work, and I’m not going to do that unless I believe you’ll benefit.
Practicing concepts through multiple, smaller assignments helps you to retain
information (e.g., Lang, 2016).
Discussion 18%
Thought papers 12%
Method presentation 5%
Proposal assignments 8% (see breakdown by assignment
in that section)
Proposal 35%
Proposal presentation 10%
Final exam 12%
Class discussion (18%). In graduate school especially, you learn not only from books and professors, but from interactions and discussions with peers. Discussing information also helps you to think about it more deeply and learn it more quickly. You are expected to contribute meaningfully to class discussions. While mere attendance is not enough to get a good grade for this component, it is imperative in that you can’t participate if you’re not here. Obviously, it also requires that you read and think about the readings. Both frequency and quality count. You will get graded on discussion each week; you can drop your lowest score. If we don’t get an adequate amount of participation from everyone, I reserve the right to add weekly quizzes. This is your class—take it seriously and share your thoughts. I look forward to learning from our discussions.
Thought papers (12%). Each week you’ll also be asked to email me a short (1-2 pages) thought paper addressing the readings for the week. These emails should have “thought paper” in the subject line and be sent by midnight Monday nights (just paste it in the message instead of attaching it). You should address each of the readings to some extent in your paper, although you may sometimes want to mainly focus on one or another. You want to show me that you’ve read, understood, and carefully processed the readings for the week. Think about how these readings relate to your thesis; bring up questions you have; relate the readings to other things you know. The purpose of these assignments is to get you thinking more deeply about the method and its application to your research and to facilitate in-class discussion. You can drop your lowest thought paper score. If you turn in one the first week of classes, you can drop your lowest two.
Discussion and thought papers will be graded on the following scale:
0 = not there
2 = attended but didn’t participate, or turned in, but not very relevant (below average)
3 = comments or questions relevant, but didn’t involve much insight (average)
4 = comments or questions relevant and insightful (good)
5 = more than one comment or question showed a significant contribution (excellent)
Method presentation (5%).
You will also present on a particular method or “hot topic.” For your topic
presentation, you should have at least four references—two of these should be
sources discussing the method itself (advantages and disadvantages) and two should
be articles that used the method. In your presentation, you should a) describe
the method; b) explain how it is used in psychology; c) talk about
considerations, advantages, disadvantages, best practices etc.; d) describe and
critique at least 2 studies that used the method (e.g., Did they use the method
well? Should they have done something else? Why was it a good method for this
question?). For some topics, I have particular questions or articles that I
would like for you to use in your presentation. It is your responsibility to
set up a brief meeting with me at least 1 week before your presentation to make
sure you are clear on what the method is and that you have the sources you
need. Presentations should be 10-15
minutes (including time for questions) and include your references in your
final slide. Send me a copy of your Powerpoints
before the presentation. Go to https://docs.google.com/a/uni.edu/spreadsheets/d/1H5jW-E9lf1iELKJ40ILDDKzDGv23wdMngNBTcyL5TlU/edit?usp=sharing
to sign up for a topic and time. NOTE: You have to do this from your UNI
google account.
Proposal Assignments (8%). In addition to the weekly thought papers, you will turn in several assignments directly related to your research proposal (i.e., topic, articles, outlines, rough draft) and provide feedback on a peer’s draft. These assignments encourage you to not wait until the last minute on your paper. They also provide you with feedback as you go along. The breakdown of these assignments and how they relate to your grade (and their due dates) are as follows:
Topic 5% Jan. 16
References 10% Feb. 1
Outline 20% Feb. 22
Complete draft 20% Apr. 3 (turn in with paper, time stamped)
Comments on peer’s draft 25% Apr. 5 (turn in with paper, time stamped)
Ethics appendix 10% Apr. 12 (with paper)
Reliability and validity app. 10% Apr. 12 (with paper)
Research proposal (35%). One of the best ways to show that you understand research design is by applying it. You will submit a complete research proposal (7-8 page introduction, complete method section, including a design section and appendices with measures, plan of analysis section, discussion of expected results/what it would mean if you didn’t find them, in APA style). You should also add appendices addressing 1) ethical considerations related to your study; and 2) reliability and validity (internal, external, and construct) in your study. Finally, when you turn in your proposal, you should also turn in your earlier draft and your peer’s comments on that draft.
I highly recommend that you make
this paper your thesis proposal. If you don’t want to do that for some reason,
that’s okay, but talk to me first. There will be assignments due during the
semester to provide you with feedback on this paper (e.g., outlines, rough
drafts). If this is your thesis proposal, then you should be getting feedback
from your thesis supervisor as you go as well. That’s fine. You want to
incorporate feedback from wherever you get it (including the class, your thesis
supervisor and possibly committee, me, and your peer reviewer). However—this
paper should not be a paper that you have written or are writing for another
class or for another (nonthesis) project with a
faculty member. If there is any question about whether it might be too
collaborative or too close to something you’ve already done, talk to me before you get started to see if it is
acceptable. The paper should be in excellent APA style. Those that aren’t,
will be returned ungraded and lose at least 1 letter grade (1 if turned in
corrected within 24 hours, -1 more for each day after).
Proposal presentation (10%). You will present your proposal to the class (background, method, expected results, what they would mean, etc.). Your presentation, which should include some audio-visual effects (e.g., PowerPoint), should last no more than 12 minutes, followed by a discussion of the proposal by the class (we will play the role of thesis committee). You can (and should) integrate any helpful comments from the class into your proposal before you turn it in. You’ll be graded on presentation style as well as knowledge about the area. Go to https://docs.google.com/spreadsheets/d/1B1VWcXbDnjb9qRHJIBUK0SXH5jccB5Ya7WhD6ps4REQ/edit#gid=0 to sign up for a presentation time. NOTE: You have to do this from your UNI google account.
Final exam (12%). There are two parts to your final exam—a take-home portion and an in-class portion. I will give you information on the take-home portions of your test well in advance of exam week. If you wish (this is a good idea), you can turn in those parts of the final early. At any rate, you must turn them in by 1pm on Monday, May 1 (the exam time). I’ll give out more information about the final as we get closer to the time.
Makeup and Late Paper Policies: Class discussion grades and presentations cannot be
made up. I will accept one minor assignment (i.e., thought paper, proposal
assignment, not counting draft and peer review) up to 24 hours late with no
penalty. Proposals will be accepted up to three days past the due date, but one
letter grade will be deducted for each day until they are turned in. Plan ahead
and don’t wait until the last minute to finish (or start) the paper, in case
something unexpected arises. No late papers will be accepted for the take-home
portion of the exam, and no makeups allowed for the exam except in extreme
circumstances (e.g., illness, familial death).
Academic Honesty Policy: Cheating and plagiarism of any kind will not be
tolerated and will result in a 0 on the assignment in question. This includes
using a paper from another class to fulfill a requirement in this class as well
as using quotes from materials without attribution, even in short assignments,
thought papers, or tests. For more
information on UNI’s academic honesty policies, see UNI’s Student Academic Ethics Policy.
If you have any questions about what is acceptable, ask.
Disability Services: The Americans with Disabilities Act of 1990 (ADA)
provides protection from illegal discrimination for qualified individuals with
disabilities. Students requesting instructional accommodations due to
disabilities must arrange for such accommodation through the Office of
Disability Services. The ODS is located at 213 Student Services Center, and the
phone number is 273-2676.
CLASS SCHEDULE
Jan. 11—Foundations and theory
T&D, Chapter 1
Meehl, P. E. (1990). Why
summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66, 195-244.
doi:10.2466/PR0.66.1
Kerr, N. L.
(1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2,
196-217. doi:10.1207/s15327957pspr0203_4
LeBel, E. P., &
Peters, K. R. (2011). Fearing
the future of empirical psychology: Bem’s (2011)
evidence of psi as a case study of deficiencies in modal research practice.
Review of General Psychology, 15, 371-379.
doi:10.1037/a0025172
Greenwald,
A. G. (2012). There
is nothing so theoretical as a good method. Perspectives on Psychological Science, 7,
99-108. doi:10.1177/1745691611434210
Jan. 17—Ethics **NOTE THIS IS A
TUESDAY, 12:30-3:15
Hertwig, R., & Ortmann, A. (2008). Deception
in experiments: Revisiting the arguments in its defense. Ethics & Behavior, 18, 59-92. doi:10.1080/10508420701712990
Becker-Blease, K. A., & Freyd, J. J.
(2006). Research
participants telling the truth about their lives: The ethics of asking and not
asking about abuse. American
Psychologist, 61, 218-226. doi:10.1037/0003-066X.61.3.218
Moreno, M.
A., Goniu, N., Moreno, P. S., & Diekema, D. (2013). Ethics
of social media research: Common concerns and practical considerations. Cyberpsychology, Behavior, and Social Networking, 16,
708-713. doi:10.1089/cyber.2012.0334
Nosek, B. A., Spies, J.
R., & Motyl, M. (2012). Scientific
utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives
on Psychological Science, 7, 615-631.
doi:10.1177/1745691612459058
Proposal
topic due. You should turn in about a
paragraph describing what you plan to test in your proposal and how you would
test it (in general—you don’t need specific measures, etc. at this point). If
you want to change your topic later, you need to talk to me first.
T&D Chapter 2
Henrich, J. Heine, S. J., & Norenzayan, A.
(2010). The
weirdest people in the world? Behavioral
and Brain Sciences, 33, 61-83. doi:10.1017/S0140525X0999152X
(just the article itself—you don’t have to read all the commentaries, though
you might want to skim some)
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
doi:10.1037/0033-2909.112.1.155
McShane, B.
B., & Bockenholt, U. (2014). You cannot step
into the same river twice: When power analyses are optimistic. Perspectives on Psychological Science, 9,
612-625. doi:10.1177/1745691614548513
For when you might need them:
Cross-cultural
research presentation—When is
cross-cultural research most useful? What particular things do researchers need
to keep in mind? What are best practices?
T&D Chapters 3 and 5
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff,
N. P. (2012). Sources
of method bias in social science research and recommendations on how to control
it. Annual Review of Psychology, 63, 539-569.
doi:10.1146/annurev-psych-120710-100452
Westfall,
J., & Yarkoni, T. (2016). Statistically
controlling for confounding constructs in harder than you think. PLoS ONE, 11, e0152719.
doi:10.1371/journal.pone.0152719
Panayides, P. (2013). Coefficient alpha:
Interpret with caution. Europe’s
Journal of Psychology, 9, 687-696. doi:10.5964/ejop.v9i4.653
Proposal
references due. Find 10 articles
for your research proposal for this class and submit an APA style reference
section. If they are not in near-perfect APA style, they will be returned
ungraded, and you will get a 0 on the assignment. Put the tentative title of your proposal at
the top of the references section.
T&D
Chapter 4
Stern, M.
J., Bilgen, I., & Dillman,
D. A. (2014). The state
of survey methodology: Challenges, dilemmas, and new frontiers in the era of
the tailored design. Field Methods,
26, 284-301. doi;
10.1177/1525822X13519561
Graham, J.
W. (2009). Missing
data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549-576.
doi:10.1146/annurev.psych.58.110405.085530
Curran, P.
G. (2016). Methods
for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology,
66, 4-19. doi:
10.1016/j.jesp2015.07.006
Experience sampling/daily diary presentation—Include information on the variety of experience
sampling methods and how they are analyzed.
T&D Chapters 7 and 9
Bless, H.,
& Burger, A. M. (2016). A closer
look at social psychologists’ silver bullet: Inevitable and evitable side
effects of the experimental approach. Perspectives
on Psychological Science, 11, 296-308. doi:10.1177/1745691615621278
Mitchell, G.
(2012). Revisiting
truth or triviality: The external validity of research in the psychological
laboratory. Perspectives on
Psychological Science, 7, 109-117. doi:10.1177/1745691611432343
Judd, C. M, Yzerbyt, V. Y., & Muller, D. (2014). Mediation
and moderation. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (2nd
ed., pp. 653-676). New York:
Cambridge University Press. doi:10.1017/cbo9780511996481.030
Optional:
Baron, R.
M., & Kenny, D. A. (1986). The moderator-mediator
variable distinction in social psychological research: Conceptual, strategic,
and statistical considerations. Journal
of Personality and Social Psychology, 51, 1173-1182.
doi:10.1037/0022-3514.51.6.1173
Placebo
presentation—What
are placebo effects and what causes them? What are best practices in placebo
research?
Feb.
22—Quasi and field experiments
T&D Chapters 10 and 11
Shadish, W. R., &
Cook, T. D. (2009). The
Renaissance of field experimentation in evaluating interventions. Annual Review of Psychology, 60, 607-629.
doi:10.1146/annurev.psych.60.110707.163544
*Kendall, P.
C., & Comer, J. S. (2011). Research methods in clinical psychology. In D.
H. Barlow (Ed.), The Oxford handbook of
clinical psychology: Updated edition (pp. 52-75). New York: Oxford
University Press. doi:10.1093/oxfordhb/9780195366884.013.0004
Small n designs presentation –What types of small n designs are often used in
psychology? What are their advantages and disadvantages?
Proposal
outline due. Turn in a complete and
detailed outline of your entire proposal (introduction, method, results, and
discussion). Include your tentative title at the top, and use good outline
form. Outlines will be graded on organization, completeness, and use of good
outlining techniques (e.g., at least 2 headings at each level, headings in
similar format).
T&D Chapters 12 and 14
Cumming, G.
(2014). The
new statistics: Why and how. Psychological
Science, 25, 7-29. doi:10.1177/0956797613504966
APA
Publications and Communications Board Working Group on Journal Article
Reporting Standards. (2008). Reporting
standards for research in psychology: Why do we need them? What might they
be? American Psychologist, 63,
839-951. doi:10.1037/0003-066X.63.9.839
Simmons, J.
P., Nelson, L. D., & Simonsohn, U. (2011). False-positive
psychology: Undisclosed flexibility in data collection and analysis allows
presenting anything as significant. Psychological
Science, 22, 1359-1366. doi:10.1177/0956797611417632
March 6
or 7—Go to Open Science Talk
March 8—Replication and
generalizability
Open Science
Collaboration. (2015). Estimating
the reproducibility of psychological science. Science, 349, aac4716.
doi:10.1126/science.aac4716
Pashler, H., & Harris,
C. R. (2012). Is the
replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7, 531-536.
doi:10.1177/1745691612463401
Maxwell, S.
E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering
from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70, 487-498. doi:10.1037/a0039400
Patil, P.,
Peng, R. D., & Leek, J. T. (2016). What should
researchers expect when they replicate studies? A statistical review of
replicability in psychological science. Perspectives
in Psychological Science, 11, 539-544. doi:10.1177/1745691616646366
Tackett, J.
L., Lilienfeld, S. O., Patrick, C. J., Johnson, S.
L., Krueger, R. F., Miller, J. D.,…Shrout, P. E.. (in press). It’s
time to broaden the replicability conversation: Thoughts for and from clinical
psychological science. Perspectives
on Psychological Science.
March
22—Presentations
7 presentations
March 29--Meta-analysis and
evaluation (and presentations)
3 presentations
T&D Chapter 16
Stanley, D.
J., & Spence, J. R. (2014). Expectations
for replications: Are yours realistic? Perspectives
in Psychological Science, 9, 305-318. doi:10.1177/1745691614528518
van Elk, M., Matzke, D., Gronau, Q. F., Guan, M., Vanderkerckhove,
J., & Wagenmakers, E.-J. (2015). Meta-analyses
are no substitute for registered replications: A skeptical perspective on
religious priming. Frontiers in
Psychology, 6, 1365. doi:10.3389/fpsyg.2015.01365
Field, A.
P., & Gillett, R. (2010). How to
do a meta-analysis. British Journal
of Mathematical and Statistical Psychology, 63, 665-694. doi:10.1348/000711010X502733
Fritz, C.
O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates:
Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141, 2-18. doi:10.1037/a0024338
April 3 – exchange rough draft with peer
T&D Chapter 15
Strunk, W.,
Jr. (1999). The elements of style.
New York: Bartleby.com
Bem, D. J. (2002). Writing the empirical journal article.
In J. M.
Darley, M. P. Zanna, & H. L. Roediger III (Eds.), The compleat academic: A career guide.
Washington, DC: American Psychological Association.
Sternberg,
R. J. (1993). How to
win acceptances by psychology journals: 21 tips for better writing. APA
Observer.
Proposal
draft due April 3 (exchange with assigned peer). Peer comments due April 5
(11:59pm). Bring in draft for peer
review. You should have a completed draft for your partner to comment on.
You’ll turn in both your draft and your partner’s comments when you turn in
your proposal.
Gosling, S.
D., & Mason, W. (2015). Internet
research in psychology. Annual Review
of Psychology, 66, 877-902. doi:10.1146/annurev-psych-010814-015321
Necka, E. A., Cacioppo, S., Norman, G. J., & Cacioppo,
J. T. (2016). Measuring
the prevalence of problematic respondent behaviors among mTurk,
campus, and community participants. PLoS ONE, 11, e0157732.
doi:10.1371/journal.pone.0157732
Paolacci, G., &
Chandler, J. (2014). Inside
the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science,
23, 184-188. doi:10.1177/0963721414531598
Lewis, K.,
Kaufman, J., Gonzalez, M., Wimmer, A., &
Christakis, N. (2008). Tastes,
ties, and time: A new social network dataset using Facebook.com. Social Networks, 30, 330-342.
doi:10.1016/j.socnet.2008.07.002
Computer
simulation presentation—How can computer
simulations add to our knowledge of psychology and human behavior? When are
they best used? What advantages and disadvantages do they offer?
Smart phone
presentation—What types of applications can smartphones be used for
(e.g., GPS, activity, sounds, photographs)? What are the advantages and
challenges of these apps and how can the challenges be addressed (think ethics
as well)?
Virtual
reality presentation—How has virtual reality been used in psychology? What
types of virtual reality have been used? What are the advantages and
disadvantages of these approaches? (Jim Blascovich is
a good researcher to start with)
Big data
mining—How can researchers use Facebook, Twitter, Google, and
other sources to learn about human behavior and personality? Are there ethical
considerations?
Proposals
due!
April 18—Priming, implicit measures,
and physiological measures. **NOTE THIS IS A TUESDAY, 12:30-3:15
Tiege-Mocigemba, S., Klauer, K. C., & Sherman, J. W. (2010). A
practical guide to implicit association tests and related tasks. In B. Gawronski & B. K. Payne (Eds.), Handbook of implicit social cognition: Measurement, theory, and
applications (pp. 117-139). New York: Guilford Press. doi: 10.13140/2.1.4889.1845
Bartlett, T.
(2017, January 5). Can we really measure implicit bias? Maybe not. The Chronicle of Higher Education. Retrieved
from http://www.chronicle.com/article/Can-We-Really-Measure-Implicit/238807/
De Houwer, J., & Moors, A. (2010). Implicit
measures: Similarities and differences. In B. Gawronski
& B. K. Payne (Eds.), Handbook of
implicit social cognition: Measurement, theory, and applications (pp.
176-196). New York: Guilford Press. doi:10.1588/imre.v16i2.232
Klatzky, R. L., &
Creswell, J. D. (2014). An intersensory interaction account of priming effects—and
their absence. Perspectives in
Psychological Science, 9, 49-58. doi:10.1177/1745691613513468
*Blascovich, J. (2014). Using physiological indexes of
psychological processes in social psychological research. In H. T. Reis &
C. M. Judd (Eds.), Handbook of research
methods in social and personality psychology (2nd ed., pp.
117-137). New York: Cambridge
University Press. doi:10.1017/cbo9780511996481.010
fMRI
presentation—What is fMRI? What does
it tell us? How can it be used in psychology? Are there things researchers
using this method need to be aware of?
Eye tracking presentation—What is eye tracking? How can it be used in psychology?
Hormone
sampling presentation—What hormones have typically been tracked in psychology?
How are these collected? What do they tell us? What are best practices in
collecting them (e.g., times of day, people for whom the results aren’t
accurate)?
April
26--Qualitative research
T&D Chapter 6, 8, and 13
*Smith, C.
P. (2000). Content analysis and narrative analysis. In H. T. Reis & C. M.
Judd (Eds.), Handbook of research methods
in social and personality psychology (pp. 313-335). New York: Cambridge University Press.
Tracy, S. J.
(2010). Qualitative
quality: Eight “big tent” criteria for excellent qualitative research. Qualitative Inquiry, 16, 837-851. doi:10.1177/1077800410383121
Semi-structured
interviews presentation—What are the advantages/disadvantages of this method? What
are some best practices? How can the data be analyzed?
Monday,
May 1 (1:00-2:50)—Final exam
(take-home portion due at 1pm).