RESEARCH
DESIGN
Spring 2014
Class Information |
Instructor Information |
PSYCH 6002 |
Helen C. Harton, Ph.D. |
Baker 315 |
Baker 357 |
W 12-2:50pm |
273-2235; harton@uni.edu |
Office Hours: WF 11:30-11:50 and F 1-3;
pretty much any time I’m around
Readings:
Trochim, W., & Donnelly, J. P. (2007). The research
methods knowledge base (3rd ed.). Cincinnati, OH:
Atomic Dog Publishing.
Kuhn, T. S. (1996). The structure
of scientific revolutions (3rd
ed). Chicago: University
of Chicago Press.
Course Description and Objectives: Research is at the heart of psychology; psychology’s
focus on research and empirical evidence is one of the things that
differentiate the field from other similar disciplines. In this course, you’ll
learn about the basics of research, explore several “hot topics” or new methods
in psychology, and become a better designer, describer, and critical reader of
research. The course is designed to give you a breadth of knowledge on research
design in psychology and to facilitate your future research career and thesis
progress.
By the end of this course, you
should
1) know
the advantages and disadvantages of the commonly used methods in psychology and
when to best apply them;
2) know the advantages and
disadvantages of several new methodologies in psychology;
3) be able to critique and
integrate previous research;
4) be
more aware of ethical concerns in research and how to deal with them;
5) be a
better scientific writer;
6) be able to design and carry
out your own research project; and
7) have
made significant progress on your thesis.
Course Format: Each week you will have several readings that you should have processed
and thought about before class. In class, we will discuss the readings and the
issues they bring up. You should contribute thoughtfully to the discussion and
build on and gently challenge the comments of other students.
Course Requirements:
Class discussion (20%). In graduate school especially, you learn not only from books and professors, but from interactions and discussions with peers. Discussing information also helps you to think about it more deeply and learn it more quickly. You are expected to contribute meaningfully to class discussions. While mere attendance is not enough to get a good grade for this component, it is imperative in that you can’t participate if you’re not here. Obviously, it also requires that you read and think about the readings. Both frequency and quality count. You will get graded on discussion each week; you can drop your lowest score.
Thought papers (15%). Each week you’ll also be asked to email me a short (1-2 pages) thought paper addressing the readings for the week. These emails should have “thought paper” in the subject line and be sent by midnight Monday nights (just paste it in the message instead of attaching it). The thought paper should address all the readings for the week at some level. Think about how these readings relate to your thesis; bring up questions you have; relate the readings to other things you know. You want to show me that you’ve read, understood, and carefully processed the readings for the week. The purpose of these assignments is to get you thinking more deeply about the method and its application to your research and to facilitate in-class discussion. You can drop your two lowest thought paper grades.
Discussion and thought papers will be graded on the following scale:
0 = not there
2 = attended but didn’t participate, or turned in, but not very relevant (below average)
3 = comments or questions relevant, but didn’t involve much insight (average)
4 = comments or questions relevant and insightful (good)
5 = more than one comment or question showed a significant contribution (excellent)
Research
proposal (45%). One of the best ways to show that you understand
research design is by applying it. You will submit a complete research proposal
(7-8 page introduction, complete method and appendices, section on ethics and
validity, plan of analysis, discussion of expected results/what it would mean
if you didn’t find them, in APA style). This paper doesn’t have to be your
thesis proposal, but if you can write about something that does develop into it
(or another research project), all the better. There will be assignments due
during the semester to provide you with feedback on this paper (e.g., outlines,
rough drafts) that will count toward the “assignment” portion of your grade. Obviously, this should not be a paper that
you have written or are writing for another class. If there is any question
about whether it might be too collaborative or too close to something you’ve
already done, talk to me before you
get started to see if it is acceptable. The paper should be in good APA
style. Those that aren’t, will be
returned ungraded and lose at least 1 letter grade (1 if turned in that same
day, -1 more for each day after).
Proposal presentation (10%). During one of the last class sessions, you will present your proposal to the class (background, method, expected results, what they would mean, etc.). Your presentation, which should include some audio-visual effects (e.g., PowerPoint), should last no more than 12 minutes, followed by a discussion of the proposal by the class (we will play the role of thesis committee). You can (and should) integrate any helpful comments from the class into your proposal before you turn it in. You’ll be graded on presentation style as well as knowledge about the area. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdFI0TXU0T3BweEljS1M1Wndmdi1hMWc&usp=sharing to sign up for a time. NOTE: You have to do this from your uni google account.
Assignments (10%). In addition to the weekly thought papers, you will turn in several assignments directly related to your research proposal (i.e., topic, articles, outlines, rough draft) and provide feedback on a peer’s draft. You will also review articles demonstrating different methods. You will also present on a particular method or “hot topic.” For your topic presentation, you should have at least five references. These should include one or more articles or chapters on the method itself (or evaluations of it) and often, depending on topic, one or more articles demonstrating good or bad use of that method. If I listed an article with the presentation topic in the class schedule, that should be one of the ones you read. In the presentation, you’ll tell us about what the method is, what its advantages and disadvantages are, when it should/shouldn’t be used, and depending on the topic, either best practices in using the method and/or how (in general) you would analyze data collected by the method. You should also provide examples (actual research examples) of good and not-so-good uses of that method. Sometimes there are additional questions you should address that I put on the class schedule. You’re giving a brief overview so that everyone learns the issues involved, but you don’t need to describe it in enough detail that we could actually go out and do it. Presentations should be 10-15 minutes and include your references in your final slide. Send me a copy of your Powerpoints before the presentation. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdFlxN2MyUFpkeTMyNTRSLTNXbmIwdGc&usp=sharing to sign up for a topic and time. NOTE: You have to do this from your UNI google account.
Makeup and Late Paper Policies: Class discussion grades and presentations cannot be
made up. Proposals will be accepted up to three days past the due date, but one
letter grade will be deducted for each day until they are turned in. Plan ahead
and don’t wait until the last minute to finish (or start) the paper, in case
something unexpected arises.
Academic Honesty Policy: Cheating and plagiarism of any kind will not be
tolerated and will result in a 0 on the assignment in question. This includes
using a paper from another class or that you have worked on with another
faculty member to fulfill a requirement in this class as well as using quotes
from materials without attribution even in short assignments or thought papers.
For more information on UNI’s academic
honesty policies, see the University Catalog as well as the information in the
Department of Psychology Graduate Student Handbook. If you have any questions
about what is acceptable, ask.
Disability Services: The Americans with Disabilities Act of 1990 (ADA)
provides protection from illegal discrimination for qualified individuals with
disabilities. Students requesting instructional accommodations due to disabilities
must arrange for such accommodation through the Office of Disability Services.
The ODS is located at 213 Student Services Center, and the phone number is
273-2676.
CLASS SCHEDULE
Jan. 15—Foundations and theory
T&D, Chapter 1
Kerr, N. L.
(1998). HARKing: Hypothesizing after the results are
known. Personality and Social Psychology
Review, 2, 196-217. doi:10.1207/s15327957pspr0203_4
Simmons, J. P., Nelson, L. D., & Simonsohn,
U. (2011). False-positive
psychology: Undisclosed flexibility in data collection and analysis allows
presenting anything as significant. Psychological
Science, 22, 1359-1366. doi:10.1177/0956797611417632
What is
the theoretical basis for your thesis? What is good and bad about theories?
What examples have you seen of HARKing? Is it really
a bad thing? What do you think of the suggestions of Kerr and of Simmons et
al.? Are they good? Are they practical? What should we do to avoid misleading
reports of research in psychology?
Jan. 22--Philosophy of science
Kuhn book
What is
Kuhn’s view of science? Do you agree? How does science change? How should it
change? What is and should be the goal of science? What are examples of
“revolutions” in psychology?
Turn in topic
description for your research proposal for class; put it in my mailbox by
Friday at noon.
Hertwig, R., & Ortmann, A.
(2008). Deception
in experiments: Revisiting the arguments in its defense. Ethics & Behavior, 18, 59-92. doi:10.1080/10508420701712990
Becker-Blease, K. A., & Freyd, J. J. (2006). Research
participants telling the truth about their lives: The ethics of asking and
not asking about abuse. American
Psychologist, 61, 218-226. doi:10.1037/0003-066X.61.3.218
Moreno, M. A., Goniu, N., Moreno, P. S.,
& Diekema, D. (2013). Ethics of social media research: Common concerns and
practical considerations. Cyberpsychology,
Behavior, and Social Networking, 16, 708-713. doi:10.1089/cyber.2012.0334
Are
there any ethical issues that would need to be addressed with your thesis? How
could you deal with them? Is deception okay? When? How should people be
debriefed? Are our current techniques adequate? Should “vulnerable populations”
receive more protection? How or why? Should we ask about things like abuse?
Debriefing
presentation—Is
debriefing effective? When or why? Briefly review research on debriefing and
give us some “best practice” suggestions.
T&D Chapter 2
Henrich, J. Heine, S. J., &
Norenzayan, A. (2010). The weirdest people in
the world? Behavioral and Brain
Sciences, 33, 61-83. doi:10.1017/S0140525X0999152X
Maxwell, S.
E., Kelley, K., Rausch, J. R. (2008). Sample size planning for statistical
power and accuracy in parameter estimation. Annual
Review of Psychology, 59, 537-563. doi:
10.1146/annurev.psych.59.103006093735
Is
sampling an issue for your thesis topic? What type of sampling will you use? Should you use? What populations have been used in your
area? Is that a problem or not and why? If there are WEIRD samples used, how is
this likely to interact with your thesis topic area? When are WEIRD samples
more or less of a problem? How large should your sample be for your thesis? How
will you determine this?
Find
10 articles for your research proposal for this class and submit an APA style
reference section. If they are not in near-perfect APA style, they will be
returned ungraded, and you will get a 0 on the assignment.
Cross-cultural
research presentation—When is cross-cultural research most useful? What
particular things do researchers need to keep in mind? What are best practices?
Feb. 12 --Reliability and validity
and scaling
T&D Chapters 3 and 5
Find an
article that describe the development of a scale related to your thesis topic
(should include multiple studies, assessing reliability and validity). Evaluate
the article and scale (and include a brief description of the scale, including
the APA style citation). How did they show the various types of reliability and
validity? Do you agree? What type of scaling is the measure? What type of
scale? Are there similar concepts that still need a scale developed? How would
you go about creating a scale? What would you correlate it with to test
reliability and validity?
Outline of
intro for proposal due
Method
variance presentation--What is the
problem of method bias? How can it be decreased? Find at least 2 articles that
dealt with this issue either well or poorly.
Podsakoff, P. M., MacKenzie, S. B.,
& Podsakoff, N. P. (2012). Sources of method bias in social
science research and recommendations on how to control it. Annual Review of Psychology, 63, 539-569.
doi:10.1146/annurev-psych-120710-100452
T&D
Chapter 4
Tourangeau, R. (2004). Survey research and societal change. Annual Review of Psychology, 55, 775-801. doi:10.1146/annurev.psych.55.090902.142040
Groves, R.
M. (2011). Three eras of survey research. Public Opinion Quarterly, 75, 861-871. doi:10.1093.poq/nfr057
Graham, J.
W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60,
549-576. doi:10.1146/annurev.psych.58.110405.085530
Find examples
of “bad” survey questions. Think about any issues that you might need to deal
with in terms of how you ask questions on your thesis. What types of issues (e.g.,
population, sampling) might come up and how would those affect what method you
choose? What are the advantages and disadvantages of survey methodology for
your topic?
Diary
presentation—Include information on
the variety of diary methods and how they are analyzed (sometimes called
“event-sampling” as well)
Social network analysis presentation—What is social network
analysis? When and why might you want to use it? How can it be analyzed?
Burt, R. S., Kilduff, M., & Tasselli, S. (2013). Social network analysis: Foundations and frontiers on advantage. Annual Review of Psychology, 64, 527-547. doi:10.1146/annurev-psych-113011-143828
T&D Chapters 7 and 9
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social
Psychology, 51, 1173-1182. doi:10.1037/0022-3514.51.6.1173
MacKinnon,
D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation
analysis. Annual Review of
Psychology, 58, 593-614. doi:10.1146/annurev.psych.58.110405.085542
Which threats
to internal validity are most likely to be a problem in your thesis? How can
you diminish them? What potential mediators and moderators might be involved in
your thesis topic area? How can we differentiate between mediators and
moderators? Should you include covariates in your thesis? Why or why not?
Outline of
proposal paper due
Placebo
presentation—What
are placebo effects and what causes them? What are best practices in placebo
research?
Price, D. D., Finniss, D. G., &
Benedetti, F. (2008). A
comprehensive review of the placebo effect: Recent advances and current
thought. Annual Review of Psychology, 59,
565-590. doi:10.1146/annurev.psych.59.113006.095941
T&D Chapters 10 and 11
Shadish, W. R., & Cook, T. D. (2009). The Renaissance of field
experimentation in evaluating interventions. Annual Review of Psychology, 60, 607-629. doi:10.1146/annurev.psych.60.110707.163544
Is your
thesis a quasi-experiment? If so, what kind? What threats to internal validity
might be a problem? Are the elements of “good design” present in your thesis?
Explain. What are the advantages and disadvantages of field research? How could
it be (or has been) used in the area of your thesis topic?
March 12—Evaluation research and
meta-analysis
T&D Chapter 16
Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative
methods for literature reviews. Annual
Review of Psychology, 52, 59-82. doi:10.1146/annurev.psych.52.1.59
Field, A. P., & Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and
Statistical Psychology, 63, 665-694. doi:10.1348/000711010X502733
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and
interpretation. Journal of Experimental
Psychology: General, 141, 2-18. doi:10.1037/a0024338
What types of
evaluation are relevant for your thesis topic? Why? Which is more
important: clinical, practical, or
statistical significance? Have there been meta-analyses on your thesis topic? If
not, is the area ready for one? What key words would you use? What mediators or
moderators would you test? Find and evaluate a meta-analysis that interests
you/is related to your topic (include citation and brief description of
method/findings). What is good and bad about what they did? Address things like
inclusion criteria as well as fixed vs. random effects, etc.
March
19—Spring break
March 26—Quantitative analysis
T&D Chapters 12 and 14
APA Publications and Communications Board Working Group on Journal
Article Reporting Standards.
(2008). Reporting standards for research in
psychology: Why do we need them? What might they be? American Psychologist, 63, 839-951. doi:10.1037/0003-066X.63.9.839
Kashy, D. A., & Kenny,
D. A. (2000). The analysis of data from dyads and groups.
In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology.
New York: Cambridge University Press. On reserve in the
library (2 hour hold).
How will you
analyze your data for your thesis? What tests will be involved? What
assumptions do those tests have? Make sure you include enough description of
your method that I can understand whether these analyses are appropriate. How
could some of the analyses that you were less familiar with be useful in
studying your topic?
Quantitative
paper critique due (see questions below).
April 2--Writing it up, peer review
T&D Chapter 15
Strunk, W., Jr. (1999). The elements
of style. New York: Bartleby.com
Bem, D. J. (2002). Writing
the empirical journal article. In J. M. Darley, M. P.
Zanna, & H. L. Roediger III (Eds.),
(2002). The compleat
academic: A career guide. Washington, DC: American Psychological Association.
Sternberg,
R. J. (1993). How to
win acceptances by psychology journals: 21 tips for better writing. APA
Observer.
What things do you need to be more careful with in
your own writing? How can you become a better scientific writer?
Bring in
rough draft of thesis proposal for peer review and comment.
April 9--Other methods: Implicit
measures and physiological measures
De Houwer, J., & Moors, A. (2010). Implicit measures: Similarities and differences. In
B. Gawronski & B. K. Keith (Eds.), Handbook of implicit social cognition:
Measurement, theory, and applications (pp. 176-196). Available
in resource room.
Sowden, P., & Barrett, P. (2006). Psychophysiological methods.
In G. M. Breakwell, S. Hammond, C. Fife-Schaw, & J. A. Smith (Eds.), Research methods in psychology (3rd ed.)
(pp. 146-159). Thousand Oaks, CA: Sage. Available in resource
room and possibly on googlebooks.
What are the
advantages and disadvantages of implicit measures? What are they really
measuring? When should they be used? What do physiological measures tell us and
what can they not tell us? How could either of these types of measures be used
in research in your thesis area? What are questions or comments do you have
about these readings?
fMRI
presentation—What is fMRI? What does
it tell us? How can it be used in psychology?
Vul, E., Harris, C., Winkielman,
P., & Pashler, H. (2009). Puzzingly high correlations in fMRI studies of emotion, personality, and
social cognition. Perspectives on
Psychological Science, 4, 274-290. doi:10.1111/j.1745-6924.2009.01125.x
ERP/EKG presentation—What are
ERP and EKGs? What do they tell us? How can they be used in psychology?
Eye tracking presentation—What is eye tracking? How can it be used in psychology?
Duchowski, A. T. (2002). A breadth-first survey of eye tracking applications. Behavior Research Methods, Instruments, and Computers,
34, 455-470. doi:10.3758/BF03195475
Hormone
sampling presentation—What hormones have typically been tracked in psychology?
How are these collected? What do they tell us? What are best practices in
collecting them (e.g., times of day, people for whom the results aren’t
accurate)?
April 16—Other
methods: Spatial analyses and internet-based research
Goodchild, M. F., Anselin, L., Appelbaum, R, P., & Herr Harthorn,
B. (2000). Toward spatially
integrated social science. International
Regional Science Review, 23, 139-159.
doi:10.1177/016001760002300201
Sui, D., & Goodchild, M. (2011). The convergence of GIS and social media: Challenges
for GIScience. International
Journal of Geographical Information Science, 25, 1727-1748. doi:10.1080/13658816.2011.604636
Skitka, L. J., & Sargis, E. G.
(2006). The
internet as psychological laboratory. Annual Review of Psychology, 57, 529-555. doi:10.1146/annurev.psych.57.102904.190048
Lewis, K., Kaufman, J., Gonzalez, M., Wimmer,
A., & Christakis, N. (2008).
Tastes, ties, and time: A new social network dataset using Facebook.com. Social Networks, 30, 330-342. doi:10.1016/j.socnet.2008.07.002
How does space
affect your thesis topic, if at all? Are there potential spatial effects either
that may be interesting in their own right or that should be controlled for?
Research using the internet brings exciting advantages, but also significant
disadvantages (e.g., ethical concerns). What do you think about research on the
internet—are there certain methods or populations or topics that shouldn’t be
studied there? What other questions or comments do you have about these
methods?
Computer
simulation presentation—How can computer
simulations add to our knowledge of psychology and human behavior? When are
they best used? What advantages and disadvantages do they offer?
mTurk
presentation—What is mTurk? When is it more/less useful?
Buhrmester, M., Kwang, T., & Gosling,
S. D. (2011). Amazon’s mechanical turk: A new course of inexpensive,
yet high-quality data? Perspectives on
Psychological Science, 6, 3-5. doi:10.1177/1745691610393980
Buhrmester, M. (2010).
Amazon
mechanical turk guide for
social scientists.
Smart phone
presentation—What types of applications can smartphones be used for?
Try one out if possible (e.g., iEar). What are its
advantages and challenges and how can the challenges be addressed (think ethics
as well)?
Raento, J., Oulasvirta, A., &
Eagle, N. (2009). Smartphones: An
emerging tool for social scientists. Sociological
Methods and Research, 37, 426-454. doi:10.11177/0049124108330005
Virtual
reality presentation—How has virtual reality been used in psychology? What
types of virtual reality have been used? What are the advantages and
disadvantages of these approaches? (Jim Blascovich is
a good researcher to start with)
Second life
presentation—How can second life be used, either to study topics or to
recruit participants? Are there ethical or methodological concerns?
Dean, E., Cook, S., Murphy, J.,
& Keating, M. (2012). The effectiveness of survey recruitment methods in Second Life.
Social Science Computer Review, 30,
324-338. doi:10.1177/0894439311410024
April 23--Qualitative research
T&D Chapter 6 and 8 and 13
Smith, C. P.
(2000). Content analysis and narrative analysis. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp.
313-335). New York: Cambridge
University Press. On reserve in the library.
What are the
advantages and disadvantages of qualitative research? What new or useful
information could qualitative studies provide on your thesis topic?
Qualitative
article critique due (see questions below).
Semi-structured
interviews presentation—What are the advantages/disadvantages of this method? What
are some best practices? How can the data be analyzed?
Bartholomew, K., Henderson, A. J. Z., & Marcia, J. E. (2000). Coded semistructured
interviews in social psychological research. . In H.
T. Reis & C. M. Judd (Eds.), Handbook
of research methods in social and personality psychology (pp. 286-312). New York: Cambridge University
Press. On reserve in the library.
April
28—Proposal presentations ****Note that this is a MONDAY
Six
proposals
Monday,
May 5 (1:00-2:50)—Proposal presentations
Four
proposals
Papers due Thursday, May 8 at noon. Turn in
rough draft and peer review comments as well.
1. What was the purpose of the article? How does
it fit into previous literature?
2. What
was/were the research question(s) or hypotheses?
3. What is the
research design (be specific)?
4.
How were the variables operationalized
or measured (mention for IVs and DVs)?
5. Are the
measures reliable and valid? What evidence do you have of that?
6. Was the
sample representative of the population of interest? How much does that matter
for this study?
7. Was there
sufficient power to obtain significant results? Include details on your power
analysis.
8. What types
of statistical analyses were done? Do
they match the research questions/hypotheses? Are there any concerns related to
these analyses?
9. What were
the results?
10. Is the
discussion of the results in line with the results? (comment
on this; don’t just say yes or no)
11. Give at least 2 strengths of the article and at
least 2 weaknesses or concerns you would have as a reviewer. For the concerns,
are there ways that the authors could or did fix or address them?
Critique of Qualitative
Research Article
1.
What type of qualitative study is it? (ethnography,
case study, focus group, etc)
2.
What is the purpose of the study? What are the research questions?
3. How does the study build on previous
literature?
4. What sources of data over what period of time
were used?
5.
What does this study tell you that a quantitative one would not be able
to?
6. Is the
discussion of the results in line with the results? (comment
on this; don’t just say yes or no)
7. Give at least 2 strengths of the article and at
least 2 weaknesses or concerns you would have as a reviewer. For the concerns,
are there ways that the authors could or did fix or address them?