RESEARCH
DESIGN
Spring 2012
|
Class Information |
Instructor Information |
|
PSYCH 6002 |
Helen C. Harton, Ph.D. |
|
Baker 315 |
Baker 357 |
|
W 12-2:50pm |
273-2235; harton@uni.edu |
Office Hours: WF 11-11:50; W 2-3; pretty much
any time I’m around
Readings:
Trochim, W., & Donnelly, J. P. (2007). The research
methods knowledge base (3rd ed.). Cincinnati, OH:
Atomic Dog Publishing.
Kuhn, T. S. (1996). The structure
of scientific revolutions (3rd
ed). Chicago: University
of Chicago Press.
Course Description and Objectives: Research is at the heart of psychology; psychology’s
focus on research and empirical evidence is one of the things that differentiates the field from other similar disciplines. In
this course, you’ll learn about the basics of research, explore several “hot
topics” or new methods in psychology, and become a better designer, describer,
and critical reader of research. The course is designed to give you a breadth of
knowledge on research design in psychology and to facilitate your future
research career and thesis progress.
By the end of this course, you
should
1) know
the advantages and disadvantages of the commonly used methods in psychology and
when to best apply them;
2) know the advantages and
disadvantages of several new methodologies in psychology;
3) be able to critique and
integrate previous research;
4) be
more aware of ethical concerns in research and how to deal with them;
5) be a
better scientific writer;
6) be able to design and carry
out your own research project; and
7) have
made significant progress on your thesis.
Course Format: Each week you will have several readings that you should have processed
and thought about before class. In class, we will discuss the readings and the
issues they bring up. You should contribute thoughtfully to the discussion and
build on and gently challenge the comments of other students.
Course Requirements:
Class discussion (25%). In graduate school especially, you learn not only from books and professors, but from interactions and discussions with peers. Discussing information also helps you to think about it more deeply and learn it more quickly. You are expected to contribute meaningfully to class discussions. While mere attendance is not enough to get a good grade for this component, it is imperative in that you can’t participate if you’re not here. Obviously, it also requires that you read and think about the readings. Both frequency and quality count. You will get graded on discussion each week; you can drop your lowest score.
Thought papers (15%). Each week you’ll also be asked to email me a short (1-2 pages) thought paper addressing the readings for the week. These emails should have “thought paper” in the subject line and be sent by midnight Monday nights (just paste it in the message instead of attaching it). Each week, I included some questions (in italics on the class schedule) that you should address, but you don’t have to address all of these and you can address other issues as well. The exception is when it specifically says to do something (e.g., find a meta-analysis on a topic). The purpose of these assignments is to get you thinking more deeply about the method and its application to your research and to facilitate in-class discussion. You can drop your lowest thought paper grade. If you do one the first week of classes and get it in on time, you can drop your two lowest.
Discussion and thought papers will be graded on the following scale:
0 = not there
2 = attended but didn’t participate, or turned in, but not very relevant (below average)
3 = comments or questions relevant, but didn’t involve much insight (average)
4 = comments or questions relevant and insightful (good)
5 = more than one comment or question showed a significant contribution (excellent)
Research proposal (30%). One of the best ways to show that you understand research design is by applying it. You will submit a complete research proposal (7-8 page introduction, complete method and appendices, plan of analysis, discussion of expected results/what it would mean if you didn’t find them, in APA style). This paper doesn’t have to be your thesis proposal, but if you can write about something that does develop into it (or another research project), all the better. There will be assignments due during the semester to provide you with feedback on this paper (e.g., outlines, rough drafts) that will count toward the “assignment” portion of your grade. Obviously, this should not be a paper that you have written or are writing for another class. If there is any question about whether it might be too collaborative or too close to something you’ve already done, talk to me before you get started to see if it is acceptable.
Proposal presentation (15%). During one of the last class sessions, you will present your proposal to the class (background, method, expected results, what they would mean, etc.). Your presentation, which should include some audio-visual effects (e.g., overheads or PowerPoint), should last no more than 12 minutes, followed by a discussion of the proposal by the class (we will play the role of thesis committee). You can (and should) integrate any helpful comments from the class into your proposal before you turn it in. You’ll be graded on presentation style as well as knowledge about the area. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdHR1UVB1TkdyZnkzNmk0NFBVYmhqZ1E to sign up for a time. NOTE: You have to do this from your uni google account.
Assignments (15%). In addition to the weekly thought papers, you will turn in several assignments directly related to your research proposal (i.e., topic, articles, outline, rough draft) and provide feedback on a peer’s draft. You may sometimes be asked to read short articles before class that we will critique in class. You will also present on a particular method or “hot topic.” For your topic presentation, you should read at least one article or chapter on the method itself as well as at least two articles that used the method (if I listed an article with the presentation topic in the class schedule, that should be one of the ones you read). In the presentation, you’ll tell us about what the method is, what its advantages and disadvantages are, when it should/shouldn’t be used, how the articles you read used it and whether that was a good use or not, and depending on the topic, either best practices in using the method and/or how (in general) you would analyze data collected by the method. Sometimes there are additional questions you should address that I put on the class schedule. You’re giving a brief overview so that everyone learns the issues involved, but you don’t need to describe it in enough detail that we could actually go out and do it. Presentations should be 10-15 minutes and include your references in your final slide. Send me a copy of your Powerpoints. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdG02bzBGckQ3MXBQemIxUzN6UHB4RlE&hl=en_US#gid=0 to sign up for a topic and time. NOTE: You have to do this from your uni google account.
Makeup and Late Paper Policies: Class discussion grades and presentations cannot be
made up. Proposals will be accepted up to three days past the due date, but one
letter grade will be deducted for each day until they are turned in. Plan ahead
and don’t wait until the last minute to finish (or start) the paper, in case
something unexpected arises.
Academic Honesty Policy: Cheating and plagiarism of any kind will not be
tolerated and will result in a 0 on the assignment in question. This includes
using a paper from another class or that you have worked on with another
faculty member to fulfill a requirement in this class as well as using quotes
from materials without attribution even in short assignments or thought papers.
For more information on UNI’s academic
honesty policies, see the University Catalog as well as the information in the
Department of Psychology Graduate Student Handbook. If you have any questions
about what is acceptable, ask.
Disability Services: The Americans with Disabilities Act of 1990 (ADA)
provides protection from illegal discrimination for qualified individuals with
disabilities. Students requesting instructional accommodations due to disabilities
must arrange for such accommodation through the Office of Disability Services.
The ODS is located at 213 Student Services Center, and the phone number is
273-2676.
CLASS SCHEDULE
Jan.
11—Foundations and theory
T&D, Chapter 1
Kerr, N. L. (1998). HARKing:
Hypothesizing after the results are known. Personality
and Social Psychology Review, 2, 196-217. doi: 10.1207/s15327957pspr0203_4
Simmons, J. P., Nelson, L. D., &
Simonsohn, U. (2011). False-positive
psychology: Undisclosed flexibility in data collection and analysis allows
presenting anything as significant. Psychological
Science, 22, 1359-1366. 10.1177/0956797611417632.
What is the theoretical basis for your thesis? What is
good and bad about theories? What examples have you seen of HARKing?
Is it really a bad thing? What do you think of the suggestions of Kerr and of
Simmons et al.? Are they good? Are they practical? What should we do to avoid
misleading reports of research in psychology? What other comments do you have
on these readings?
Jan.
18---Philosophy of science
Kuhn book
What is Kuhn’s view of science? Do you agree? How does
science change? How should it change? What is and should be the goal of science?
What are examples of “revolutions” in psychology? What other comments do you
have on the ideas in the book?
Turn in topic description for your thesis (research
proposal).
Hertwig, R., & Ortmann,
A. (2008). Deception
in experiments: Revisiting the arguments in its defense. Ethics & Behavior, 18, 59-92. doi: 10.1080/10508420701712990
Becker-Blease,
K. A., & Freyd, J. J. (2006). Research participants
telling the truth about their lives: The ethics of asking and not asking about
abuse. American Psychologist, 61, 218-226.
doi:
10.1037/0003-066X.61.3.218
Barchard, K. A., Williams,
J. (2008). Practical advice for conducting ethical online experiments and
questionnaires for United States psychologists. Behavior Research Methods, 40, 1111-1128. doi: 10.3758/BRM.40.4.1111
Are there any ethical issues that would need to be
addressed with your thesis? How could you deal with them? Is deception okay?
When? How should people be debriefed? Are our current techniques adequate?
Should “vulnerable populations” receive more protection? How or why? Should we
ask about things like abuse? What other comments do you have on these readings?
Debriefing presentation—Is debriefing effective?
When or why? Briefly review research on debriefing and give us some “best
practice” suggestions.
T&D Chapter 2
Henrich, J. Heine, S. J., &
Norenzayan, A. (2010). The weirdest people in
the world? Behavioral and Brain
Sciences, 33, 61-83. doi:10.1017/S0140525X0999152X
Maxwell, S. E., Kelley, K., Rausch, J. R. (2008).
Sample size planning for statistical power and accuracy in parameter
estimation. Annual Review of Psychology,
59, 537-563. doi:
10.1146/annurev.psych.59.103006093735
Is sampling an issue for your thesis topic? What type
of sampling will you use? Should you use? What
populations have been used in your area? Is that a problem or not and why? If
there are WEIRD samples used, how is this likely to interact with your thesis
topic area? When are WEIRD samples more or less of a problem? How large should
your sample be for your thesis? How will you determine this? What other
comments do you have on the readings for this week?
Find 10 articles for thesis (research proposal), submit
in APA style
Cross-cultural research presentation—When is cross-cultural research most useful? What
particular things do researchers need to keep in mind? What are best practices?
Feb. 8
--Reliability and validity and scaling
T&D Chapters 3 and 5
Find an article that describe the development of a
scale related to your thesis topic (should include multiple studies, assessing
reliability and validity). Evaluate the article and scale (and include a brief
description of the scale, including the APA style citation). How did they show
the various types of reliability and validity? Do you agree? What type of
scaling is the measure? What type of scale? Are there similar concepts that
still need a scale developed? How would you go about creating a scale? What
would you correlate it with to test reliability and validity? What other
comments or questions do you have on reliability/validity/scaling?
Outline of intro for proposal
T&D Chapter 4
Tourangeau, R. (2004). Survey research and
societal change. Annual Review of
Psychology, 55, 775-801. doi:
10.1146/annurev.psyc.55.090902.142040
Graham, J. W. (2009). Missing data analysis: Making it
work in the real world. Annual Review of
Psychology, 60, 549-576. doi:
10.1146/annurev.psych.58.110405.085530
Find examples of “bad” survey questions. Think about
any issues that you might need to deal with in terms of how you ask questions
on your thesis. What types of issues (e.g., population, sampling) might come up
and how would those affect what method you choose? What are the advantages and
disadvantages of survey methodology for your topic? What other comments or
questions do you have based on these readings?
Diary presentation—Include information on the variety of diary methods and how they are
analyzed (sometimes called “event-sampling” as well)
Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54,579-616.
doi:
10.1146/annurev.psych.54.101601.145030
T&D Chapters 7 and 9
Baron, R. M., & Kenny, D. A.
(1986). The moderator-mediator
variable distinction in social psychological research: Conceptual, strategic,
and statistical considerations. Journal
of Personality and Social Psychology, 51, 1173-1182. doi: 10.1037/0022-3514.51.6.1173
MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S.
(2007). Mediation analysis. Annual Review of Psychology, 58, 593-614. doi: 10.1146/annurev.psych.58.110405.085542
Which threats to internal validity are most likely to
be a problem in your thesis? How can you diminish them? What potential
mediators and moderators might be involved in your thesis topic area? How can
we differentiate between mediators and moderators? Should you include
covariates in your thesis? Why or why not? What other comments or questions do
you have on these readings?
Outline of thesis proposal
Placebo presentation--What are best practices in placebo research?
Price, D. D., Finniss,
D. G., & Benedetti, F. (2008).
A comprehensive review of the placebo effect: Recent advances and current
thought. Annual Review of Psychology, 59,
565-590.
Method variance presentation--What is the problem of method bias? How can it be
decreased? Find at least 2 articles that dealt with this issue either well or
poorly.
Podsakoff, P. M., MacKenzie, S. B.,
Lee, J., Podsakoff, N. P. (2003). Common method
biases in behavioral research: A critical review of the literature and
recommended remedies. Journal of Applied
Psychology, 88, 879-903. doi:
10.1037/00212-9010.88.5.879.
Feb. 29
-----Quasi-experiments
T&D Chapters 10 and 11
Shadish, W. R., & Cook, T. D. (2009). The Renaissance of field
experimentation in evaluating interventions. Annual Review of Psychology, 60, 607-629. doi: 10.1146/annurev.psych.60.110707.163544
Is your thesis a quasi-experiment? If so, what kind?
What threats to internal validity might be a problem? Are the elements of “good
design” present in your thesis? Explain. What are the advantages and
disadvantages of field research? How could it be (or has been) used in the area
of your thesis topic? What other comments/questions did this week’s readings
bring up for you?
March
7—Evaluation research and meta-analysis
T&D Chapter 16
Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative
methods for literature reviews. Annual
Review of Psychology, 52, 59-82. doi:10.1146/annurev.psych.52.1.59
Field, A. P., & Gillett, R.
(2010). How to
do a meta-analysis. British
Journal of Mathematical and Statistical Psychology, 63, 665-694. doi:10.1348/000711010X502733
What types of evaluation are relevant for your thesis
topic? Why? Which is more important:
clinical, practical, or statistical significance? Have there been
meta-analyses on your thesis topic? If not, is the area ready for one? What key
words would you use? What mediators or moderators would you test? Find and
evaluate a meta-analysis that interests you/is related to your topic (include
citation and brief description of method/findings). What is good and bad about
what they did? Address things like inclusion criteria as well as fixed vs.
random effects, etc.
March 14—Spring break
March
21—Quantitative analysis
T&D Chapters 12 and 14
Kashy, D. A., & Kenny, D. A. (2000). The
analysis of data from dyads and groups. In H. T. Reis
& C. M. Judd (Eds.), Handbook of
research methods in social and personality psychology. New York:
Cambridge University Press. On reserve in the library (2 hour
hold).
How will you analyze your data for your thesis? What
tests will be involved? What assumptions do those tests have? Make sure you
include enough description of your method that I can understand whether these
analyses are appropriate. How could some of the analyses that you were less
familiar with be useful in studying your topic? What other questions/issues
came up for you with this week’s readings? Find a quantitative article related
to your thesis (include citation) and answer the quantitative critique
questions below.
March 28--Writing it up, peer review
T&D Chapter 15
Strunk, W., Jr. (1999). The elements of style. New
York: Bartleby.com
Bem, D. J. (2002). Writing the empirical journal article.
In J. M.
Darley, M. P. Zanna, & H. L. Roediger III (Eds.), (2002). The compleat
academic: A career guide. Washington, DC: American Psychological
Association.
Sternberg, R. J. (1993). How to win acceptances
by psychology journals: 21 tips for better writing. APA Observer.
Bring in rough draft of thesis proposal for peer
review and comment. No thought paper this week!
April
4--Other methods: Implicit measures and physiological measures
Fazio, R. H., & Olson, M. A.
(2003). Implicit measures in
social cognition research: Their meaning and use. Annual Review of Psychology, 54, 297-327. doi: 10.1146/annurev.psych.54.101601.145255
De Houwer,
J., Teige-Mocigemba, S., Spruyt,
A., & Moors, A. (2009).
Implicit measures: A normative analysis and review. Psychological Bulletin, 135, 347-368. doi: 10.1037/a0014211
Sowden, P., & Barrett, P. (2006). Psychophysiological methods. In G. M. Breakwell, S.
Hammond, C. Fife-Schaw, & J. A. Smith (Eds.), Research methods in psychology (3rd
ed.) (pp. 146-159). Thousand Oaks, CA: Sage. Available in resource room and possibly on googlebooks.
What are the advantages and disadvantages of implicit
measures? What are they really measuring? When should they be used? What do
physiological measures tell us and what can they not tell us? How could either
of these types of measures be used in research in your thesis area? What are
questions or comments do you have about these readings?
fMRI
presentation—What is fMRI? What does it tell us? How can it be used in
psychology?
Vul, E., Harris, C., Winkielman, P., & Pashler, H.
(2009). Puzzingly high correlations in fMRI studies of emotion, personality, and social cognition.
Perspectives on Psychological Science, 4,
274-290. doi:
10.1111/j.1745-6924.2009.01125.x
ERP/EKG
presentation—What are ERP and EKGs? What do they tell us? How can they
be used in psychology?
Eye tracking
presentation—What is eye tracking? How can it be used in psychology?
Duchowski, A. T. (2002). A breadth-first
survey of eye tracking applications. Behavior
Research Methods, Instruments, and Computers, 34, 455-470. doi:10.3758/BF03195475
Hormone sampling presentation—What hormones have typically been tracked in psychology?
How are these collected? What do they tell us? What are best practices in
collecting them (e.g., times of day, people for whom the results aren’t
accurate)?
April
11—Other methods: Spatial analyses and internet-based
research
Goodchild, M. F., Anselin,
L., Appelbaum, R, P., & Herr Harthorn,
B. (2000). Toward spatially
integrated social science. International
Regional Science Review, 23, 139-159.
doi: 10.1177/016001760002300201
Skitka, L. J., & Sargis,
E. G. (2006). The
internet as psychological laboratory. Annual Review of Psychology, 57, 529-555. doi: 10.1146/annurev.psych.57.102904.190048
Lewis, K., Kaufman, J., Gonzalez,
M., Wimmer, A., & Christakis, N. (2008). Tastes, ties, and time: A new social network dataset
using Facebook.com. Social Networks, 30, 330-342.
doi:10.1016/j.socnet.2008.07.002
How does space affect your thesis topic, if at all?
Are there potential spatial effects either that may be interesting in their own
right or that should be controlled for? Research using the internet brings exciting
advantages, but also significant disadvantages (e.g., ethical concerns). What
do you think about research on the internet—are there certain methods or
populations or topics that shouldn’t be studied there? What other questions or
comments do you have about these methods?
Computer simulation presentation—How can computer simulations add to our knowledge of
psychology and human behavior? When are they best used? What advantages and
disadvantages do they offer?
mTurk presentation—What is mTurk? When is it
more/less useful?
Buhrmester, M., Kwang,
T., & Gosling, S. D. (2011).
Amazon’s mechanical turk: A
new course of inexpensive, yet high-quality data? Perspectives on Psychological Science, 6, 3-5. doi: 10.1177/1745691610393980
Buhrmester, M. (2010). Amazon
mechanical turk guide for
social scientists.
Smart phone presentation—What
types of applications can smartphones be used for?
Try one out if possible (e.g., iEar). What are its
advantages and challenges and how can the challenges be addressed (think ethics
as well)?
Raento, J., Oulasvirta,
A., & Eagle, N. (2009). Smartphones: An emerging tool for social scientists. Sociological Methods and Research, 37, 426-454.
doi:
10.11177/0049124108330005
Virtual reality presentation—How has virtual reality been used in psychology? What
types of virtual reality have been used? What are the advantages and
disadvantages of these approaches? (Jim Blascovich is
a good researcher to start with)
Two proposals
April
18--Qualitative research
T&D Chapter 6 and 8 and
13
Smith, C. P. (2000). Content
analysis and narrative analysis. In H. T. Reis &
C. M. Judd (Eds.), Handbook of research methods
in social and personality psychology (pp. 313-335). New York: Cambridge University Press. On
reserve in the library.
What are the advantages and disadvantages of
qualitative research? What new or useful information could qualitative studies
provide on your thesis topic? Find and critique an article using qualitative
methods that is related to your thesis topic. Use the question for the
qualitative critique below.
Semi-structured interviews presentation—What are the
advantages/disadvantages of this method? What are some best practices? How can
the data be analyzed?
Bartholomew, K., Henderson, A. J.
Z., & Marcia, J. E. (2000). Coded semistructured interviews in social
psychological research. . In H. T. Reis & C. M.
Judd (Eds.), Handbook of research methods
in social and personality psychology (pp. 286-312). New York: Cambridge University Press. On
reserve in the library.
Two proposals
April 25—Proposal presentations
Six proposals
Mon April 30 (1:00-2:50)—Proposal presentations
Four proposals
Papers due Fri. May 4 at noon. Turn in
rough draft and peer review comments as well.
1. Considering the research design and approach,
what are the underlying assumptions concerning the nature of science and how we
know what we know?
2. What is
(are) the research question(s)?
3. What is the
research design?
4. How are the
variables operationalized or measured?
5. Are the measures
reliable and valid?
6. Is the
sample representative of the population of interest?
7. Is there
sufficient power to obtain significant results?
8. What are the
statistical analyses? Do they match the
research question?
9. What are the
results?
10. Is the
discussion of the results in line with the results?
Critique of Qualitative
Research Article
1.
What type of qualitative study is it? (ethnography,
case study, focus group, etc)
2. Considering the research design and approach,
what are the underlying assumptions concerning the nature of science and how we
know what we know?
3.
What is the purpose of the study specifically?
4. What sources of data over what period of time
were used to “hear the participants’ voices”?
5.
How do you evaluate the quality of the study?
6.
What did we learn from the study?