RESEARCH
DESIGN
Spring 2013
|
Class Information |
Instructor Information |
|
PSYCH 6002 |
Helen C. Harton, Ph.D. |
|
Baker 315 |
Baker 357 |
|
W 12-2:50pm |
273-2235; harton@uni.edu |
Office Hours: WF 11-11:50; W 2-3; pretty much
any time I’m around
Readings:
Trochim, W., & Donnelly, J. P. (2007). The research
methods knowledge base (3rd ed.). Cincinnati, OH:
Atomic Dog Publishing.
Kuhn, T. S. (1996). The structure
of scientific revolutions (3rd
ed). Chicago: University
of Chicago Press.
Course Description and Objectives: Research is at the heart of psychology; psychology’s
focus on research and empirical evidence is one of the things that differentiates the field from other similar disciplines. In
this course, you’ll learn about the basics of research, explore several “hot
topics” or new methods in psychology, and become a better designer, describer,
and critical reader of research. The course is designed to give you a breadth
of knowledge on research design in psychology and to facilitate your future
research career and thesis progress.
By the end of this course, you
should
1) know
the advantages and disadvantages of the commonly used methods in psychology and
when to best apply them;
2) know the advantages and
disadvantages of several new methodologies in psychology;
3) be able to critique and
integrate previous research;
4) be
more aware of ethical concerns in research and how to deal with them;
5) be a
better scientific writer;
6) be able to design and carry
out your own research project; and
7) have
made significant progress on your thesis.
Course Format: Each week you will have several readings that you should have processed
and thought about before class. In class, we will discuss the readings and the
issues they bring up. You should contribute thoughtfully to the discussion and
build on and gently challenge the comments of other students.
Course Requirements:
Class discussion (20%). In graduate school especially, you learn not only from books and professors, but from interactions and discussions with peers. Discussing information also helps you to think about it more deeply and learn it more quickly. You are expected to contribute meaningfully to class discussions. While mere attendance is not enough to get a good grade for this component, it is imperative in that you can’t participate if you’re not here. Obviously, it also requires that you read and think about the readings. Both frequency and quality count. You will get graded on discussion each week; you can drop your lowest score.
Thought papers (15%). Each week you’ll also be asked to email me a short (1-2 pages) thought paper addressing the readings for the week. These emails should have “thought paper” in the subject line and be sent by midnight Monday nights (just paste it in the message instead of attaching it). The thought paper should address all the readings for the week at some level. Think about how these readings relate to your thesis; bring up questions you have; relate the readings to other things you know. You want to show me that you’ve read, understood, and carefully processed the readings for the week. In addition, some weeks there will be a specific assignment (e.g., find a meta-analysis on a topic). The purpose of these assignments is to get you thinking more deeply about the method and its application to your research and to facilitate in-class discussion. You can drop your lowest thought paper grade.
Discussion and thought papers will be graded on the following scale:
0 = not there
2 = attended but didn’t participate, or turned in, but not very relevant (below average)
3 = comments or questions relevant, but didn’t involve much insight (average)
4 = comments or questions relevant and insightful (good)
5 = more than one comment or question showed a significant contribution (excellent)
Research proposal (45%). One of the best ways to show that you understand research design is by applying it. You will submit a complete research proposal (7-8 page introduction, complete method and appendices, section on ethics and validity, plan of analysis, discussion of expected results/what it would mean if you didn’t find them, in APA style). This paper doesn’t have to be your thesis proposal, but if you can write about something that does develop into it (or another research project), all the better. There will be assignments due during the semester to provide you with feedback on this paper (e.g., outlines, rough drafts) that will count toward the “assignment” portion of your grade. Obviously, this should not be a paper that you have written or are writing for another class. If there is any question about whether it might be too collaborative or too close to something you’ve already done, talk to me before you get started to see if it is acceptable. The paper should be in good APA style. Those that aren’t, will be returned ungraded and lose at least 1 letter grade (1 if turned in that same day, -1 more for each day after).
Proposal presentation (10%). During one of the last class sessions, you will present your proposal to the class (background, method, expected results, what they would mean, etc.). Your presentation, which should include some audio-visual effects (e.g., PowerPoint), should last no more than 12 minutes, followed by a discussion of the proposal by the class (we will play the role of thesis committee). You can (and should) integrate any helpful comments from the class into your proposal before you turn it in. You’ll be graded on presentation style as well as knowledge about the area. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdHQwbUYySDVDeW40V0pmMWxKb1BFWmc to sign up for a time. NOTE: You have to do this from your uni google account.
Assignments (10%). In addition to the weekly thought papers, you will turn in several assignments directly related to your research proposal (i.e., topic, articles, outlines, rough draft) and provide feedback on a peer’s draft. You may sometimes be asked to read short articles before class that we will critique in class. You will also present on a particular method or “hot topic.” For your topic presentation, you should read at least one article or chapter on the method itself as well as at least two articles that used the method (if I listed an article with the presentation topic in the class schedule, that should be one of the ones you read). In the presentation, you’ll tell us about what the method is, what its advantages and disadvantages are, when it should/shouldn’t be used, how the articles you read used it and whether that was a good use or not, and depending on the topic, either best practices in using the method and/or how (in general) you would analyze data collected by the method. Sometimes there are additional questions you should address that I put on the class schedule. You’re giving a brief overview so that everyone learns the issues involved, but you don’t need to describe it in enough detail that we could actually go out and do it. Presentations should be 10-15 minutes and include your references in your final slide. Send me a copy of your Powerpoints. Go to https://docs.google.com/a/uni.edu/spreadsheet/ccc?key=0AvGm2QdOV41CdHp6WDI2QkltMGl1LWdSTXQ4cEx1THc to sign up for a topic and time. NOTE: You have to do this from your UNI google account.
Makeup and Late Paper Policies: Class discussion grades and presentations cannot be
made up. Proposals will be accepted up to three days past the due date, but one
letter grade will be deducted for each day until they are turned in. Plan ahead
and don’t wait until the last minute to finish (or start) the paper, in case
something unexpected arises.
Academic Honesty Policy: Cheating and plagiarism of any kind will not be
tolerated and will result in a 0 on the assignment in question. This includes
using a paper from another class or that you have worked on with another
faculty member to fulfill a requirement in this class as well as using quotes
from materials without attribution even in short assignments or thought papers.
For more information on UNI’s academic
honesty policies, see the University Catalog as well as the information in the
Department of Psychology Graduate Student Handbook. If you have any questions
about what is acceptable, ask.
Disability Services: The Americans with Disabilities Act of 1990 (ADA)
provides protection from illegal discrimination for qualified individuals with
disabilities. Students requesting instructional accommodations due to
disabilities must arrange for such accommodation through the Office of
Disability Services. The ODS is located at 213 Student Services Center, and the
phone number is 273-2676.
CLASS SCHEDULE
Jan. 16—Foundations and theory
T&D, Chapter 1
Kerr, N. L.
(1998). HARKing: Hypothesizing after the results are
known. Personality and Social Psychology
Review, 2, 196-217. doi:10.1207/s15327957pspr0203_4
Simmons, J. P., Nelson, L. D., & Simonsohn,
U. (2011). False-positive
psychology: Undisclosed flexibility in data collection and analysis allows
presenting anything as significant. Psychological
Science, 22, 1359-1366. doi:10.1177/0956797611417632
What is
the theoretical basis for your thesis? What is good and bad about theories?
What examples have you seen of HARKing? Is it really
a bad thing? What do you think of the suggestions of Kerr and of Simmons et
al.? Are they good? Are they practical? What should we do to avoid misleading
reports of research in psychology?
In class: Panel of 2nd year students
on thesis; video on syllabus; then do on-line discussion in elearning.
You need to participate in each of the forums at least once (more is better).
No thought paper this week.
Jan. 23---Philosophy of science
Kuhn book
What is
Kuhn’s view of science? Do you agree? How does science change? How should it
change? What is and should be the goal of science? What are examples of
“revolutions” in psychology?
Turn in
topic description for your thesis (research proposal for class); put it in my
mailbox by Friday at noon.
In class: Mary Losch will
lead discussion. Send your thought papers this week to both of us.
Hertwig, R., & Ortmann, A.
(2008). Deception
in experiments: Revisiting the arguments in its defense. Ethics & Behavior, 18, 59-92. doi:10.1080/10508420701712990
Becker-Blease, K. A., & Freyd, J. J. (2006). Research
participants telling the truth about their lives: The ethics of asking and
not asking about abuse. American
Psychologist, 61, 218-226. doi:10.1037/0003-066X.61.3.218
Barchard, K. A., Williams, J. (2008). Practical advice for conducting
ethical online experiments and questionnaires for United States psychologists.
Behavior Research Methods, 40,
1111-1128. doi:10.3758/BRM.40.4.1111
Are there
any ethical issues that would need to be addressed with your thesis? How could
you deal with them? Is deception okay? When? How should people be debriefed?
Are our current techniques adequate? Should “vulnerable populations” receive
more protection? How or why? Should we ask about things like abuse?
Debriefing
presentation—Is
debriefing effective? When or why? Briefly review research on debriefing and
give us some “best practice” suggestions.
T&D Chapter 2
Henrich, J. Heine, S. J., &
Norenzayan, A. (2010). The weirdest people in
the world? Behavioral and Brain
Sciences, 33, 61-83. doi:10.1017/S0140525X0999152X
Maxwell, S.
E., Kelley, K., Rausch, J. R. (2008). Sample size planning for statistical
power and accuracy in parameter estimation. Annual
Review of Psychology, 59, 537-563. doi: 10.1146/annurev.psych.59.103006093735
Is
sampling an issue for your thesis topic? What type of sampling will you use? Should you use? What populations have been used in your
area? Is that a problem or not and why? If there are WEIRD samples used, how is
this likely to interact with your thesis topic area? When are WEIRD samples
more or less of a problem? How large should your sample be for your thesis? How
will you determine this?
Find
10 articles for thesis (research proposal), submit in APA style. If they are
not in near-perfect APA style, they will be returned ungraded, and you will get
a 0 on the assignment.
Cross-cultural
research presentation—When is cross-cultural research most useful? What
particular things do researchers need to keep in mind? What are best practices?
Feb. 13 --Reliability and validity
and scaling
T&D Chapters 3 and 5
Find an
article that describe the development of a scale related to your thesis topic
(should include multiple studies, assessing reliability and validity). Evaluate
the article and scale (and include a brief description of the scale, including
the APA style citation). How did they show the various types of reliability and
validity? Do you agree? What type of scaling is the measure? What type of
scale? Are there similar concepts that still need a scale developed? How would
you go about creating a scale? What would you correlate it with to test
reliability and validity?
Outline
of intro for proposal
Method
variance presentation--What is the problem of method bias? How can it be
decreased? Find at least 2 articles that dealt with this issue either well or
poorly.
Podsakoff, P. M., MacKenzie, S. B.,
& Podsakoff, N. P. (2012). Sources of method bias in social
science research and recommendations on how to control it. Annual Review of Psychology, 63, 539-569.
doi:10.1146/annurev-psych-120710-100452
T&D
Chapter 4
Tourangeau, R. (2004). Survey research and societal change. Annual Review of Psychology, 55, 775-801. doi:10.1146/annurev.psych.55.090902.142040
Graham, J.
W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60,
549-576. doi:10.1146/annurev.psych.58.110405.085530
Find examples
of “bad” survey questions. Think about any issues that you might need to deal
with in terms of how you ask questions on your thesis. What types of issues
(e.g., population, sampling) might come up and how would those affect what
method you choose? What are the advantages and disadvantages of survey
methodology for your topic?
Diary
presentation—Include information on
the variety of diary methods and how they are analyzed (sometimes called
“event-sampling” as well)
Bolger, N., Davis, A., & Rafaeli, E.
(2003). Diary methods: Capturing
life as it is lived. Annual Review of
Psychology, 54, 579-616. doi:10.1146/annurev.psych.54.101601.145030
T&D Chapters 7 and 9
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology,
51, 1173-1182. doi:10.1037/0022-3514.51.6.1173
MacKinnon,
D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation
analysis. Annual Review of
Psychology, 58, 593-614. doi:10.1146/annurev.psych.58.110405.085542
Which threats
to internal validity are most likely to be a problem in your thesis? How can
you diminish them? What potential mediators and moderators might be involved in
your thesis topic area? How can we differentiate between mediators and
moderators? Should you include covariates in your thesis? Why or why not?
Outline of
thesis proposal
Placebo
presentation--What are best practices
in placebo research?
Price, D. D., Finniss, D. G., &
Benedetti, F. (2008). A
comprehensive review of the placebo effect: Recent advances and current
thought. Annual Review of Psychology, 59,
565-590. doi:10.1146/annurev.psych.59.113006.095941
T&D Chapters 10 and 11
Shadish, W. R., & Cook, T. D. (2009). The Renaissance of field
experimentation in evaluating interventions. Annual Review of Psychology, 60, 607-629. doi:10.1146/annurev.psych.60.110707.163544
Is your
thesis a quasi-experiment? If so, what kind? What threats to internal validity
might be a problem? Are the elements of “good design” present in your thesis?
Explain. What are the advantages and disadvantages of field research? How could
it be (or has been) used in the area of your thesis topic?
March 13—Evaluation research and
meta-analysis
T&D Chapter 16
Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative
methods for literature reviews. Annual
Review of Psychology, 52, 59-82. doi:10.1146/annurev.psych.52.1.59
Field, A. P., & Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and
Statistical Psychology, 63, 665-694. doi:10.1348/000711010X502733
What types of
evaluation are relevant for your thesis topic? Why? Which is more
important: clinical, practical, or
statistical significance? Have there been meta-analyses on your thesis topic?
If not, is the area ready for one? What key words would you use? What mediators
or moderators would you test? Find and evaluate a meta-analysis that interests
you/is related to your topic (include citation and brief description of
method/findings). What is good and bad about what they did? Address things like
inclusion criteria as well as fixed vs. random effects, etc.
March
20—Spring break
March 27—Quantitative analysis
T&D Chapters 12 and 14
Kashy, D. A., & Kenny,
D. A. (2000). The analysis of data from dyads and groups.
In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology.
New York: Cambridge University Press. On reserve in the
library (2 hour hold).
How will you
analyze your data for your thesis? What tests will be involved? What
assumptions do those tests have? Make sure you include enough description of
your method that I can understand whether these analyses are appropriate. How
could some of the analyses that you were less familiar with be useful in
studying your topic? Find a quantitative article related to your thesis
(include citation) and answer the quantitative critique questions below.
April 3--Writing it up, peer review
T&D Chapter 15
Strunk, W., Jr. (1999). The elements
of style. New York: Bartleby.com
Bem, D. J. (2002). Writing
the empirical journal article. In J. M. Darley, M. P.
Zanna, & H. L. Roediger III (Eds.),
(2002). The compleat
academic: A career guide. Washington, DC: American Psychological
Association.
Sternberg,
R. J. (1993). How to
win acceptances by psychology journals: 21 tips for better writing. APA
Observer.
Bring in
rough draft of thesis proposal for peer review and comment. No thought paper
this week!
April 10--Other methods: Implicit
measures and physiological measures
Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition research: Their
meaning and use. Annual Review of
Psychology, 54, 297-327. doi:10.1146/annurev.psych.54.101601.145255
De Houwer, J., Teige-Mocigemba,
S., Spruyt, A., & Moors, A. (2009). Implicit measures: A normative analysis and review. Psychological Bulletin, 135, 347-368. doi:10.1037/a0014211
Sowden, P., & Barrett, P. (2006). Psychophysiological methods.
In G. M. Breakwell, S. Hammond, C. Fife-Schaw, & J. A. Smith (Eds.), Research methods in psychology (3rd ed.)
(pp. 146-159). Thousand Oaks, CA: Sage. Available in resource
room and possibly on googlebooks.
What are the
advantages and disadvantages of implicit measures? What are they really
measuring? When should they be used? What do physiological measures tell us and
what can they not tell us? How could either of these types of measures be used
in research in your thesis area? What are questions or comments do you have
about these readings?
fMRI
presentation—What is fMRI? What does
it tell us? How can it be used in psychology?
Vul, E., Harris, C., Winkielman,
P., & Pashler, H. (2009). Puzzingly high correlations in fMRI studies of emotion, personality, and
social cognition. Perspectives on
Psychological Science, 4, 274-290. doi:10.1111/j.1745-6924.2009.01125.x
ERP/EKG presentation—What are
ERP and EKGs? What do they tell us? How can they be used in psychology?
Eye tracking presentation—What is eye tracking? How can it be used in psychology?
Duchowski, A. T. (2002). A breadth-first survey of eye tracking applications. Behavior Research Methods, Instruments, and
Computers, 34, 455-470. doi:10.3758/BF03195475
Hormone
sampling presentation—What hormones have typically been tracked in psychology?
How are these collected? What do they tell us? What are best practices in
collecting them (e.g., times of day, people for whom the results aren’t
accurate)?
April 17—Other
methods: Spatial analyses and internet-based research
Goodchild, M. F., Anselin, L., Appelbaum, R, P., & Herr Harthorn,
B. (2000). Toward spatially
integrated social science. International
Regional Science Review, 23, 139-159.
doi:10.1177/016001760002300201
Skitka, L. J., & Sargis, E. G.
(2006). The
internet as psychological laboratory. Annual Review of Psychology, 57, 529-555. doi:10.1146/annurev.psych.57.102904.190048
Lewis, K., Kaufman, J., Gonzalez, M., Wimmer,
A., & Christakis, N. (2008).
Tastes, ties, and time: A new social network dataset using Facebook.com. Social Networks, 30, 330-342. doi:10.1016/j.socnet.2008.07.002
How does
space affect your thesis topic, if at all? Are there potential spatial effects
either that may be interesting in their own right or that should be controlled
for? Research using the internet brings exciting advantages, but also
significant disadvantages (e.g., ethical concerns). What do you think about
research on the internet—are there certain methods or populations or topics
that shouldn’t be studied there? What other questions or comments do you have
about these methods?
Computer
simulation presentation—How can computer
simulations add to our knowledge of psychology and human behavior? When are
they best used? What advantages and disadvantages do they offer?
mTurk
presentation—What is mTurk? When is it more/less useful?
Buhrmester, M., Kwang, T., &
Gosling, S. D. (2011). Amazon’s
mechanical turk: A new
course of inexpensive, yet high-quality data? Perspectives on Psychological Science, 6, 3-5. doi:10.1177/1745691610393980
Buhrmester, M. (2010).
Amazon
mechanical turk guide for
social scientists.
Smart phone
presentation—What types of applications can smartphones be used for?
Try one out if possible (e.g., iEar). What are its
advantages and challenges and how can the challenges be addressed (think ethics
as well)?
Raento, J., Oulasvirta, A., &
Eagle, N. (2009). Smartphones: An
emerging tool for social scientists. Sociological
Methods and Research, 37, 426-454. doi:10.11177/0049124108330005
Virtual
reality presentation—How has virtual reality been used in psychology? What
types of virtual reality have been used? What are the advantages and
disadvantages of these approaches? (Jim Blascovich is
a good researcher to start with)
April 24--Qualitative research
T&D Chapter 6 and 8 and 13
Smith, C. P.
(2000). Content analysis and narrative analysis. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp.
313-335). New York: Cambridge
University Press. On reserve in the library.
What are the
advantages and disadvantages of qualitative research? What new or useful
information could qualitative studies provide on your thesis topic? Find and
critique an article using qualitative methods that is related to your thesis
topic. Use the question for the qualitative critique below.
Semi-structured
interviews presentation—What are the advantages/disadvantages of this method? What
are some best practices? How can the data be analyzed?
Bartholomew, K., Henderson, A. J. Z., & Marcia, J. E. (2000). Coded semistructured
interviews in social psychological research. . In H.
T. Reis & C. M. Judd (Eds.), Handbook
of research methods in social and personality psychology (pp. 286-312). New York: Cambridge University
Press. On reserve in the library.
Two
proposals
MONDAY,
April 29—Proposal presentations
Six
proposals
Monday,
May 6 (1:00-2:50)—Proposal presentations
Three
proposals
Papers
due Thursday, May 9 at 5pm. Turn in rough draft and
peer review comments as well.
1. What is the purpose of the article? How does
it fit into previous literature?
2. What is
(are) the research question(s)?
3. What is the
research design (be specific)?
4. How are the
variables operationalized or measured?
5. Are the
measures reliable and valid? What evidence do you have of that?
6. Is the
sample representative of the population of interest? How much does that matter
for this study?
7. Is there sufficient
power to obtain significant results?
8. What types
of statistical analyses were done? Do
they match the research questions/hypotheses?
9. What are the
results?
10. Is the
discussion of the results in line with the results?
Critique of Qualitative
Research Article
1.
What type of qualitative study is it? (ethnography,
case study, focus group, etc)
2.
What is the purpose of the study specifically?
3. How does the study build on previous
literature?
4. What sources of data over what period of time
were used to “hear the participants’ voices”?
5.
What does this study tell you that a quantitative one would not be able
to?
6.
What did we learn from the study? What were the results?