Assignment 8: Coding papers for your meta-analysis

Modern Research Methods

Objectives

By the end of this assignment, you should:

• know how to code primary sources for inclusion in a meta-analysis.

This assignment is due Thursday, November 18 at noon.

Like Assignment 7, you do not need to “turn in” anything on Canvas as part of this assignment. Your exercises for this assignment are to complete the MA_data spreadsheet for your group final project. You will only produce one spreadsheet per group.

1. Go through the papers you’ve already selected for inclusion on the MA_data spreadsheet and enter data for calculating an effect size. Specifically, for each paper, complete the following six steps:
   
   1-Download the paper in pdf form.
   2-Put it in the folder titled MA_papers
   3-Relabel the paper with the unique_id
   4-Skim the paper to determine whether it satisfies your inclusion criteria. If it does not, note this in the eligibility_decision column, and you are done. Start at Step 1 with a new paper.
   5-Find the relevant data in the paper for coding the effect size, and highlight it using your pdf reader.
   6-Enter the data for that paper in your MA data spreadsheet. The codebook for the MA_data spreadsheet is provided below. Remember, each effect size is one row, but each paper may have multiple effect sizes.
   
   Coding all of the papers you have identified in your literature search is necessary to complete your meta-analysis, but that is likely beyond the scope of this class project. Therefore, each student must enter data for 5 papers This will mean looking at many more than 5 papers . You should keep repeating the six steps above until you end up with 5 papers that satisfy your inclusion criteria and have enough data to calculate an effect size (if this is a problem for your MA, please chat with me).
   
   Your final MA_data spreadsheet should have enough information to calculate an effect size for each of the effect sizes in each of your 5 papers. Some papers may have just 1 effect size (rows), others may have many. Therefore you will likely enter many more than 5 effect sizes. If you’re not sure what this means, please talk with me.
   
   

Codebook for “MA data” spreadsheet:
Blue columns

• coder - your name (this same person does not need to code the effect size data as who entered the paper in the “relevant_studies” spreadsheet)
• unique_id - lastnameYear (same as “relevant_studies” spreadsheet)
• long_cite - APA citation (same as “relevant_studies” spreadsheet)
• paper_eligibility - After looking at the full text of the paper, does it satisfy your inclusion criteria? (“include”, “exclude”, “?”)
• exclusion_reason - If you marked “exclude” for paper_eligibility, explain why. This is where you should note if you can’t get access to the paper.
• short_cite - This is a shorter version of “long_cite” and will be used for plotting later on. It’s everything up to the year in the long_cite column (e.g. “Schug, M. G., Shusterman, A., Barth, H., & Patalano, A. L. (2016)”)
• source_of_data - Where did you get the data for the red columns from in the paper?
• expt_num - What experiment in the paper does this effect size come from? If the paper only has one experiment put “1”.
• expt_condition - Some papers will have multiple conditions in the same experiment that satisfy your inclusion criteria. Add a descriptive label here that will help us identify the condition later on. This value will likely be specific to the particular paper. (e.g. “betting”).
• dependent_measure - If your criteria specifies multiple dependent measures, indicate which dependent measure this effect sizes is for (e.g., “liking” vs. “betting”). Note you make have multiple effect sizes for the same experiment for different measures. These should go on separate lines because they are separate effect sizes.

Red columns

• n_1 - number of participants, or sample size. This should be the number of people who actually participated in the experiment (after exclusions).
• x_1 - mean of dependent measure in group 1
• x_2 - mean of dependent measure in group 2
  
• SD_1  SD_2- standard deviations for x_1 and x_2 above.
• t - t-value comparing x_1 and x_2
• d - d value (effect size)

Green columns

• The remaining green values are specific to your MA, but make sure you standardize the coding values across group members

1. Extra credit: You will earn extra credit for each paper coded over and above the minimum of 5 papers.