Assignment 4: Reproducibility with OSF and Git

Objectives

By the end of this assignment, you should:

understand how to reproduce analyses from a repository on OSF
understand how create a Git repo on Github
understand how to use the git commands add and commit to commit changes
understand how to use the git commands push and pull to make and receive changes

This assignment is due Thursday, October 7th at noon. Please turn your .html AND .Rmd files into Canvas. Your .Rmd file should knit without an error before turning in the assignment. Exercises 1-2 involve writing R code. The remaining exercises involve Git/Github. The exercises walk you through various steps for working with a repository. You only need to include something in the assignment you turn in when prompted (e.g., when it says something like, What does git print out?).

Load packages:

library(tidyverse)
library(knitr)
library(janitor)

In the first few exercises, we’ll try to reproduce some analyses from a published paper using the data from the OSF repo. In particular, we’ll try to reproduce the main analyses from Experiment 1A of Zettersten & Lupyan (2020).

[a] What is the key question in this paper? How do they test it?
[b] The link to the OSF repo for this paper is provided in the Results section of Experiment 1A. Go to the repo. Find the relevant data file. In this case, the data for the main experiments in the paper are located in a file called CRN_data.txt
[c] Import CRN_data.txt from OSF into RStudio. There are two ways to do this. One way is to import it directly from the web. To do this, go to the CRN_data.txt page on OSF, right click (Control-click) on the “Download” button and select “copy link address”. The path to the file is the copied link address. Use this address to read the data into R (similar to how we’ve done in the previous assignments). Alternatively, you can download the file to your local computer and read the data into RStudio by providing the path to the file on your local computer. Save the data to a data frame called zl_data.
[d] Use glimpse to look at the data. How many variables are there? To find out what the variables mean, you can look at the codebook provided.
[e] Let’s clean up the data and save it to a new data frame (zl_clean). Use the clean_names function from the janitor package to clean the variable names. Then, select only the variables that we’re going to need (experiment, subject, age, condition, block_num, is_right).
[f] Create a new data frame (zl_exp1a) that only includes data from the experiment we need (Experiment 1A).

[a] Key question: How do linguistic labels influence category learning? Specifically, does nameability improve category learning?
How they tested this: Zetterson and Lupyan (2020) tested this by manipulating the nameability (low nameability and high nameability conditions) of the category features in a task of learning novel categories.
[d] There are 77 variables.

DATA_PATH<- "https://osf.io/a4dzb/download"
zl_data <- read_csv(DATA_PATH)

zl_clean <- zl_data %>%
  clean_names() %>%
  select(experiment, subject, age, condition, block_num, is_right)

zl_exp1a <- zl_clean %>%
  filter(experiment == "1A")

Now, we’re ready to try to reproduce some of the numbers reported in the paper for Zettersten & Lupyan (2020) Experiment 1A, using the tidyverse functions that we’ve learned.

[a] How many participants were there in Experiment 1A? Does this match what is reported in the paper?
[b] What was the mean age of participants in Experiment 1A? Does this match what is reported in the paper?
[c] What was the overall accuracy of participants in the High Nameability condition and the Low Nameability condition? Does this match what is reported in the paper? (Note: Like in Assignment 3, you can calculate proportions with the summary function, sum(is_right)/n())
[d] Recreate Figure 4a in the paper as closely as possible using ggplot. You can ignore the error bars on the points and the different line type (dashed vs. solid).

[a] There were 50 participants.

distinct(zl_exp1a, subject) %>%
  nrow()

## [1] 50

[b] The mean age was 32.02 years.

distinct(zl_exp1a, subject, .keep_all = T) %>%
  summarize(mean_age = mean(age))

## # A tibble: 1 × 1
##   mean_age
##      <dbl>
## 1     32.0

[c] The accuracy was 0.84 in the high condition, and 0.677 in the low condition.

ms_by_overall <- zl_exp1a %>%
  group_by(condition) %>%
  summarize(prop_right = sum(is_right)/n())

ms_by_overall

## # A tibble: 2 × 2
##   condition prop_right
##   <chr>          <dbl>
## 1 high           0.84 
## 2 low            0.677

[d]

ms_by_condition <- zl_exp1a %>%
  group_by(condition, block_num) %>%
  summarize(prop_right = sum(is_right)/n())

## `summarise()` has grouped output by 'condition'. You can override using the `.groups` argument.

ggplot(ms_by_condition, aes(x = block_num, y = prop_right, color = condition)) +
  geom_point() +
  ylim(.5,1) +
  scale_color_manual(values = c('red', "blue")) +
  geom_line() +
  geom_hline(yintercept = 0.50, color="gray")+
  scale_x_continuous(breaks=c(1,2,3)) +
  xlab("Block Number") +
  ylab("Accuracy") +
  theme_classic(base_size = 12)

The next exercises concern creating and cloning a new Github repo. The slides from lab describe the steps for completing these exercises.

Let’s start by creating a new repository (“repo”) on Github.

[a] Navigate to Github, and create an account if you do not already have one.
[b] Create a new Public Github Repo called “MRM-Assignment-04”, initialized with a README.

Look at the contents of your repository (in the “Code” tab). You should see that the only thing it contains is a single read me file (“README.md”). Right now the only place this repo exists is in the “cloud” (specifically, on the Github website).

Next, we’re going to make a copy of our repository so that it also exists locally as a project inside RStudio Cloud. To use git locally, you need to have the software downloaded on your local computer. I recommend skipping this step, and instead using a version of Rstudio that lives on the cloud, called RStudio Cloud, that has git pre-installed on it. To get started in RStudio Cloud, go to the website and click “Get started for free”. Signup for a free account. Once you’re logged in go to “New Project” -> “New Rstudio Project”. The RStudio interface will load, and you’re good to go! To use git in RStudio, you use Terminal. Terminal is an interface for interacting with the system shell, and you can find it next to the “Console” tab in RStudio.

[a] Use git clone <URL> to create a local copy of your repo (where <URL> is the URL of the repo you want to clone, which typically looks like: https://github.com/<USERNAME>/<REPO_NAME> (where <USERNAME> is the username of your Github account).
[b] Look in the files pane of RStudio Cloud. You should see a file with the name of your repository and your read me file inside. Change your working directory in Terminal to be the repo using cd <REPO NAME>. This is important.

Next, set up your Git environment (you only have to do this once per computer you’re using). Run the following lines in Terminal:
[a] git config --global user.email "<EMAIL>" (where <EMAIL> is the email you used to create your Github account; be sure to include the quotes around your email).
[b] git config --global user.name "<NAME>" (where <NAME> is your name).

In the last few exercises, we’ll make a basic change to the repo, commit the change, and push it to Github.

[a] Modify the file README.md by adding a description of the repo. In this case, describe the repo by copying and pasting the assignment objectives (the text in yellow box at the top of this document) into the read me file. To do this, open README.md from Files pane on the right, and edit the text. Save your changes.
[b] In the Terminal window, type git status to check that Git has detected the change (make sure your current working directory is the repository). What does git print out?
[c] Use git add README.md to add the file README.md to the current change.
[d] Use git status again to check that the change is ready to be commited. What does git print out now?

[b]

On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

[d]

On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   README.md

Now, we’re ready to commit the changes in our repository.
[a] Use git commit -m '<COMMIT_MSG>' (where <COMMIT_MSG> is a short message describing the change you made, e.g. “update readme”).
[b] Use git status again to check that the change is committed. What does git print out now?

[b]

On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)
nothing to commit, working directory clean

Let’s add something else to our rep.
[a] Find a picture of an animal on the internet that starts with the first letter of your favorite tidyverse function (e.g., M utate -> monkey , F ilter -> flamingo, etc.). Download the picture (in any format, e.g. .jpg, .png, .pdf). Rename the picture to include the function name, animal name, and your first name. (e.g. “filter_flamingo_molly.pdf”)
[b] Upload this picture to your project in RStudio cloud.
[c] Use git add <PICTURE FILENAME> to add your animal file to the current change.
[d] Use git status again to check that the change is ready to be committed. What does git print out now?

[d]

On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        new file:   arrange_axolotl_jaeah.jpg

Let’s commit again and add a message that describes the changes we’ve made since the last time we committed.
[a] Use git commit -m '<COMMIT_MSG>' (where <COMMIT_MSG> is a short message describing the change you made).
[b] Use git status again to check that the change is committed. What does git print out now?

[b]

On branch master
Your branch is ahead of 'origin/master' by 2 commits.
  (use "git push" to publish your local commits)
nothing to commit, working directory clean

Now, we’re ready to push the changes in our repository to Github.
[a] In order to be able to push, you need to give yourself credentials, called a personalized access token. To create the token, follow the steps, here. [b] Use git push to push the change to Github. You will need to enter your Github username and password. The password is the personalized access token you just created, i.e., a long string of numbers and letters (not your Github password you create in Exercise 3). [c] Use git status again to check that you’re in sync with Github. What does git print out? [d] Navigate to the project on Github and check that your changes were successfully pushed. What is the link to your repo on Github?

[c]

On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean

[d] The git repository link: https://github.com/jaeahk/MRM-Assignment-04.

Assignment 4: Reproducibility with OSF and Git - SOLUTIONS

Modern Research Methods