snairdesai / cjd_epi

Analysis of evolving epidemiological trends in Creutzfeldt-Jakob disease in the United States

Purpose and Collaboraters

The purpose of the CJD_Epi repository is to document source code, raw data, temporary files, and outputs from our analysis of recent trends in the evolving epidemiology of Creutzfeldt-Jakob Disease in the United States.

The co-authors for this project are:

• Matthew A. Crane (Institutional Affiliations: Johns Hopkins University School of Medicine & USC Schaeffer Center; Contact: [email protected]; Website: https://healthpolicy.usc.edu/author/matthew-a-crane/)
• Sameer Nair-Desai (Institutional Affiliations: Stanford University & Meridian Collective; Contact: [email protected]; Website: https://sameernairdesai.wordpress.com/)
• John A. Romley (Institutional Affiliations: USC Schaeffer Center, USC Price School of Public Policy, & USC School of Pharmacy; Website: https://priceschool.usc.edu/people/john-a-romley/)
• John C. Probasco (Institutional Affiliations: Johns Hopkins University School of Medicine; Website: https://www.hopkinsmedicine.org/profiles/details/john-probasco)
• Alison Gemmill (Institutional Affiliations: Johns Hopkins University Bloomberg School of Public Health; Website: https://publichealth.jhu.edu/faculty/3843/alison-gemmill)

Template Instructions

Note: The template from this repository has been forked from gentzkow/template. All credit to Matthew Gentzkow and his team. See the License below for further reference.

Table of Contents

• Requirements
• Setup
• Running Package Scripts in Other Languages
• Adding Packages
• Command Line Usage
• User Configuration
• Windows Differences
• License

---

Requirements

Note: The application requirements and setup instructions outlined below are intended to serve general users. To build the repository as-is, the following applications are required:

• R
• Stata
• Python
• git
• git lfs
• LyX
• A TeX distribution for your local OS (for example, MacTeX for MacOS).

You may download the latest versions of each. By default, the Setup instructions below will assume their usage. Note that some of these applications must also be invocable from the command line. See the Command Line Usage section for details on how to set this up. You must set up a personal GitHub account to clone private repositories on which you are a collaborator. For public repositories (such as template), Git will suffice. You may need to set up Homebrew if git and git-lfs are not available on your local computer.

If you are using MacOS, ensure your terminal is operating in bash rather than the default zsh. MacOS users who are running template on an M1 chip will instead want to use Rosetta as their default terminal. You can find instructions on how to shift from zsh to Rosetta here and here. WindowsOS users (with Version 10 or higher) will need to switch to bash from PowerShell. To do this, you can run bash from within a PowerShell terminal (you must have installed git first).

Once you have met these OS and application requirements, clone a team repository from GitHub and proceed to Setup.

---

Setup

1. Create a config_user.yaml file in the root directory. An example can be found in the /setup directory. If this step is skipped, the default config_user.yaml will be copied over when running check_setup.py below. You might skip this step if you do not want not to specify any external paths, or want to use default executable names. See the User Configuration section below for further details.

2. Initialize git lfs. From the root of the repository, run:

   git lfs install
   ./setup/lfs_setup.sh
   git lfs pull

This will not affect files that ship with the template. The first command will initialize git lfs for usage. The second command will instruct git lfs to handle files with extensions such as .pdf, .png, etc. The third command will download large files from the remote repository to your local computer, if any exist. See here for more on how to modify your git lfs settings.

3. If you already have conda setup on your local machine, feel free to skip this step. If not, this will install a lightweight version of conda that will not interfere with your local Python and R installations.

NOTE: If you do not wish to install conda, proceed to steps 6 - 8 (installing conda is recommended).

Install miniconda to be used to manage the R/Python virtual environment, if you have not already done this. If you have homebrew (which can be download here) miniconda can be installed as follows:

    brew install --cask miniconda

Once you have installed conda, you need to initialize conda by running the following commands and restarting your terminal:

    conda config --set auto_activate_base false
    conda init $(echo $0 | cut -d'-' -f 2)

4. Next, create a conda environment with the commands:

    conda config --set channel_priority strict
    conda env create -f setup/conda_env.yaml

The default name for the conda environment is template. This can be changed by editing the first line of /setup/conda_env.yaml. To activate the conda virtual environment, run:

    conda activate <project_name>

The conda environment should be active throughout setup, and whenever executing modules within the project in the future. You can deactivate the environment with:

conda deactivate <project_name>

5. Fetch gslab_make submodule files. We use a Git submodule to track our gslab_make dependency in the /lib/gslab_make folder. After cloning the repository, you will need to initialize and fetch files for the gslab_make submodule. One way to do this is to run the following bash commands from the root of the repository:

   git submodule init
   git submodule update

Once these commands have run to completion, the /lib/gslab_make folder should be populated with gslab_make. For users with miniconda, proceed to step 7.

6. For users who do not want to install miniconda, follow the instructions in /setup/dependencies.md to manually download all required dependencies. Ensure you download the correct versions of these packages. Proceed to step 7.

7. Run the script /setup/check_setup.py. One way to do this is to run the following bash command from the /setup directory (note that you must be in the /setup directory for the script to run successfully):

   python check_setup.py

8. To build the repository, run the following bash command from the root of repository:

   python run_all.py
   

---

Adding Packages

Note: These instructions are relevant for users who have installed miniconda. If you have not done so, consult /setup/dependencies.md.

Python

Add any required packages to /setup/conda_env.yaml. If possible add the package version number as well. If there is a package that is not available from conda, add this to the pip section of the yaml file. In order to not re-run the entire environment setup you can download these individual files from conda with the command:

conda install -c conda-forge --name <environment name> <package_name=version_number>

R

Add any required packages that are available via CRAN to /setup/conda_env.yaml. These must be prepended with r-. If there is a package that is only available from GitHub and not from CRAN, add this package to /setup/setup_r.r (after copying this script from /extensions). These individual packages can be added in the same way as Python packages above (with the r- prepend).

Stata

Install Stata dependencies using /setup/download_stata_ado.do (copy download_stata_ado.do from /extensions to /setup first). We keep all non-base Stata ado files in the lib subdirectory, so most non-base Stata ado files will be versioned. To add additional Stata dependencies, use the following bash command from the setup subdirectory:

stata-mp -e download_stata_ado.do

---

Command Line Usage

For instructions on how to set up command line usage, refer to the repo wiki.

By default, the repository assumes these executable names for the following applications:

application : executable
python      : python
git-lfs     : git-lfs
lyx         : lyx
r           : Rscript
stata       : stata-mp (will need to be updated if using a version of Stata that is not Stata-MP)

Default executable names can be updated in config_user.yaml. For further details, see the User Configuration section below.

---

User Configuration

config_user.yaml contains settings and metadata such as local paths that are specific to an individual user and should not be committed to Git. For this repository, this includes local paths to external dependencies as well as executable names for locally installed software.

Required applications may be set up for command line usage on your computer with a different executable name from the default. If so, specify the correct executable name in config_user.yaml. This configuration step is explained further in the repo wiki.

---

Running Package Scripts in Other Languages

By default, this template is set up to run Python scripts. The template is, however, capable of running scripts in other languages too (make-scripts are always in Python, but module scripts called by make-scripts can be in other languages).

The directory /extensions includes the code necessary to run the repo with R and Stata scripts. Only code that differs from the default implementation is included. For example, to run the repo using Stata scripts, the following steps need to be taken.

1. Replace /analysis/make.py with /extensions/stata/analysis/make.py and /data/make.py with /extensions/stata/data/make.py.
2. Copy contents of /extensions/stata/analysis/code to /analysis/code and contents of /extensions/stata/data/code to /data/code.
3. Copy .ado dependencies from /extensions/stata/lib/stata to /lib/stata. Included are utilities from the repo gslab_stata.
4. Copy setup script from /extensions/stata/setup to /setup.

---

Windows Differences

The instructions in template are applicable to Linux and Mac users. However, with just a few tweaks, this repo can also work on Windows.

If you are using Windows, you may need to run certain bash commands in administrator mode due to permission errors. To do so, open your terminal by right clicking and selecting Run as administrator. To set administrator mode on permanently, refer to the repo wiki.

The executable names are likely to differ on your computer if you are using Windows. Executable names for Windows generally resemble:

application : executable
python      : python
git-lfs     : git-lfs
lyx         : LyX#.# (where #.# refers to the version number)
r           : Rscript
stata       : StataMP-64 (will need to be updated if using a version of Stata that is not Stata-MP or 64-bit)

To download additional ado files on Windows, you will likely have to adjust this bash command:

stata_executable -e download_stata_ado.do

stata_executable refers to the name of your Stata executable. For example, if your Stata executable was located in C:\Program Files\Stata15\StataMP-64.exe, you would want to use the following bash command:

StataMP-64 -e download_stata_ado.do

---

License

MIT License

Copyright (c) 2019 Matthew Gentzkow

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

cjd_epi's Issues

Reconcile results with MC draft version

The purpose of this issue (#18) is to reconcile the outputs produced here with those from MC's spot-check. Specifically, we want to make the following edits to the pipeline prior to submission:

• Do not multiply the age-adjusted rates by 100 in the following lines of clean_data.R (see code below):

CJD_Epi/data/code/clean_data.R

Lines 96 to 106 in e15b7d0

	dplyr::mutate(icd_code = icd_input,
	us_time_period_pop = format(as.numeric(us_time_period_pop), big.mark = ","),
	raw_death_rate = as.numeric(raw_death_rate) * 100,
	raw_death_rate_ci_low = as.numeric(raw_death_rate_ci_low) * 100,
	raw_death_rate_ci_hi = as.numeric(raw_death_rate_ci_hi) * 100,
	raw_death_rate_se = as.numeric(raw_death_rate_se) * 100,
	age_adj_death_rate = as.numeric(age_adj_death_rate) * 100,
	age_adj_death_rate_ci_low = as.numeric(age_adj_death_rate_ci_low) * 100,
	age_adj_death_rate_ci_hi = as.numeric(age_adj_death_rate_ci_hi) * 100,
	age_adj_death_rate_se = as.numeric(age_adj_death_rate_se) * 100,
	time_period_death_share = readr::parse_number(time_period_death_share)) %>%

Specify in the figure label that this is rate per 100,000 (not per 1K), and present all sample years on x-axis.

• Revert to Alison's specified settings for Joinpoint table (email thread), rather than using the decisions made in #14 (comment). Check that doing so aligns with the results in the new draft version.

• Check that the t-tests/z-tests computed in #16 use the correct raw/age-adjusted demographic rates.

• Ensure the entire pipeline structure works as expected.

@snairdesai has been assigned.

cc @mcrane9

Initial build of Figure B

Purpose

---

The purpose of this issue (#4) is to construct an initial build of Figure B: Age-Adjusted Rate of CJD Deaths by Gender (Male and Female). This follows from #2. The relevant development branch is init_fig_b.

@snairdesai has been assigned to complete.

Updates t-test differences with revised data pulls

The purpose of this issue follows in spirit from #16. We will update our t-test and z-score constructions using revised data. @snairdesai has been assigned to complete.

Revision to build of Table 1

Purpose

---

Follows #7. The purpose of this issue (#14) is to revise our initial build of Table 1: Joinpoint Analysis of CJD Mortality Trends by Gender and Race/Ethnicity. Specifically, we want to change our estimation strategy away from homoskedastic errors in order to present 95% CIs for our results. The relevant development branch is revise_tab1.

@snairdesai has been assigned to complete.

Initial build of main text figures and tables

Purpose

---

The purpose of this issue (#2) is to track and delegate the construction of our proposed main text figures. Currently (as of 12/30/2022), these figures/tables are as follows:

• #3: Figure A: Raw Number of CJD Deaths by Gender (Male and Female)
• #4: Figure B: Age-Adjusted Rate of CJD Deaths by Gender (Male and Female)
• #5: Figure C: Age-Specific Rate of CJD Deaths for Males
• #6: Figure D: Age-Specific Rate of CJD Deaths for Females
• #7: Table 1: Joinpoint Analysis of CJD Mortality Trends by Gender and Race/Ethnicity

We will be opening separate issues for each of these figures/tables, with @snairdesai assigned to each. The goal here is not to build publication-ready figures/tables, but rather get a baseline sense of the raw data and explore other interesting trends.

Initial write-up of methods in draft

The purpose of this issue (#8) is to complete an initial write-up of the Methods section for our research letter. These edits will be made directly in the Word Document posted, but I will begin by posting proposed text directly in this thread for review by @mcrane9.

@snairdesai has been assigned to complete this item.

Initial build of Figure D

Purpose

---

The purpose of this issue (#6) is to construct an initial build of Figure D: Age-Specific Rate of CJD Deaths for Females. This follows from #2. The relevant development branch is init_fig_d.

@snairdesai has been assigned to complete.

Initial build of Figure C

Purpose

---

The purpose of this issue (#5) is to construct an initial build of Figure C: Age-Specific Rate of CJD Deaths for Males. This follows from #2. The relevant development branch is init_fig_c.

@snairdesai has been assigned to complete.

Initializing proper template format

The purpose of this initial issue is to reformat the gentzkow/template format to align with our project needs. We are using this format to aid with project reproducibility. More information on the benefits of this approach can be found in the repository's README.md.

@snairdesai has been assigned to complete.

Constructing t-test/z-test differences in means (within group, pre- vs. post-period)

The purpose of this issue (#16) is to construct t-tests and z-tests to determine significant differences in mortality rates between the start of the sample period (2007) and the end of the sample period (2020), within each subgroup of interest (i.e., 2007 vs. 2020 age-adjusted + crude mortality for females, 2007 vs. 2020 age-adjusted + crude mortality for males, 2007 vs. 2020 crude mortality for 55 - 64 years old; 2007 vs. 2020 crude mortality for 65 - 74 years old; and 2007 vs. 2020 crude mortality for 75+ years old).

I will also double-check the p-values outputted from Joinpoint to ensure they make intuitive sense to the reader.

Initial build of Figure A

Purpose

---

The purpose of this issue (#3) is to construct an initial build of Figure A: Raw Number of CJD Deaths by Gender (Male and Female). This follows from #2. The relevant development branch is init_fig_a.

@snairdesai has been assigned to complete.

Initial build of Table 1

Purpose

---

The purpose of this issue (#7) is to construct an initial build of Table 1: Joinpoint Analysis of CJD Mortality Trends by Gender and Race/Ethnicity. This follows from #2. The relevant development branch is init_tab_1.

@snairdesai has been assigned to complete.