rebeccajohnson88 / rights_towhat_chapter

Code related to processing DC DPH hearings

IEP filings and disiputes

Code related to processing (1) filings from IEP disputes in DC, Texas, and Ohio, (2) decisions from filings that go to a hearing.

rights_towhat_chapter's Issues

To do

Creating the data:

• Merge in demographics for matched schools
• Manually match non-match
• Do aggregation for multiple-campus PCSs
• Do CRDC data for ieps/schools not in main sample

Descriptive analyses:

• Get code for shape files by ward/school
• Plot the rate of complaints and presence of complaints against ward demographics

Loose ends:

• Rerun with non-extracted rows of the pdf to pull cols

Summarizing DC's complaint rate compared to other jurisdictions

@rgarg21 I ended up moving stuff to this repo to keep things more organized!

• File: https://github.com/rebeccajohnson88/rights_towhat_chapter/blob/master/code/06_statelevel_filingtrends.Rmd
• Input files: the count of complaints data and count of disability students (IEPs) data is read in from a link with fread; the other data is in the raw_input directory

The general order of things will be:

• Rowbind the complaints data from 2006-2007 to 2017-2018; I left a comment in the code but the variable naming convention changes slightly so this will involve some reconciliation of names. The most important cols to retain if you end up subsetting to get to rbind are ones with the "DPC" prefix, the year, and the state

• Clean the total student counts data read in from csv: this has consistent colnames but is wide format with different cols for different school years; should probably clean up colnames a bit and convert to long format

• Clean the IEP data read in from online (I think has one more year than complaints data so can exclude last year)- similar to dataset 1, the naming conventions change slightly. If subsetting, the most important cols from this are the total Part B (3-21; may need to sum 3-5 and 6-21 in later years), state, and year

• With the clean data, construct the relevant rates for all years (note the two denominators in the .rmd)

• Visualize!

Files needed

From my understanding, these are the files that I should need to be able to run the script. I can try once I have these, and I'll let you know if I need anything else.

processed_filings.csv
dc_ccd.csv
dc_ccd_pull2.csv
nces_filings_fuzzymatch.csv
manual_nonmatch_dc.csv
EducationDataPortal_03.07.2020_disability.csv
EducationDataPortal_03.08.2020_schools.csv
nces_civilrightsdf_fuzzymatch.csv

Initial Queries

• Loaded, cleaned and merged data
• Next steps:

1. Which columns from dataset are pertinent
2. Do we want to re-code these in any way
3. Further things

Data Files To Update

dc_ccd_pull2.csv
dc_ccd.csv

The top two files are the CCD data which only has columns until 2017 -2018.

EducationDataPortal_03.07.2020_disability.csv

The above file is the disability data which only spans from 2013 - 2015.

EducationDataPortal_03.08.2020_schools.csv

The final file is the school's data and it spans from 2013 - 2016.

See if you can match any of the remaining schools

In this chunk, I summarize the schools I could not match after (1) fuzzy matching, (2) manual matching. Seeing if you can match any of these additional schools:

filings_crosswalk_notmatched = filings_crosswalk.loc[~filings_crosswalk.school_against_cleaned.isin(fm_cc_filings.original_name.tolist() +
                            manualmatch.original_name[manualmatch.matched_manually == 1].tolist())].copy()

#print(filings_crosswalk_notmatched[['school_against_cleaned']].sort_values(by = 
 #                                       "school_against_cleaned").to_latex(index = False))