uwescience / datasci_course_materials

Public repository for course materials for the Data Science at Scale Specialization at Coursera

datasci_course_materials

Public repository for course materials for the Data Science at Scale specialization offered by Coursera and the University of Washington.

datasci_course_materials's Issues

URL in twitterstreams.py needs updating

The URL on line 54 should be changed to:
https://stream.twitter.com/1.1/statuses/sample.json

as:

https://stream.twitter.com/1/statuses/sample.json

leads to:
Unknown URL. See Twitter Streaming API documentation at http://dev.twitter.com/pages/streaming_api

updated twitter endpoint

datasci_course_materials/assignment1/twitterstream.py

Line 54 in daace27

url = "https://stream.twitter.com/1/statuses/sample.json"

hello! I'd like to share that the url has changed to "https://stream.twitter.com/1.1/statuses/sample.json"

Thanks :-)
Verona

Assignment 1: Oauth2 does not work with python 3

Error: joestump/python-oauth2#147

solution:
joestump/python-oauth2#158

possible PR solutions:

refactor assignment 1 based on an oauth library that supports python 3 syntax

Matrix.db corrupt?

Assignment 4: awsinstructions.md is outdated

There's a thread about this in the course's Discussions section:

https://www.coursera.org/learn/data-results/discussions/bxh9ibjcEeWJPQ76oZDWrw

This repository is over its data quota

When cloning the repository:

Downloading capstone/blight/data/detroit-311.csv (6.2 MB)
Error downloading object: capstone/blight/data/detroit-311.csv (542a42d): Smudge error: Error downloading capstone/blight/data/detroit-311.csv (542a42df00192aec8198bf895aad6cdf4ddab3b4c00242af2dfec09322fc42bf): batch response: This repository is over its data quota. Purchase more data packs to restore access.

Errors logged to /home/lieu/dev/datasci_course_materials/.git/lfs/logs/20190722T100133.191553284.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: capstone/blight/data/detroit-311.csv: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

Assignment 1 for Windows

I'm putting this here for posterity. No real action required.

Issue Summary:
Windows users not using the course's image may encounter an infuriatingly frustrating issue if they out their file using the command:

python twitterstream.py > output.txt

This defaults the encoding Windows-1252 and the output looks something like:

This will inevitably cause JSON parsing to fail.

Solution:

In Powershell, pipe in Out-File and force UTF-8. So something like:

python twitterstream.py | Out-File -Encoding "UTF8" output.txt

In command prompt, you actually need to convert the default encoding temporarily:

chcp 65001

Then run the command as normal. Then convert back:

chcp 1252

Assignment 1 - twitterstream.py

twitterstream.py does not pass tweets to json file when using the search url example.

Works: url = "https://stream.twitter.com/1.1/statuses/sample.json"
Does not work: url = "https://api.twitter.com/1.1/search/tweets.json?q=microsoft"

Assignment 1 - twitterstream.py

Sorry if this isn't the correct way to do this, I'm new to GitHub. The twitterstream.py needs to have the fetchsamples() function updated. The url defined in the function is an old url. Per the post below, this is needs to be updated as follows:

Current: url = "https://stream.twitter.com/1/statuses/sample.json"
Should be: url = "https://stream.twitter.com/1.1/statuses/sample.json"

Post showing new URL: https://twittercommunity.com/t/retiring-old-v1-streaming-api-endpoints-on-march-31st-2016/60351

Assignment 2 - Part d - mismatch in instructions

Assignment materials in github does NOT match the online posted instructions for Data Manipulation course.

Online instructions
https://www.coursera.org/learn/data-manipulation/programming/AxbQn/twitter-sentiment-analysis

(d) count: Write a SQL statement to count the number of unique documents containing the word "law" or containing the word "legal" (If a document contains both law and legal, it should only be counted once)

github instructions:

https://github.com/uwescience/datasci_course_materials

datasci_course_materials/assignment2.html

(d) count: Write a SQL statement to count the number of documents containing the word "parliament"

The grader is looking for counts of law/legal not parliament. Therefore, online instructions are the ones to use if there is a discrepency.

Also typo in Problem 2

Part (g) matrix shows 0 0 -3 but the matrix is showing row at 2,2,-3.

Update of assign1 twitterstream.py needed for py3

Since python 3 is now standard for most environments, it would be helpful to have the twitterstream.py file updated to use urllib instead of urllib2. i've spent a few hours on this and still can't get it to work.

Thank you,
Jennifer

git pull

error running twitterstream.py to pull twitter data in assignment 1

Getting an error on line 58 trying to run twitterstream.py to pull twitter data:
print line.strip()

The error reads:
[pylint] E0001:invalid syntax (, line 58).

I'm using Python 3.6.2.