newb having some trouble about nfldb

[ochawkeye]

@roodhouse have a look at #21 and let me know if it helps. If that doesn't get you going I'm sure I can help set your environment up.

[roodhouse]

Thank you. Not sure what it did. But now when I try to run nfldb-update it says 'nfldb-update' is not recongizned as an internal or external command, operable program or batch file."

I first ran it as python nfldb-update and got the same message.. Thank you for walking me through this.

[roodhouse]

and I am using Windows..

[roodhouse]

Based on your answer to the question you reference I tried : C:\Python27\Scripts> pip install nfldb

It did a few things but ended with "ImportError: No module named _vendor"

Am I missing files? I downloaded a bunch of stuff, so forgive me if I missed something obvious.

[BurntSushi]

Can you open Python IDLE (it is an interactive interpreter) and try typing:

import nfldb

And hit enter. Does it work? If so, then nfldb is installed.

If nfldb-update can't be found, then either nfldb is not installed or you are experiencing PATH issues. @ochawkeye I think nfldb-update should end up in the bin or scripts directory?

[BurntSushi]

The pytz error means you haven't installed nfldb's dependencies. If you installed nfldb via pip, then this should have been done automatically.

I have no idea what that pip error means. It's almost as if your pip install is borked, which is very troubling.

[roodhouse]

From Python I get "ImportError: No module named pytz"

I see nfldb-update in scripts folder for python.

[BurntSushi]

If you're completely lost, I would suggest removing all Python stuff on your machine and starting over. Focus on installing pip and getting 'pip install nfldb' to work.

Installing the Python code should be the easiest part, compared to setting up the database. :(

[BurntSushi]

Can you run nfldb-update from inside the scripts directory?

[BurntSushi]

How did you install nfldb if pip isn't working for you?

[roodhouse]

tell me how to install nfldb via pip and I can tell you if I did it. I really don't remember and starting over is something I have been coming back to.

I cant run nfldb from anywhere at this point.

I have no idea how installed if pip wasn't working.

[roodhouse]

everything was rolling until after this step...

nfldb=# \d
List of relations
Schema | Name | Type | Owner
--------+-------------+-------+-------
public | drive | table | nfldb
public | game | table | nfldb
public | meta | table | nfldb
public | play | table | nfldb
public | play_player | table | nfldb
public | player | table | nfldb
public | team | table | nfldb
(7 rows)

I saw that message in my prompt. Since then I have been lost..

[BurntSushi]

If pip is installed, then follow the instructions here: https://github.com/BurntSushi/nfldb/wiki/Windows-Install

These look like decent instructions for installing pip: https://pip.pypa.io/en/latest/installing.html

[BurntSushi]

@ochawkeye Windows is really annoying. Do you know of any way to bundle everything up into one executable?

[roodhouse]

I am sorry, but I have been on those websites for the past 4 hours. I thought I followed the instructions correctly...

[roodhouse]

Thinks went askew for me on step 12 of the windows install page you linked

[BurntSushi]

You aren't really on step 12 though because nfldb just isn't properly installed.

It seems like you got the database setup, which is great and usually the most difficult part.

I'm not sure how else I can help you with the detail you've given so far. If you start over, I'd suggest keeping a more detailed record of the steps you went through and what exactly went wrong.

I know it's frustrating, but the Python experience on Windows is unfortunately pretty grim.

[roodhouse]

No worries.

I will start over. Uninstall everything I installed yesterday. Keep notes
through the reinstall and follow ochawkeye's window's installation step by
step. When I resurface I will let you know how it went and where I stand..

Thanks for yall's help!

On Tue, Jul 29, 2014 at 10:44 PM, Andrew Gallant [email protected]
wrote:

You aren't really on step 12 though because nfldb just isn't properly
installed.

It looms like you got the database setup, which is great and usually the
most difficult part.

I'm not sure how else I can help you with the detail you've given so far.
If you start over, I'd suggest keeping a more detailed record of the steps
you went through and what exactly went wrong.

I know it's frustrating, but the Python experience on Windows is
unfortunately pretty grim.

—
Reply to this email directly or view it on GitHub
#33 (comment).

[ochawkeye]

Apologies for offering to help and then promptly falling asleep - ah, the life of someone with a ~2 year old.

Nuking it and starting over is an extreme step and I'm pretty sure we could get you going with what you have. But my schedule is kind of booked this morning for the next 6 hours or so. If you do start over and arrive at the same problem, we can have a TeamViewer session for me to review your environment.

[BurntSushi]

@ochawkeye is awesome. :D

[roodhouse]

HA! 2 kids under 3.5 and a 7 month pregnant wife here.

All 3 girls.

I am currently nuking it, but I will be in touch again later today. If I
am still in trouble we can do a teamview.

Thanks man!

On Wed, Jul 30, 2014 at 6:59 AM, Andrew Gallant [email protected]
wrote:

@ochawkeye https://github.com/ochawkeye is awesome. :D

—
Reply to this email directly or view it on GitHub
#33 (comment).

[ochawkeye]

A couple of things I would be interested in reviewing:

Do the following commands from a command prompt do the same thing for you?

C:\Users\Ben>c:\Python27\python.exe
Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

C:\Users\Ben>python
Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

Same question for pip:

C:\Users\Ben>c:\Python27\Scripts\pip.exe install
You must give at least one requirement to install (see "pip help install")

C:\Users\Ben>pip install
You must give at least one requirement to install (see "pip help install")

What version of Python are you running? (From my example above you can see I have Python 2.7.6 32 bit)

[teamfball]

I’m also a novice and struggled a bit. But I found that installing with pip to be key.
Take a look at both install instructions, not just the one for windows.

Installing the nfldb Python module
If you've made it this far, congratulations—the hard part is over! The nfldb Python module should be installed through pip from PyPI:
pip2 install nfldb
It will automatically install dependencies for you (which include nflgame, psycopg2, pytz and enum34).

Also I reverted to Ochawkeye’s tutoring here,

issue BurntSushi/nflgame#29

Good luck.

[ochawkeye]

BurntSushi/nflgame#29

[roodhouse]

@teamfball thank you, I will take a look.

@ochawkeye, when I get to that point in my reinstalls I will see if those
commands works.

Before I do anything else though.....

Do I need nflgame up and running in order to have nfldb up and running?

On Wed, Jul 30, 2014 at 7:22 AM, ochawkeye [email protected] wrote:

BurntSushi/nflgame#29 BurntSushi/nflgame#29

—
Reply to this email directly or view it on GitHub
#33 (comment).

[BurntSushi]

@roodhouse If you're using pip to install nfldb (you should be), then nflgame will get installed automatically. In theory, you shouldn't even need to know that nflgame is on your system.

[ochawkeye]

All 3 girls.

@roodhouse Man I feel for you. Frozen has already been played twice beginning to end here and it's not even 10am. And I only have one girl. 2 month old son can't grow up fast enough for me 👪

[roodhouse]

frozen. do you want to build a snowman? ride your bike around the hall?

I am done with the windows install page. I am going to copy my notes
below. First I will attach them to this email string to see if that works
to cut down on clutter.

As you can see I have the same result..

On Wed, Jul 30, 2014 at 9:54 AM, ochawkeye [email protected] wrote:

All 3 girls.
@roodhouse https://github.com/roodhouse Man I feel for you. Frozen has
already been played twice beginning to end here and it's not even 10am. And
I only have one girl. 2 month old son can't grow up fast enough for me [image:
👪]

—
Reply to this email directly or view it on GitHub
#33 (comment).

1. Uninstall Python 2.7.8
2. Uninstall Python 2.7 pscopg2-2.5.3
3. Uninstall Python 2.7 pip-1.5.6
4. Uninstall PostgreSQL 9.3 (x86)
   • note: The Data directory (c:\program files(x86)\postgresql\9.3\data) has not been removed

Windows Install
not switching b/w the main install page and the windows install page

1. manually installing pyscopg
   v. psycopg2-2.5.3.win32-py2.7.exe
   a. python 2.7 not found in directory.
2. intalling python 2.7.8 via windows x86 msi installer
3. manually installing pyscopg
   v. psycopg2-2.5.3.win32-py2.7.exe
4. Diverting from instructions and installing compiled pip binary frm question #21
   v. pip-1.5.6.win32-py2.7.exe
5. Download/install PostgreSQL
   a. an existing data dir has been found @ c:....\9.3\data set
   to use port 5432. This dir & its config will be used for
   this instl
   b. Stack builder 3.1.1 launched. Not sure what to install
   from this. Installed the Add-ons category, I dont know
   why.
   c. skipped the intstall of these add-ons bc it wanted a
   postgresql password and i dont have one yet.
6. Uninstalling PostgreSQL
   a. something doesnt seem right re: 5a & I was never prompted
   to create a password as step 4 of windows install instructions
   said i would be.
   b. got same warning as i did about in unstall step 4 note.
   c. attempting to manually delete all files located in postgresql folder
   d. reboot
7. Download/install PostgreSQL
   a. did ask me to set up password
   b. warning msg: A non-fatal error occured whilst loading
   database modules. Please ck the instl long in C:....\Temp
   for details.
   c. no msg this time about Stack Builder
8. Navigating to "C:\Program Files (x86)\PostgreSQL\9.3\bin" vis cmd prompt
9. Step 5 seemed to work.
10. Step 6 seemed to work.
11. step 7 seemed to work.
12. step 8 seemed to work. I now see nfldb=> in my cmd prompt
13. step 9 worked.
14. for step 10 leaving nfldb=> and going back to C:\Program Files (x86)\PostgreSQL\9.3\bin>
    a. using this to import: C:\Program Files (x86)\PostgreSQL\9.3\bin>psql.exe -U nfldb nfldb < C:\Users\Rugh\Desktop\nfldb\nfldb.sql
    b. database started @ 10:05, ended @ 10.14
    c. Got these Warnings:
    WARNING: no privileges could be revoked for "public"
    REVOKE
    WARNING: no privileges could be revoked for "public"
    REVOKE
    WARNING: no privileges were granted for "public"
    GRANT
    WARNING: no privileges were granted for "public"
    GRANT
15. step 11 already done, somehow files from previous install returned..
16. in config.ini set time zone like this (could be wrong):
    [US/Central]

    The time zone to use in the PostgreSQL session by default.

17. in config.ini placed password here:
    ....

    indexes and types.

    user = nfldb
    password = xxxxxx#....
18. step 13 file also has returned.. but i am moving it to a new location
19. beginning step 14:
    a. using this: C:\Python27\Scripts>python "C:\Users\Rugh\Desktop\nfldb\top-ten-qbs.py"
    1. Here is what happened
    c:\Python27\Scripts>python "C:\Users\Rugh\Desktop\nfldb\top-ten-qbs.py"
    Traceback (most recent call last):
    File "C:\Users\Rugh\Desktop\nfldb\top-ten-qbs.py", line 1, in
    import nfldb
    File "C:\Python27\lib\site-packages\nfldb__init__.py", line 61, in
    from nfldb.db import pdoc as db_pdoc
    File "C:\Python27\lib\site-packages\nfldb\db.py", line 14, in
    import pytz
    ImportError: No module named pytz
20. trying step 15
    a. got same message from my step 19 above

[roodhouse]

all attaching did was paste my notes here.. sorry about that..

I am waiting instruction now..

[ochawkeye]

Looks like you've installed pip but have not installed nfldb. Try this from a command prompt:

c:\python27\scripts\pip.exe install nfldb

[roodhouse]

after doing that here is my next error..

c:\Python27\Scripts>pip.exe install nfldb
Traceback (most recent call last):
File "c:\Python27\Scripts\pip-script.py", line 9, in
load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
File "c:\Python27\pip.py", line 17445, in main
bootstrap(tmpdir=tmpdir)
File "c:\Python27\pip.py", line 17415, in bootstrap
cert.write(pkgutil.get_data("pip._vendor.requests", "cacert.pem"))
File "c:\Python27\lib\pkgutil.py", line 578, in get_data
loader = get_loader(package)
File "c:\Python27\lib\pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "c:\Python27\lib\pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "c:\Python27\lib\pkgutil.py", line 430, in iter_importers
import(pkg)
ImportError: No module named _vendor

On Wed, Jul 30, 2014 at 10:37 AM, ochawkeye [email protected]
wrote:

Looks like you've installed pip but have not installed nfldb. Try this
from a command prompt:

c:\python27\scripts\pip.exe install nfldb

—
Reply to this email directly or view it on GitHub
#33 (comment).

[ochawkeye]

Ho - lee crap :( Scary!

You installed the 32-bit version of pip. Do you have the 32-bit version of Python?
Can confirm from c:\python27\python.exe

[roodhouse]

c:\Python27>python.exe
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win
32

yet i am running a 64 bit ver of windows......?

[ochawkeye]

yet i am running a 64 bit ver of windows......?

That's just fine. So am I. There are some packages I use that are only stable in 32-bit. I'm going to upgrade from 2.7.6 to 2.7.8 to see if it breaks anything.

I'm happy to remote into your system to see if I can't identify the problem as well. If you want me to do that you can use that TeamViewer link I sent you and send the credentials directly to me: [email protected]

[roodhouse]

let me know if anything breaks for you with 2.7.8 and then if not we can remote... cool?

[ochawkeye]

Nope, no ill effects there...

C:\Users\Ben>python
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

C:\Users\Ben>pip uninstall nfldb
Uninstalling nfldb:
  d:\python27\lib\site-packages\nfldb-0.2.0-py2.7.egg-info
  d:\python27\lib\site-packages\nfldb\__init__.py
  d:\python27\lib\site-packages\nfldb\__init__.pyc
  d:\python27\lib\site-packages\nfldb\category.py
  d:\python27\lib\site-packages\nfldb\category.pyc
  d:\python27\lib\site-packages\nfldb\db.py
  d:\python27\lib\site-packages\nfldb\db.pyc
  d:\python27\lib\site-packages\nfldb\query.py
  d:\python27\lib\site-packages\nfldb\query.pyc
  d:\python27\lib\site-packages\nfldb\sql.py
  d:\python27\lib\site-packages\nfldb\sql.pyc
  d:\python27\lib\site-packages\nfldb\team.py
  d:\python27\lib\site-packages\nfldb\team.pyc
  d:\python27\lib\site-packages\nfldb\types.py
  d:\python27\lib\site-packages\nfldb\types.pyc
  d:\python27\lib\site-packages\nfldb\version.py
  d:\python27\lib\site-packages\nfldb\version.pyc
  d:\python27\scripts\nfldb-update
  d:\python27\share\doc\nfldb\doc\index.html
  d:\python27\share\doc\nfldb\doc\nfldb-condensed.pdf
  d:\python27\share\doc\nfldb\doc\nfldb-condensed.png
  d:\python27\share\doc\nfldb\doc\nfldb-short.pdf
  d:\python27\share\doc\nfldb\doc\nfldb-short.png
  d:\python27\share\doc\nfldb\doc\nfldb.pdf
  d:\python27\share\doc\nfldb\doc\nfldb.png
  d:\python27\share\doc\nfldb\longdesc.rst
  d:\python27\share\doc\nfldb\readme.md
  d:\python27\share\doc\nfldb\unlicense
  d:\python27\share\nfldb\config.ini.sample
Proceed (y/n)? y
  Successfully uninstalled nfldb

C:\Users\Ben>pip install nfldb
Downloading/unpacking nfldb
  Running setup.py (path:c:\users\ben\appdata\local\temp\pip_build_Ben\nfldb\set
up.py) egg_info for package nfldb

Requirement already satisfied (use --upgrade to upgrade): nflgame>=1.2.2 in d:\p
ython27\lib\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): psycopg2 in d:\python2
7\lib\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): enum34 in d:\python27\
lib\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): pytz in d:\python27\li
b\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): httplib2 in d:\python2
7\lib\site-packages (from nflgame>=1.2.2->nfldb)
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in d:\p
ython27\lib\site-packages (from nflgame>=1.2.2->nfldb)
Installing collected packages: nfldb
  Running setup.py install for nfldb

Successfully installed nfldb
Cleaning up...

I don't think we have a problem with nfldb (yet). We need to get past this pip issue; Does pip freeze work for you? (You will need the full path if you haven't setup your ENVIRONMENT VARIABLES) c:\python27\scripts\pip.exe freeze

C:\Users\Ben>pip freeze
beautifulsoup4==4.3.2
enum34==1.0
httplib2==0.9
nfldb==0.2.0
nflgame==1.2.6
psycopg2==2.5.3
pytz==2014.4
requests==2.3.0
selenium==2.42.1
virtualenv==1.11.6

[roodhouse]

no here is the result...

c:\Python27\Scripts>pip.exe freeze
Traceback (most recent call last):
File "c:\Python27\Scripts\pip-script.py", line 9, in
load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
File "c:\Python27\pip.py", line 17445, in main
bootstrap(tmpdir=tmpdir)
File "c:\Python27\pip.py", line 17415, in bootstrap
cert.write(pkgutil.get_data("pip._vendor.requests", "cacert.pem"))
File "c:\Python27\lib\pkgutil.py", line 578, in get_data
loader = get_loader(package)
File "c:\Python27\lib\pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "c:\Python27\lib\pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "c:\Python27\lib\pkgutil.py", line 430, in iter_importers
import(pkg)
ImportError: No module named _vendor

---

One thing to note is that I did not follow any steps on the regular nfldb page. Therefore the step on that page that begins with:

Installing the nfldb Python module

If you've made it this far, congratulations—the hard part is over! The nfldb Python module should be installed through pip from PyPI:

has not been done.

[ochawkeye]

That's the exact step we're trying to do (which equates to step 3 in the Windows installation page). But with a non-functioning pip we're not able to do it...

[BurntSushi]

Could you uninstall only pip and then try reinstalling it using this method?

You should only have to run the first command, python get-pip.py (note that you have to download get-pip.py from the link I gave you. After you run that, could you try pip install nfldb again?

I'm not seeing anything helpful on the Internet about your weird _vendor import error.

@ochawkeye Do you have an updated pip? 1.5.6 I think is the latest.

[ochawkeye]

Are you positive you installed pip‑1.5.6.win32‑py2.7.exe? Mine came with a _vendor subfolder.

You should be able to confirm the file name you downloaded in your Downloads folder.

[roodhouse]

So at this point installing the nfldb Python module is redundant?

Where should I be able to locate these?

It will automatically install dependencies for you (which include nflgame, psycopg2, pytz and enum34).

[ochawkeye]

So at this point installing the nfldb Python module is redundant?

Not at all - you haven't been able to successfully install the python module because the install method pip is failing. Once you get pip to work, installing nfldb will be a breeze.

[BurntSushi]

So at this point installing the nfldb Python module is redundant?

No. You can't install nfldb without a functioning pip.

Let me explain it to you.

Python is the programming language that nfldb was written in. This is required in order to run Python programs that use nfldb.

pip is a tool, written in Python, that installs other Python libraries. One such library is nfldb. Without pip, you cannot easily install nfldb. Without Python, you cannot have pip.

Therefore, at this point in time, since your pip is not working, that must be fixed before we can move on to anything else.

Once it is working, you should be able to install nfldb with pip install nfldb. This will also install pytz, nflgame, etc.

[roodhouse]

@BurntSushi got it

@ochawkeye i did install the correct pip. and i too have a _vendor folder

[roodhouse]

going to uninstall & reinstall pip now..

[roodhouse]

i want to make sure i am installing pip correctly from the link above...

i copy and paste all this code into a file with ext .py?

where does the code begin and end? with the 3 " marks? or with ZIPFILE? ends with the final 3 " marks? or ends with the last line of text "main ()"?

[BurntSushi]

@roodhouse Right click on the link and click "Save link as..." or "Save target as..."

Avoid copy and pasting. That can introduce errors.

[roodhouse]

@BurntSushi right click? how?

[BurntSushi]

Uh...

Here: https://bootstrap.pypa.io/get-pip.py <-- Right click on that link. There should be an option to "Save target as..." (Or "Save link as...")

[roodhouse]

sorry.. bad joke.

I downloaded it as get-pip.py ... now i need to run it in cmd prompt with c:\python27\scripts>python get-pip.py ?

[BurntSushi]

:P Yup, that sounds right.

You've uninstalled your previous pip, right?

[roodhouse]

yes i have. i got this:

c:\Python27\Scripts>python get-pip.py

python: can't open file 'get-pip.py': [Errno 2] No such file or directory

so i should do it like this instead: c:\Python27\Scripts>python < C:\Users\Rugh\Desktop\nfldbget-pip.py ?

[roodhouse]

looks like it did not save correctly..

c:\Python27\Scripts>python < c:\users\rugh\desktop\nfldb\get-pip.py
File "", line 3

^

SyntaxError: invalid syntax

i am going to re-download it with the 2nd link you provided

[roodhouse]

c:\Python27\Scripts>python < c:\users\rugh\desktop\nfldb\get-pip.py
Requirement already up-to-date: pip in c:\python27\lib\site-packages
Cleaning up...

[ochawkeye]

Get rid of that <.

c:\python27\python.exe c:\users\rugh\desktop\nfldb\get-pip.py

[roodhouse]

this:

c:\Python27\Scripts>python c:\users\rugh\desktop\nfldb\get-pip.py
Requirement already up-to-date: pip in c:\python27\lib\site-packages
Cleaning up...

then this:
c:\Python27>python pip.py install nfldb
Requirement already up-to-date: pip in c:\python27\lib\site-packages
Downloading/unpacking install
Could not find any downloads that satisfy the requirement install
Some externally hosted files were ignored (use --allow-external install to all
ow).
Cleaning up...
No distributions at all found for install
Storing debug log for failure in C:\Users\Rugh\pip\pip.log

[BurntSushi]

It looks like you didn't uninstall pip (or the pip uninstaller didn't do a very good job).

Go into C:/Python27/lib/site-packages and just delete the pip and nfldb directories. (The directories may be called pip-something-something, but delete them just the same. Then retry python C:/.../get-pip.py.

[roodhouse]

i am erasing folders called "pip" "nfldb" & "pip-1.5.6.dist-info" along with files called "nfldb-0.2.0py2.7.egg-info"

[roodhouse]

now this:

c:\Python27\Scripts>python c:\users\rugh\desktop\nfldb\get-pip.py
Downloading/unpacking pip
Installing collected packages: pip
Successfully installed pip
Cleaning up...

and then this:

c:\Python27>python pip.py install nfldb
Requirement already up-to-date: pip in c:\python27\lib\site-packages
Downloading/unpacking install
Could not find any downloads that satisfy the requirement install
Some externally hosted files were ignored (use --allow-external install to all
ow).
Cleaning up...
No distributions at all found for install
Storing debug log for failure in C:\Users\Rugh\pip\pip.log

[ochawkeye]

How about c:\python27\scripts\pip.exe install nfldb?

[roodhouse]

just tried this with this result:

c:\Python27\Scripts>python pip.exe install nfldb
Traceback (most recent call last):
File "C:\Python27\lib\runpy.py", line 162, in run_module_as_main
"main", fname, loader, pkg_name)
File "C:\Python27\lib\runpy.py", line 72, in run_code
exec code in run_globals
File "pip.exe__main.py", line 9, in
File "C:\Python27\pip.py", line 17445, in main
bootstrap(tmpdir=tmpdir)
File "C:\Python27\pip.py", line 17415, in bootstrap
cert.write(pkgutil.get_data("pip._vendor.requests", "cacert.pem"))
File "C:\Python27\lib\pkgutil.py", line 578, in get_data
loader = get_loader(package)
File "C:\Python27\lib\pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "C:\Python27\lib\pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "C:\Python27\lib\pkgutil.py", line 430, in iter_importers
import(pkg)
ImportError: No module named _vendor

and what you suggested with this result:

c:\Python27\Scripts>python pip.exe install nfldb
Traceback (most recent call last):
File "C:\Python27\lib\runpy.py", line 162, in run_module_as_main
"main", fname, loader, pkg_name)
File "C:\Python27\lib\runpy.py", line 72, in run_code
exec code in run_globals
File "pip.exe__main.py", line 9, in
File "C:\Python27\pip.py", line 17445, in main
bootstrap(tmpdir=tmpdir)
File "C:\Python27\pip.py", line 17415, in bootstrap
cert.write(pkgutil.get_data("pip._vendor.requests", "cacert.pem"))
File "C:\Python27\lib\pkgutil.py", line 578, in get_data
loader = get_loader(package)
File "C:\Python27\lib\pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "C:\Python27\lib\pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "C:\Python27\lib\pkgutil.py", line 430, in iter_importers
import(pkg)
ImportError: No module named _vendor

[roodhouse]

however, this time my _vendor folder is in fact missing...

[roodhouse]

wait.. no it is not... it is w/in pip.. sorry

[ochawkeye]

I'm available for a TeamViewer session if you have time.

[roodhouse]

how long will you be available for? i have to run out real quick..

[roodhouse]

nevermind, lets do it

[roodhouse]

email sent..

[roodhouse]

awesome.

@ochawkeye thank you!
@BurntSushi thank you!

[BurntSushi]

@roodhouse Got it? Yay!

@ochawkeye I am fiercely curious what the issue was...

[ochawkeye]

@BurntSushi T'was a rouge file named 'pip.py' in the Python execution folder. I guess that file was being given preferential treatment over the actual pip module and was overriding the commands that were trying to be sent to pip. Similar to if I* were to create an 'nflgame.py' file and paste it into my working directory, Python would try to use that instead of the actual nflgame module.

*not that I ever did that when I first started playing with nflgame and it took me two days to figure out what the heck was wrong with your "junk" python code

[BurntSushi]

@ochawkeye Ah ha! Nice find.

Yeah, 99% of all Python problems are due to its insane import resolution semantics. Gah.

One reason to move to Python 3 is that pip is now included in every Python 3.4 distribution.

[ochawkeye]

I only hope the frustration of the past couple of days hasn't scared away @roodhouse! nflgame|db|vid|fan, and Python in general, can be a very rewarding hobby.

[iliketowel]

First of all, thanks so much for putting this together, this is some amazing stuff, and I've wanted to work with data like this for a while now.

Hey, I'm reading this now, and I'm basically in the same predicament. I have gotten the nfldb db loaded in postgresql (for ease, I'll call it SQLnfldb), but may be issues with the python nfldb (PYTnfldb). I got the get-pip.py file downloaded and I ran it. I thought it installed okay, but when I tried to run the test example I get an empty space

import nfldb
db = nfldb.connect()
q = nfldb.Query(db)
q.game(season_year=2013, season_type='regular')
<nfldb.query.Query object at 0x03562A70>

I tried pressing through this (I wasn't sure if this was an error or not), and when I ran

for pp in q.sort('passing_yds').limit(10).as_aggregate():
print pp.player, pp.passing_yds

It seems to work okay for 2012 one time, but since then, I get nothing. I just wanted to confirm if I should be seeing the "Query object at 0x0#######" or if that's a sign that something is wrong, or if there's another issue in play.

Thanks again.

[ochawkeye]

Strings in Python are case sensitive, so you need to watch out for that season_type='Regular'

Running this code:

import nfldb
db = nfldb.connect()
q = nfldb.Query(db)
q.game(season_year=2013, season_type='Regular')

for pp in q.sort('passing_yds').limit(10).as_aggregate():
    print pp.player, pp.passing_yds

should give you this result:

Peyton Manning (DEN, QB) 5477
Drew Brees (NO, QB) 5139
Matthew Stafford (DET, QB) 4647
Matt Ryan (ATL, QB) 4515
Philip Rivers (SD, QB) 4478
Tom Brady (NE, QB) 4338
Andy Dalton (CIN, QB) 4296
Carson Palmer (ARI, QB) 4274
Ben Roethlisberger (PIT, QB) 4147
Joe Flacco (BAL, QB) 3912

Changing that one character in my code:

import nfldb
db = nfldb.connect()
q = nfldb.Query(db)
q.game(season_year=2013, season_type='regular')

for pp in q.sort('passing_yds').limit(10).as_aggregate():
    print pp.player, pp.passing_yds

now gives me a Python error:

Traceback (most recent call last):
  File "P:\Projects\Home Computer\Fantasy Football\2013\scratch7.py", line 6, in <module>
    for pp in q.sort('passing_yds').limit(10).as_aggregate():
  File "C:\Python27\lib\site-packages\nfldb\query.py", line 925, in as_aggregate
    cur.execute(q)
  File "C:\Python27\lib\site-packages\psycopg2\extras.py", line 223, in execute
    return super(RealDictCursor, self).execute(query, vars)
psycopg2.DataError: invalid input value for enum season_phase: "regular"
LINE 8:                 WHERE (((game.season_type = 'regular') AND (...

Is this what you're seeing?

[iliketowel]

No... It's going through. It just seems to be... empty. I just restarted IDLE

I did this:

import nfldb
db = nfldb.connect()
q = nfldb.Query(db)
q.game(season_year=2013, season_type='Regular')

And got this message: <nfldb.query.Query object at 0x03518030>

I then ran

for pp in q.sort('passing_yds').limit(10).as_aggregate():
print pp.player, pp.passing_yds

and got the same results as you
Peyton Manning (DEN, QB) 5477
Drew Brees (NO, QB) 5139
Matthew Stafford (DET, QB) 4647
Matt Ryan (ATL, QB) 4515
Philip Rivers (SD, QB) 4478
Tom Brady (NE, QB) 4338
Andy Dalton (CIN, QB) 4296
Carson Palmer (ARI, QB) 4274
Ben Roethlisberger (PIT, QB) 4147
Joe Flacco (BAL, QB) 3912

But now, when I ran

q.game(season_year=2012, season_type='Regular')
for pp in q.sort('passing_yds').limit(10).as_aggregate():
print pp.player, pp.passing_yds

underneath, I got a blank (usually I had been hitting enter and then seeing results), and then (>>>)
with program waiting for me to make a command.

[iliketowel]

I guess I haven't asked what I suppose should be an obvious question, do I need to re-enter

import nfldb
db = nfldb.connect()
q = nfldb.Query(db)
q.game(season_year=2013, season_type='Regular')

every time?

And again, thanks to yourself and burntsushi, this is super fun stuff.

[ochawkeye]

Ahhh...I'm beginning to understand. The IDLE shell has it's uses, but it is not much more than a learning tool for Python.

Try clicking File->New File from the menu. Paste all of that code you posted above into the Untitled document window that opens and save the file with an appropriate name - maybe something like top-ten-qbs.py. Now, in the same window where you wrote your code, click Run->Run Module (or simply click F5). Control will switch back to the shell window that was open in the background and all of your Python code will execute instead of just line by line as you were typing it into the shell.

But you don't really need IDLE to do any of this. With that same file you just created (which you could have created with the text editor of your preference), you can fire up a command prompt and enter the following:

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\OCHawkeye> python c:\path\to\my\file\top-ten-qbs.py
Drew Brees (NO, QB) 5177
Matthew Stafford (DET, QB) 4965
Tony Romo (DAL, QB) 4903
Tom Brady (NE, QB) 4799
Matt Ryan (ATL, QB) 4719
Peyton Manning (DEN, QB) 4667
Andrew Luck (IND, QB) 4374
Aaron Rodgers (GB, QB) 4303
Josh Freeman (UNK, UNK) 4065
Carson Palmer (ARI, QB) 4018

C:\Users\OCHawkeye>

[ochawkeye]

do I need to re-enter ... every time?

I think you just asked a question asked by every beginner programmer. The answer, of course, is a resounding NO! That's probably why you're doing this in the first place. Sure, you could look up each of the yardage totals for each of those QBs on NFL.com and create the table yourself - or you could have the Python code do the work for you.

If you find yourself typing redundant code over and over again, there is a good chance you could be consolidating that in to a much more succinct set of instructions.

Say you wanted to find out who led the league in passing in 2013, 2012, and 2011. You could always go the route of copying and pasting your working example above, changing the season_year value each time.

import nfldb
db = nfldb.connect()

q = nfldb.Query(db)
q.game(season_year=2013, season_type='Regular')
for pp in q.sort('passing_yds').limit(10).as_aggregate():
    print pp.player, pp.passing_yds
print '-'*79
q = nfldb.Query(db)
q.game(season_year=2012, season_type='Regular')
for pp in q.sort('passing_yds').limit(10).as_aggregate():
    print pp.player, pp.passing_yds
print '-'*79
q = nfldb.Query(db)
q.game(season_year=2011, season_type='Regular')
for pp in q.sort('passing_yds').limit(10).as_aggregate():
    print pp.player, pp.passing_yds

There sure is a lot of code that is written and rewritten again and again there. It could easy be re-factored into the following:

import nfldb
db = nfldb.connect()

def top_10_qb_passing_yds(db, yr):
    q = nfldb.Query(db)
    q.game(season_year=yr, season_type='Regular')
    for pp in q.sort('passing_yds').limit(10).as_aggregate():
        print pp.player, pp.passing_yds

for year in [2013, 2012, 2011]:
    top_10_qb_passing_yds(db, year)
    print '-'*79

I see now that there is another way to interpret your question. The answer to that version is:

Do I need to re-enter <> every time?:
import nfldb - nope, you only need to import 1 time
db = nfldb.connect() - nope, you are only establishing a single connection to the database
q = nfldb.Query(db) - yep, if you want to run a new query, you have to re-enter this
q.game(season_year=2013, season_type='Regular') - yep, if you ran a new query above, this is what you would use to filter the query of the entire database down to a single year and season-type.

[ochawkeye]

Do I need to re-enter <> every time?: (continued)

On the other hand, if you're doing more/different stuff with the same query, then no, you don't need to re-enter the top lines every time.

For example, if I'm doing multiple sorts of the same data pulled from the database, then I only need pull the data from the database the one time. The following is perfectly valid code.

import nfldb
db = nfldb.connect()

q = nfldb.Query(db)
q.game(season_year=2013, season_type='Regular')
for pp in q.sort('passing_yds').limit(3).as_aggregate():
    print pp.player, pp.passing_yds
print '-'*79
for pp in q.sort('rushing_yds').limit(3).as_aggregate():
    print pp.player, pp.rushing_yds
print '-'*79
for pp in q.sort('receiving_yds').limit(3).as_aggregate():
    print pp.player, pp.receiving_yds

Peyton Manning (DEN, QB) 5477
Drew Brees (NO, QB) 5139
Matthew Stafford (DET, QB) 4647
-------------------------------------------------------------------------------
LeSean McCoy (PHI, RB) 1607
Matt Forte (CHI, RB) 1341
Jamaal Charles (KC, RB) 1288
-------------------------------------------------------------------------------
Josh Gordon (CLE, WR) 1646
Calvin Johnson (DET, WR) 1489
Antonio Brown (PIT, WR) 1438

[iliketowel]

I tried to run the update module, and it looks like it worked somewhat, but I got some errors too.

I get an error saying
"no module named httplib2"
and a separate error saying
python.exe -m nflgame.update_players --no-block' failed (exit status 1)

otherwise, it seemed to load okay.

[iliketowel]

for year in [2013, 2012, 2011]:
top_10_qb_passing_yds(db, year)
print '-'*79

I was about to ask was the 'print '-'*79 was, then I ran it and realized it was a separator.

I have another question, which may be something that's a totally different "issue", or an answer or already exists

When I ran that top 10 QBs for 2012 I got this result in the 10

Josh Freeman (UNK, UNK) 4065

Is there a simple method to refer to the team he played for during that season (TB), rather than UNK, (which is his current situation)?

Thanks again.

[ochawkeye]

httplib2 is a dependency of nfldb and should have been installed if you used pip to do the nfldb installation.

What does it say if you try to pip install nfldb now? Should look like the following:

C:\Users\Ben>pip install nfldb
Requirement already satisfied (use --upgrade to upgrade): nfldb in d:\python27\l
ib\site-packages
Requirement already satisfied (use --upgrade to upgrade): nflgame>=1.2.2 in d:\p
ython27\lib\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): psycopg2 in d:\python2
7\lib\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): enum34 in d:\python27\
lib\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): pytz in d:\python27\li
b\site-packages (from nfldb)
Requirement already satisfied (use --upgrade to upgrade): httplib2 in d:\python2
7\lib\site-packages (from nflgame>=1.2.2->nfldb)
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in d:\p
ython27\lib\site-packages (from nflgame>=1.2.2->nfldb)
Cleaning up...

Notice the line

Requirement already satisfied (use --upgrade to upgrade): httplib2 in d:\python2
7\lib\site-packages (from nflgame>=1.2.2->nfldb)

[iliketowel]

Huh... now I get an error message 'pip' is not a recognized as an internal or external command.

last time, I installed by doing python pip.exe install nfldb (or maybe pip.exe install nfldb) but from the directory of c:\python27\Scripts

I tried doing it from that directory and it worked, the first 5 were the same as yours, but I didn't have the httplib or beautifulsoup lines.

[ochawkeye]

Is there a simple method to refer to the team he played for during that season (TB), rather than UNK, (which is his current situation)?

Interesting question and one I'm not immediately able to provide an answer for. Of course, it can be a complex scenario with what you call a player that ends up hopping from team to team to team, but truth is that nfldb "knows" he played for Tampa Bay that year even though that fact is not explicitly tied to Josh Freeman's meta data anywhere.

import nfldb
db = nfldb.connect()

q = nfldb.Query(db)
q.game(season_year=2012, season_type='Regular')
q.play_player(team='TB')

for pp in q.sort('passing_yds').limit(3).as_aggregate():
    print '%s - %s yards passing' % (pp.player, pp.passing_yds)
for pp in q.sort('rushing_yds').limit(3).as_aggregate():
    print '%s - %s yards rushing' % (pp.player, pp.rushing_yds)
for pp in q.sort('receiving_yds').limit(3).as_aggregate():
    print '%s - %s yards receiving' % (pp.player, pp.receiving_yds)

Josh Freeman (UNK, UNK) - 4065 yards passing
Dan Orlovsky (DET, QB) - 51 yards passing
Mike Williams (BUF, WR) - 28 yards passing
Doug Martin (TB, RB) - 1454 yards rushing
LeGarrette Blount (PIT, RB) - 151 yards rushing
Josh Freeman (UNK, UNK) - 135 yards rushing
Vincent Jackson (TB, WR) - 1384 yards receiving
Mike Williams (BUF, WR) - 996 yards receiving
Doug Martin (TB, RB) - 472 yards receiving

I'm sure @BurntSushi can give a thorough explanation.

[ochawkeye]

Re: 'pip' is not a recognized as an internal or external command

Just need to add C:\Python27\Scripts to your system's PATH ENVIRONMENT VARIABLE
see #21

[BurntSushi]

@iliketowel With regards to the team for Josh Freeman. This gets a bit hairy.

Here is a central truth about the data in nfldb: its meta data about a player is always current. The data is meant to capture information about the player as he exists this moment. This means that only an active roster spot will give a player a team. This meta data is what you get when you use pp.player. Namely, it retrieves player meta for the player statistic pp.

With that said, every individual statistic for a player also has a team attached to it. This is historical data, so that the proper team for every player stays fixed.

So that means, if you're listing individual play statistics, you can always print the right team. For example:

import nfldb

db = nfldb.connect()
q = nfldb.Query(db)

q.game(season_year='2012', season_type='Regular', week=1)
q.player(full_name='Josh Freeman')
q.play_player(passing_yds__ge=15)

for pp in q.as_play_players():
    print pp.player.full_name, pp.team, pp.passing_yds

And the output:

[andrew@Liger nfldb] python2 33.py                                                      
Josh Freeman TB 15                                                                      
Josh Freeman TB 33                                                                      
Josh Freeman TB 21                                                                      
Josh Freeman TB 15

Notice that the team is outputted as pp.team instead of from pp.player. This means that team here is a property of the statistic itself rather than meta data about the player. For example, if you changed pp.team to pp.player.team, then it would say UNK instead because it is accessing current knowledge about the player.

Now, finally, we can get to your particular example. It is difficult because you are aggregating results over a season. A player doesn't necessarily have the same team over a season, so when you aggregate statistics, the pp.team field gets dropped. So let's try it:

import nfldb

db = nfldb.connect()
q = nfldb.Query(db)

q.game(season_year='2012', season_type='Regular')
q.sort('passing_yds').limit(10)

for pp in q.as_aggregate():
    print pp.player.full_name, pp.team, pp.passing_yds

And now the output:

[andrew@Liger nfldb] python2 33.py 
Drew Brees None 5177
Matthew Stafford None 4965
Tony Romo None 4903
Tom Brady None 4799
Matt Ryan None 4719
Peyton Manning None 4667
Andrew Luck None 4374
Aaron Rodgers None 4303
Josh Freeman None 4065
Carson Palmer None 4018

That's not very nice, so therefore, the example takes an easier path: it just shows you the team that the player currently belongs to.

If you'd like to tumble down the rabbit hole and fix this for real, then you need to find all teams that a player played for. There are lots of ways to do this, but basically, you'd want to find all the individual plays and accrue all unique teams in those plays for that player. For example, last year, Trent Richardson played on a couple teams. We could discover this by looking at the team field on all of his statistics:

import nfldb

db = nfldb.connect()
q = nfldb.Query(db)

q.game(season_year='2013', season_type='Regular')
q.player(full_name='Trent Richardson')

teams = set()
for pp in q.as_play_players():
    teams.add(pp.team)
print ', '.join(teams)

And that outputs:

[andrew@Liger nfldb] python2 33.py 
IND, CLE

Just as you'd expect.

[iliketowel]

Thanks again. I figured out the other issue I was having. Because I installed NFLGame first, I didn't use pip at the time. I did it on a different computer and after some minor issues (I'm finding issues with trying to install when using PostGRESQL 64 bit version), I was able to get this up and running.

For now I'm mostly playing with this data in the POSTGRESQL DB GUI, and because I work in visual analytics I'm trying to create fun and interesting visualizations out of the information. I may end up putting some of this stuff up on Tableau (and Tableau Public), but I'd like to confirm how/if you want to be credited.

For now, I added fields to confirm direction of the play, and whether it was in shotgun. I'm also working to see if I can figure out a way to see how players doing against starters in preseason (to see if there is any carryover), but that's difficult to do, because there's no real way to tell when starters exit pre-season games.

[BurntSushi]

Yes, please do put things up on Tableau and share them with us when you do. :-) (Opening a new issue or adding it to the wiki is perfectly acceptable.)

As far as credit goes... Having a shout out to the project (not me) is always appreciated. It helps increase awareness and hopefully attracts more folks. But of course, nfldb and associated projects are in the public domain like SQLite, so you could in theory copy the code, rebrand it, claim you wrote it and sell it, and it'd all be nice and legal. :-)