brettcannon / caniusepython3 Goto Github PK

Can I Use Python 3?

License: Apache License 2.0

Python 100.00%

caniusepython3's Issues

Package name normalization is not handled correctly

For example:

$ caniusepython3 --projects wsgi_intercept
Finding and checking dependencies ...
No handlers could be found for logger "ciu"

You have 0 projects blocking you from using Python 3!

$ caniusepython3 --projects wsgi-intercept
Finding and checking dependencies ...
No handlers could be found for logger "ciu"

You need 1 project to transition to Python 3.
Of that 1 project, 1 has no direct dependencies blocking its transition:

  wsgi-intercept

What should we do with deprecated packages?

For eg, https://github.com/omab/django-social-auth is deprecated in favor of python-social-auth. I see it is not overridden. Any plan to do something about them as those packages will never Python 3 support?

Add tests to cover just_name() rename failure

e3dd493 should not have been needed. Figure out where the hole in the testing is and plug it.

pyramid_mako (among others) is not recognized as py3 compatible

Seems some packages are not correctly recognized even though they have the required classifiers on PyPI.

For example pyramid_mako.

One thing I noticed is that on the website entering pyramid_mako turns out as pyramid-mako (underscore converted to dash). Maybe it does not find that on PyPI...?

AttributeError for download_urls

> caniusepython3 -r requirements.txt 
No handlers could be found for logger "ciu"
Finding and checking dependencies ...
[WARNING] Stale overrides: set([u'xlwt', u'jmespath'])
Traceback (most recent call last):
  File "$HOME/.local/bin/caniusepython3", line 11, in <module>
    sys.exit(main())
  File "$HOME/.local/venvs/caniusepython3/lib/python2.7/site-packages/caniusepython3/__main__.py", line 179, in main
    check(projects_from_cli(args))
  File "$HOME/.local/venvs/caniusepython3/lib/python2.7/site-packages/caniusepython3/__main__.py", line 167, in check
    blockers = dependencies.blocking_dependencies(projects, pypi.all_py3_projects())
  File "$HOME/.local/venvs/caniusepython3/lib/python2.7/site-packages/caniusepython3/dependencies.py", line 81, in blocking_dependencies
    dist = distlib.locators.locate(project)
  File "$HOME/.local/venvs/caniusepython3/lib/python2.7/site-packages/distlib/locators.py", line 361, in locate
    result.download_urls = versions.get('urls', {}).get(version, set())
AttributeError: 'dict' object has no attribute 'download_urls'

Add a change log

With the imminent release of version 2.0 it's time to start keeping a change log for major changes that will affect users. Unfortunately with GitHub using Markdown and PyPI using reST, it's already annoying to maintain two files. My current thinking is to make PyPI host the docs for the use of caniusepython3 and then have README.md host the change log and any other development details.

numpy already supports Python 3

Numpy already supports Python3. And this CLI was working fine. Recently, something has changed and it gives following output.

$ caniusepython3 --projects numpy
Finding and checking dependencies ...
[WARNING] Stale overrides: set([u'reportlab'])

You need 1 project to transition to Python 3.
Of that 1 project, 1 has no direct dependencies blocking its transition:

  numpy

Not sure if it is the case with packages that specify version number in PyPi like https://pypi.python.org/pypi/numpy/ gives 404 so we have to use https://pypi.python.org/pypi/numpy/1.8.1

This issue was fist reported here chhantyal/py3readiness#24

MemoryError when circular dependencies are specified

Found when checking kotti_tinymce:

mattr@helios in ~/git λ: caniusepython3 -p kotti_tinymce
Finding and checking dependencies ...
[WARNING] Stale overrides: set([u'xlwt'])
Traceback (most recent call last):
  File "/home/mattr/.virtualenvs/test-env/bin/caniusepython3", line 11, in <module>
    sys.exit(main())
  File "/home/mattr/.virtualenvs/test-env/local/lib/python2.7/site-packages/caniusepython3/__main__.py", line 179, in main
    check(projects_from_cli(args))
  File "/home/mattr/.virtualenvs/test-env/local/lib/python2.7/site-packages/caniusepython3/__main__.py", line 167, in check
    blockers = dependencies.blocking_dependencies(projects, pypi.all_py3_projects())
  File "/home/mattr/.virtualenvs/test-env/local/lib/python2.7/site-packages/caniusepython3/dependencies.py", line 114, in blocking_dependencies
    return reasons_to_paths(reasons)
  File "/home/mattr/.virtualenvs/test-env/local/lib/python2.7/site-packages/caniusepython3/dependencies.py", line 45, in reasons_to_paths
    path.append(parent)
MemoryError

TypeError: parse_requirements() missing 1 required keyword argument: 'session'

Hello!

caniusepython3==2.2.0 and pip 6.0.6

Traceback (most recent call last):
  File "/vagrant/project/tests/py3_compatibility_test.py", line 11, in test_dependencies
    './project/configure/requirements.txt',
  File "/home/vagrant/pyenv/versions/latest-2/lib/python2.7/site-packages/caniusepython3/__init__.py", line 41, in check
    dependencies = main.projects_from_requirements(requirements_paths)
  File "/home/vagrant/pyenv/versions/latest-2/lib/python2.7/site-packages/caniusepython3/__main__.py", line 37, in projects_from_requirements
    for req in reqs:
  File "/home/vagrant/pyenv/versions/latest-2/lib/python2.7/site-packages/pip/req/req_file.py", line 19, in parse_requirements
    "parse_requirements() missing 1 required keyword argument: "
TypeError: parse_requirements() missing 1 required keyword argument: 'session'

Override ipaddress

It's a backport of the builtin python3 module, so probably could be ignored in some way?

TypeError thrown when looking at all pypi projects.

While running a very simple but long running program I received this stack.

Program Code:

from collections import Counter
from pprint import pprint

import caniusepython3
from caniusepython3.dependencies import blocking_dependencies

all_py_projects = list(caniusepython3.pypi.all_projects())
py3_projects = caniusepython3.pypi.all_py3_projects()
all_blockers = blocking_dependencies(
    all_py_projects,
    py3_projects)
blocker_list = []
for blockers_subset in all_blockers:
    for blocker in blockers_subset:
        blocker_list.append(blocker)
blockers_counter = Counter(blocker_list)
pprint(blockers_counter.most_common())

Just as a warning this takes my poor laptop a couple hours to run.

Stack

Traceback (most recent call last):
  File "top_python3_blockers.py", line 15, in <module>
    py3_projects)
  File "/Users/alexlord/.virtualenvs/porting_authority/lib/python3.4/site-packages/caniusepython3-3.1.0-py3.4.egg/caniusepython3/dependencies.py", line 98, in blocking_dependencies
  File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 549, in result_iterator
    yield future.result()
  File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 395, in result
    return self.__get_result()
  File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 354, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/thread.py", line 54, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexlord/.virtualenvs/porting_authority/lib/python3.4/site-packages/caniusepython3-3.1.0-py3.4.egg/caniusepython3/dependencies.py", line 60, in dependencies
  File "/Users/alexlord/.virtualenvs/porting_authority/lib/python3.4/site-packages/distlib/database.py", line 375, in run_requires
    return self._get_requirements('run_requires')
  File "/Users/alexlord/.virtualenvs/porting_authority/lib/python3.4/site-packages/distlib/database.py", line 371, in _get_requirements
    env=self.context))
  File "/Users/alexlord/.virtualenvs/porting_authority/lib/python3.4/site-packages/distlib/metadata.py", line 874, in get_requirements
    result.extend(d['requires'])
TypeError: string indices must be integers

I'm going to re-run the sample code using python -m pdb to see if I can figure out what's happening.

Make passing in overrides data as optional

For jezdez/caniusepython3.com#3

Consider making caniusepython3 a script

I.e. don't use py_modules in setup.py but instead use scripts and then simply rename caniusepython3.py to caniusepython3. Since it is fully self-contained and there is no API beyond the CLI the only thing that would be an issue is the tests, but those could even be integrated into the single file.

Otherwise look into using Setuptools' automatic script creation.

Require user specify some argument that leads to specifying projects

Not specifying anything isn't an error right now but leads to pointless output.

Push Git tags to Github

I was just trying to find out whether there are API incompatibilities between 1.2.1 and 2.x as I'm using some of it in caniusepython3.com. Sadly I couldn't find the Git tags for the project. Mind if you push them?

Break out overrides data into own JSON file

Will require making the project a package (which wouldn't be so bad). Can keep it zip_safe by using pkgutil.get_data() to read the JSON file out of the zip file.

Consider switching to Fastly copy of metadata

See https://github.com/jezdez/caniusepython3.com/blob/master/ciupy3/__init__.py on how to do it.

PyPI

Can this be on PyPI?

Error when using distlib==0.2.0

So I'm not sure where the bug is, but I ran into this bug when running caniusepython3:

kevin@myserver:~$ /usr/local/bin/caniusepython3 -r requirements.txt
Finding and checking dependencies ...
Traceback (most recent call last):
  File "/usr/local/bin/caniusepython3", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/caniusepython3/__main__.py", line 159, in main
    check(projects_from_cli(args))
  File "/usr/local/lib/python2.7/dist-packages/caniusepython3/__main__.py", line 145, in check
    blockers = dependencies.blocking_dependencies(projects, pypi.all_py3_projects())
  File "/usr/local/lib/python2.7/dist-packages/caniusepython3/dependencies.py", line 80, in blocking_dependencies
    dist = distlib.locators.locate(project)
  File "/usr/local/lib/python2.7/dist-packages/distlib/locators.py", line 361, in locate
    result.download_urls = versions.get('urls', {}).get(version, set())
AttributeError: 'dict' object has no attribute 'download_urls'

I fixed it by rolling back to distlib==0.1.9, though that gave me these errors

Finding and checking dependencies ...
[WARNING] recaptcha not found  
[WARNING] django-recaptcha-field not found
[ERROR] Failed to get external data for https://www.red-dove.com/pypi/projects/E/email-reply-parser/project.json: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/distlib/util.py", line 744, in _get_external_data
    resp = urlopen(url)
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 444, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found

You need 17 projects to transition to Python 3.
Of those 17 projects, 15 have no direct dependencies blocking their transition:

  django-celery-transactions   
  django-concurrent-server
  django-discover-jenkins
  django-include-strip-tag
  django-ipware
  django-storages
  docx
  hellosign-python-sdk
  oauth2
  pdfminer
  pykss
  reportlab
  lepl (which is blocking rfc6266)
  m2crypto (which is blocking tender_multipass)
  xlsxcessive

Not sure if this is a problem with distlib, my setup, or maybe freezing the required version of distlib in the requirements. Happy to report more as necessary. If it helps, I'm running python 2.7.3 in ubuntu 12.04

File bugs against projects missing trove classifier in overrides.json

~~mox3~~ (can't find source repo)
pbr (code, pull request which will get ignored, pull request by @Steap)
pylint (code, bug)
pyopenssl (code, pull request; also already updated with 2.x classifiers)
python-novaclient (code, bug)
rsa (code, pull request)
zc.recipe.egg (code, pull request)

What about non-existent projects?

Currently you can search for any project and caniusepython3 will report it's Python3 compatible... If you press on "Check again" in the result page will 404 you.

Mention caniusepython3.com in the docs?

I hope that is okay with you, I thought it would help spread the word a bit.

Use unittest2

Now it just seems silly to be working around it not being installed in the tests if I'm going to pull in mock as well.

Identify supported versions from wheel format

Just wondering if it's possible to detect supported version from wheel format.

For example, this https://pypi.python.org/pypi/reportlab doesn't show version in classifier but uploaded wheel shows what versions of Python are supported.

Unknown projects should be considered ported

If someone uses caniusepython3 with some dependencies that are private to their project, we have no way of knowing they are ported, so instead of erroneously saying that they can never switch, but the onus on them and say they can until they move their own private dependencies over.

Parallelize fetching of projects

While the total set of project is unknown and thus can't necessarily make everything concurrent, at least making the fetching of direct dependencies concurrent would be helpful.

[feature] an option to suggest close/drop-in alternative

There is this package called oauth2 which seems to be very popular. Unfortunately it's abandoned and has no activity since 2011 https://github.com/simplegeo/python-oauth2

There are nice alternative packages like this one https://github.com/idan/oauthlib, which are actively developed, as well as popular. Can we somehow add a feature to let people that there is close/drop-in alternative which supports Python 3?

In above example, oauth2 should not be holding people back. What do you think?

Look into making this a Python2/3 compatible package

I think this would go a long way to help switchers to check their Python 2 based projects whether they're ready. Only providing a Python 3 compatible version seems like a bottleneck that can easily be prevented.

Marking packages which have alternatives as supporting Python3 is very confusing

I just spent 10 minutes trying to understand how python-memcached supports python3, before I realized there's actually a fork which does.

It seems to be there should be some sort of alternate presentation of this info.

Drop Python 2.6 support

It's 7 years old, so it's past security fixes and just plain old buggy compared to Python 2.7.

Fix logging format

https://bitbucket.org/pypa/distlib/issue/46/logging-messages-not-being-formatted

Ignore/warn for local packages in requirements.txt

File paths listed in a requirements.txt return a None at some point and cause an AttributeError later on.

Traceback:

$ caniusepython3 -r requirements.txt
Finding and checking dependencies ...
WARNING:root:Stale overrides: set([u'pyopenssl'])
Traceback (most recent call last):
  File "/usr/local/bin/caniusepython3", line 11, in <module>
    sys.exit(main())
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/caniusepython3/__main__.py", line 119, in main
    blockers = ciu.blocking_dependencies(projects, ciu.all_py3_projects())
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/caniusepython3/__init__.py", line 169, in blocking_dependencies
    for project in projects if project.lower() not in py3_projects]
AttributeError: 'NoneType' object has no attribute 'lower'

Make tests less flaky in the face of PyPI timing out

E.g., make https://travis-ci.org/brettcannon/caniusepython3/jobs/85333311 not be a failure.

Use an environment variable to signify changing the name of the command

To use the icanhazpython3 hack, you need to edit setup.py, but that doesn't work if you use pip. Probably better to let people set an environment variable like CIU_ICANHAZPYTHON3 to a true value to flag they want the alternative name used during installation.

Timeouts when querying PyPI

STR:

Clone https://github.com/mozilla/treeherder
Update to revision 985cd86a6de3b33a5025b7152be4f29804456eba
From the root of the repo, run: caniusepython3 -r requirements/common.txt

Expected:
Command succeeds.

Actual:

Finding and checking dependencies ...
Traceback (most recent call last):
  File "/usr/bin/caniusepython3", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/caniusepython3/__main__.py", line 179, in main
    check(projects_from_cli(args))
  File "/usr/lib/python2.7/site-packages/caniusepython3/__main__.py", line 167, in check
    blockers = dependencies.blocking_dependencies(projects, pypi.all_py3_projects())
  File "/usr/lib/python2.7/site-packages/caniusepython3/pypi.py", line 112, in all_py3_projects
    for result in map(projects_matching_classifier, py3_classifiers()):
  File "/usr/lib/python2.7/site-packages/caniusepython3/pypi.py", line 98, in projects_matching_classifier
    for result in client.browse([classifier]))
  File "/usr/lib/python2.7/xmlrpclib.py", line 1240, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1599, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/xmlrpclib.py", line 1280, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1313, in single_request
    return self.parse_response(response)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1490, in parse_response
    return u.close()
  File "/usr/lib/python2.7/xmlrpclib.py", line 799, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "<class 'psycopg2.extensions.QueryCanceledError'>:canceling statement due to statement timeout\n">

Run using Python 2.7.11 on MSYS2.

Add overrides for pyephem

Unfortunately pyephem uses the different name ephem on Python 3. Otherwise it is the exact same project (import ephem). See Python3 Version issue.

To confuse matters further, pyephem now has Python 3 classifiers

Add support for installing this as a wheel

Misidentified projects: boto, botocore, jinja2

I would submit a patch for this, but I'm not sure what the problem is.

boto, botocore, and jinja2 all explicitly support Python 3, but caniusepython3 claims they don't. It doesn't seem appropriate to add them to the overrides because they do have some form of Programming Language :: Python :: 3 classifier.

Override pil with pillow

Fix requirements.txt parsing

Currently the pkg_resources.parse_requirements function is used. Sadly this isn't the correct way to parse pip requirements files (since it's a non-standard, just like pkg_resources).

Either use pip's own functionality (https://github.com/pypa/pip/blob/develop/pip/req/req_file.py#L12) or a 3rd party app like https://github.com/davidfischer/requirements-parser (which probably only implements half of it though)

Add a test for -v

0c146de should never have happened. Shouldn't need to be anything more than verifying that 'INFO' is found in the output.

Outputs `No handlers could be found for logger "ciu"`

Note No handlers could be found for logger "ciu" in the output below. I haven't yet looked into why it's showing up, but it would be nice if it didn't.

$ python setup.py caniusepython3
running caniusepython3
Finding and checking dependencies ...
No handlers could be found for logger "ciu"

You need 3 projects to transition to Python 3.
Of those 3 projects, 2 have no direct dependencies blocking their transition:

  gevent
  paste (which is blocking pastescript)

Add test for not using editable deps from a requirements.txt file

Semantics added in 7c27dfb from @jezdez.

Dynamically fetch valid Python 3 trove classifiers

The guarding code for detecting new releases of Python is somewhat silly when the entire list of trove classifiers can be fetched and then searched to find all valid Python 3 classifiers.

Import caniusepython3.pypi directly

Now necessary from command and dependency thanks to just_name(); otherwise might need to move the function to init.

TypeError: 'int' object is not iterable

I ran caniusepython3 -r requirements.txt against my requirements.txt and the following error resulted:

Traceback (most recent call last):
  File "/usr/local/bin/caniusepython3", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/caniusepython3/__main__.py", line 159, in main
    check(projects_from_cli(args))
  File "/usr/local/lib/python2.7/dist-packages/caniusepython3/__main__.py", line 86, in projects_from_cli
    projects.extend(projects_from_requirements(parsed.requirements))
  File "/usr/local/lib/python2.7/dist-packages/caniusepython3/__main__.py", line 39, in projects_from_requirements
    for req in reqs:
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1631, in parse_requirements
    req = InstallRequirement.from_line(line, comes_from, prereleases=getattr(options, "pre", None))
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 172, in from_line
    return cls(req, comes_from, url=url, prereleases=prereleases)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 108, in __init__
    self.prereleases = any([is_prerelease(x[1]) and x[0] != "!=" for x in self.req.specs])
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 720, in is_prerelease
    return any([any([y in set(["a", "b", "c", "rc", "dev"]) for y in x]) for x in parsed])
TypeError: 'int' object is not iterable

For context, this is on Python 2.7.6 on Ubuntu 14.04.

"""Module Docstring"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

def awesome():
    print("awesome")

Running pylint --load-plugins=caniusepython3.pylint_checker --py3k sample.py --report=no yields the following:

************* Module t5
W:  7, 0: native string literal (native-string)

brettcannon / caniusepython3 Goto Github PK

caniusepython3's Issues

Recommend Projects

Recommend Topics

Recommend Org