darvid / hydrogen Goto Github PK

So I have some thoughts floating around for how I'd like to do this, and a very primitive skeleton of what I think I'd like the structure of Hydrogen to look like is in next.

I'll try not to make this too exhaustive, but I have a few goals and related use cases that would be nice to have, and easier to implement in code that isn't 2 years old, undocumented, and only half working:

• I'd like a framework that allows us to add package managers and requirements specs for different platforms or package managers at any time. I have an initial sketch of this in the next branch, but I'd like to use stevedore and a plugin/driver pattern. This should be minimal effort, once we decide on structure of the BasePackageManager and BasePackageRequirement ABCs.
• I'd like pip-tools integration. I use pip-compile and pip-sync for nearly all my projects now, with what I'd like to think of as a very maintainable pattern consisting of:
  • Adding requirements/ and requirements/src/ directories.
  • Adding a couple of Makefile targets and variables for defining different requirements files and source files. See https://gist.github.com/darvid/3c51f6e67cee0eb8101141e143a683c6
  • Direct dependencies (1st-level) are pinned in the respective src/*.in files. make requirements is run whenever requirements change, or in CI. Both source and "compiled" requirements files are stored in VCS.
  • Obviously this is a very specific and opinionated workflow, and shouldn't be the standard or default way that Hydrogen manages pip requirements. But I'd like some semblance of support for calling the pip-tools API and resolving requirements from a .in file to a .txt file. Ideally the requirements/src/* directory would go away, and you'd just have main/qa/test/whatever arbitrary groups you want in the requirements.yml file, and Hydrogen would build separate requirements files, and optionally resolve dependencies recursively with pip-tools (and generate a sweet annotated requirements file as well).
• Actual Python 3 support, sane packaging, actual unit tests, documentation on RTD, flake8, CI, etc. I can set all these up, it's not a huge amount of effort, but worth mentioning.

In terms of package managers, we should probably add plain old NPM/package.json support in addition to Bower.

/thread

I know I'm making a bunch of noise about a project you've probably not thought about in quite a while, and if you would prefer, I'll be happy to fork this and maintain that, but...

In current HEAD, there exists a problem with creating the default requirements.yml from output of pip freeze, rather than from reading in a requirements.txt. I intend to fix the issue, but I wanted to open this issue to track it.

The setup:
Create an empty directory and cd to it. Initialize an empty virtualenv/pyenv. Run hydrogen install --save pylama.

The problem:
GroupedRequirements.load_pip_requirements calls an instance of Requirements.load using the output of pip freeze as the filename parameter. This fails because Requirements.load attempts to open this parameter.

The fix:
Lolidunno. I'm still learning the code, so I'm not sure what the easiest way to fix this is. Obviously , we need the pip freeze output to instantiate a number of requirements, but I haven't dug deep enough to figure out where that happens yet (or if I need to create a function that does it explicitly for this purpose)