gitpython-developers / gitpython

GitPython is a python library used to interact with Git repositories.

Gitoxide: A peek into the future…

I started working on GitPython in 2009, back in the days when Python was 'my thing' and I had great plans with it. Of course, back in the days, I didn't really know what I was doing and this shows in many places. Somewhat similar to Python this happens to be 'good enough', but at the same time is deeply flawed and broken beyond repair.

By now, GitPython is widely used and I am sure there is a good reason for that, it's something to be proud of and happy about. The community is maintaining the software and is keeping it relevant for which I am absolutely grateful. For the time to come I am happy to continue maintaining GitPython, remaining hopeful that one day it won't be needed anymore.

More than 15 years after my first meeting with 'git' I am still in excited about it, and am happy to finally have the tools and probably the skills to scratch that itch of mine: implement git in a way that makes tool creation a piece of cake for most.

If you like the idea and want to learn more, please head over to gitoxide, an implementation of 'git' in Rust.

(Please note that gitoxide is not currently available for use in Python, and that Rust is required.)

GitPython

GitPython is a python library used to interact with git repositories, high-level like git-porcelain, or low-level like git-plumbing.

It provides abstractions of git objects for easy access of repository data often backed by calling the git command-line program.

DEVELOPMENT STATUS

This project is in maintenance mode, which means that

• …there will be no feature development, unless these are contributed
• …there will be no bug fixes, unless they are relevant to the safety of users, or contributed
• …issues will be responded to with waiting times of up to a month

The project is open to contributions of all kinds, as well as new maintainers.

REQUIREMENTS

GitPython needs the git executable to be installed on the system and available in your PATH for most operations. If it is not in your PATH, you can help GitPython find it by setting the GIT_PYTHON_GIT_EXECUTABLE=<path/to/git> environment variable.

• Git (1.7.x or newer)
• Python >= 3.7

The list of dependencies are listed in ./requirements.txt and ./test-requirements.txt. The installer takes care of installing them for you.

INSTALL

GitPython and its required package dependencies can be installed in any of the following ways, all of which should typically be done in a virtual environment.

From PyPI

To obtain and install a copy from PyPI, run:

pip install GitPython

(A distribution package can also be downloaded for manual installation at the PyPI page.)

From downloaded source code

If you have downloaded the source code, run this from inside the unpacked GitPython directory:

pip install .

By cloning the source code repository

To clone the the GitHub repository from source to work on the code, you can do it like so:

git clone https://github.com/gitpython-developers/GitPython
cd GitPython
./init-tests-after-clone.sh

On Windows, ./init-tests-after-clone.sh can be run in a Git Bash shell.

If you are cloning your own fork, then replace the above git clone command with one that gives the URL of your fork. Or use this gh command (assuming you have gh and your fork is called GitPython):

gh repo clone GitPython

Having cloned the repo, create and activate your virtual environment.

Then make an editable install:

pip install -e ".[test]"

In the less common case that you do not want to install test dependencies, pip install -e . can be used instead.

With editable dependencies (not preferred, and rarely needed)

In rare cases, you may want to work on GitPython and one or both of its gitdb and smmap dependencies at the same time, with changes in your local working copy of gitdb or smmap immediatley reflected in the behavior of your local working copy of GitPython. This can be done by making editable installations of those dependencies in the same virtual environment where you install GitPython.

If you want to do that and you want the versions in GitPython's git submodules to be used, then pass -e git/ext/gitdb and/or -e git/ext/gitdb/gitdb/ext/smmap to pip install. This can be done in any order, and in separate pip install commands or the same one, so long as -e appears before each path. For example, you can install GitPython, gitdb, and smmap editably in the currently active virtual environment this way:

pip install -e ".[test]" -e git/ext/gitdb -e git/ext/gitdb/gitdb/ext/smmap

The submodules must have been cloned for that to work, but that will already be the case if you have run ./init-tests-after-clone.sh. You can use pip list to check which packages are installed editably and which are installed normally.

To reiterate, this approach should only rarely be used. For most development it is preferable to allow the gitdb and smmap dependencices to be retrieved automatically from PyPI in their latest stable packaged versions.

Limitations

Leakage of System Resources

GitPython is not suited for long-running processes (like daemons) as it tends to leak system resources. It was written in a time where destructors (as implemented in the __del__ method) still ran deterministically.

In case you still want to use it in such a context, you will want to search the codebase for __del__ implementations and call these yourself when you see fit.

Another way assure proper cleanup of resources is to factor out GitPython into a separate process which can be dropped periodically.

Windows support

See Issue #525.

RUNNING TESTS

Important: Right after cloning this repository, please be sure to have executed the ./init-tests-after-clone.sh script in the repository root. Otherwise you will encounter test failures.

Install test dependencies

Ensure testing libraries are installed. This is taken care of already if you installed with:

pip install -e ".[test]"

If you had installed with a command like pip install -e . instead, you can still run the above command to add the testing dependencies.

Test commands

To test, run:

pytest

To lint, and apply some linting fixes as well as automatic code formatting, run:

pre-commit run --all-files

This includes the linting and autoformatting done by Ruff, as well as some other checks.

To typecheck, run:

mypy

CI (and tox)

Style and formatting checks, and running tests on all the different supported Python versions, will be performed:

• Upon submitting a pull request.
• On each push, if you have a fork with GitHub Actions enabled.
• Locally, if you run tox (this skips any Python versions you don't have installed).

Configuration files

Specific tools are all configured in the ./pyproject.toml file:

• pytest (test runner)
• coverage.py (code coverage)
• ruff (linter and formatter)
• mypy (type checker)

Orchestration tools:

• Configuration for pre-commit is in the ./.pre-commit-config.yaml file.
• Configuration for tox is in ./tox.ini.
• Configuration for GitHub Actions (CI) is in files inside ./.github/workflows/.

Contributions

Please have a look at the contributions file.

INFRASTRUCTURE

• User Documentation
• Questions and Answers
• Please post on Stack Overflow and use the gitpython tag
• Issue Tracker
  • Post reproducible bugs and feature requests as a new issue. Please be sure to provide the following information if posting bugs:
    • GitPython version (e.g. import git; git.__version__)
    • Python version (e.g. python --version)
    • The encountered stack-trace, if applicable
    • Enough information to allow reproducing the issue

How to make a new release

1. Update/verify the version in the VERSION file.
2. Update/verify that the doc/source/changes.rst changelog file was updated. It should include a link to the forthcoming release page: https://github.com/gitpython-developers/GitPython/releases/tag/<version>
3. Commit everything.
4. Run git tag -s <version> to tag the version in Git.
5. Optionally create and activate a virtual environment. (Then the next step can install build and twine.)
6. Run make release.
7. Go to GitHub Releases and publish a new one with the recently pushed tag. Generate the changelog.

Projects using GitPython

• PyDriller
• Kivy Designer
• Prowl
• Python Taint
• Buster
• git-ftp
• Git-Pandas
• PyGitUp
• PyJFuzz
• Loki
• Omniwallet
• GitViper
• Git Gud

LICENSE

3-Clause BSD License, also known as the New BSD License. See the LICENSE file.

Two files exclusively used for fuzz testing are subject to a separate license, detailed here. These files are not included in the wheel or sdist packages published by the maintainers of GitPython.

gitpython's Issues

Repo.blame don't return multi lines.

According to API reference,

---

The blame information for the given file at the given revision.

Parm rev: revision specifier, see git-rev-parse for viable options.
Returns: list: [git.Commit, list: []] A list of tuples associating a Commit object with a list of lines that changed within the given commit. The Commit objects will be given in order of appearance.

---

blame method returns list of lines but I think the method always returns only one line.
The method pick up the first line of lines which were written in same commit (other lines are skipped and method don't return its information).

The behavior come from the lien 620 of git/repo/base.py.
The variable 'info' is initialized only when a line is first line of same commit.
But 'info' will be None after processing a line (the lien is not commit information).

We have to initialize 'info' to process subsequent lines.

FetchInfo.commit breaks if you fetch a tag to anywhere but refs/tags/

As an example, use a refspec of refs/*:refs/remotes/origin/*. Behavior:

  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/tag.py", line 26, in commit
    obj = self.object
  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/symbolic.py", line 158, in _get_object
    return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/symbolic.py", line 112, in dereference_recursive
    hexsha, ref_path = cls._get_ref_info(repo, ref_path)
  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/symbolic.py", line 139, in _get_ref_info
    raise ValueError("Reference at %r does not exist" % ref_path)
ValueError: Reference at 'refs/tags/766da90/tags/1.0' does not exist

I'd guess there's an implicit assumption that a tag lives in refs/tags/, and the ref path is reconstructed based on that rather than using the correct ref path which the code has already (refs/remotes/766da90/tags/1.0).

cron and pull()

hi im writing a python script which i want to backup my git repositories every day...
so im trying to run it as a cron job under my user profile...
it runs fine from the command line...

although when cron tries to run it i get an error in the syslog: (CRON) error (grandchild failed with exit status 1)..

upon testing it seems is the pull() call where it is having an issue...

my only thought is that it maybe be a permissions/key issue as im trying to pull from a remote...

has anybody come across this issue?

am i doing something wrong?

any help would be great as it would be really cool to automate this...

thanks

Review 'delete' classmethod for refs

Currently references have a class method for deleting them. An instance method would be so much more convenient.
This is quite an api change, so it should be introduced under a new easy to remember name.
Review the types, there might be more which do it like that, and I am not sure where this is coming from. Is it 0.1 heritage ?

Remote Progress Parsing

Parsing of progress when pushing and fetching from remotes is implemented, but it relies on functionality of send-pack and receive-pack which comes and goes as git changes.

For instance, in git 1.6.4 , pushing to a remove over ssh would send progress information, but a subversion later it is gone without replacement as it will check whether stderr is connected to a tty, and not send progress if this is not the case. Fetch progress is implemented as well, but the currently its unknown whether it works properly.

In git 1.7.X, there is an option to receive-pack and send-pack which can enforce the progress information to be sent. During my last tests, I was unable to pass the flags though or they didn't propagate through the commands.

Once the behavior was figured out, tests should be written and improved to assure progress parsing works as expected.

BadObject on repo after git.gc()

Example code:

>>> r = Repo.init('/tmp/foo/', bare=False)
>>> print >> open('/tmp/foo/bar.txt', 'w'), 'some content'
>>> r.index.add(['bar.txt'])
>>> r.index.commit('commit bar.txt')
>>> print r.git.status() # all clear?
# On branch master
nothing to commit (working directory clean)
>>> r.git.gc()
''
>>> r.iter_commits() # kabooom!
...
BadObject: 39154e2f45f96bdbbb86aa4f1ffc72f224b88f31

Tested on GitPython 0.3.2.RC1

os.getlogin() - Inappropriate ioctl for device (mod_wsgi)

Using gitpython from within a mod_wsgi web app results in the following exception when trying to commit:

  File "...GitPython-0.3.2.RC1-py2.6.egg/git/index/base.py", line 887, in commit 
    return Commit.create_from_tree(self.repo, tree, message, parent_commits, head) 
  File "...GitPython-0.3.2.RC1-py2.6.egg/git/objects/commit.py", line 302, in create_from_tree 
    committer = Actor.committer(cr) 
  File "...GitPython-0.3.2.RC1-py2.6.egg/git/util.py", line 354, in committer 
    return cls._main_actor(cls.env_committer_name, cls.env_committer_email, config_reader) 
  File "...GitPython-0.3.2.RC1-py2.6.egg/git/util.py", line 327, in _main_actor 
    default_email = get_user_id() 
  File "...GitPython-0.3.2.RC1-py2.6.egg/git/util.py", line 120, in get_user_id 
    username = os.getlogin() 
OSError: [Errno 25] Inappropriate ioctl for device 

A possible explanation might be that since apache/mod_wsgi are not started by a login shell, no call to setlogin() has been made (in order for getlogin() to succeed a previous call to setlogin() must have been made). Perhaps the problem is in mod_wsgi running in daemon mode. Either way, a more robust get_user_id() function would be nice.

A non-portable solution:

120: username = pwd.getpwuid(os.getuid()).pw_name

These could also be helpful:
http://defect.opensolaris.org/bz/show_bug.cgi?id=3595
http://www.agmweb.ca/blog/andy/2172/ (comment by Graham Dumpleton)

git-submodule handling of branch changes

When the submodule update detects a branch-change, it seems that it tries to remove the branch it has checked out before switching to the new branch, which obviously fails as the currently checked-out branch cannot be deleted. Apparently it tries to clean up, without noticing that it tries to clean up its own branch - it seems to make incorrect assumptions.

Additionally, it should have fetched/updated the submodule before trying to do anything with the paths, as this can trigger new branches to come in (and that you want to switch to subsequently).

To make things crazier: if you tried to help it to get along, it will manage to delete its source branch, but as it still sees the change and tries to switch to a new branch, it will do all the wrong things and try to work with a non-existing branch (the one it deleted).

Changing branch of submodule 'shotgun_events_0.9' from refs/heads/0.9 to refs/heads/custom
GitCommandError: 'git cherry origin/0.9 0.9' returned exit status 128: fatal: Unknown commit 0.9

NOTE: it seems to help if the --to-latest-revision flag is given. However, I would expect it to be clearer when it tries to do what. The -n flag might help here as well.

GitPython 0.3.1 is not installable from PyPI anymore

GitPython 0.3.1 can't be installed anymore, causing dependencies problem for softwares relying on that version of GitPython :

% pip install git-python==0.3.1
Downloading/unpacking git-python==0.3.1
  Could not find any downloads that satisfy the requirement git-python==0.3.1
No distributions at all found for git-python==0.3.1

Assertion fails when fetching with prune=True

I'm occasionally hitting an assertion failure when calling remote.fetch(prune=True). It appears to fail only when some remote-tracking branch is actually pruned. For example:

File "/usr/local/lib/python2.7/dist-packages/GitPython-0.3.2.RC1-py2.7.egg/git/remote.py", line 593, in fetch
return self._get_fetch_info_from_stderr(proc, progress or RemoteProgress())
File "/usr/local/lib/python2.7/dist-packages/GitPython-0.3.2.RC1-py2.7.egg/git/remote.py", line 539, in _get_fetch_info_from_stderr
assert len(fetch_info_lines) == len(fetch_head_info), "len(%s) != len(%s)" % (fetch_head_info, fetch_info_lines)
AssertionError: len("fcf7e70bc1df0fad8b08d088df46c6d778b4579f\tnot-for-merge\tbranch 'master' of github.com:foo/bar\n"]) != len([' 4a2be28..fcf7e70 master -> foo/master', ' x deleted -> foo/release'])

Apparently when the remote-tracking branch release was pruned, it was removed from .git/FETCH_HEAD but was still mentioned in the fetch info lines. Maybe lines starting with ' x [deleted]' should simply be removed from the fetch info lines?

How can I run the test suite?

I tried install.py test, but that jut says running test and nothing more. How is the test suite supposed to be run?

Traceback when ssh VisualHostKey is used

https://bugzilla.redhat.com/show_bug.cgi?id=773640 is where this bug came in.

Description of problem:
If you have VisualHostKey turned on in .ssh/config and you try to access a git
repo via ssh in GitPython, GitPython will fail.

Version-Release number of selected component (if applicable):
GitPython-0.3.2-0.1.RC1.fc14.noarch
(Rebuilt from the current sources in master)

How reproducible:
Everytime

Steps to Reproduce:

1. echo 'VisualHostKey yes' >> ~/.ssh/config
2. git clone ssh://git.fedorahosted.org/git/fas
3. python
4. import git
5. repo = git.Repo('fas')
6. repo.remotes.origin.fetch()

Actual results:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in
repo.remotes.origin.fetch()
File "/usr/lib/python2.7/site-packages/git/remote.py", line 593, in fetch
return self._get_fetch_info_from_stderr(proc, progress or RemoteProgress())
File "/usr/lib/python2.7/site-packages/git/remote.py", line 539, in
_get_fetch_info_from_stderr
assert len(fetch_info_lines) == len(fetch_head_info), "len(%s) != len(%s)"
% (fetch_head_info, fetch_info_lines)
AssertionError: len(["0fd7e783141916845432f94d8ee7c2e8a34c02f9\t\tbranch
'master' of ssh://fedorapeople.org/~toshio/public_git/gitsync\n"]) !=
len(['Host key fingerprint is 07:d4:02:db:9f:70:d5:2d:7f:1b:6a:df:83:73:95:1d',
'+--[ RSA 2048]----+', '| .... .. . |', '| +. .. o .|', '| .
+.. o |', '| = . Eo|', '| S + . B|', '| . o
oo|', '| . o o|', '| o +.|', '| o .|',
'+-----------------+', '', '', ' = [up to date] master ->
origin/master'])

Expected results:
The remote repository is fetched

Using gitpython on windows outside git bash raises an exception

Because "git.cmd" is in PATH and not "git"
I wrote an email to mailing list about it.

The exception:

import git
git.Git().help()

WindowsError Traceback (most recent call last)

C:\Program Files\Support Tools in ()

C:\python\lib\site-packages\gitpython-0.3.1-py2.6.egg\git\cmd.pyc in (_args, *kwargs)
217 if name[:1] == '':
218 raise AttributeError(name)
--> 219 return lambda _args, *_kwargs: self._call_process(name, _args, *_kwargs)
220
221 @Property

C:\python\lib\site-packages\gitpython-0.3.1-py2.6.egg\git\cmd.pyc in _call_process(self, method, _args, *_kwargs)
428 call.extend(args)
429
--> 430 return self.execute(call, **_kwargs)
431
432 def _parse_object_header(self, header_line):

C:\python\lib\site-packages\gitpython-0.3.1-py2.6.egg\git\cmd.pyc in execute(self, command, istream, with_keep_cwd, with_extended_output, with_exceptions, as_process, output_stream, *_subprocess_kwargs)
307 stdout=PIPE,
308 close_fds=(os.name=='posix'),# unsupported on linux
--> 309 *_subprocess_kwargs
310 )
311 if as_process:

C:\python\python26.zip\subprocess.py in init(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags)
593 p2cread, p2cwrite,
594 c2pread, c2pwrite,
--> 595 errread, errwrite)
596
597 if mswindows:

C:\python\python26.zip\subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite)
802 env,
803 cwd,
--> 804 startupinfo)
805 except pywintypes.error, e:
806 # Translate pywintypes.error to WindowsError, which is

Fix

Changing line 427 in cmd.py from "git" to "git.cmd" solved this locally.
Need a better solution, e.g.

• try: except WindowError:
• check if platform.system() == "Windows"
• search for git or git.cmd in PATH
• etc.

Incorrect handling of backslashes in Git configuration

Steps to reproduce the issue:

import git
config = git.Repo().config_writer()
config.add_section('test')
config.set_value('test', 'test', r'some\data')

Now if you try to read this value using a regular (non Python) git config, Git complains that the configuration file is invalid:

fatal: bad config file line 11 in .git/config

Indeed, if you open .git/config you can see that the value is written as:

[test]
  test = some\data

While the git-config configuration states that:

String values may be entirely or partially enclosed in double quotes. You need to enclose variable values in double quotes if you want to preserve leading or trailing whitespace, or if the variable value contains comment characters (i.e. it contains # or ;). Double quote " and backslash \ characters in variable values must be escaped: use " for " and \ for .

That is, the backslashes are not escaped in the configuration file.

This also cause issues in while reading, because values are not un-escaped.

Some files are never closed

I'm working on a script using GitPython to loop over lots of Git repos, pull remote changes, etc...

At some point, I noticed that doing repo.is_dirty() was opening some files which never got closed until the process exits, which in turns causes the tool to crash with "too many open files" after iterating over enough repos:

import os

import git

# Put here whatever Git repos you might have in the current folder
# Listing more than one makes the problem more visible
repos = ["ipset", "libhtp"]

raw_input("Check open files with `lsof -p %s | grep %s`" % (os.getpid(),
                                                            os.getcwd()))

for name in repos:
    repo = git.Repo(name)
    repo.is_dirty()

    del repo

raw_input("Check open files again")                 # files are still open

I tried digging deeper down the GitPython stack, but couldn't find the actual cause.

In case that's helpful, below is the same as the previous snippet, but using directly the lowest-level gitdb objects I could find to open the files:

import os

import git
from gitdb.util import hex_to_bin

# Put here whatever Git repos you might have in the current folder
# Listing more than one makes the problem more visible
repos = ["ipset", "libhtp"]

raw_input("Check open files with `lsof -p %s | grep %s`" % (os.getpid(),
                                                            os.getcwd()))

raw_input("Check open files again")                 # files are still open

for name in repos:
    repo = git.Repo(name)
    sha = hex_to_bin("71acab3ca115b9ec200e440188181f6878e26f08")

    for db in repo.odb._dbs:
        try:
            for item in db._entities:               # db._entities opens *.pack files
                item[2](sha)                        # opens *.idx files

        except AttributeError as e:
            # Not all DBs have this attribute
            pass

    del repo

raw_input("Check open files again")                 # files are still open

But that's just where the files are open (well, in fact they are open even lower down but I didn't manage to go deeper), not where they are supposed to be closed.

remove async dependency from gitpython

Considering that async is difficult to use in a globally locked python, it would be best to remove its dependency and the complexity that comes with it.

The single- and multi-item API should stay, if possible, but only at moderate costs.

Brokenness fetching from a github repo which has merged a pull request

Try fetching from, for example, git://github.com/kergoth/oe-core-template. The reason it blows up is as far as I can tell, the remote end has a ref (not a tag) which is outside of refs/heads/ and refs/tags/ (it's refs/pull/1/head and refs/pull/1/merge). When it fetches these remotes, the fetch_line has no ref type specified at all:

9e30b7d61417455edbbc4617eb9eef9341255bb5 not-for-merge 'refs/pull/1/merge' of git://github.com/kergoth/oe-core-template

And results in, as you'd expect:

File "/home/kergoth/.local/lib/python2.7/site-packages/GitPython-0.3.1-py2.7.egg/git/db/cmd/base.py", line 119, in
for err_line,fetch_line in zip(fetch_info_lines, fetch_head_info))
File "/home/kergoth/.local/lib/python2.7/site-packages/GitPython-0.3.1-py2.7.egg/git/db/cmd/base.py", line 442, in _from_line
raise TypeError("Cannot handle reference type: %r" % ref_type_name)
TypeError: Cannot handle reference type: "'refs/pull/1/head'"

Config file cannot be parsed if spaces are used instead of tabs

Apply another monkey-patch to not only deal with tabs, but with spaces too.

'NoneType' object has no attribute 'groups'

Exception Type: AttributeError
Exception Value:
'NoneType' object has no attribute 'groups'
Exception Location: /usr/local/lib/python2.6/dist-packages/git/diff.py in _index_from_patch_format, line 301
Python Executable: /usr/bin/python
Python Version: 2.6.6

this happens when trying to "Swap two inputs" .

commit = repo.commit()
diff = commit.diff('HEAD~1', create_patch=True, R=True)

Incorrect parsing of git diff output for mode changes

I'm using GitPython-0.3.2.RC1-py2.7, with git version 1.7.4.1, on linux.

In my code I do something like:
diff_list = old_commit.diff(new_commit, create_patch=True, w=True)

With GIT_PYTHON_TRACE set to 1 I see this executes:
git diff -w edeca5290f8a3dbe7d78375fc061de47e7d626bc 413a379a86a63a3bba970c52c56cfa3fa2183907 --abbrev=40 --full-index -p -M

git produces some output, an interesting part of which is:

...
+
+
 if __name__ == "__main__":
     main()
\ No newline at end of file
diff --git a/src/app/names.pyw b/src/app/names.pyw
old mode 100644
new mode 100755
diff --git a/src/lib/pyccp/config/lapdesk.py b/src/lib/pyccp/config/lapdesk.py
new file mode 100644
index 0000000000000000000000000000000000000000..f870d902f92ac6101f8f4d8c5d4e063c78a053a1
--- /dev/null
+++ b/src/lib/pyccp/config/lapdesk.py
@@ -0,0 +1,50 @@
+# -*- coding: utf-8 -*-
+'''
+Script Name : lapdesk.py
...

The diff object that should represent the names.pyw entry, when printed, looks like:

%s
==========
lhs: None
rhs: None

I notice that this doesn't have an "index" line - could this be the issue?

Handle indented comments in git config files

It's perfectly valid for a git config file to contain comments inside
sections, e.g.:

[core]
        # This is a shared repository
        sharedRepository = true

The git tools all parse this fine while GitPython was choking on it.

Error in documentation http://packages.python.org/GitPython/0.2/tutorial.html

Query the active branch, query untracked files or whether the repository data has been modified:
Next to that in code block
repo.is_dirty() is supposed to be repo.is_dirty becasue repo.is_dirty is bool object cannot be callable.

Archive method does not support paths parameter

In the following code you are not able to supply what paths you would like to archive (and not all the repo).

repo = git.Repo("/git/repo")
repo.archive(ostream=temparchive, treeish="master", format="tar")

Meanwhile, if you use the git archive command you can suplly what paths you would like to archive.
An extra kw args could be the list of paths you want in the tar file.

Do diffs detect renamed files properly?

I'm using following code for detecting changes between commits:

diff = commit.diff(*parents, R = True, M = True)

Everything works ok in except of renames. When I use M = True it should detect renames instead of add / deletions. However, when I use it, it doesn't return anything. Is this a bug or did I just misunderstand how to use it?

The thing I noticed in git log -M is that instead of R it shows me R100 (not really sure whether it's normal or not):

commit a8b61b9d18350a3d44e50fc9614279a2654b646d
Author: ondrowan <...>
Date: Sat Nov 5 12:16:21 2011 +0100

Moved thing

R100 thing newdir/moved_thing

Submodule Support

Add a new Submodule module and class which encapsulates a git submodule and allows it to be handled.

The class needs to be designed first based on the current implementation, but feel free to improve on the existing git submodule design if possible.

When working on it, don't forget to update the Tree class to return your submodule type when it is being traversed ( see git.objects.Tree._iter_from_data ).

Note: This ticket involves quite some amount of work, and it might be worth splitting it into sub-tasks.

Docs Review

Overhaul docs by check examples, check looks, check for typos. The docs review should be done before any release is getting official.

Also make sure all api docs are being generated, hence some resT syntax errors need fixing.

Silently opens repo for ancestor directory

Let /git be a git repo, suppose path /git/foo/bar exists (file or directory).

>>> git.Repo('/git/foo/bar')
<git.Repo "/git/.git">

I think this is very dangerous behaviour. For example, in a testing environment one might use

shutil.rmtree(some_repo.working_dir)

where, if some_repo was subject to the behaviour above, created from a path in a project's working directory, unhappiness might ensue.

Unable to fetch a remote tracking ref from a remote

_from_line in Remote blows up, as it can't handle ref type of "remote-tracking". Easily fixed by handling them the same as the ref_type_name "branch", as far as I can tell.

Error while calling is_dirty on repo on a windows machine

Check the following gist to understand the problem: http://bit.ly/LH4QKd

Review Exception Types

Currently, whenever something git-specific happens, git-python will throw a more or less generic Python exception, such as ValueError or TypeError, including a very git specific message though.

Client Code is less readable that way, as it will handle these general exceptions very specifically although this is not indicated by the exception type.

Review the all raise statements and introduce custom Exception types which are derived from the type you currently see. This way client code will be more readable without breaking existing code.

IndexFile.diff without default index

Implement IndexFile diff so that temporary indices can be used as well ( using file swapping ).

Typo in the documentation

In tutorial.rst

new_repo = repo.init("path/for/new/repo")

should be

new_repo = Repo.init("path/for/new/repo")

git-submodule: implement keep_going flag

Sometimes, there is a submodule which we do not control, and which will fail to update or to be handled in any way. In that case, it might be suitable to just ignore it and keep going.
Such a flag should be implemented in a way which catches all repository errors on a high-level and logs a warning accordingly.

Object cache inconsistent after fetch

If I create a new remote, call remote.fetch() and then try to obtain one of the newly-fetched commits by its sha, I get a BadObject exception deep inside gitdb:

Traceback (most recent call last):
File "/home/foobar/ggit/ggit/repo.py", line 134, in foreach
results[i] = fn(items[i], m, _args, *_kargs)
File "/home/foobar/ggit/gg", line 185, in mergepull
sb = repo.commit(headRev)
File "/usr/local/lib/python2.7/dist-packages/GitPython-0.3.2.RC1-py2.7.egg/git/repo/base.py", line 378, in commit
return self.rev_parse(str(rev)+"^0")
File "/usr/local/lib/python2.7/dist-packages/GitPython-0.3.2.RC1-py2.7.egg/git/repo/fun.py", line 151, in rev_parse
obj = name_to_object(repo, rev[:start])
File "/usr/local/lib/python2.7/dist-packages/GitPython-0.3.2.RC1-py2.7.egg/git/repo/fun.py", line 89, in name_to_object
return Object.new_from_sha(repo, hex_to_bin(hexsha))
File "/usr/local/lib/python2.7/dist-packages/GitPython-0.3.2.RC1-py2.7.egg/git/objects/base.py", line 64, in new_from_sha
oinfo = repo.odb.info(sha1)
File "/usr/local/lib/python2.7/dist-packages/gitdb-0.5.4-py2.7-linux-x86_64.egg/gitdb/db/base.py", line 256, in info
return self._db_query(sha).info(sha)
File "/usr/local/lib/python2.7/dist-packages/gitdb-0.5.4-py2.7-linux-x86_64.egg/gitdb/db/base.py", line 243, in _db_query
raise BadObject(sha)
BadObject: BadObject: 4634b83087e4744ee980a17a63952412374b1b95

I can work around the problem by calling repo.odb.update_cache() after the fetch but before looking up the commit.

It would be nice if Remote.fetch() did this automatically to ensure cache consistency.

How do you deal with commit that are not utf-8 ?

When trying to get commit with fuc*** french caracter : é or è
Then i've got some python error as it seems to think that de the commit.message is in utf8.

How can i deal with that ?

IndexFile.remove: shell command length limit might be hit

On windows, there can be a command line length overflow as the shell might be used there to make the git command call as we pass the paths directly as argv. This is as we use git-rm to be able to remove whole directories easily.

Alternatively, the index could be manipulated in-python or using git-update-index, and the working tree adjustments would have to be hand implemented then which might be some work to get it right ( and well tested ).

GitPython should tolerate comparisons between None and Commits

Using GitPython 0.3.1, I find myself often catching this exception :

  File "/home/mike/.virtualenvs/qgb/lib/python2.6/site-packages/GitPython-0.3.1-py2.6.egg/git/objects/base.py", line 80, in __eq__
    return self.binsha == other.binsha
AttributeError: 'NoneType' object has no attribute 'binsha'

It occurs when I try to compare a None object to a commit. I think this could be enhanced in objects/base.py by returning False if "other" has no 'binsha' attribute.

Prints to stderr from library

git/objects/commit.py has a few places where it will print to stderr if unable to decode a stream from the repo. This has ill effect on client callers to the library. In my case, I'm making use of GitPython in a git update hook, and this stderr print is making its way back to the user's console and confusing them when they push.

Can we make these prints optional?

Unable to use GitPython with Gevent.

Hi, I'm trying to use GitPython in a gevent project and it would seem that I am unable to.
Here is an example of the problem.

Thanks,
Dan

Assertion on creating or adding files in bare repositories

I can create new files from in-memory bytestreams in non-bare
repositories easily enough:

        repo = git.Repo (repopath)
        istream = IStream ("blob", len (filedata), StringIO (filedata))
        repo.odb.store (istream)
        blob = git.Blob (repo, istream.binsha, 0100644, filename)
        repo.index.add ([git.IndexEntry.from_blob (blob),])
        repo.index.commit (comment)

However this fails with bare repositories with an assertion error
around repo.working_tree_dir.

/home/allura/scm/git/p/jcltest/code.git/ in ()
----> 1 repo.index.add ([git.IndexEntry.from_blob (blob),])

/home/allura/anvil/local/lib/python2.7/site-packages/GitPython-0.3.0_beta2-py2.7.egg/git/index/util.pyc in
set_git_working_dir(self, _args, *_kwargs)
73 def set_git_working_dir(self, _args, *_kwargs):
74 cur_wd = os.getcwd()
---> 75 os.chdir(self.repo.working_tree_dir)
76 try:
77 return func(self, _args, *_kwargs)

/home/allura/anvil/local/lib/python2.7/site-packages/GitPython-0.3.0_beta2-py2.7.egg/git/repo/base.pyc in working_tree_dir(self)
174 :raise AssertionError: If we are a bare repository"""
175 if self._working_tree_dir is None:
--> 176 raise AssertionError( "Repository at %r is bare and does not have a working tree directory" % self.git_dir )
177 return self._working_tree_dir/home/allura/scm/git/p/jcltest/code.git/ in ()
----> 1 repo.index.add ([git.IndexEntry.from_blob (blob),])

/home/allura/anvil/local/lib/python2.7/site-packages/GitPython-0.3.0_beta2-py2.7.egg/git/index/util.pyc in set_git_working_dir(self, _args, *_kwargs)
73 def set_git_working_dir(self, _args, *_kwargs):
74 cur_wd = os.getcwd()
---> 75 os.chdir(self.repo.working_tree_dir)
76 try:
77 return func(self, _args, *_kwargs)

/home/allura/anvil/local/lib/python2.7/site-packages/GitPython-0.3.0_beta2-py2.7.egg/git/repo/base.pyc in working_tree_dir(self)
174 :raise AssertionError: If we are a bare repository"""
175 if self._working_tree_dir is None:
--> 176 raise AssertionError( "Repository at %r is bare and does not have a working tree directory" % self.git_dir )
177 return self._working_tree_dir

diff does not show changed lines

I have a problem with the diff capabilities of 03.2.RC1:

If I do repo.git.diff('origin/master') it prints a nice diff:
diff --git a/base_settings.py b/base_settings.py
index d6c2ddb..c572c31 100644
--- a/base_settings.py
+++ b/base_settings.py
@@ -22,7 +22,7 @@ DATABASES = {
}
}

-TIME_ZONE = 'Europe/Zurich'
+TIME_ZONE = 'America/Chicago'

 LANGUAGE_CODE = 'en'

diff --git a/generated_requirements.txt b/generated_requirements.txt
index e69de29..f77b16b 100644
--- a/generated_requirements.txt
+++ b/generated_requirements.txt
@@ -0,0 +1 @@
+cmsplugin-flickr==0.1.2
\ No newline at end of file
diff --git a/settings.py b/settings.py
index 73aa9b1..e72486a 100644
--- a/settings.py
+++ b/settings.py
@@ -1 +1,14 @@
 from base_settings import *
+FLICKR_API_KEY = "132"
+FLICKR_API_SECRET = "123"
+SECRET_KEY = "rEWcYWl8KngUNSaELWKDqm2RGod3CYlEHmNCf5rwPj2D52BBYIYKcJt7yAPuJh0R"
+LANGUAGES = [
+    ("en", "English"),
+    ("de", "German")
+]
+SIMPLE_SSO_SECRET = "UJ7zUNDJxzEWr8tmBLPy1T9LfhWaS7rIdcHUCUF7l5naoN42amDNy8CK6x7UNMtO"
+SIMPLE_SSO_KEY = "FBUQ21ULQsYfLGJgbchGPQc9uDozXJ73mtW4g9ob517BWOA1ZYQGG9kmjyW7zaOV"
+SIMPLE_SSO_SERVER = "http://192.168.2.69:8000/auth/"
+INSTALLED_APPS += [
+    "cmsplugin_flickr"
+]
\ No newline at end of file

If I run:

for d in repo.index.diff(repo.remotes.origin.refs.master.commit):
    print d
    print d.diff

I do not get the lines changed:

base_settings.py
=======================================================
lhs: 100644 | c572c313d74e7d2df63704e4712d782fd7e731e8
rhs: 100644 | d6c2ddbea1926853b82c43ecd21e2088735ca32f

generated_requirements.txt
=======================================================
lhs: 100644 | f77b16bda1cf332bb17baaa02abb74cac51c5374
rhs: 100644 | e69de29bb2d1d6434b8b29ae775ad8c2e48c5391

settings.py
=======================================================
lhs: 100644 | e72486aea3e660427e9eb77310c7ab3682d277ec
rhs: 100644 | 73aa9b146ba9f7e49eb43b1f7e3d190e7baf871b

Is this a bug or am i using it wrong?

Improve IterableList Type

Adjust git.utils.IterableList to be more convenient.

Implement the 'contains' method to allow something like:
"upsteam" in repo.heads

Concurrency Safety Review

make sure the ODB gets locks on resources it needs before altering them, assuring multiple instances won't step onto each others feet. The LockFile implementation used should be configurable ( i.e. blocking vs. failing ).

Review the index as well - it is now possible to change it in memory, and these changes should be marked by start-stop calls that end with writing the index back to disk, maybe, to simulate some sort of transaction.

git.GitCmdObjectDB consuming system's resources

Using GitPython==0.3.2.RC1 with Python 2.7.1 on a Fedora 15 with "/etc/security/limits.conf" nproc set to 25000 for the user running the test.

After repetitive calls to different repositories using git.GitCmdObjectDB, the system resources decrease reaching a "OSError: [Errno 24] Too many open files" exception.

This does not happen with the default odbt (gitdb).

Digging into the problem, lsof outputs lots of "pipe" processes. Possibly, the calls to git command using subprocess modure aren't properly deallocated.

Is this a bug?

Thanks,

Handle Too Many Open Files

Currently the gitdb will memory map pack indices, and loose objects for reading. If there are too many mapped files, which could happen on large databases, one has to gracefully unload existing memory maps and reload them later.
This could be as easy as deleting the data caches, which are memory maps most of the time.

This task could be done by the PackedDB, and additionally by the git-db which handles multiple object databases.

TestSystem: Coverage Report

Enable easy use of nosetests --with-coverage and make it a part of the development workflow.

How about integrating the generated html coverage report into the documentation as done in the MRV Development Framework ?

Empty (bare) repository causes error

Hi,

I cannot instanciate a Repo object with a bare repository:

progval@Andromede:~/repositories$ python
Python 2.7.3rc2 (default, Mar 21 2012, 06:59:11) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import git
>>> git.Repo('testing.git').tree()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 399, in tree
    return self.head.commit.tree
  File "/usr/local/lib/python2.7/dist-packages/git/refs/symbolic.py", line 168, in _get_commit
    obj = self._get_object()
  File "/usr/local/lib/python2.7/dist-packages/git/refs/symbolic.py", line 161, in _get_object
    return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
  File "/usr/local/lib/python2.7/dist-packages/git/refs/symbolic.py", line 115, in dereference_recursive
    hexsha, ref_path = cls._get_ref_info(repo, ref_path)
  File "/usr/local/lib/python2.7/dist-packages/git/refs/symbolic.py", line 142, in _get_ref_info
    raise ValueError("Reference at %r does not exist" % ref_path)
ValueError: Reference at 'refs/heads/master' does not exist

Regards,
Valentin Lorentz

Git-Helper-Command-Termination Windows

git commands on windows may only be killed using the /F options which probably is like SIGKILL. This is bad as the process might be doing something, leaving the resource locked and/or in an inconsistent state. Without the /F option, git does not terminate itself, probably it listens to SIGINT which is not sent by TASKKILL.
We can't really help it, but may be at some point git handles windows signals properly.

See whether its git that doesn't do it right, or whether there is a way to send different signals on windows that git will understand.

Diff fails on empty repository.

If you have an empty repository (git init in an empty folder), then add files through GitPython, the repository isn't registered as "dirty" and diffs come back empty. For example:

>>> r = Repo('.')
>>> r.index.add(["README"])
>>> r.is_dirty
False
>>> r.index.diff(None)
[]

The index.entries property shows the newly added file, but there doesn't seem to be a way to turn it into "this file has been added, but not committed".

FetchInfo.commit breaks if you fetch a ref to a location outside of refs/remotes/

Example with a refspec of refs/remotes/origin/tags/*:refs/tags/*:

  File "/home/kergoth/.local/lib/python2.6/site-packages/git/remote.py", line 183, in commit
    return self.ref.commit
  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/symbolic.py", line 165, in _get_commit
    obj = self._get_object()
  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/symbolic.py", line 158, in _get_object
    return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/symbolic.py", line 112, in dereference_recursive
    hexsha, ref_path = cls._get_ref_info(repo, ref_path)
  File "/home/kergoth/.local/lib/python2.6/site-packages/git/refs/symbolic.py", line 139, in _get_ref_info
    raise ValueError("Reference at %r does not exist" % ref_path)
ValueError: Reference at 'refs/remotes/1.0' does not exist

In this case, I fetched a tag with a refspec rather than the ordinary tag fetching, but as you can see from the error, the code looks like it assumes that any fetched ref has to live in refs/remotes, which isn't the case. I suspect this would also occur fetching an ordinary branch to a path outside of refs/remotes/, but I haven't yet tested that case.