felipec / git-remote-bzr

Transparent bidirectional bridge between Git and Bazaar for Git

'git-remote-bzr' is the semi-official Bazaar bridge from the Git project, once installed, it allows you to clone, fetch and push to and from Bazaar repositories as if they were Git ones:

git clone "bzr::lp:ubuntu/hello"

To enable this, simply add the 'git-remote-bzr' script anywhere in your $PATH:

wget https://raw.github.com/felipec/git-remote-bzr/master/git-remote-bzr -O ~/bin/git-remote-bzr
chmod +x ~/bin/git-remote-bzr

That’s it :)

Obviously you will need Bazaar installed.

Notes

Remember to run git gc --aggressive after cloning a repository, specially if it’s a big one. Otherwise lots of space will be wasted.

The oldest version of Bazaar supported is 2.0. Older versions are not tested.

Branches vs. repositories

Bazaar’s UI can clone only branches, but a repository can contain multiple branches, and 'git-remote-bzr' can clone both.

For example, to clone a branch:

git clone bzr::bzr://bzr.savannah.gnu.org/emacs/trunk emacs-trunk

And to clone the whole repository:

git clone bzr::bzr://bzr.savannah.gnu.org/emacs emacs

The second command would clone all the branches contained in the emacs repository, however, it’s possible to specify only certain branches:

git config remote-bzr.branches 'trunk, xwindow'

Some remotes don’t support listing the branches contained in the repository, in which cases you need to manually specify the branches, and while you can specify the configuration in the clone command, you might find this easier:

git init emacs
git remote add origin bzr::bzr://bzr.savannah.gnu.org/emacs
git config remote-bzr.branches 'trunk, xwindow'
git fetch

Caveats

Limitations of the remote-helpers' framework apply. In particular, these commands don’t work:

• git push origin :branch-to-delete

• git push origin old:new (it will push 'old') (patches available)

• git push --dry-run origin branch (it will push) (patches available)

Other projects

There are other 'git-remote-bzr' projects out there, do not confuse this one, this is the one distributed officially by the Git project:

• bzr-git’s git-remote-bzr

• lelutin 's git-remote-bzr

For a comparison between these and other projects go here.

Contributing

Send your patches to the mailing list [email protected] (no need to subscribe).

git-remote-bzr's Issues

Track multiple remote bzr branches in one git repository

Hello,

Following the @Robru reply on #8, I can not track several remote bzr branches in my git repository.

I expected the git remote url to point to a remote bzr repository, under witch I can push my branches.

I expected something like git push perso feature/something to concatenate the git config remote.perso.url with the branch name to build the bzr URL.

It could be great to be able to track several remote branches within one git repository, and do things like:

Start by a clone:

$ git clone bzr::lp:something
Cloning into 'something'...
Checking connectivity... done.

Create a test branch

$ git checkout -b test master
$ echo "Test file" > test
$ git add test
$ git commit -m "Test commit"

Create my personal bzr remote repository and push my test branch

$ git remote add perso bzr::lp:~mylogin/something
$ git push -u perso test
<push the local test branch to remote bzr::lp:~mylogin/something/test>

Merge someone else feature:

$ git remote add someoneelse bzr::lp:~someoneelse/something
$ git fetch someoneelse
$ git merge someoneelse/feature/something
$ git push perso test

Regards.

Create a new remote bzr branch when it doesn't exist on push

When pushing, if the remote bzr branch does not exist, this error appears:

$ git push bzr-dequis bzr-test
Traceback (most recent call last):
  File "/home/dx/bin/git-remote-bzr", line 981, in <module>
    sys.exit(main(sys.argv))
  File "/home/dx/bin/git-remote-bzr", line 967, in main
    do_export(parser)
  File "/home/dx/bin/git-remote-bzr", line 682, in do_export
    branch = get_remote_branch(name)
  File "/home/dx/bin/git-remote-bzr", line 787, in get_remote_branch
    remote_branch = bzrlib.branch.Branch.open(branches[name],
KeyError: 'bzr-test'
error: failed to push some refs to 'bzr::bzr+ssh://dequis.org/home/dx/web/bzr/bitlbee'

I tried patching it, my (failed) attempt is at dequis@7365028 - a bit hacky (getting url from sys.argv[2]) and doesn't really work, other than creating an empty branch that can't get pushed to, while never reporting any errors on "git push":

$ git push bzr-dequis bzr-test
To bzr::bzr+ssh://dequis.org/home/dx/web/bzr/bitlbee
   bf4be5b..bf4be5b  bzr-test -> bzr-test

That short commit hash is my current HEAD commit. It shows this exact same output every time, even if the remote doesn't exist or after running it twice in a row.

The bzr log output in the bzr-test directory in the remote repo is empty.

The bzr log output in .git/bzr/bzr-dequis/clone/bzr-test shows 1046 revisions, the last one being HEAD.

I have no idea what I'm doing.

does not handle conversion of file into symlink

git-remote-bzr doesn't appear to handle conversion of a file into a symlink.

I've set up a simplified test case bzr repository which exhibits the issue:

bzr branch http://people.debian.org/~vagrant/bugs/731671-testcase
git clone bzr::731671-testcase 731671-testcase-git

ls -l 731671-testcase*
731671-testcase:
total 4
lrwxrwxrwx 1 vagrant vagrant 3 Jun 8 23:22 bar -> foo
-rw-r--r-- 1 vagrant vagrant 20 Jun 8 23:22 foo

731671-testcase-git:
total 8
-rw-r--r-- 1 vagrant vagrant 20 Jun 8 23:23 bar
-rw-r--r-- 1 vagrant vagrant 20 Jun 8 23:23 foo

This appears very similar to the bug reported on the git-bzr package in Debian (which was recently removed from Debian):

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731671

Debian packaging

I have created a rudimentary Debian packaging directory for as much of what I can grok from the HOWTOs, and added a make package rule.

git clone --depth 1 (or getting a specific rivision) doesn't work

git clone --depth 1 doesn't work. It should have method to get a specific revision which will be passed as an argument to bzr.

Example: git clone bzr::lp:~terminix -r 344

Thanks.

can't push to older bzr formats (pack-0.92)

It was reported in https://bugs.debian.org/714038 that git-remote-bzr failed to push to a repository with an older bzr format (pack-0.92). Cloning the repository worked without issues.

I think the relevent error message is:

bzrlib.errors.IncompatibleRepositories: RemoteRepository(bzr+ssh://bazaar.launchpad.net/%2Bbranch/foxtrotgps/.bzr/)
is not compatible with
CHKInventoryRepository('file:///home/pabs/devel/misc/foxtrotgps/.git/bzr/.bzr/repository/')
different rich-root support

port from bzr to breezy

Forwarding https://bugs.debian.org/923096

Canonical has mostly abandoned bzr and a new fork called breezy has replaced it. Please port git-remote-bzr to breezy so that distros that have transitioned to breezy can include git-remote-bzr.

pushing tags not supported

git-remote-bzr seems to import tags from bzr, but doesn't seem able to push them to bzr:

git push --tags
To bzr::/tmp/tmp.Uz3tBCm7B1/test
! [remote rejected] test-tags -> test-tags (pushing tags not supported)

It's clearly a known issue, don't know how feasible it would be to implement.

It would definitely be nice, as tagging is maybe the only thing I still need to call bzr directly for...

Thanks so much for git-remote-bzr!

live well,
vagrant

Cloning a local Bazaar repo fails: "This version of fast-import does not support feature done"

Z:\>git version --build-options
git version 2.10.1.windows.1
sizeof-long: 4
machine: x86_64

Z:\>python --version
Python 2.7.12

Z:\>git clone -v "bzr::file:///Z:/Bazaar/" Bzr2Git
Cloning into 'Bzr2Git'...
.atal: This version of fast-import does not support feature done
fast-import: dumping crash report to Z:/Bzr2Git/.git/fast_import_crash_2264
fatal: Error while running fast-import

Z:\>Traceback (most recent call last):
  File "Z:\git-remote-bzr", line 1022, in <module>
    sys.exit(main(sys.argv))
  File "Z:\git-remote-bzr", line 1006, in main
    do_import(parser)
  File "Z:\git-remote-bzr", line 437, in do_import
    export_branch(repo, name)
  File "Z:\git-remote-bzr", line 359, in export_branch
    modified_final = export_files(cur_tree, modified)
  File "Z:\git-remote-bzr", line 277, in export_files
    print d
IOError: [Errno 22] Invalid argument
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

fast-import crash report:
    fast-import process: 2264
    parent process     : 1
    at 2016-10-17 10:24:47 +0000

fatal: This version of fast-import does not support feature done
.

Most Recent Commands Before Crash
---------------------------------
* feature done

Active Branch LRU
-----------------
    active_branches = 0 cur, 5 max

  pos  clock name
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Inactive Branches
-----------------

Marks
-----

-------------------
END OF CRASH REPORT

Speeding up import of branches?

First, thanks for your excellent work!

I am using this to mirror all branches from merge-proposals for my project from launchpad to GitHub.

I have two directories:

• bzr_repo contains a shared repository made with bzr init-repo. I check out the branches I want to mirror into this.
• git_repo contains the git repository that pushes to https://github.com/widelands/widelands.

Let's say I want to mirror the branch my_branch. My tool first branches the launchpad source branch into bzr_repo/my_branch. I then execute git remote add my_branch bzr::../bzr_repo/my_branch in the git repository. Doing a fetch on this new remote then triggers the import with continuous updates: "progress revision 4600 ....". This takes a long time, roughly 2 hours.

Now comes my question: If I repeat the same process with another branch my_branch2 that only differs by one commit, the git fetch takes exactly as long again. bzr branch is clever and understands that little needs to be fetched and after the process has terminated, git also understands that only one commit is different (judging by the gitk branch relationship graphs).

Is there no way to speed up importing of my_branch2?

How to get it work with a bzr repository that hasn't been pushed to launchpad and only resides on my computer

git clone "bzr:file://~/Documents/Projects/myapp" doesn't work.
It says : fatal: I don't handle protocol 'bzr:file'

How to fetch different branches from launchpad?

Maybe this is a bug, or maybe I am doing something wrong. For example with the project lp:do

git clone bzr::lp:do
cd do
git fetch bzr::lp:~psybers/do/do-vala

I get:

fatal: bad object 0000000000000000000000000000000000000000
error: bzr::lp:~psybers/do/do-vala did not send all necessary objects

Allow passing options to bzr

In order to fetch a bzr repository on a host with self-signed SSL certificates, I have to do the following with bzr:

bzr -Ossl.cert_reqs=none branch https://URL NAME

However, there does not seem to be a way to clone such a repository with git-remote-bzr. I tried adding the option to the "git clone" call, but that just resulted in git complaining.

Add a configure or setuptools/setup.py script to check for bzr

It'd be more elegant to check whether bzr is installed in a build script. Due to the fact that the Makefile doesn't contain any too complicated commands and dependencies it could be replaced with a setuptools/setup.py script or a configure script could be introduced. It'd do that with autoconf even if might appear overkill for the size of the project.

I could provide the necessary changes as a pull request. Interested?

error importing launchpad branch

hi
i have used git-remote-bzr successfully to import a few branches of my project from launchpad/bzr to git, however with this particular branch i have a problem:

% git remote add kalivedadoc bzr::lp:kaliveda/kalivedadoc
% git fetch kalivedadoc
progress revision 100 'master' (100/944)
progress revision 200 'master' (200/944)
progress revision 300 'master' (300/944)
progress revision 400 'master' (400/944)
progress revision 500 'master' (500/944)
progress revision 600 'master' (600/944)
progress revision 700 'master' (700/944)
progress revision 800 'master' (800/944)
progress revision 900 'master' (900/944)
fatal: Empty path component found in input
fast-import: dumping crash report to .git/fast_import_crash_16343
fatal: Error while running fast-import
% Traceback (most recent call last):
  File "/usr/bin/git-remote-bzr", line 1022, in <module>
    sys.exit(main(sys.argv))
  File "/usr/bin/git-remote-bzr", line 1006, in main
    do_import(parser)
  File "/usr/bin/git-remote-bzr", line 437, in do_import
    export_branch(repo, name)
  File "/usr/bin/git-remote-bzr", line 359, in export_branch
    modified_final = export_files(cur_tree, modified)
  File "/usr/bin/git-remote-bzr", line 277, in export_files
    print d
IOError: [Errno 32] Broken pipe

I assume there is something nasty in the bzr branch which git-remote-bzr can't handle?
Thanks for any help

true roundtrip git -> bzr -> git (different repo) does not keep git hashes as stable as it could

If you take a perfectly ordinary git branch with just regular commits and merges, well behaved authors and commit messages, push to a bzr branch, then import that bzr branch into a new git repository, you can end up with an exact replica of the first git branch.

However, several things can make the second git branch functionally equivalent to the first, but with different sha1 hashes. For example:

• rebases and cherry picks will have a different CommitDate and AuthorDate; bzr only has one native date field, so on reimport to git one will have been lost.
• an empty email in the source (<>) will be re-exported as <Unknown>, an empty name as unknown.
• commit messages can be weird. I don't know how it can be provoked with the regular UI, but at least with fast-import, you can have a git commit message that does not end in a newline; git-remote-bzr will then add one.

I have test cases in this branch: https://github.com/zmanuel/git-remote-bzr/tree/recommit-roundtrip-test

Now, I realize the following things:

• This is probably just a me problem. Usually, the marks cache should take care of keeping a bzr and git revision properly linked.
• Any workflow that relies on such roundtrip stability is foolish and certainly doomed; a single affected revision where it fails 'poisons' all its children.
• The stability of git-remote-bzr is bounded by the stability of git's fast-export/import, and guess what? That one already is unstable and an import of a fresh export can very well be different.

So I definitely won't complain if you close this as 'out of scope', or 'fundamentally impossible' or 'no, we don't want anyone to even entertain this idea'. I get that :)

But dammit, it's a matter of principle, export followed by import should be just the identity if the export format allows. So for me, on the git->bzr side, I made it so that the raw git properties of a commit that can't be reconstructed are saved to custom revision properties, and on the bzr->git process, they are restored. The branch is here: https://github.com/zmanuel/git-remote-bzr/recommit-roundtrip
So add another reason not to do this: 'we don't want the bzr properties to be polluted'. In the code's defence, it only does that when needed, so the perfectly regular branches mentioned in the introduction would be unaffected.

carriage return in git commit message causes push to bzr to fail

Windows line endings (\r\n) are fine, they are probably filtered by bzrlib itself at some point, but a plain \r makes it through and causes an exception:

23:53 $ git push
Traceback (most recent call last):
  File "/home/manuel/bin/git-remote-bzr", line 1022, in <module>
    sys.exit(main(sys.argv))
  File "/home/manuel/bin/git-remote-bzr", line 1008, in main
    do_export(parser)
  File "/home/manuel/bin/git-remote-bzr", line 708, in do_export
    parse_commit(parser)
  File "/home/manuel/bin/git-remote-bzr", line 676, in parse_commit
    builder.commit(data.decode('utf-8', 'replace'))
  File "/usr/lib/python2.7/dist-packages/bzrlib/vf_repository.py", line 193, in commit
    self._validate_unicode_text(message, 'commit message')
  File "/usr/lib/python2.7/dist-packages/bzrlib/repository.py", line 142, in _validate_unicode_text
    raise ValueError('Invalid value for %s: %r' % (context, text))
ValueError: Invalid value for commit message: u'four\rno, three'
error: failed to push some refs to 'bzr::/alt/home/nocrypt/manuel/Tron/git/git-remote-bzr/test/trash directory.main/bzrrepo'

PR #22 contains the test that produces the above error and a fix; \r's are just filtered out.

Offending commits in the wild are for example in https://github.com/godotengine/godot, I lost the precise commit :( I can dig it up again if needed.

clone from launchpad fails with PermissionDenied

For example, when trying to clone inkscape:

git clone "bzr::lp:inkscape"
bzrlib.errors.PermissionDenied: Permission denied: "Cannot create 'backu'. Only Bazaar branches are allowed."

I get this error on debian testing, with bzr-lib 2.6.0

No instruction about the following error: Could not determine revno for {x} because its ancestry shows a ghost at {y}

• What
  There is no instruction about the following error: Could not determine revno for {xx} because its ancestry shows a ghost at {y}

Could anyone please tell me what I should do next when I got the error? I have no idea...

# Bazaar 2.6.0dev2
# git version 2.4.1

% git clone "bzr://xxx/trunk"

Clonage dans 'trunk'...
Traceback (most recent call last):
  File "/usr/local/bin/git-remote-bzr", line 1022, in <module>
    sys.exit(main(sys.argv))
  File "/usr/local/bin/git-remote-bzr", line 1004, in main
    do_list(parser)
  File "/usr/local/bin/git-remote-bzr", line 804, in do_list
    branch.revision_id_to_dotted_revno(revid)
  File "<string>", line 4, in revision_id_to_dotted_revno_read_locked
  File "/usr/lib/python2.7/dist-packages/bzrlib/branch.py", line 399, in revision_id_to_dotted_revno
    return self._do_revision_id_to_dotted_revno(revision_id)
  File "/usr/lib/python2.7/dist-packages/bzrlib/branch.py", line 413, in _do_revision_id_to_dotted_revno
    revno = self.revision_id_to_revno(revision_id)
  File "<string>", line 4, in revision_id_to_revno_read_locked
  File "/usr/lib/python2.7/dist-packages/bzrlib/branch.py", line 3013, in revision_id_to_revno
    raise errors.GhostRevisionsHaveNoRevno(revision_id, e.revision_id)
bzrlib.errors.GhostRevisionsHaveNoRevno: Could not determine revno for {[email protected]} because its ancestry shows a ghost at {[email protected]}

switch to Python 3

git-remote-bzr was removed from Debian due to the transition away from Python 2 triggered by the impending Python 2 EOL. Please migrate the codebase to Python 3 so that git-remote-bzr can be used after 2020.

add compatibility layer between .bzrignore and .gitignore or .git/info/excludes

Since git looks at .gitignore and .git/info/excludes and bzr looks at .bzrignore, it might be a good idea to automatically symlink .git/info/excludes to .bzrignore.

In some .bzrignore files there is stuff like ./autom4te.cache but in .gitignore that should be autom4te.cache so maybe there needs to be some automated conversion from .bzrignore to .git/info/excludes instead.