Problem pushing to remote Mercurial repo on Bitbucket, push to local file:// repo works normally about git

[vedranmiletic]

@cawka is also interested in this.

[vedranmiletic]

Sorry for closing/opening the issue, I'm slowly getting familiar with GitHub interface.

[felipec]

So, there are two ways I know:

1. Specify the password in the URL:

https://user:[email protected]/user/repo

2. Set the password in hgrc:

http://stackoverflow.com/questions/2584407/how-to-save-username-and-password-with-mercurial

Then you can clone with the url:

bb://user/repo

But for the option 2) you need the latest version (I just pushed a fix for that).

Also, you can use SSH instead of HTTPS.

[vedranmiletic]

@felipec thank you for your response. I will try it tomorrow.

[felipec]

I'm going to mark this as closed. Reopen if you have issues.

[vedranmiletic]

I just tried the file from hg-next with git 1.8.2 and with old data I still get the same issue:

$ git push bitbucket master
searching for changes
no changes found
Traceback (most recent call last):
  File "/home/vedranm/bin/git-remote-hg", line 899, in 
    sys.exit(main(sys.argv))
  File "/home/vedranm/bin/git-remote-hg", line 887, in main
    do_export(parser)
  File "/home/vedranm/bin/git-remote-hg", line 745, in do_export
    parse_reset(parser)
  File "/home/vedranm/bin/git-remote-hg", line 716, in parse_reset
    node = parser.repo.changelog.node(mark_to_rev(from_mark))
  File "/usr/lib/python2.7/dist-packages/mercurial/changelog.py", line 193, in node
    return super(changelog, self).node(rev)
  File "/usr/lib/python2.7/dist-packages/mercurial/revlog.py", line 310, in node
    return self.index[rev][7]
IndexError: revlog index out of range

After cleaning out old data:

Traceback (most recent call last):                                                                                                         
  File "/home/vedranm/bin/git-remote-hg", line 899, in 
    sys.exit(main(sys.argv))
  File "/home/vedranm/bin/git-remote-hg", line 887, in main
    do_export(parser)
  File "/home/vedranm/bin/git-remote-hg", line 778, in do_export
    parser.repo.push(peer, force=True, revs=p_revs)
  File "/usr/lib/python2.7/dist-packages/mercurial/localrepo.py", line 1844, in push
    ret = remote.unbundle(cg, remoteheads, 'push')
  File "/usr/lib/python2.7/dist-packages/mercurial/wireproto.py", line 306, in unbundle
    ret, output = self._callpush("unbundle", cg, heads=heads)
  File "/usr/lib/python2.7/dist-packages/mercurial/httppeer.py", line 200, in _callpush
    r = self._call(cmd, data=fp, headers=headers, **args)
  File "/usr/lib/python2.7/dist-packages/mercurial/httppeer.py", line 170, in _call
    fp = self._callstream(cmd, **args)
  File "/usr/lib/python2.7/dist-packages/mercurial/httppeer.py", line 118, in _callstream
    resp = self.urlopener.open(req)
  File "/usr/lib/python2.7/urllib2.py", line 407, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 520, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 439, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 894, in http_error_401
    url, req, headers)
  File "/usr/lib/python2.7/dist-packages/mercurial/url.py", line 431, in http_error_auth_reqed
    self, auth_header, host, req, headers)
  File "/usr/lib/python2.7/urllib2.py", line 869, in http_error_auth_reqed
    response = self.retry_http_basic_auth(host, req, realm)
  File "/usr/lib/python2.7/urllib2.py", line 875, in retry_http_basic_auth
    user, pw = self.passwd.find_user_password(realm, host)
  File "/usr/lib/python2.7/dist-packages/mercurial/url.py", line 36, in find_user_password
    raise util.Abort(_('http authorization required'))
mercurial.error.Abort: http authorization required

Which you mentioned. Thanks a lot for your effort.

It would be nice if we could make that error mesage nicer. I will cook up a patch and post it here.

[vedranmiletic]

Bitbucket is here: https://bitbucket.org/rivanvx/ns-3-dev-git-remote-hg-experiment

$ git push bitbucket master 
searching for changes
no changes found
searching for changes
sending (progressbar removed)
remote: adding changesets                                                                                                                  
remote: adding manifests                                                                                                                   
remote: adding file changes                                                                                                                
remote: added 9252 changesets with 19341 changes to 3293 files (+1 heads)                                                                  
remote: bb/acl: rivanvx is allowed. accepted payload.                                                                                      
warning: helper reported unexpected status of refs/headsmaster
Everything up-to-date

Is +1 heads thing expected?

Also, now I have both master and default branch. is this expected?

[felipec]

'master' is usually a fake pointer, it might point to a bookmark, or a branch head, but 'branch/default' is the branch pointer. You can for example push to another branch by checking out 'branch/foo'.

But you can push, right?

[vedranmiletic]

Yeah, I can push. Massive props for that.

I'm just worried about +1 heads thing. The problem is, from what I see, that changesets got duplicated somehow. I have over 18000 of them, compared to over 9000 (no pun intended) that ns-3-dev repo has.

[felipec]

What exactly did you do? What do you mean by "cleaning out old data"?

[vedranmiletic]

Sorry, stupid me. I should have done

$ git push bitbucket branch/default

instead of

$ git push bitbucket master

as you said. Let me try that.

[felipec]

It doesn't matter, both should do the same thing. Pushing the 'master' branch would just push to the current branch, which I assume is 'default'. But it doesn't matter, either way it should not duplicate every single commit.

[vedranmiletic]

OK, this doesn't work:

$ git push bitbucket branch/default
searching for changes
no changes found
error: src refspec branch/default does not match any.

I'm still new with Git apparently. On the other hand, per http://blog.supermatter.com/post/761870270/git-push-a-local-branch-to-a-different-remote-branch I tried:

$ git push bitbucket master:branch/default
searching for changes
no changes found
searching for changes
no changes found
warning: helper reported unexpected status of refs/headsmaster
Everything up-to-date

However, remote default remains the same, as in local master doesn't get pushed to remote default.

[vedranmiletic]

Also, cleaning old data meant

$ rm -rf .git/hg

[felipec]

Ok, that is the correct way to clear the remote-hg data, but did you do 'git fetch' (or pull) after that?

And you can't push branch/default, you need the check it out first:

% git checkout branches/default

Refspecs such as master:branch/default don't really work with remote helpers at the moment.

Finally, the warning "helper reported unexpected status of refs/headsmaster" is a bit worrying, some extra logging might be needed to understand what is going on.

[vedranmiletic]

OK, let me retry that with clean repository. I will convert hg to git again and try adding some commits and pushing.

[vedranmiletic]

Some weirdness:

$ git push bitbucket branches/default 
searching for changes
no changes found
searching for changes
sending [                         <=>                       ] 116019/33190 -44s

Notice the -44s and 116019/33190?

[vedranmiletic]

Unfortunately, changesets are again duplicated.

$ git push bitbucket branches/default 
searching for changes
no changes found
searching for changes
remote: adding changesets                                                                                                                  
remote: adding manifests                                                                                                                   
remote: adding file changes                                                                                                                
remote: added 9278 changesets with 19341 changes to 3293 files (+1 heads)                                                                  
remote: bb/acl: rivanvx is allowed. accepted payload.                                                                                      
To hg::https://rivanvx:**********************@bitbucket.org/rivanvx/ns-3-dev-git-remote-hg-experiment
 * [new branch]      branches/default -> branches/default

Really, doing:

hg clone https://bitbucket.org/rivanvx/ns-3-dev-git-remote-hg-experiment

yields a repo with over 18000 changesets. Any ideas?

[felipec]

I just did the same to a fork of ns-3-dev:

% git push
searching for changes
no changes found
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: bb/acl: felipec is allowed. accepted payload.
warning: helper reported unexpected status of refs/headsmaster
Everything up-to-date

I'm using fc/remotes/hg not hg-next though. But I'm thinking the problem might be while mixing repositories.

[vedranmiletic]

But it's the same repo, right? Even though it's at different location.

Thanks a lot for testing. To be precise, I usually do it like this: I clone ns-3-dev from bitbucket with git. This repo misses some changests from http://code.nsnam.org/ns-3-dev, which I pull using git. This stuff works. I try to push it to bitbucket then, and this is where it breaks.

[vedranmiletic]

Minimal sequence of commands that breaks is:

$ git clone hg::http://code.nsnam.org/ns-3-dev
$ cd ns-3-dev
$ git checkout branches/default
$ git remote add bitbucket hg::https://bitbucket.org/rivanvx/ns-3-dev-git-remote-hg-experiment
$ git push bitbucket branches/default

Of course, in step 4 I also add my username and password.

[felipec]

You need to run git fetch bitbucket before pushing. It shouldn't be creating duplicates either way though.

[vedranmiletic]

Doesn't fetch happen automatically? Seems to me it does.

[felipec]

Where? In git remote add? That just adds pointers. And git push only pushes, doesn't fetch.

Either way, this is a different bug, either this bug should be renamed and updated, or you need to open a new bug.

I'm trying that sequence of commands. Let's see.

[vedranmiletic]

OK. Could you please open a new bug when we understand what's going on?

[felipec]

OK, so here's what's going on. When you issue the push, the first thing that happens is that that a new mercurial clone is fetched, but remote-hg has no information about it, and neither does git. So git thinks it's empty, and pushes all the commits. Ideally, this should create exactly the same mercural repository, but doesn't always happen.

In this particular case, the repositories diverge when there's a file rename, which git doesn't store, and everything after that is a very similar version of the original, but not equal, so you have duplicated commits. However, remote-hg has a compatibility mode with hg-git, which does store file renames, and so on, so it's possible that using this mode, the whole mercurial repository is created again through git exactly the same, and you would see no duplication.

But really, there's no point in trying. After you add a new remote, you should always fetch, that's all.

[vedranmiletic]

Good, thanks for the info, I will retry the correct way. Can you somehow limit people from doing wrong things by enforcing it in code, i.e. make the script fail if user does something like what I did?