Commit messages became unreadable (Chinese text) about fast-export

[frej]

I want migrate from mercurial to git, and this tool works very well indeed,
except all the commit messages(which were wrote in Chinese) became unreadable.

What encoding have you used for the Chinese commit messages?

I tried to reproduce your problem by creating a dummy repository where I
created commit messages containing letters (åäö) which encodes
differently in utf-8 and latin-1 (using hg --encoding). The resulting
logs displays correctly in an utf-8 terminal both before and after
conversion to git. Unless you can give me an exact recipe to reproduce
the problem I don't think it is a fast-export problem.

To salvage the situation I would try to play with the
i18n.commitEncoding or i18n.logOutputEncoding (check 'git config
--help') to see if I could trick git into converting your messages on
the fly. Otherwise have a look at 'git filter-branch' by which you could
massage the messages using iconv after the initial conversion.

[frej]

No feedback from reporter for half a year, closing...

[zedxxx]

I have the same issue.

Commit messages in my mercurial repo stored in cp1251 (default charset encoding for Russian) and it's impossible correct import it to git.

So, I make some little changes in hg2git.py:

def get_changeset(ui,repo,revision,authors={}):
  node=repo.lookup(revision)
  (manifest,user,(time,timezone),files,desc,extra)=repo.changelog.read(node)
  # +++ Do convert local encoding to UTF8
  user = user.decode('cp1251').encode('utf8')  
  desc = desc.decode('cp1251').encode('utf8')  
  # +++ End encoding convertions
  tz="%+03d%02d" % (-timezone / 3600, ((-timezone % 3600) / 60))
  branch=get_branch(extra.get('branch','master'))
  return (node,manifest,fixup_user(user,authors),(time,tz),files,desc,branch,extra)

I think you need add some option to set correct source encoding for all text fields that you read from changelog.

[frej]

I think you need add some option to set correct source encoding for all
text fields that you read from changelog.

Patches welcome :)

[zedxxx]

So, maybe you should reopen this issue?

[frej]

So, maybe you should reopen this issue?

Could you provide a recipe for creating a repo which exhibits this
issue (or point to a public hg repo)?

[zedxxx]

https://bitbucket.org/sas_team/sas.planet.src

[zedxxx]

My pull request: #39

[frej]

Fixed by e87c9cb, closing issue.