jeffkaufman / icdiff

improved colored diff

Icdiff

Improved colored diff

Installation

Download the latest icdiff and put it on your PATH.

Alternatively, install with packaging tools:

# pip
pip install icdiff

# apt
sudo apt install icdiff

# homebrew
brew install icdiff

# aur
yay -S icdiff

# nix
nix-env -i icdiff

Usage

icdiff [options] left_file right_file

Show differences between files in a two column view.

Options

  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --cols=COLS           specify the width of the screen. Autodetection is Unix
                        only
  --encoding=ENCODING   specify the file encoding; defaults to utf8
  -E MATCHER, --exclude-lines=MATCHER
                        Do not diff lines that match this regex. Not
                        compatible with the 'line-numbers' option
  --head=HEAD           consider only the first N lines of each file
  -H, --highlight       color by changing the background color instead of the
                        foreground color.  Very fast, ugly, displays all
                        changes
  -L LABELS, --label=LABELS
                        override file labels with arbitrary tags. Use twice,
                        one for each file
  -N, --line-numbers    generate output with line numbers. Not compatible with
                        the 'exclude-lines' option.
  --no-bold             use non-bold colors; recommended for solarized
  --no-headers          don't label the left and right sides with their file
                        names
  --output-encoding=OUTPUT_ENCODING
                        specify the output encoding; defaults to utf8
  -r, --recursive       recursively compare subdirectories
  -s, --report-identical-files
                        report when two files are the same
  --show-all-spaces     color all non-matching whitespace including that which
                        is not needed for drawing the eye to changes.  Slow,
                        ugly, displays all changes
  --tabsize=TABSIZE     tab stop spacing
  -t, --truncate        truncate long lines instead of wrapping them
  -u, --patch           generate patch. This is always true, and only exists
                        for compatibility
  -U NUM, --unified=NUM, --numlines=NUM
                        how many lines of context to print; can't be combined
                        with --whole-file
  -W, --whole-file      show the whole file instead of just changed lines and
                        context
  --strip-trailing-cr   strip any trailing carriage return at the end of an
                        input line
  --color-map=COLOR_MAP
                        choose which colors are used for which items. Default
                        is --color-map='add:green_bold,change:yellow_bold,desc
                        ription:blue,meta:magenta,separator:blue,subtract:red_
                        bold'.  You don't have to override all of them:
                        '--color-map=separator:white,description:cyan

Using with Git

To see what it looks like, try:

git difftool --extcmd icdiff

To install this as a tool you can use with Git, copy git-icdiff into your PATH and run:

git icdiff

You can configure git-icdiff in Git's config:

git config --global icdiff.options '--highlight --line-numbers'

Using with subversion

To try it out, run:

svn diff --diff-cmd icdiff

Using with Mercurial

Add the following to your ~/.hgrc:

[extensions]
extdiff=

[extdiff]
cmd.icdiff=icdiff
opts.icdiff=--recursive --line-numbers

Or check more in-depth setup instructions.

Setting up a dev environment

Create a virtualenv and install the dev dependencies. This is not needed for normal usage.

virtualenv venv
source venv/bin/activate
pip install -r requirements-dev.txt

Running tests

./test.sh python3

Making a release

• Update ChangeLog with all the changes since the last release
• Update __version__ in icdiff
• Run tests, make sure they pass
• git commit -a -m "release ${version}"
• git push
• git tag release-${version}
• git push origin release-${version}
• A GitHub Action should be triggered due to the release tag being pushed, and will upload to PyPI.

License

This file is derived from difflib.HtmlDiff which is under license. I release my changes here under the same license. This is GPL compatible.

icdiff's Issues

recursive option broken in v1.6.0

Traceback (most recent call last):
File "/usr/local/bin/icdiff", line 553, in
start()
File "/usr/local/bin/icdiff", line 472, in start
diff_recursively(options, a, b, options.encoding)
File "/usr/local/bin/icdiff", line 496, in diff_recursively
os.path.join(b, child))
TypeError: diff_recursively() takes exactly 4 arguments (3 given)

Clarify significance of left_file/right_file argument ordering

The order of the filename arguments is significant, but it's not documented. None of the existing tests demonstrate expected output when the file names are swapped.

Consider these two files: StackTest.cmp and StackTest.out

Whether the --whole-file option is enabled or not, icdiff StackTest.cmp StackTest.out fails to indicate StackTest.cmp has two more lines than StackTest.out. Diffs between line 2 in both files are highlighted in yellow.

When I swap the filename args (icdiff StackTest.out StackTest.cmp) it's clear one file has two more lines than the other. However, now all the yellow highlights on line 2 have been replaced by whole-line red/green.

Is there a way to get all this information in a single run?

Missed changes in files with Windows newlines

With revision 824927d:

$ icdiff <(echo -n -e "hello i'm hiding\\x0d") <(echo -n -e "\\x0d")
/dev/fd/63                                                                                            /dev/fd/62                                                                                           

$ diff <(echo -n -e "hello i'm hiding\\x0d") <(echo -n -e "\\x0d")
1c1
< hello i'm hiding
\ No newline at end of file

---
> 
\ No newline at end of file

Scrollable diff

Great tool - thanks for sharing!

It'd be great, if your tool would provide the diff in a "scrollable way". Just like the git diff command. I'm not a bash-expert - but I mean the same behavior as if you read a file with less or a manpage.

Typo on line 577

574    for x in [a, b]:
575        if os.path.isdir(x):
576            codec_print("error: %s is a directory; did you mean to "
577                        "pass --recursive?" % x, optoins)

optoins -> options

link to latest binary in readme is broken

Download the latest icdiff binary and put it on your PATH.

"git icdiff" does not work with path with spaces

git icdiff filename works fine with paths without spaces. When there is a space, I get:

fatal: ambiguous argument 'samples/Common/Resources/JSON': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

I tried "", '' or adding \ , none of them works.

Document the ability to use this script as the default git diff external program

Hi,
after I saw this can be used as a difftool I wanted to try to take it a step further and use icdiff as the default program for git diff.

Frankly, I wasn't even sure this would be possible and it also took me some time. For those reasons I think it makes sense to document this "process" so others can benefit from:

• knowing it can be done
• save time

Generalized steps are described here: http://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#External-Merge-and-Diff-Tools

Here are the exact steps:

• create a script named extDiff on the path and put the following contents in it:

#!/bin/sh
[ $# -eq 7 ] && icdiff "$2" "$5"

• make the extDiff script executable with chmod +x extDiff
• make the script the default program for git diff with: git config --global diff.external extDiff

After this is done, wherever git uses the diff view, icdiff will be invoked. I'm using this for a day now and it works just fine.

I didn't want to open a pull request bc I don't know where (and if at all) would you prefer to put the instructions (in the readme, wiki, somewhere else..). So if you don't mind, I'd let this to you.

Hope you like the idea and let me know if I can be of any help for getting this "in"!

Feature request: exclude files matching a pattern

Would you be open to adding the ability to exclude a pattern?

For example, from diff on OS X:

-x PAT  --exclude=PAT
              Exclude files that match PAT.

This is useful to diff two folders while excluding, for example, the .git subdirectory.

Use as default SVN diff

Has anyone successfully got this to work as the diff tool for SVN?

When I try to set it with svn diff --diff-cmd icdiff I get:

Usage: icdiff [options] left_file right_file
icdiff: error: no such option: -u
svn: E200012: 'icdiff' returned 2

Is it because the signature is different to icdiff LEFT RIGHT?

icdiff 1.7.3 reports 1.7.2 version

Not a big one, but after having installed 1.7.3, icdiff --version reports icdiff version 1.7.2.

See source code.

Error since last pull

Sorry I don't know python, I can just report the error :
(used from svn)

Traceback (most recent call last):
  File "/bin/icdiff", line 561, in <module>
    start()
  File "/bin/icdiff", line 476, in start
    diff_files(options, a, b, options.encoding)
  File "/bin/icdiff", line 556, in diff_files
    codec_print(line, options)
  File "/bin/icdiff", line 483, in codec_print
    sys.stdout.write(s.encode(options.output_encoding))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 32: ordinal not in range(128)

exit code 1 if there is difference

DIAGNOSTICS

  An  exit status of 0 means no differences were found, 1 means some dif-
  ferences were found, and 2 means trouble.

http://unixhelp.ed.ac.uk/CGI/man-cgi?diff
http://stackoverflow.com/questions/6971284/what-are-the-error-exit-values-for-diff

Installation via pip

It would be nice to upload icdiff to PyPI for easier installation on Linux.

Feature request: add support for piplining ("-" as argument)

I usually use diff in such a way (compare stdout with a given file):

python "B.py" < B-small-attempt0.in | diff - B-small.out

however seems that icdiff donot support this kind of operation:

$ python "B.py" < B-small-attempt0.in | icdiff - B-small.out
Traceback (most recent call last):
  File "/usr/local/bin/icdiff", line 9, in <module>
    load_entry_point('icdiff==1.7.5', 'console_scripts', 'icdiff')()
  File "/usr/local/lib/python2.7/dist-packages/icdiff.py", line 511, in start
    diff_files(options, a, b)
  File "/usr/local/lib/python2.7/dist-packages/icdiff.py", line 580, in diff_files
    lines_a = read_file(a, options)
  File "/usr/local/lib/python2.7/dist-packages/icdiff.py", line 550, in read_file
    with codecs.open(fname, encoding=options.encoding, mode="rb") as inf:
  File "/usr/lib/python2.7/codecs.py", line 878, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: '-'

Thus could you please add the support for - as argument ?

Not reporting minimal diff when letter is changed within word

When I change a letter in a word, icdiff doesn't report the minimal difference, which is:
1 letter removed -> 1 letter added, and not 1 word deleted -> 1 word added.

(icdiff version 1.7.2)

file1

aaa

file2

aba

icdiff shows:

file1              | file2
aaa (all in red)   | aba (all in green)

expected:

file1              | file2
aaa (only second 'a' in red)   | aba (only 'b' in green)

how to use it as a module

I want to import icdiff in my python module, but it is giving error.

[jpriyal-mba13:~/Desktop] jpriyal% python
Python 2.7.10 (default, Jul 14 2015, 19:46:27) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import icdiff
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named icdiff
>>> 

Importing icdiff module to my python code is giving error

Hi,

I have installed icdiff in Mac, its working fine from terminal. But when I am importing it in my python code it is giving error.

ImportError: No module named icdiff

How to import this module, is it not supported ??

Not reporting minimal diff when words added

When I add words to a line, icdiff doesn't report the minimal difference, which is:
0 changes -> 2 words added, and not 1 word deleted -> 3 words added.

(icdiff version 1.7.2)

file1

aaa

file2

aaa bbb ccc

icdiff shows:

file1              | file2
aaa (all in red)   | aaa bbb ccc (all in green)

expected:

file1              | file2
aaa (no colours)   | aaa bbb ccc ('bbb ccc' in green)

setup script and distribution

This package could use regular setuptools infrastructure to be installable, also could be registered on pypi, so installation could be even easier.

Doesn't work with solarized colorscheme

Hi,
I'm on OS X Yosemite using Terminal.app with solarized color scheme.

I tried using icdiff for git diff, but for some reason this does not work well with this colorscheme.
The lines and characters are highlighted but that's barely noticeable.
For some reason, red color is displayed just fine.

Here's the screenshot where I've pointed out lines and chars that should be green or yellow, but that's not visible:

What's weird, regular git diff is displayed just fine. Here's the "proof":

Also, things work just fine and colors are displayed properly in a different terminal (with "regular" colors).
This clearly indicates a problem with solarized colorscheme I'm using.

The reason I'm bringing this up as an issue here is:

• In ~2 years using the colorscheme, I've never had a single issue with displaying colors
• it's a relatively popular colorscheme (see github stars). I never got the feeling the project is "hacked" or buggy

So do you have any idea what could be wrong here? Can I help troubleshooting in any way?
Thanks!

Flag for tab-size

There should be a flag to specify the number of spaces for a tab character.

Clarify copyrights

Hi,

I am packaging icdiff for Debian and I am looking to sort out some minor licensing issues.
The Python license linked by you in README.md specifies that the Python copyright:

Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation; All Rights Reserved

must be retained in derivative based on it (section 2). This is missing in the script header. Also, section 3 states that

In the event Licensee prepares a derivative work that is based on
or incorporates Python or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python.

It would be nice to have such a brief summary.

Also, the UTF-8 decoder capability and stress test file (tests/b/1 and tests/input-3.txt) seems like it was not prepared by you but by Markus Kuhn. The version of the file you are distributing with icdiff does not include any license information, but a newer version does (https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt, CC-BY). It would be nice to have this file updated or the license specified by you.

Error with align in right column

I have error with separating screen on two columns after updating to commit 365dfef

Right column must be aligned like green text. All lines endings with \n

compare directories recursively

Currently, passing two directories into icdiff gives

Traceback (most recent call last):
  File "/usr/local/bin/icdiff", line 536, in <module>
    start()
  File "/usr/local/bin/icdiff", line 519, in start
    lines_a = open(a, "U").readlines()
IOError: [Errno 21] Is a directory: 'server/'

Since it is advertised as a tool to compare two files, that is perfectly fine (except for that one could wish for a more user friendly error message) . However, it would be sooooo awesome, if icdiff would be able to compare full directory trees like diff with -r/--recursive.

line wrap break Chinese characters for long lines.

for a file containing a long line of Chinese characters, icdiff might show undesirable characters when it tries to wrap the line.

for example, we have a file:

测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试

if we copy the file to another name and make a change. and the icdiff those two files, and if the width of the console is even(132 characters for example), the output would be:

abd                                                               abc
测试行abd测试测试行abd测试测试行abd测试测试行a  测试行abc测试测试行abc测试测试行abc测试测试行a
bd测试测试行abd测试测试行abd测试测试行abd测试�  bc测试测试行abc测试测试行abc测试测试行abc测试�
�试行abd测试测试行abd测试                                �试行abc测试测试行abc测试

while a desirable output should be like:

abd                                                               abc
测试行abd测试测试行abd测试测试行abd测试测试行a  测试行abc测试测试行abc测试测试行abc测试测试行a
bd测试测试行abd测试测试行abd测试测试行abd测试  bc测试测试行abc测试测试行abc测试测试行abc测试
测试行abd测试测试行abd测试                                测试行abc测试测试行abc测试

image for readme

Feature request: support to read diff file with icdiff

icdiff is awesome to compare with two files and be used with git.

Sometimes we just get the diff file and it would be much better to use icdiff to read that diff file.

GH Repo version behind with PyPi

It seems like the PyPi version is at version 1.8.5.

Github seems to be at version 1.8.1. Why is the github repo. behind the PyPi release?

I used meld (a competing diff utility) to compare the two icdiff files and there are about 20 lines that have changed.

git icdiff does not work in Linux Mint 16 Cinnamon 64-bit

I have added git-icdiff to my path. I open a terminal using ctrl + alt + t. Now in my local repository if I do a git icdiff, the terminal flashes and the diff doesn't shows up. However, if I run it using git difftool --extcmd icdiff, it works perfectly fine.

I also noticed git icdiff works fine from a virtual terminal (i.e. by pressing ctrl + alt + F1).

Encoding issue for python2

With 93cffff there is an encoding issue for python 2.

If you put a non-ascii char in a file (eg å) then icdiff will fail. I tried using python 3 instead, and that worked fine.

Traceback (most recent call last):
  File "icdiff", line 553, in <module>
    start()
  File "icdiff", line 474, in start
    diff_files(options, a, b, options.encoding)
  File "icdiff", line 548, in diff_files
    print(line)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 112: ordinal not in range(128)

Handle file not found error

[ec2-user@startrek captain]$ icdiff bootstrap_9.sh bootstrap_10.sh
Traceback (most recent call last):
  File "/usr/local/bin/icdiff", line 603, in <module>
    start()
  File "/usr/local/bin/icdiff", line 511, in start
    diff_files(options, a, b)
  File "/usr/local/bin/icdiff", line 580, in diff_files
    lines_a = read_file(a, options)
  File "/usr/local/bin/icdiff", line 550, in read_file
    with codecs.open(fname, encoding=options.encoding, mode="rb") as inf:
  File "/usr/lib64/python2.7/codecs.py", line 884, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: 'bootstrap_9.sh'

Module version AttributeError for options

>>> from icdiff import diff
>>> diff({}, '/var/tmp/xxx.py', '/var/tmp/xxxl.py') # is there a way to skip options parameter
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "icdiff.py", line 529, in diff
    diff_files(options, a, b)
  File "icdiff.py", line 562, in diff_files
    if options.labels:
AttributeError: 'dict' object has no attribute 'labels'

Take standard input

When making changes to a file in Vim, I find myself wanting to see what changes I have made in my buffer. One way to to this is :w !diff % - where % is the name of the current file and - is the contents of the current buffer passed in via standard input. Currently, icdiff requires both to be a file.

fullwidth characters support not working

$ icdiff --version
icdiff version 1.5.1

$ cat a
你好你好你好你好你好

$ cat b
你

$ icdiff a b
a                                                                    b                                                                   
你好你好你好你好你好                                       你

vim file causes encoding error

Specifically, I get this:

error: file '.../.ClassName.h.un~' not valid with encoding 'utf-8': at 3-4.

Documentation update: mac installation works on linux :)

Hi Jeff,
I found this tool very useful, I used to fallback to vimdiff to perform the same taks because I sometimes found diff a bit difficult to read.
I've installed it on a fedora linux with no issues, so I'd like to recommend to update the mac install section to mac/linux install.

inline additions not highlighted on git-icdiff

Changes in case of additional text on an existing line is not highlighted in case of git-icdiff.

Deleting characters in a line are highlighted as expected.

Solarized Dark iTerm Colors for changes grey

Changes are being displayed in grey.
What would be the preferred way of defining what colors icdiff uses based on the current selected colorscheme ?

ctrl+d closes pager when scrolling below end of output.

When I do use git diffwith long output I use vim ctrl+d and ctrl+u shortcuts (which also work in less) to scroll half page up/down.
After scrolling below the end of output ctrl+d does nothing.

But in the case of git icdiffit closes the program. So I'm incidentally not only closing icdiff but also terminal after second press.

This behaviour can be treated as a bug or as a feature. But form the point of view of Unix UX - I'd say it would be better if it worked like git diff does.

Add support for invoking icdiff on strings from python

Brew says "no available formula for icdiff"

pip install instructions don't work

$ pip install git+https://github.com/jeffkaufman/icdiff.git
Collecting git+https://github.com/jeffkaufman/icdiff.git
  Cloning https://github.com/jeffkaufman/icdiff.git to /tmp/pip-430nkex7-build
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-430nkex7-build/setup.py", line 3, in <module>
        from icdiff import __version__
    ImportError: No module named 'icdiff'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-430nkex7-build/

I wonder if this might be because you forgot to rename icdiff to icdiff.py when merging #69 ?
46d5a72

difference not caught with `--recursive`

./icdiff --recursive tests/a/c/f tests/b/c/f doesn't produce output on one machine I'm testing, because filecmp.cmp("tests/a/c/f", "tests/b/c/f") is returning True. If I pass the shallow=False flag to filecmp.cmp then it returns False and everything works. The easy fix is to always pass shallow=False, but what's really going on?

The docs say:

Unless shallow is given and is false, files with identical os.stat() signatures are taken to be equal.

In my case the os.stat() signatures are:

>>> os.stat(a)
posix.stat_result(
    st_mode=33188,
    st_ino=30750980,
    st_dev=16777220L,
    st_nlink=1,
    st_uid=163798,
    st_gid=5000,
    st_size=2,
    st_atime=1418672570,
    st_mtime=1418658075,
    st_ctime=1418658075)
>>> os.stat(b)
posix.stat_result(
    st_mode=33188,
    st_ino=30750985,
    st_dev=16777220L,
    st_nlink=1,
    st_uid=163798,
    st_gid=5000,
    st_size=2,
    st_atime=1418672570,
    st_mtime=1418658075,
    st_ctime=1418658075)

Those are very similar, but not identical. Specifically, st_ino is different. What's the code in filecmp?

def cmp(f1, f2, shallow=1):
    ...
    s1 = _sig(os.stat(f1))
    s2 = _sig(os.stat(f2))
    ...
    if shallow and s1 == s2:
        return True
...
def _sig(st):
    return (stat.S_IFMT(st.st_mode),
            st.st_size,
            st.st_mtime)

So it's only looking at the mode (permissions), size, and modification time.

This seems like a bit of a doc bug with cmp(), but switching us to set shallow=False will fix it.

Ignore binary diff?

Diff binary file won't give useful output currently.
How about ignore binary files and simply says "This is a binary file"?

how can i use it in svn

i want use it at 'svn diff'.

Feature Request: auto-detect encodings

I use icdiff with git. It's awesome! But some of my project files are not in utf8 and I can not view diff with that files. Would it be more reliable not to throw exception but to convert bad symbols as escape sequences or stub chars?
$ git diff HEAD~
error: file '/tmp/R97Zaw_http.c' not valid with encoding 'utf-8': <invalid continuation byte> at 74350-74351

document --context

You mention --context when describing --numlines but never explain --context

Passing '--no-bold' along with '--highlight' looses highlight

Expected

Actual

Is this behavior intentional?

Can we please get this on homebrew?

Thanks so much for making this program! It is awesome!

Can we please get it on homebrew (OS X package manager) so that installation is easier?

Thanks