Unsupervised joint alignment of complex images
http://vis-www.cs.umass.edu/code/congealingcomplex/
All code provided under a BSD-style license. Terms of license can be
found at the top of each source file.
README contents:
--------------------------------
1. Overview
2. Quick Guide
3. Cars
4. Full Details
5. Additional Notes
1. Overview
--------------------------------
The source code is given for congealReal.cpp and funnelReal.cpp,
producing binaries congealReal and funnelReal. Both are written in
C++ and require the OpenCV library (http://opencvlibrary.sourceforge.net).
On a Linux machine, the following Makefile commands will produce the
binaries, following the conventions from
http://opencvlibrary.sourceforge.net/CompileOpenCVUsingLinux (refer to
the first URL for other environments).
IFLAGS = `pkg-config --cflags opencv` -O2
LFLAGS = `pkg-config --libs opencv`
all: congealReal funnelReal
congealReal: congealReal.cpp
gcc $(IFLAGS) $(LFLAGS) -o congealReal congealReal.cpp
funnelReal: funnelReal.cpp
gcc $(IFLAGS) $(LFLAGS) -o funnelReal funnelReal.cpp
Depending on the settings congealReal is run with, it may bring up
images in new windows. Press any key to continue with the program.
2. Quick Guide
--------------------------------
congealReal images.list images.model
This command will read in a list of image filenames, one per line,
from images.list, perform congealing, and save the sequence of
distribution fields to the file images.model.
funnelReal images.list images.model images_aligned.list
This command will read in a list of image filenames, one per line,
from images.list, align each image by funneling it according to the
sequence of distribution fields in images.model, then save the aligned
images using the filenames specified in images_aligned.list (which
should be in the same order as images.list).
3. Cars
--------------------------------
The set of car images used in the ICCV paper is provided to try with
the source code.
After uncompressing cars.tgz, one can run the following commands from
the cars/ directory.
congealReal carsFn.txt cars.train -o animSeq.txt -d display -v visualize -g carsOutFn.txt -outer 176 132 -inner 120 76 -nonrand -verbose
This will display the resulting images in new windows and save these
to the directory display, produce visualizations of the final
distribution field entropy and highest posterior probability cluster
representatives to visualize, generate aligned images and save them to
the directory final, use a 176x132 outer image and 120x76 inner image,
compute the entropy at all points within the inner window, and output
the entropy at each iteration.
congealReal carsFn.txt cars.train -a animSeq.txt animations -outer 176 132 -inner 120 76
This will produce frames for animation of the congealing done by the
previous command, saving the images grouped in 5x5 panels and entropy
of the distribution field, at each iteration of congealing, to the
directory animations.
funnelReal carsFn.txt cars.train carsOut.fn -outer 176 132 -inner 120 76 -o params.txt
This will align the car images using the funnel learned from the
congealing and save the aligned images to the directory final and the
transformation parameters used to align each image to params.txt.
In the case of the cars, the funneling is duplicating the result of
congealing with the -g option, so this is just for illustration.
Funneling would be used in instances where it is not feasible to
congeal all images in a data set at once, or when congealing has been
done on an initial set of images and subsequently new images are found
that need to be aligned.
4. Full Details
--------------------------------
congealReal.cpp : congealing for complex, realistic images
using soft clusters of SIFT descriptors
usage : congealReal <list of image filenames> <model output file> ...
[options]
<list of image filenames> is a list of filenames of the images
to process
<model output file> is the filename to which the sequence of
distribution fields should be written to (for use later in
funneling)
options :
-o filename
output the transformations at each iteration to the
specified file, in order to create an animation later
-a filename directory
create a frame (for animation) using the transformations
given in the specified file, and write the result to the
specified directory (must be used alone, and no
congealing will be done)
-v directory
create visualizations of highest probability patches for
each cluster and of entropy of final distribution field,
writing images to the specified directory
-g directory or list of filenames
generate the final aligned images. if the argument is a
directory name, the images will be written to the
specified directory using the original filenames (this
assumes the original filenames were relative filenames,
and appends them to the specified directory).
otherwise, it is assumed the argument is the name of a
file containing a list of filenames to use for the
aligned images
-d directory
display the final transformations in 5x5 panels and
write images to specified directory (press ESC to skip
display of panels)
-outer w h
resize images to w by h for congealing computations
(default 150x150)
-inner w h
use an inner window of size w by h, within which to
calculate likelihood for congealing (must be smaller
than outer dimensions by at least the size of the window
for which SIFT descriptor is calculated over) (default
100x100)
-loc n
sample n pixel locations at which to calculate
likelihood for congealing (default 6,000)
-nonrand
use all points within inner window rather than sampling
(will ignore -loc if provided)
-clusters k
use k clusters of SIFT descriptors (default 12)
-verbose
print out entropy for each iteration of congealing
funnelReal.cpp : funneling for complex, realistic images
using sequence of distribution fields learned from congealReal
usage : funnelReal <list of image filenames> ...
<model file from congealing> ...
<output directory or list of output filenames> [options]
<list of image filenames> is a list of filenames of the images
to process
<model file from congealing> is the file containing the
sequence of distribution fields from congealing
<output directory or list of output filenames> if this is the
name of a directory, the aligned images will be written to
this directory (making the assumption to the filenames
provided in the first argument are relative. If it is not
the name of the directory, then it should be the name of a
file containing the filenames to use for the aligned
images, in order corresponding to the first argument
options :
-o filename
output the final parameter values used to generate
aligned images
-outer w h
resize images to w by h for funneling computations
(default 150x150). this must match the values used in
congealing
-inner w h
use an inner window of size w by h, within which to
calculate likelihood for congealing (must be smaller
than outer dimensions by at least the size of the window
for which SIFT descriptor is calculated over) (default
100x100). this must match the values used in congealing
5. Additional Notes
--------------------------------
Currently, the SIFT descriptor computation remains as described in the
ICCV paper. Important points are that it is computed over 8x8 patches
split into 4x4 subregions, yielding a 32 dimensional vector, and the
patches are not re-oriented to the dominent edge orientation of the
patch.
congealReal.cpp contains a constant variable maxIters = 100, and will
terminate if the number of iterations of congealing exceeds this
number. In practice, congealing takes approximately 20 to 30
iterations. Of course, this will vary depending on your particular
data, so you may wish to increase this number of necessary.
congealReal.cpp also contains a constant variable maxFrameIndex = 5.
This number determines how many sets of 5x5 images it will create when
making frames for animation.
funnelReal.cpp contains a constant variable maxProcessAtOnce = 600,
and will attempt to simultaneously funnel at most maxProcessAtOnce
images together. This number should be set based on memory
constraints, though there should not be any significant slowdown if a
smaller number of images are funneled together in one round.