Coder Social home page Coder Social logo

tomkyle / negatives-linear-tiff Goto Github PK

View Code? Open in Web Editor NEW
7.0 3.0 1.0 137 KB

Converts RAW/NEF/CR2 files into linear TIFF files, using GNU Parallel for maximum speed. Other features: B/W grayscaling, TIF resizing and ZIP compression.

Home Page: https://tomkyle.github.io/negatives-linear-tiff

License: Other

Shell 100.00%
linear-algebra tiff photography dcraw parallel homebrew

negatives-linear-tiff's Introduction

negatives/linear-tiff

Build Status

Converts RAW/NEF/CR2 files into linear 16bit-TIFF files.

  • Mirroring for negatives digitalized on their emulsion side.
  • Resizing when megapixel power is not everything.
  • Grayscaling: B/W lovers save up to 60% disk space.
  • Cropping: According to your Raw converter crop.
  • Orientation: According to your custom image rotation in Raw converter.
  • ICC profiles: Output images get a linear gamma profile.
    No ‘custom gamma colorspace’ in Photoshop!

What happens inside? LibRaw's dcraw_emu creates a linear 16bit TIFF file from your photographed negative raw file. ImageMagick's mogrify then does the color-profiling, B/W grayscaling, mirroring, resizing, cropping, orientation, and ZIP compression. GNU Parallel uses all CPU cores to speed up the whole thing.

Homebrew Installation (MacOS)

The linear-tiff bash script can be installed by a Homebrew formula, which itself is part of the tomkyle/homebrew-negatives tap.

# Variant 1:
# Install tap first, formula second:
$ brew tap tomkyle/negatives
$ brew install linear-tiff

As “tapping” first is not neccessarily needed, you can install the formula directly:

# Variant 2
$ brew install tomkyle/negatives/linear-tiff

Usage

Open your terminal application and go to your images directory. linear-tiff will work in the current working directory. Run linear-tiff --help or -h to display help text. See Options and Examples.

$ linear-tiff [options] [-a | file(s)]

Options

-a, --all

All images — process any RAW/NEF/CR2 file in working directory, using GNU Parallel.

-c, --crop

Crop output image, if input file's meta data carry crop information. Useful when cropping film holder parts away with your Raw converter—The output image will then be cropped as well, even if the crop info is stored a XMP sidecar file. Currently, no crop angles are supported.

--debug

Switch on debugging mode, showing everything that's going on.

-d, --desaturate

Desaturate colors — recommended for B/W negatives. dcraw's TIFF output is converted to 16-bit grayscale, with a linear gamma 1.0 ICC profile applied (Gray-elle-V4-g10.icc). Grayscaling saves up to 60% in file size.

-m, --mirror value

Mirror the image vertically and/or horizontally. Possible values are horizontal horizontal, vertical or even both (guess what). Short values are h and v. Examples: -m horizontal, -m v, -m both

-h, --help

Display help text

--orientation

Enable image rotation. Rotate your RAW files before running linear-tiff with your Raw Converter. “Orientation” metadata given in XMP file overrides any according RAW file entry.

-o, --output path

Output directory — default is current working directory. Example: -o results

--rating stars

Omit images that do not reach this star rating threshold. At least in Adobe Camera Raw, rejected images have -1 stars. To omit these (i.e. process only non-rejected files), pass rating parameter like so: --rating 0. To process only starred images, set to --rating 1

-w, --width pixel

Resize image — pixel width for larger side, preserving aspect ratio. Example: -r 3000

-v, --verbous

Verbous mode — show some more information under way.

Deprecated Options

-f, --flipflop (deprecated)

Mirror the image vertically and/or horizontally. Possible values are flop horizontal, flip vertical or even flipflop (guess what). Example: -f flop. This option will be replaced by -m, --mirror as of version 2.

-r, --resize pixel (deprecated)

Resize image — pixel width for larger side, preserving aspect ratio. Example: -r 3000. This option will be replaced by -w, --width as of version 2.

Examples

Convert all images or list of images in current working directory using defaults:

# All images:
$ linear-tiff -a 
$ linear-tiff --all

# List of images:
$ linear-tiff DSC0001.NEF DSC0002.NEF DSC0003.NEF

Desaturate B/W negatives:

# These are equal:
$ linear-tiff -d DSC0001.CR2 DSC0002.CR2
$ linear-tiff --desaturate DSC0001.CR2 DSC0002.CR2

# And those as well:
$ linear-tiff -ad
$ linear-tiff -a --desaturate
$ linear-tiff --all -d
$ linear-tiff --all --desaturate

Resize images:

# These are equal:
$ linear-tiff -r 2048 DSC0001.CR2 DSC0002.CR2
$ linear-tiff -a --resize 2048

Fullstack Conversion:

  • Create B/W versions of all images.
  • Resize and mirror horizontally.
  • Output goes into ‘fullstack’ directory.
  • If the raw file has been cropped in e.g. Adobe Camera Raw, the output will be cropped as well.
  • Skip rejected files.
  • Verbous output is showing what's going on.
# These are equal:
$ linear-tiff -advc -r 2048 -f flop -o fullstack --rating 0
$ linear-tiff --all --rating 0 --desaturate --crop --resize 2048 --mirror horizontal --output fullstack --verbous

Changelog

New Features

v.1.1.6

  • Orientation: Once you have copied your RAW files to your HDD, you will like to rotate any upright/portrait images. linear-tiff will consider this orientation meta data. Just turn it on with option --orientation.
  • Faster conversion: LibRaw's dcraw_emu is now used to extract the linear data. It is a whole lot faster than the original dcraw.

v1.1.4

  • Star rating filter: Photo managers like Adobe Camera Raw let their users reject bad images or rate better ones with ‘stars’. This release introduces a new CLI option --rating that requires a minimum star rating.

v1.1.2

  • Crop output file: The output file is now cropped when the input file's meta data have crop information stored, even in an XMP sidecar file.
    • Currently, crops with angles other than 0 are not supported. The reason, I simply did not experiment with it. Has someone experience on this? issue#8
    • And there are issues with cropping raw files that are not Nikon NEF. Cropping works in generally, but the resulting pixel size still differs from from the expected values in your Raw converter by a few pixels. Workaround: use resize option. issue#8

v1.1.0

  • Long option names for those preferring self-explanatory options like --resize, --desaturate and so on.
  • Improved code quality: Wrapped main features in single functions; Adhere to Bash best practices and coding standards.

Upcoming Features

These features go into the current major version 1:

  • Custom configuration files: Would it not be fine if users could store their favourite options in a configuration file? ~/.negativesrc or ~/linear-tiff.conf or even an INI, YAML or JSON? Head over to issue #7.

Roadmap to version 2

  • The -r option will be renamed to -w, as -r is more natural for the upcoming Rating filter, and so is -w for width.

  • The -f option will be renamed to -m, as the actually performed image action is mirroring horizontally or vertically. The option values flip, flop and flipflop will then become something like V, H or VH.

  • New batch mode trigger: New sub-command all will replace the current -a flag, like so: linear-tiff batch <options>.

  • Orientation: This option will be enabled per default.

Issues and FAQ

To see the full list, head over to the issues page.

Cropping the output image: The output cropping is in fact a “shaving”, when image size displayed in the Raw Converter is different than both the sensor size and draw's output image size. linear_tiff creates the shaving information by the pixel dimensions of the embedded JPG thumbnail and dcraw's output file. Currently Nikon NEF and Canon CR2 are supported. See issue#14 for what's going on.

linear-tiff does not find my Raw photos in batch mode.
Currently, linear-tiff uses a regex to locate RAW files by these extensions: NEF (Nikon), CR2 (Canon), and RAW (Contax, Kodak, Leica, Panasonic). Leave a comment on issue#2 to “order” your favourite file extension. I'll happily lengthen the regex to your needs 😃

I get a error message “mogrify: delegate library support not built-in”
ImageMagick must be compiled with litte-cms2 support. See issue#1 for details.

Development and Contribution

$ git clone https://github.com/tomkyle/negatives-linear-tiff.git

negatives-linear-tiff's People

Contributors

tomkyle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

mikaelsundell

negatives-linear-tiff's Issues

New feature: take account of cropping in Camera Raw et al.

After shooting your negatives it may be useful to crop the RAW files first to get rid of unneeded parts of the negative holder. linear_tiff should consider this crop data and crop the output image accordingly. Example workflow:

  1. User shoot negative, crop RAW in Camera Raw. Crop data is now in XMP sidecar file.
  2. Run linear_tiff:
    • linear_tiff runs dcraw. The resulting TIFF file should be (surely is?) uncropped.
    • linear_tiff runs mogrify and crops the output according to the crop data in XMP file.

The crop data are stored like this: http://www.exiv2.org/tags-xmp-crs.html and can be extracted.

I am not sure about how Lightroom and other RAW converters handle this.

Check input file with dcraw -i

In function convertRawPhotos, we need to check first if the very image can be decoded at all. Currently, the script relies on the common Raw file extensions known as working well. Buuuut:

From dcraw documentation: “dcraw -i: Identify files … Exit status is 0 if dcraw can decode the last file, 1 if it can't.”

Alternatively, as I am working on using libraw's dcraw_emu, one should try the raw-identify command.

raw-identify: The only LibRaw call it uses is open_file(); further code prints the values of the fields of the imgdata structure. The output of raw-identify (without switches) is virtually identical to the output of dcraw -i.”

linear-tiff does not find my Raw photos in batch mode.

Not neccessarily your fault. While dcraw itself can defacto handle every existing RAW photo file extension, linear-tiff uses the find command with a regex to get a list of files to work on. So we have a question of “how long is the regex”. — For now, these most-occuring file extensions are implemented in linear-tiff:

  • NEF (Nikon)
  • CR2 (Canon)
  • RAW (Contax, Kodak, Leica, Panasonic)

Leave me a comment below to “order” your favourite file extension.

"Warning: Hmmm. Mogrifying failed somehow on" file

Dearest developer,

in case you're still maintaining this awesome tool, I'd like to report an error.

$ linear-tiff --debug --desaturate -o result -v P1610369.RW2


/opt/homebrew/bin/linear-tiff: line 567: [: : integer expression expected
P1610369.RW2 is a Panasonic DMC-GH4 image.

Filename: P1610369.RW2
Timestamp: Thu Sep  7 20:43:34 2023
Camera: Panasonic DMC-GH4 ID: 0x0
Normalized Make/Model: =Panasonic/DMC-GH4= CamMaker ID: 47

EXIF:
	MinFocal: 0.0 mm
	MaxFocal: 0.0 mm
	MaxAp @MinFocal: f/0.0
	MaxAp @MaxFocal: f/0.0
	CurFocal: 33.0 mm
	MaxAperture @CurFocal: f/5.6
	FocalLengthIn35mmFormat: 66 mm
	LensMake:
	Lens:


Makernotes:
	DriveMode: -1
	FocusMode: -1
	MeteringMode: -1
	AFPoint: -1
	ExposureMode: -1
	ExposureProgram: -1
	ImageStabilization: -1
	Lens:
	LensFormat: 0, 	LensMount: 10, 	FocalType: 0, Undefined
	LensFeatures_pre:
	LensFeatures_suf:
	MinFocal: 0.0 mm
	MaxFocal: 0.0 mm
	MaxAp @MinFocal: f/0.0
	MaxAp @MaxFocal: f/0.0
	MinAp @MinFocal: f/0.0
	MinAp @MaxFocal: f/0.0
	MaxAp: f/0.0
	MinAp: f/0.0
	CurFocal: 0.0 mm
	CurAp: f/0.0
	MaxAp @CurFocal: f/0.0
	MinAp @CurFocal: f/0.0
	TeleconverterID: 0
	Teleconverter:
	AdapterID: 0
	Adapter:
	AttachmentID: 0
	Attachment:

ISO speed: 200
Shutter: 1/4.0
Aperture: f/16.0
Focal length: 33.0 mm
Flash exposure compensation: 0.00 EV
Embedded ICC profile: no
Number of raw images: 1
Thumb size:  1920 x 1440
Full size:   4816 x 3472
Raw inset, width x height: 4608 x 3456 left: 8 top: 8
Image size:  4624 x 3472
Output size: 4624 x 3472
Image flip: 0
Raw colors: 3
Filter pattern: BGGRBGGRBGGRBGGR
black: 143
Highlight linearity limits: 4079 4079 4079 4079
Makernotes WB data:               coeffs                  EVs
  As shot                   435 256 728 0    0.76  0.00  1.51  0.00
  Tungsten (Incandescent)    442  256  731  256    0.79  0.00  1.51  0.00
  Flash                      621  256  452  256    1.28  0.00  0.82  0.00
  Fine Weather               607  256  475  256    1.25  0.00  0.89  0.00
  Cloudy                     641  256  437  256    1.32  0.00  0.77  0.00
  Shade                      682  256  404  256    1.41  0.00  0.66  0.00
  Illuminant A               442  256  731  256    0.79  0.00  1.51  0.00
  D55                        582  256  480  256    1.18  0.00  0.91  0.00
  ISO Studio Tungsten        442  256  731  256    0.79  0.00  1.51  0.00
  Camera Auto               1743 1024 2915 1024    0.77  0.00  1.51  0.00

Camera2RGB matrix (mode: 1):
1.7620	-0.5566	-0.2055
-0.2834	1.7385	-0.4551
0.0192	-0.4958	1.4767

XYZ->CamRGB matrix:
0.7122	-0.2108	-0.0512
-0.3155	1.1201	0.2231
-0.0541	0.1423	0.5045

Derived D65 multipliers: 2.436786 0.940606 1.562026

Linear TIFF
dcraw_emu -w -t 0 -H 5 -q 3 -o 4 -4 -T P1610369.RW2 ✔
Meta tags
exiftool -overwrite_original -quiet -mwg:Description= -mwg:Creator= -Artist<mwg:Creator -Make<exif:Make -Model<exif:model -Software<exif:software ✔
Mangle output file /opt/homebrew/bin/linear-tiff: line 654: mogrify_shave_options_array[*]: unbound variable
/opt/homebrew/bin/linear-tiff: line 656: mogrify_shave_options_array[@]: unbound variable
Warning: Hmmm. Mogrifying failed somehow on P1610369.RW2.tiff

The file gets converted to .tiff, but every command seems to be ignored.

I hope I didn't misunderstand it, and it's not connected to #1

Resize output using embedded thumbnail?

While dcraw outputs 6016x4016px for my Nikon D5500, Camera Raw and other raw converters display and work with 6000x4000, exactly the size of the embedded thumbnail. Not only should linear-tiff consider this thumbnail size on cropping but on output resizing as well.

Thumbnail extraction: not exiftool but dcraw?

The buildMogrifyShaveOptions function in linear_tiff checks embedded thumbnails with exiftool's binary data extraction. The metatags used are -JpgFromRawor -PreviewImage, depending on the Raw input file's filetype. This is not very elegant.

It seems dcraw can do this as well, and certainly knows better how to extract any thumbnail data. Perhaps worth a try.

dcraw -c -e "filename.CR2" | cat > "filename.jpg"

Feature: --debug mode

In addition to verbous mode triggered with --verbous, we should also have a --debug option.

New Feature: Star rating filter

Many photo managers like Lightroom or Bridge let their users reject bad images or rate better ones with ‘stars’. linear-tiff should get a new CLI option flag to set a minimum rating level.

Dependency injection for functions

So this script is refactored to a main function and some helper functions as well. Nice. All of them on the global level. Not so nice, as the code blocks are not self-containing any longer.

To make sure everything is self-containing again and to prepare for unit testing, all functions should receive the stuff they need as parameter.

function Foo {
  echo "Foo!"
}

function main {
    local local_foo=$1
    shift
    # Call dependency. Output should be "Foo!"
    ($local_foo)
}
main Foo "${@}"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.