In following bib file misses the comma after the field value (American Mineralogist) for journal:

❯ cat my2.bib
@Article{Dove:1997,
  author  = {Dove, Martin T.},
  title   = {{Theory of displacive phase transitions in minerals}},
  doi     = {10.2138/am-1997-3-401},
  issn    = {0003-004X},
  journal = {American Mineralogist}
  month   = {apr},
  number  = {3-4},
  pages   = {213--244},
  url     = {https://pubs.geoscienceworld.org/ammin/article/82/3-4/213-244/43266},
  volume  = {82},
  year    = {1997}
}

❯ bibtool my2.bib

  month   = {apr},
__^
*** BibTool WARNING (line 7 in ./my2.bib): Missing ',' assumed.

@Article{	  dove:1997,
  author	= {Dove, Martin T.},
  title		= {{Theory of displacive phase transitions in minerals}},
  doi		= {10.2138/am-1997-3-401},
  issn		= {0003-004X},
  journal	= {American Mineralogist},
  month		= {apr},
  number	= {3-4},
  pages		= {213--244},
  url		= {https://pubs.geoscienceworld.org/ammin/article/82/3-4/213-244/43266},
  volume	= {82},
  year		= {1997}
}

❯ bibtool -s my2.bib

@Article{	  dove:1997,
  author	= {Dove, Martin T.},
  title		= {{Theory of displacive phase transitions in minerals}},
  doi		= {10.2138/am-1997-3-401},
  issn		= {0003-004X},
  journal	= {American Mineralogist},
  month		= {apr},
  number	= {3-4},
  pages		= {213--244},
  url		= {https://pubs.geoscienceworld.org/ammin/article/82/3-4/213-244/43266},
  volume	= {82},
  year		= {1997}
}

v2.67 and commit 4120397 are affected.

Could the message also be printed when sorting is enabled? There is still the switch -q for quiet when someone does not want to see the message.

I might be looking for the wrong keywords, but I could not find an option to check the required fields for each entry (e.g., author, title, journal, year for @article). Is there a way to do this with bibtool?

Perhaps I am mistaken, but it seems from the documentation in A.11.3. (and my limited testing) that the pattern supported by the useful rename.field {old=new if field=pattern} syntax provides no means of renaming a field to another, provided the target field does not exist, but only if it does.

I am trying to delete all number fields and create pages fields for those missing them, iff the originating field did not have one. That is, entries without any page numbers, such as those with only an article number (which I have under the numbers field), should have their number field renamed pages. I then delete the number field.

That is, I would like the following or an equivalent to work:

-- 'delete.field {number}' -- 'rename.field {number=pages ifnotpages}'

Is there a workaround that I could use to accomplish this and if not could this feature be added?

The first three lines look like a leftover from a faild sed run and break the build. Deleting them fixes it for me. This is also valid for the tarball downloaded from the homepage.

Fails to build with latest glibc:

rewrite.c: In function 'get_regex_syntax':
rewrite.c:1351:5: error: duplicate case value
     case RE_SYNTAX_POSIX_EGREP: return "posix_egrep";/*                      */
     ^~~~
rewrite.c:1349:5: note: previously used here
     case RE_SYNTAX_EGREP:       return "egrep";    /*                        */
     ^~~~

These are now the same, regex.h has:

# define RE_SYNTAX_EGREP                                                \
  ((RE_SYNTAX_POSIX_EXTENDED | RE_INVALID_INTERVAL_ORD | RE_NEWLINE_ALT) \
   & ~(RE_CONTEXT_INVALID_OPS | RE_DOT_NOT_NULL))

/* POSIX grep -E behavior is no longer incompatible with GNU.  */
# define RE_SYNTAX_POSIX_EGREP                                          \
  RE_SYNTAX_EGREP

Not sure the best way to handle this.

gcc -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2  -Iinclude  -DREGEX -Iregex-0.12 -I.. -c rewrite.c -o rewrite.o
rewrite.c:135:5: error: conflicting types for 'foreach_addlist'
 int foreach_addlist(fct)      /*                        */
     ^
In file included from rewrite.c:36:0:
include/bibtool/rewrite.h:40:7: note: previous declaration of 'foreach_addlist' was here
  void foreach_addlist _ARG((int (*fct)(String,String)));/* rewrite.c         */
       ^
makefile:479: recipe for target 'rewrite.o' failed

This can be fixed by

sed -i '135s+int+void+' rewrite.c

I am curious why bibtool capitalizes all fields in a bibfile when using the biblatex resource.

As far as I know, biblatex/biber also do not care about capitalized field or entry types.

I could imagine that this can increase readability for fields which are compounds of several words such as indexshorttitle (-> IndexShortTitle).

However, I would assume this is only personal preference and therefore wonder why this is part of the default file.

Furthermore, for the crossref mappings in the last part of the file lib/biblatex.rsc lowercase words are used again.

Tested on version 2.61 (Cygwin):
If I put a comment inside a construct that spans several lines, BibTool ERROR: Symbol expected is thrown.

Example:

sort.order{* =           % * is the wildcard for entry types
	 author
	#sortkey
	#title
	}

Bug or feature?

Is there a way to resolve crossref fields within entries, as done by bibexport -a?

Many (modern) text editors support lint like tools. They usually collect the returned messages in a separate window (in vim it is called the quickfix window). Futhermore line markers are added to automatically inform the user without too much distraction about possible issues.

In the vim plugin world there exist at least three popular ones for this purpose:

1. https://github.com/w0rp/ale
2. https://github.com/vim-syntastic/syntastic
3. https://github.com/neomake/neomake

Ale right now uses bibclean for bib files.

I was wondering whether bibtool could also be used or modified that it can be used for this?

When enclosing a field-value in double pairs of curly braces (to prevent BibTeX to adjust any capitalization), bibtool seems to no longer parse the value correctly. For example, if a bibtex-entry contains the field

title = {{Some title with an acronym like MCMC}}

then %-3.0T(title) will always return the whole title instead of only the first three words.

Of course one could change the above line to

title = {Some title with an acronym like {MCMC}}

so that bibtool again parses correctly, but this would be very inconvenient and time-consuming when entries are obtained automatically from a database (in my case zbmath.org, which always surrounds the fields title, journal and fjournal by double pairs of braces).

Could you please change the behaviour of bibtool so that such double pairs are recognized and the enclosed values parsed correctly?

Since 5fc54f2, biblatex support is broken:

*** BibTool WARNING: Unknown entry type `suppollection'. Mapping ignored.

{article subperiodical} journaltitle = periodical
_________________________________________________^
*** BibTool ERROR: Symbol expected.

}
_^
*** BibTool WARNING:  (line 298 in /tmp/bibtool/lib/biblatex.rsc): Symbol does not start with a letter

Used command line BIBTOOL=$PWD/lib ./bibtool -r biblatex -i ~/phd/own.bib -o foo.bib

Also, why does biblatex.rsc define capitalized key names? The current BibLaTeX manual uses lowercase ones, perhaps that changed at some point.

Hello,

is it possible to sort my bib file by year (& author) and also to keep the bib keys untouched?

This does not work:
bibtool -- sort.format="{%d(year)%N(author)}" -S sort.cased=Off input.bib -o output.bib

Hi.

Seems biblatex entries not supported. I get message:

@Online{RProject,
_^
*** BibTool ERROR:  (line 127 in ./bibs/en-online.bib): Unknown entry type

*** BibTool WARNING: Skiping to next '@'

Bibtex support only a misc entry for this. I can't use new.field.type { misc = online} because I use misc for other cases.

Thx.

Having a bibtex-entry like

@book{,
  title         = {Übertreibungen in Richtung Wahrheit. Stenogramme, Glossen, Aphorismen},
  editor        = {Lütkehaus, Ludger},
  author        = {Anders, Günther},
  year          = {2002},
  publisher     = {C.H. Beck},
  address       = {München},
  isbn          = {3-406-47612-0},
}

The citation key is built by

key.format="{%-1n(author)#%-1n(editor)}{%n(year)#%n(date)}{%-W(title)}"

and outputs correctly günther2002übertreibungen. How do I get a ascii citation key (guenther2002ueberlegungen)?

Old style font commands (\bf, \it, \sc, \tt, \rm) have been deprecated for years. Several packages have dropped support for them now, most notably the KOMA classes. As a result, doc/c_lib.tex cannot be compiled with current TeXlive any more, at least not without adding the enabledeprecatedfontcommands option (which is only a band aid).

BibTcl/bibtcl.tex and doc/ref_card.tex contain old style font commands, as well, but they don't use KOMA classes so that they still compile.

So please remove old style font commands where necessary, or better: everywhere :)

[This is part of getting the BibTool package to build again on Fedora 25 with the updated, current TeXlive.]

my.bib:

@article{Koch:1984,
  author  = {Koch, E.},
  doi     = {10.1107/S0108767384001227},
  issn    = {0108-7673},
  journal = {Acta Crystallographica Section A Foundations of Crystallography},
  month   = {sep},
  number  = {5},
  pages   = {593--600},
  title   = {The implications of normalizers on group–subgroup relations between space groups},
  volume  = {40},
  year    = {1984}
}
@Article{Dove:1997,
  author  = {Dove, Martin T.},
  title   = {Theory of displacive phase transitions in minerals},
  doi     = {10.2138/am-1997-3-401},
  issn    = {0003-004X},
  journal = {American Mineralogist},
  month   = {apr},
  number  = {3-4},
  pages   = {213--244},
  volume  = {82},
  year    = {1997}
}
@article{Koch:1984,
  author  = {Koch, E.},
  doi     = {10.1107/S0108767384001227},
  issn    = {0108-7673},
  journal = {Acta Crystallographica Section A Foundations of Crystallography},
  month   = {sep},
  number  = {5},
  pages   = {593--600},
  title   = {The implications of normalizers on group–subgroup relations between space groups},
  volume  = {40},
  year    = {1984}
}

% Semantic checks
check.rule { month "^[\"{]1?[0-9][\"}]$" }
check.rule { month "" "\@ \$: Month has to be a suitable number\n" }

❯ bibtool -r .bibtoolrsc my.bib > /dev/null
*** BibTool: Article koch:1984: Month has to be a suitable number
*** BibTool: Article dove:1997: Month has to be a suitable number
*** BibTool: Article koch:1984: Month has to be a suitable number

I would like to specify that this is a warning not an error similar to

*** BibTool ERROR:  (line 3 in ./z.bib): Unexpected character encountered
*** BibTool WARNING:  (line 5 in ./z.bib): 17 non-space characters ignored.

The output should be like

❯ bibtool -r .bibtoolrsc my.bib > /dev/null
*** BibTool WARNING: Article koch:1984: Month has to be a suitable number
*** BibTool WARNING: Article dove:1997: Month has to be a suitable number
*** BibTool WARNING: Article koch:1984: Month has to be a suitable number

This is useful when bibtool is run as a linter and the messages are categorized into errors and warnings.

Biblatex provides a data field called xdata, which is like crossref in that it inherits data from some other entry but unlike crossref in that the semantics are not that of a parent-child relationship.

At any rate, it would be nice if the xdata field were treated liked crossref in that you could declare

select.crossrefs = on

and have BibTool follow xdata references for the purposes of stuff like key generation, for example, since it is a field designed for inheriting data.

I'm not sure if it would make more sense to have a separate option (something like select.xdata) or to have them both controlled by the same option (select.crossrefs).

Hello,

it seems I'm not able to extract entries. bibtool -x t.aux simply doesn't do anything, with or without the -o option. Any idea?

Kind regards
Stefan

Are any kind of stresstest performed during development ? If so, would it be possible to provide it within the upstream source tarball ?

The tarball under http://www.gerd-neugebauer.de/software/TeX/BibTool/BibTool-2.58.tar.gz does not include bibtool.tex, but a symlink to it, that points to nowhere.

The format of generated keys seems to differ between versions 2.55 and 2.56: suffixes added for disambiguity start from "b" in 2.55 and from "a" in 2.56. Is there a way to enforce the old behavior?

I'm working on a book with c. 20 other authors and each uses a different version of bibtool. This makes it infeasible that everyone upgrades.

Names with additions (von, van, de) can be sourrounded by curly brackets to enforce a certain sorting, e.g. author={{van Gogh}, Vincent}. However, when applying key.format = { %2n(author):%-2d(year) } during key generation, BibTool does not use the name within curly brackets. It just prints :%-2d(year).

The current PDF file uploaded to the main BibTool webpage (at http://www.gerd-neugebauer.de/software/TeX/BibTool/bibtool.pdf) appears to be missing (the link refers to a zero-byte file). I am, however, still able to access the manual from CTAN.

It would be really nice (and, easy) if you could make a video that shows what can be done with this software and how to do it. This is a kind suggestion (and, a hopeful request). Thank you.

bibtoolrsc

% print.line.length        = 1000000
print.align.key          = 0
print.indent             = 2
print.newline            = 0
print.use.tab            = off
suppress.initial.newline = on
symbol.type              = lower
pass.comments            = on

% Manage keys manually:
preserve.keys            = on
preserve.key.case        = on

% BibLaTeX additions from
new.entry.type{Online}
new.entry.type{Thesis}
new.entry.type{Software}

% Sort records
sort                     = on
sort.macros              = {off}
sort.format              = { %s($type) %s(year) %s(month) {%N(author) # %N(editor)} }

Example bib file:

❯ cat demo.bib
@Software{Singular:2018,
  author        = {Decker, Wolfram and Greuel, Gert-Martin and Pfister, Gerhard and Sch\"onemann, Hans},
  title         = {{\sc Singular} {4-1-1} --- {A} computer algebra system for polynomial computations},
  year          = {2018},
  howpublished  = {\url{http://www.singular.uni-kl.de}},
  url           = {http://www.singular.uni-kl.de/}
}
@Software{cgal,
  author        = {The CGAL Project},
  title         = {The Computational Geometry Algorithms Library {(CGAL)}},
  url           = {https://www.cgal.org/},
}
@Online{DB_COD:2018,
  author        = {Grazulis, S., Chateigner, D., Downs, R. T., Yokochi, A. T., Quiros, M., Lutterotti, L., Manakova, E., Butkus, J., Moeck, P. AND Le Bail},
  title         = {Crystallography Open Database (COD)},
  date          = {2003/},
  url           = {http://www.crystallography.net/cod/},
  urldate       = {2018-05-05},
  annote        = {395.186 entries in the COD}
}
@Article{Bernal:1960,
  author        = {Bernal, J. D.},
  title         = {{Geometry of the Structure of Monatomic Liquids}},
  doi           = {10.1038/185068a0},
  issn          = {0028-0836},
  journal       = {Nature},
  month         = {jan},
  number        = {4706},
  pages         = {68--70},
  volume        = {185},
  year          = {1960}
}

❯ bibtool -v demo.bib
--- BibTool: Trying /Users/kiryph/.bibtoolrsc
--- BibTool: Set resource print.align.key = 0
--- BibTool: Set resource print.indent = 2
--- BibTool: Set resource print.newline = 0
--- BibTool: Set resource print.use.tab = off
--- BibTool: Set resource suppress.initial.newline = on
--- BibTool: Set resource symbol.type = lower
--- BibTool: Set resource pass.comments = on
--- BibTool: Set resource preserve.keys = on
--- BibTool: Set resource preserve.key.case = on
--- BibTool: Set resource new.entry.type = Online
--- BibTool: Set resource new.entry.type = Thesis
--- BibTool: Set resource new.entry.type = Software
--- BibTool: Set resource sort = on
--- BibTool: Set resource sort.macros = off
--- BibTool: Set resource sort.format =  %s($type) %N(year) month {%N(author) # %N(editor)}
--- BibTool: Trying ./demo.bib
--- BibTool: Reading demo.bib
++++--- BibTool: Done with demo.bib
@Article{Bernal:1960,
  author        = {Bernal, J. D.},
  title         = {{Geometry of the Structure of Monatomic Liquids}},
  doi           = {10.1038/185068a0},
  issn          = {0028-0836},
  journal       = {Nature},
  month         = {jan},
  number        = {4706},
  pages         = {68--70},
  url           = {http://www.nature.com/doifinder/10.1038/185068a0},
  volume        = {185},
  year          = {1960}
}
@Software{Singular:2018,
  author        = {Decker, Wolfram and Greuel, Gert-Martin and Pfister,
                  Gerhard and Sch\"onemann, Hans},
  title         = {{\sc Singular} {4-1-1} --- {A} computer algebra system for
                  polynomial computations},
  year          = {2018},
  howpublished  = {\url{http://www.singular.uni-kl.de}},
  url           = {http://www.singular.uni-kl.de/}
}
@Software{cgal,
  author        = {The CGAL Project},
  title         = {The Computational Geometry Algorithms Library {(CGAL)}},
  url           = {https://www.cgal.org/}
}
@Online{DB_COD:2018,
  author        = {Grazulis, S., Chateigner, D., Downs, R. T., Yokochi, A.
                  T., Quiros, M., Lutterotti, L., Manakova, E., Butkus, J.,
                  Moeck, P. AND Le Bail},
  title         = {Crystallography Open Database (COD)},
  date          = {2003/},
  url           = {http://www.crystallography.net/cod/},
  urldate       = {2018-05-05},
  annote        = {395.186 entries in the COD}
}

IMHO it looks like bibtool cannot sort the new entry types. They are treated as the same type and sorted by the other criteria.

key.c:255:12: Fehler: »null« nicht deklariert (erste Benutzung in dieser Funktion)
return null; /* /
^~~~
key.c:255:12: Anmerkung: jeder nicht deklarierte Bezeichner wird nur einmal für jede Funk
ion, in der er vorkommt, gemeldet
make: ** [makefile:487: key.o] Fehler 1

Having

define null ((void*)0)' key.c

fixes it.

debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=848947
bureport-848947.zip

Please provide the upstream source balls in more compressed formats as bz2 and xz besides gz .

Is it possible to provide an official ld-version-script for the libtool library ?
For ld-version-script information, a goot start is provided in the GNULib documentation:
https://www.gnu.org/software/gnulib/manual/html_node/LD-Version-Scripts.html

Bibtool considers two records as duplicates in following circumstance

Ad 2: See resource check.double.
Double entries are identified if their sort key is identical.
#50 (comment)

Would it be possible to add additional warnings, if two records have the same

• bibkey (useful when the sorting is customized or sorting is not enabled), or
• doi number
• arxiv id
• isbn number

The doi number check could be added to the default duplicate detection with the auto fix capabilities of bibtool (deactivate with ### or delete record).
A duplicate bibkey can also mean the user has chosen the same bibkey for different records and therfore the bibkey must be renamed but surely the record should not be deleted.

These additions would also increase the usefulness of bibtool as a linting tool as asked int #52 .

Hi, it seems there is a problem with BibTool processing entries that have been marked via JabRef.
If you mark an entry in the JabRef GUI program, it adds a data field of the type __markedentry to the entry. If you try to process this file with BibTool, it returns something like:

*** BibTool WARNING:  (line 5 in ./unified-test.bib): 48 non-space characters ignored.

  __markedentry            = {[adamliter:3]}
___^
*** BibTool ERROR:  (line 23 in ./unified-test.bib): Symbol does not start with a letter
*** BibTool WARNING: Skiping to next '@'

and it will not print the offending entry in the output.
Is there a fix for this? I'm trying to use BibTool to sanitize a .bib file maintained with JabRef.

Is there any possibility to have complex select statement combinations?
I am currently missing something like

((select {keywords "firstkeyword"} AND select {keywords "secondkeyword"}) OR select{keyword "alwaysselected"})

I can currently only achieve this with pipes on the commandline, combining the output of consecutive invocations of bibtool.
A regex might be a solution for OR cases in common field definitions.

AND/OR(currently default) and maybe XOR would be all logic operators needed (NOT selectors like existing with select.non should also work).

Thank you for the great tool!

In the installation instructions present in the documentation (e.g., README.md, install.tex), please mention that bibtool has been packaged for a couple of distributions (e.g., Debian based distributions as Ubuntu), so that potential users can consider to install the bibtool package effectively distributed by their favourite distribution before installing from the upstream source. This is certain the best for every one: the upstream maintainer, the packages maintainers, and the final users.

The compilation fails with

gcc -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2  -Iinclude  -DREGEX -Iregex-0.12 -I.. -DHAVE_LIBKPATHSEA -c main.c
main.c:81:24: schwerwiegender Fehler: bibtool/io.h: Datei oder Verzeichnis nicht gefunden

and indeed that file is missing in bibtool/include/bibtool.

In the section entitled `Sorting and Merging' (1.2.1), the sentence introducing the reverse ASCII order sorting seems to lack the option part.

Is there a way to remove fields that don't belong to a predefined set? I would like to define a set of fields, e.g. {author,title,year,publisher}, and let BibTool remove any field that is not a member of this set.

I'm dealing with large collections of BibTeX entries from various domains, and it's frightening how inventive scholars are. Therefore excluding non-standard fields one by one with delete.field is not an option.

Hello, I've linked some snippets to help explain what I am observing:

https://gist.github.com/jeffreywildman/a559d893a062c2dd481b

The output of bibtool adds an additional space between "HELLO" and the hash symbol in the lines containing 1) a string in double quotes, followed by 2) a hash symbol. In this particular example, they happen to be lines containing the title and publisher fields.

I recognize that this additional whitespace does not affect the content of the fields, but this was unexpected behavior to me. If this behavior is expected, are there bibtool options to remove the additional whitespace? Thanks in advance.

Hi.

I try select some entries. But when I specify out file the bibtool returns error.

bibtool -- select{URL "pdf"} bibs/*.bib

It's ok.

bibtool -- select{URL "pdf"} bibs/*.bib -o /tmp/out.bib

select{URL
__________^
*** BibTool ERROR: Unexpected end of braces.

*** BibTool WARNING: File pdf} not found.

I like section 1.1. Related Programs in the documentation.

I would add following opensource software projects which other might find interesting as well

• betterbib (github, pypi)
  It offers for example the possibility to complete records via online query, it can abbreviate or unabbreviate journal names, and get records by doi, e.g. $ cat dois.txt | xargs -I '{}' betterbib-doi2bibtex '{}' and more.
• BibDesk (sourceforge.net)
  I consider this as a good opensource alternative GUI to JabRef for macOS user which does not require Java (not preinstalled on macOS).
• python-bibtexparser (github, pypi)
  Bibtex parser for Python 2.7 and 3.3 or newer. A python script offers more possibilities than the bibtoolrsc file.
• bibtex-tidy (github, npm) Cleaner and Formatter for BibTeX files written in JavaScript. It can be used in the browser at https://flamingtempura.github.io/bibtex-tidy/.
• bibcure (github, pypi) Bibcure helps in boring tasks by keeping your bibfile up to date and normalized...also allows you to easily download all papers inside your bibtex

Text Editors

Emacs

Handling bib files in Emacs seems to be superior to Vim. There is the already mentioned bibtex-mode which is part of GNU Emacs. However, there is at least following additional option:

• ebib (github, website)
  Ebib is a program for managing BibTeX and BibLaTeX databases that runs inside Emacs.

Vim

I use vim and therefore I am interested in vim solutions. Unfortunately, I do not know any active ftplugin for vim with a similar feature set as ebib has. Some projects I noticed:

• fzf-bibtex (github) actually uses bibtool and provides also additional command line tools such as bibtool-ls, bibtool-cite.
• tbibtools (vim.org) sort bib files, reformat bib files, list keys in a bib file, show bib entries matching a regular expression

This is a personal choice and I know not everybody follows it,
but I prefer to have unlimited line width, ensuring one field per line.
My editor does line breaks dynamically.

Would it be possible to add support for unlimited line width?
Zero or negative values could be used to indicate unlimited line width,
i.e. --print.line.length=0 really means --print.line.length=∞

Hi,
I just have a comment.
I use bibtool to filter my bibtex entries and it works quite well. The only annoying thing was that the output of the bibtex-key ignores the case from the bib file.

The manual says "Some times it can be desirable to preserve the case of the key as given (even so BibTEX does not mind)." (page 27)

This is not true for my bibtex version (0.99d). Bibtex cannot find the entry with key "Examplekey" when it is specified in the latex document with \cite{examplekey}.

While this can be fixed with a bibtools resource file, I believe it would save many users some time if "preserve.key.case = on" would be set by default.

Cheers!

the following command is not sorting entries by year

bibtool -S -sort.format="%(year)" -i publications.bib

Hi @ge-ne,
thanks a lot for sharing bibtool.
If you don't mind me asking here a couple of questions which I've searched over on google but was unable to find some coherent examples.

My use case contains of creating .bib files for my work and occasionally merge them to a bigger .bib file. The citekeys I use are of the following pattern author_year_unique_consecutive_letter, without the underscores.

Example:

doe2018
doe2018b
doe2018c
doe2018d
etc...

I would like eventually to achieve the following when merging .bib files to the main bigger one.

1. search all entries and if there's a missing field such as for instance journal then add a user defined text

I tried the following command looking at the manual but seems not to produce the desired output.
bibtool 'add.field={journal="(journal){%N(journal)}{--no-journal--}"}' ./biblio.bib

2. before merging everything to the bigger .bib file check for duplicate entries, meaning find entries where they have the same key and same title and resolve those by deleting the excessive entries, in the case where there's only duplicate keys but the titles among those entries are different then extract those entries and dump them in an additional file called duplicates for manual inspection, avoiding merging them altogether in the bigger .bib file.

Thanks!

I guess the index engine in https://github.com/ge-ne/bibtool/blob/master/INSTALL should be makeindex instead of makeidx

When I clone this repository and run make, I get an endless stream of messages the first one Too many open files:

~/Library/Caches/Homebrew/bib-tool--git master*
❯ git remote -v
origin	https://github.com/ge-ne/bibtool.git (fetch)
origin	https://github.com/ge-ne/bibtool.git (push)

❯ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

❯ make
makefile:29: makefile: Too many open files
makefile:71: warning: overriding commands for target `configure'
makefile:71: warning: ignoring old commands for target `configure'
makefile:74: warning: overriding commands for target `lint'
makefile:74: warning: ignoring old commands for target `lint'
makefile:77: warning: overriding commands for target `dist'
makefile:77: warning: ignoring old commands for target `dist'
makefile:88: warning: overriding commands for target `ctan'
makefile:88: warning: ignoring old commands for target `ctan'
makefile:71: warning: overriding commands for target `configure'
makefile:71: warning: ignoring old commands for target `configure'
makefile:74: warning: overriding commands for target `lint'
makefile:74: warning: ignoring old commands for target `lint'
makefile:77: warning: overriding commands for target `dist'
makefile:77: warning: ignoring old commands for target `dist'
makefile:88: warning: overriding commands for target `ctan'
makefile:88: warning: ignoring old commands for target `ctan'
...

A more recent GNU make simply stops:

❯ gmake --version
GNU Make 4.2.1
Built for x86_64-apple-darwin16.5.

❯ gmake
makefile:29: *** Too many open files.  Stop.

My naive guess is that my file system is case insensitive which means the line

include makefile

has the effect that the Makefile simply includes itself again and again.

I use bibtool to generate new cite keys with following argument:
-f '%-1w(author)%4d(year)%-1W(title)'
In ~/.bibtoolsrc I have :
fmt.word.separator = "+-<=>*/"

But the key generated is
author2000abc-def
I want to make - as the separator and have just the key author2000abc.

The original bib record is in form of
title = "The abc-def of blablabla. "

Could please help?
Thanks

The tests crossref_map_[12] and expand_xdata_[123] emit a segmentation fault on the architectures i386, armhf, and armel. See Debian bugreport 827102.
I could reproduce the issue on the Debian porters. Please let me know if you can reproduce the issue on your side, if more help is needed to step forward.

By default key generation seems to be turned on as son as you have defined key.format. I would like to have my standard key.format in my ~/.bibtoolsrc. But if that is the case, even defined keys are overwritten by a normal bibtools-run. So I would be very happy if it would be possible to have key.format defined without having key generation turned on. That's why I defined key.format and turned off keygeneration by default in my ~/.bibtoolsrc and thought that it would be possible to turn on keygeneration by bibtool -k. _Having preserve.keys=on in ~/.bibtoolsrc, bibtool -k does not regenerate keys for existing keys._ To demontrate the behavior I wrote a little minimal-example in bash (not using ~/.bibtoolsrc, because I don't want to overwrite yours). Have you got an solution for my problem?

#!/bin/bash
rsc=$(tempfile -s .rsc)

bib="@book{key,
author={Hans Christian Andersen},
title={Das hässliche Entlein},
year={1843}}"

echo $bib | bibtool -r $rsc

echo "I want to define my key.format in my main ~/.bibtoolsrc. For
demonstration purposes, I define it in a temporary rsc-file. I didn't expect
that definition to turn on key-generation."

echo 'key.format="%-1n(author)%n(year)%-W(title)"'>$rsc

echo -e "\n\nThe definition of key.format seems to turn on key-generation:"
echo $bib | bibtool -r $rsc

echo -e "\n\nSo I add preserve.keys, so that I don't overwrite existing keys"
echo 'preserve.keys=on'>>$rsc
echo $bib | bibtool -r $rsc

echo -e "\n\nBut sometimes I need to overwrite existing keys, so I add -k to
bibtool and expect it to do the key-generation. But that doesn't happen."
echo $bib -k| bibtool -r $rsc

echo "How can I define my key.format in the main ~/.bibtoolsrc keeping the 
default behavior of key-generation?"

rm $rsc