nvchecker (short for new version checker) is for checking if a new version of some software has been released.
Contents
- Dependency
- Running
- Version Record Files
- Version Source Files
- Configuration Section
- Global Optons
- Search in a Webpage
- Find with a Command
- Check AUR
- Check GitHub
- Check BitBucket
- Check GitLab
- Check PyPI
- Check RubyGems
- Check NPM Registry
- Check Hackage
- Check CPAN
- Check Packagist
- Check Local Pacman Database
- Check Arch Linux official packages
- Check Debian Linux official packages
- Check Ubuntu Linux official packages
- Check Anitya (release-monitoring.org)
- Check Android SDK
- Manually updating
- Version Control System (VCS) (git, hg, svn, bzr)
- Other
- Bugs
- Footnotes
Dependency
- Python 3.5+
- Python library: structlog
- One of these Python library combinations (ordered by preference):
- tornado + pycurl
- aiohttp
- tornado
- All commands used in your version source files
Running
To see available options:
./nvchecker --help
Run with one or more software version source files:
./nvchecker source_file
You normally will like to specify some "version record files"; see below.
JSON logging
With --logger=json
or --logger=both
, you can get a structured logging
for programmatically consuming. You can use --json-log-fd=N
to specify the
file descriptor to send logs to (take care to do line buffering). The logging
level option (-l
or --logging
) doesn't take effect with this.
The JSON log is one JSON string per line. The following documented events and fields are stable, undocumented ones may change without notice.
- event=updated
- An update is detected. Fields
name
,old_version
andversion
are available.old_version
maybenull
. - event=up-to-date
- There is no update. Fields
name
andversion
are available. - event=no-result
- No version is detected. There may be an error. Fields
name
is available. - level=error
- There is an error. Fields
name
andexc_info
may be available to give further information.
Version Record Files
Version record files record which version of the software you know or is available. They are simple key-value pairs of (name, version)
separated by a space:
fcitx 4.2.7 google-chrome 27.0.1453.93-200836 vim 7.3.1024
Say you've got a version record file called old_ver.txt
which records all your watched software and their versions, as well as some configuration entries. To update it using nvchecker
:
./nvchecker source.ini
See what are updated with nvcmp
:
./nvcmp source.ini
Manually compare the two files for updates (assuming they are sorted alphabetically; files generated by nvchecker
are already sorted):
comm -13 old_ver.txt new_ver.txt # or say that in English: comm -13 old_ver.txt new_ver.txt | awk '{print $1 " has updated to version " $2 "."}' # show both old and new versions join old_ver.txt new_ver.txt | awk '$2 != $3'
nvtake
Command
The This command helps to manage version record files. It reads both old and new version record files, and a list of names given on the commandline. It then update the versions of those names in the old version record file.
This helps when you have known (and processed) some of the updated software, but not all. You can tell nvchecker that via this command instead of editing the file by hand.
This command will help most if you specify where you version record files are in your config file. See below for how to use a config file.
Version Source Files
The software version source files are in ini format. Section names is the name of the software. Following fields are used to tell nvchecker how to determine the current version of that software.
See sample_source.ini
for an example.
Configuration Section
A special section named __config__
is special, it provides some configuration options.
Relative path are relative to the source files, and ~
and environmental variables are expanded.
Currently supported options are:
- oldver
- Specify a version record file containing the old version info.
- newver
- Specify a version record file to store the new version info.
- proxy
- The HTTP proxy to use. The format is
host:port
, e.g.localhost:8087
. - max_concurrent
- Max number of concurrent jobs. Default: 20.
- keyfile
- Specify an ini config file containing key (token) information. This file should contain a
keys
section, mapping key names to key values. See specific source for the key name(s) to use.
Global Options
The following options apply to all checkers.
- prefix
- Strip the prefix string if the version string starts with it. Otherwise the version string is returned as-is.
- from_pattern, to_pattern
- Both are Python-compatible regular expressions. If
from_pattern
is found in the version string, it will be replaced withto_pattern
.
If both prefix
and from_pattern
/to_pattern
are used,
from_pattern
/to_pattern
are ignored. If you want to strip the prefix
and then do something special, just use from_pattern`
/to_pattern
. For
example, the transformation of v1_1_0
=> 1.1.0
can be achieved with
from_pattern = v(\d+)_(\d+)_(\d+)
and to_pattern = \1.\2.\3
.
Search in a Webpage
Search through a specific webpage for the version string. This type of version finding has these fields:
- url
- The URL of the webpage to fetch.
- encoding
- (Optional) The character encoding of the webpage, if
latin1
is not appropriate. - regex
A regular expression used to find the version string.
It can have zero or one capture group. The capture group or the whole match is the version string.
When multiple version strings are found, the maximum of those is chosen.
- proxy
- The HTTP proxy to use. The format is
host:port
, e.g.localhost:8087
. - user_agent
- The
User-Agent
header value to use. Use something more like a tool (e.g.curl/7.40.0
) in Europe or the real web page won't get through because cookie policies (SourceForge has this issue). - sort_version_key
- Sort the version string using this key function. Choose between
parse_version
andvercmp
. Default value isparse_version
.parse_version
usepkg_resources.parse_version
.vercmp
usepyalpm.vercmp
.
Find with a Command
Use a shell command line to get the version. The output is striped first, so trailing newlines do not bother.
- cmd
- The command line to use. This will run with the system's standard shell (i.e.
/bin/sh
).
Check AUR
Check Arch User Repository for updates.
- aur
- The package name in AUR. If empty, use the name of software (the section name).
- strip-release
- Strip the release part.
Check GitHub
Check GitHub for updates. The version returned is in date format %Y%m%d
, e.g. 20130701
.
- github
- The github repository, with author, e.g.
lilydjwg/nvchecker
. - branch
- Which branch to track? Default:
master
. - use_latest_release
- Set this to
true
to check for the latest release on GitHub. An annotated tag creates a "release" on GitHub. It's not the same with git tags, which includes both annotated tags and lightweight ones. - use_max_tag
- Set this to
true
to check for the max tag on GitHub. Unlikeuse_latest_release
, this option includes both annotated tags and lightweight ones, and return the biggest one sorted bypkg_resources.parse_version
. - include_tags_pattern
- Only include tags matching this pattern. This option must be used together
with
use_max_tag
. - ignored_tags
- Ignore certain tags while computing the max tag. Tags are separate by
whitespaces. This option must be used together with
use_max_tag
. This can be useful to avoid some known badly versioned tags, so the newer tags won't be "overridden" by the old broken ones. - sort_version_key
- Sort the version string using this key function. Choose between
parse_version
andvercmp
. Default value isparse_version
.parse_version
usepkg_resources.parse_version
.vercmp
usepyalpm.vercmp
. - proxy
- The HTTP proxy to use. The format is
host:port
, e.g.localhost:8087
.
An environment variable NVCHECKER_GITHUB_TOKEN
or a key named github
can be set to a GitHub OAuth token in order to request more frequently than
anonymously.
Check BitBucket
Check BitBucket for updates. The version returned
is in date format %Y%m%d
, e.g. 20130701
.
- bitbucket
- The bitbucket repository, with author, e.g.
lilydjwg/dotvim
. - branch
- Which branch to track? Default is the repository's default.
- use_max_tag
- Set this to
true
to check for the max tag on BitBucket. Will return the biggest one sorted bypkg_resources.parse_version
. - ignored_tags
- Ignore certain tags while computing the max tag. Tags are separate by
whitespaces. This option must be used together with
use_max_tag
. This can be useful to avoid some known badly versioned tags, so the newer tags won't be "overridden" by the old broken ones. - sort_version_key
- Sort the version string using this key function. Choose between
parse_version
andvercmp
. Default value isparse_version
.parse_version
usepkg_resources.parse_version
.vercmp
usepyalpm.vercmp
.
Check GitLab
Check GitLab for updates. The version returned is in date format %Y%m%d
, e.g. 20130701
.
- gitlab
- The gitlab repository, with author, e.g.
Deepin/deepin-music
. - branch
- Which branch to track? Default:
master
. - use_max_tag
- Set this to
true
to check for the max tag on BitBucket. Will return the biggest one sorted bypkg_resources.parse_version
. - ignored_tags
- Ignore certain tags while computing the max tag. Tags are separate by
whitespaces. This option must be used together with
use_max_tag
. This can be useful to avoid some known badly versioned tags, so the newer tags won't be "overridden" by the old broken ones. - sort_version_key
- Sort the version string using this key function. Choose between
parse_version
andvercmp
. Default value isparse_version
.parse_version
usepkg_resources.parse_version
.vercmp
usepyalpm.vercmp
. - host
- Hostname for self-hosted GitLab instance.
- token
- GitLab authorization token used to call the API.
Authenticated only.
To set a authorization token, you can set:
- a key named
gitlab_{host}
in the keyfile (wherehost
is formed the same as the environment variable, but all lowercased). - an environment variable
NVCHECKER_GITLAB_TOKEN_{host}
must provide that token. Thehost
part is the uppercased version of thehost
setting, with dots (.
) and slashes (/
) replaced by underscores (_
), e.g.NVCHECKER_GITLAB_TOKEN_GITLAB_COM
. - the token option
Check PyPI
Check PyPI for updates.
- pypi
- The name used on PyPI, e.g.
PySide
.
Check RubyGems
Check RubyGems for updates.
- gems
- The name used on RubyGems, e.g.
sass
.
Check NPM Registry
Check NPM Registry for updates.
- npm
- The name used on NPM Registry, e.g.
coffee-script
.
Check Hackage
Check Hackage for updates.
- hackage
- The name used on Hackage, e.g.
pandoc
.
Check CPAN
Check MetaCPAN for updates.
- cpan
- The name used on CPAN, e.g.
YAML
. - proxy
- The HTTP proxy to use. The format is
host:port
, e.g.localhost:8087
.
Check Packagist
Check Packagist for updates.
- packagist
- The name used on Packagist, e.g.
monolog/monolog
.
Check Local Pacman Database
This is used when you run nvchecker
on an Arch Linux system and the program always keeps up with a package in your configured repositories for Pacman.
- pacman
- The package name to reference to.
- strip-release
- Strip the release part.
Check Arch Linux official packages
This enables you to track the update of Arch Linux official packages, without needing of pacman and an updated local Pacman databases.
- archpkg
- Name of the Arch Linux package.
- strip-release
- Strip the release part.
Check Debian Linux official packages
This enables you to track the update of Debian Linux official packages, without needing of apt and an updated local APT database.
- debianpkg
- Name of the Debian Linux source package.
- suite
- Name of the Debian release (jessie, wheezy, etc, defaults to sid)
- strip-release
- Strip the release part.
Check Ubuntu Linux official packages
This enables you to track the update of Ubuntu Linux official packages, without needing of apt and an updated local APT database.
- ubuntupkg
- Name of the Ubuntu Linux source package.
- suite
- Name of the Ubuntu release (xenial, zesty, etc, defaults to None, which means no limit on suite)
- strip-release
- Strip the release part.
Check Anitya
This enables you to track updates from Anitya (release-monitoring.org).
- anitya
distro/package
, wheredistro
can be a lot of things like "fedora", "arch linux", "gentoo", etc.package
is the package name of the chosen distribution.
Check Android SDK
This enables you to track updates of Android SDK packages listed in sdkmanager --list
.
- android_sdk
- The package path prefix. This value is matched against the
path
attribute in all <remotePackage> nodes in an SDK manifest XML. The first match is used for version comparisons. - repo
- Should be one of
addon
orpackage
. Packages inaddon2-1.xml
useaddon
and packages inrepository2-1.xml
usepackage
.
Manually updating
This enables you to manually specify the version (maybe because you want to approve each release before it gets to the script).
- manual
- The version string.
Version Control System (VCS) (git, hg, svn, bzr)
Check a VCS repo for new commits. The version returned is currently not related to the version of the software and will increase whenever the referred VCS branch changes. This is mainly for Arch Linux.
- vcs
- The url of the remote VCS repo, using the same syntax with a VCS url in PKGBUILD (Pacman's build script). The first VCS url found in the source array of the PKGBUILD will be used if this is left blank. (Note: for a blank
vcs
setting to work correctly, the PKGBUILD has to be in a directory with the name of the software under the path where nvchecker is run. Also, all the commands, if any, needed when sourcing the PKGBUILD need to be installed). - use_max_tag
- Set this to
true
to check for the max tag. Currently only supported forgit
. This option returns the biggest tag sorted bypkg_resources.parse_version
. - ignored_tags
- Ignore certain tags while computing the max tag. Tags are separate by
whitespaces. This option must be used together with
use_max_tag
. This can be useful to avoid some known badly versioned tags, so the newer tags won't be "overridden" by the old broken ones.
Other
More to come. Send me a patch or pull request if you can't wait and have written one yourself :-)
Bugs
- Finish writing results even on Ctrl-C or other interruption.