ppelleti / whence

A command-line program to determine where a file was downloaded from, based on extended attributes.

Web browsers often use extended attributes to store the URL that a file was downloaded from. Sometimes I have a file lying around and want to know where it was downloaded from, so I wrote the whence command as an easy way to access this information on several major platforms (FreeBSD, Linux, MacOS, and Windows).

Usage

Usage: whence [OPTIONS] FILE ...

  -j, --json                  Print results in JSON format.
  -h, --help                  Print this message and exit.
  -v, --version               Print the version number of whence and exit.

Example

bash$ whence wget-1.20.tar.gz
wget-1.20.tar.gz:
  URL         https://ftp.gnu.org/gnu/wget/wget-1.20.tar.gz
  Referrer    https://ftp.gnu.org/gnu/wget/
  Application Firefox
  Date        Sun Jun  7 11:30:18 PDT 2020

Download and install

Pre-built binaries

For release 0.9.2:

• FreeBSD, arm64
• Linux, armv7l
• Linux, x86_64
• MacOS, x86_64
• Windows, x86_64

Building from source

Run ./build.sh to build.

On Windows, MinGW is assumed. I haven't attempted to get it working with MSVC.

On MacOS, SQLite 3 is required, but it should already be present as part of the operating system.

Installation

On Windows, copy whence.exe to a directory on your PATH.

On other platforms, copy whence to /usr/local/bin and copy whence.1 to /usr/local/share/man/man1. You can use the supplied install-whence.sh script to do this, which uses install to get the file permissions correct.

Extended Attributes

Linux and FreeBSD

On Linux and FreeBSD, the following extended attributes from Common Extended Attributes are used:

• user.xdg.origin.url
• user.xdg.referrer.url
• user.xdg.origin.email.from
• user.xdg.origin.email.subject
• user.xdg.origin.email.message-id
• user.xdg.publisher

MacOS

On MacOS, the following extended attributes are used:

• com.apple.metadata:kMDItemWhereFroms
• com.apple.quarantine

com.apple.metadata:kMDItemWhereFroms is an array of strings, stored as a binary property list. The array contains two strings (URL, Referrer) for web pages, and three strings (From, Subject, Message-ID) for email messages.

com.apple.quarantine is a string containing four fields, separated by semicolons. The fourth field is a UUID which is a key into the SQLite database ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2. This database can be used to look up the URL and Referrer if com.apple.metadata:kMDItemWhereFroms is absent.

Windows

On Windows, the NTFS alternate data stream named Zone.Identifier is used. The contents of the stream look like this:

[ZoneTransfer]
ZoneId=3
ReferrerUrl=https://mirrors.ocf.berkeley.edu/gnu/make/
HostUrl=https://mirrors.ocf.berkeley.edu/gnu/make/make-4.2.tar.bz2

The ZoneId number is mapped to a name by looking in the registry.

License

whence is distributed under the terms of the MIT License.

Change Log

0.9.2

• Support XDG attributes on MacOS.
• Support com.apple.metadata:kMDItemDownloadedDate on MacOS.
• Generate correct Unicode escapes in JSON strings.
• Support Unicode filenames on Windows.
• Added install-whence.sh script for convenience on UNIX.
• On UNIX, do not assume ANSI color is supported. Instead, check the TERM environment variable against a list of terminals known to support color.
• Do not use ANSI color if NO_COLOR environment variable is set.

0.9

• Initial release.

whence's Issues

Naming conflict with ksh(1) builtin

In ksh(1) (and possibly other shells), whence is a built-in command that behaves similarly to POSIX's command built-in.

I doubt this merits a name-change, but it might be worth footnoting for any ksh(1) users who may be unaware of the whence built-in's existence (both in the README and in the man page).