csuyangpeng / bazel-remote

A remote HTTP/1.1 cache for Bazel

A remote build cache for Bazel

bazel-remote is a HTTP/1.1 server that is intended to be used as a remote build cache for Bazel. The cache contents are stored in a directory on disk. One can specify a maximum cache size and bazel-remote will automatically enforce this limit and clean the cache by deleting files based on their last access time. The cache supports HTTP basic authentication with usernames and passwords being specified by a .htpasswd file.

Cache entries are set and retrieved by key, and there are two types of keys that can be used:

1. Content addressed storage (CAS), where the key is the lowercase SHA256 hash of the stored value. The REST API for these entries is: /cas/<key> or with an optional but ignored cache pool name: /<pool>/cas/<key>.
2. Action cache, where the key is an arbitrary 64 character lowercase hexadecimal string. Bazel uses the SHA256 hash of an action as the key, to store the metadata created by the action. The REST API for these entries is: /ac/<key> or with an optional cache pool name: /<pool>/ac/<key>.

Values are stored via HTTP PUT requests, and retrieved via GET requests. HEAD requests can be used to confirm whether a key exists or not.

Project status: bazel-remote has been serving TBs of cache artifacts per day since April 2018, both on commodity hardware and AWS servers. Outgoing bandwidth can exceed 15 Gbit/s on the right AWS instance type.

Usage

The cache can be configured via command line flags, environment variables or a YAML configuration file. See ./config/config_test.go for the configuration format.

$ ./bazel-remote --help
NAME:
   bazel-remote - A remote build cache for Bazel

USAGE:
   bazel-remote [global options] command [command options] [arguments...]

DESCRIPTION:
   A remote build cache for Bazel.

COMMANDS:
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --config_file value    Path to a YAML configuration file. If this flag is specified then all other flags are ignored. [$BAZEL_REMOTE_CONFIG_FILE]
   --dir value            Directory path where to store the cache contents. This flag is required. [$BAZEL_REMOTE_DIR]
   --max_size value       The maximum size of the remote cache in GiB. This flag is required. (default: -1) [$BAZEL_REMOTE_MAX_SIZE]
   --host value           Address to listen on. Listens on all network interfaces by default. [$BAZEL_REMOTE_HOST]
   --port value           The port the HTTP server listens on. (default: 8080) [$BAZEL_REMOTE_PORT]
   --htpasswd_file value  Path to a .htpasswd file. This flag is optional. Please read https://httpd.apache.org/docs/2.4/programs/htpasswd.html. [$BAZEL_REMOTE_HTPASSWD_FILE]
   --tls_enabled          This flag has been deprecated. Specify tls_cert_file and tls_key_file instead. [$BAZEL_REMOTE_TLS_ENABLED]
   --tls_cert_file value  Path to a pem encoded certificate file. [$BAZEL_REMOTE_TLS_CERT_FILE]
   --tls_key_file value   Path to a pem encoded key file. [$BAZEL_REMOTE_TLS_KEY_FILE]
   --idle_timeout value   The maximum period of having received no request after which the server will shut itself down. Disabled by default. (default: 0s) [$BAZEL_REMOTE_IDLE_TIMEOUT]
   --help, -h             show help

Docker

Prebuilt Image

We publish docker images to DockerHub that you can use with docker run. The below command will start the remote cache on port 9090 with the default maximum cache size of 5 GiB.

$ docker pull buchgr/bazel-remote-cache
$ docker run -v /path/to/cache/dir:/data -p 9090:8080 buchgr/bazel-remote-cache

Note that you will need to change /path/to/cache/dir to a valid directory where the docker container can write to and read from. If you want the docker container to run in the background pass the -d flag right after docker run.

You can change the maximum cache size by appending the --max_size=N flag with N being the max. size in Gibibytes.

Build your own

The below command will build a docker image from source and install it into your local docker registry.

$ bazel run :bazel-remote-image

Build a standalone Linux binary

$ bazel build :bazel-remote

Authentication

In order to pass a .htpasswd and/or server key file(s) to the cache inside a docker container, you first need to mount the file in the container and pass the path to the cache. The below example also configures TLS which is technically optional but highly recommended in order to not send passwords in plain text.

$ docker run -v /path/to/cache/dir:/data \
-v /path/to/htpasswd:/etc/bazel-remote/htpasswd \
-v /path/to/server_cert:/etc/bazel-remote/server_cert \
-v /path/to/server_key:/etc/bazel-remote/server_key \
-p 9090:8080 buchgr/bazel-remote-cache --tls_enabled=true \
--tls_cert_file=/etc/bazel-remote/server_cert --tls_key_file=/etc/bazel-remote/server_key \
--htpasswd_file /etc/bazel-remote/htpasswd --max_size=5

Configuring Bazel

Please take a look at Bazel's documentation section on remote caching

http://192.168.1.102:8080/status

{
 "CurrSize": 1700386382,
 "MaxSize": 10737418240,
 "NumFiles": 20726,
 "ServerTime": 1553143020
}