Coder Social home page Coder Social logo

blazer's Introduction

Blazer

GoDoc Build Status

Blazer is a Golang client library for Backblaze's B2 object storage service. It is designed for simple integration with existing applications that may already be using S3 and Google Cloud Storage, by exporting only a few standard Go types.

It implements and satisfies the B2 integration checklist, automatically handling error recovery, reauthentication, and other low-level aspects, making it suitable to upload very large files, or over multi-day time scales.

import "github.com/kurin/blazer/b2"

Except see https://www.backblaze.com/blog/backblaze-b2-s3-compatible-api/; you probably don't need this library anymore.

Examples

Getting started

id := "B2_ACCOUNT_ID"
key := "B2_ACCOUNT_KEY"

ctx := context.Background()

// b2_authorize_account
b2, err := b2.NewClient(ctx, id, key)
if err != nil {
	log.Fatalln(e)
}

buckets, err := b2.ListBuckets(ctx)
if err != nil {
	log.Fatalln(e)
}

Copy a file into B2

func copyFile(ctx context.Context, bucket *b2.Bucket, src, dst string) error {
	f, err := os.Open(src)
	if err != nil {
		return err
	}
	defer f.Close()

	obj := bucket.Object(dst)
	w := obj.NewWriter(ctx)
	if _, err := io.Copy(w, f); err != nil {
		w.Close()
		return err
	}
	return w.Close()
}

If the file is less than 100MB, Blazer will simply buffer the file and use the b2_upload_file API to send the file to Backblaze. If the file is greater than 100MB, Blazer will use B2's large file support to upload the file in 100MB chunks.

Copy a file into B2, with multiple concurrent uploads

Uploading a large file with multiple HTTP connections is simple:

func copyFile(ctx context.Context, bucket *b2.Bucket, writers int, src, dst string) error {
	f, err := os.Open(src)
	if err != nil {
		return err
	}
	defer f.Close()

	w := bucket.Object(dst).NewWriter(ctx)
	w.ConcurrentUploads = writers
	if _, err := io.Copy(w, f); err != nil {
		w.Close()
		return err
	}
	return w.Close()
}

This will automatically split the file into writers chunks of 100MB uploads. Note that 100MB is the smallest chunk size that B2 supports.

Download a file from B2

Downloading is as simple as uploading:

func downloadFile(ctx context.Context, bucket *b2.Bucket, downloads int, src, dst string) error {
	r := bucket.Object(src).NewReader(ctx)
	defer r.Close()

	f, err := os.Create(dst)
	if err != nil {
		return err
	}
	r.ConcurrentDownloads = downloads
	if _, err := io.Copy(f, r); err != nil {
		f.Close()
		return err
	}
	return f.Close()
}

List all objects in a bucket

func printObjects(ctx context.Context, bucket *b2.Bucket) error {
	iterator := bucket.List(ctx)
	for iterator.Next() {
		fmt.Println(iterator.Object())
	}
	return iterator.Err()
}

Grant temporary auth to a file

Say you have a number of files in a private bucket, and you want to allow other people to download some files. This is possible to do by issuing a temporary authorization token for the prefix of the files you want to share.

token, err := bucket.AuthToken(ctx, "photos", time.Hour)

If successful, token is then an authorization token valid for one hour, which can be set in HTTP GET requests.

The hostname to use when downloading files via HTTP is account-specific and can be found via the BaseURL method:

base := bucket.BaseURL()

blazer's People

Contributors

alpn avatar arithmetric avatar armhold avatar csos95 avatar edwardbetts avatar fd0 avatar garrmcnu avatar kingmar avatar kjk avatar kula avatar kurin avatar mzukowski-reef avatar renatocron avatar tscs37 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.