Coder Social home page Coder Social logo

google-cloud-storage-bulk's Introduction

google-cloud-storage-bulk

build status code coverage code style styled with prettier made with lass license npm downloads

google cloud storage bulk upload

Table of Contents

Install

npm:

npm install google-cloud-storage-bulk

yarn:

yarn add google-cloud-storage-bulk

Usage

const GCS = require('google-cloud-storage-bulk');

const gcs = new GCS({
  projectId,
  bucketName: 'my-bucket',
  concurrency: 100,
  hashStrategy: 'file',
  retries: 3,
  subdirectory: 'application-name',
  uploadOptions: {}
});

// ... in async function
await gcs.uploadFiles(someDirectory);

Options

  • projectId - (required) - google cloud project id (see project reference for more details)
  • bucketName - (required) - cloud storage bucket name
  • concurrency - (optional - default: 250) - upload concurrency limits
  • retries - (optional - default: 3) - upload retry attempts
  • hashStrategy - (required - options: none, file, subdirectory) - described in detail below
  • subdirectory - (optional) - subdirectory within cloud storage bucket to push content into
  • uploadOptions - (optional) - extends @google/cloud-storage upload options

hashStrategy

none

No hashing to file structure, just push content as is. Google cloud does handle metageneration that you can take advantage of for versioning.

Before:
|_directory_to_upload
  |_a.js
  |_b.js
  |_c.css
  
After:
|_subdirectory
  |_a.js
  |_b.js
  |_c.css

file

Hash per file currently using sha1 algorithm.

Before:
|_directory_to_upload
  |_a.js
  |_b.js
  |_c.css
  
After:
|_subdirectory
  |_a.a9993e364706816aba3e25717850c26c9cd0d89d.js
  |_b.924f61661a3472da74307a35f2c8d22e07e84a4d.js
  |_c.bcb8c41b803b91661b5e6ee45362f47df368a731.css
  |_asset-manifest.json <-- references initial file to their new hash complement

subdirectory

Hash content of the directory being uploaded and use this directory hash as the bucket subdirectory.

Before:
|_directory_to_upload
  |_a.js
  |_b.js
  |_c.css
  
After:
|_subdirectory
  |_924f61661a3472da74307a35f2c8d22e07e84a4d
    |_a.js
    |_b.js
    |_c.css

Contributors

Name Website
Shaun Warman https://shaunwarman.com

License

MIT © Shaun Warman

google-cloud-storage-bulk's People

Watchers

James Cloos avatar Shaun Warman avatar  avatar

google-cloud-storage-bulk's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.