Coder Social home page Coder Social logo

nestjs / azure-storage Goto Github PK

View Code? Open in Web Editor NEW
84.0 84.0 33.0 762 KB

Azure Storage module for Nest framework (node.js) ☁️

Home Page: https://nestjs.com

License: MIT License

TypeScript 75.53% JavaScript 24.47%
azure azure-storage cloud javascript nest nestjs nodejs storage typescript

azure-storage's People

Contributors

alpcanaydin avatar caucik avatar dependabot[bot] avatar kamilmysliwiec avatar ksivamuthu avatar manekinekko avatar markpieszak avatar renovate-bot avatar renovate[bot] avatar sinedied avatar tony133 avatar wodcz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

azure-storage's Issues

Add command isn't working as expected

Is there an existing issue for this?

  • I have searched the existing issues

Current behavior

  1. nest new add-issue
  2. cd add-issue
  3. nest add @nestjs/azure-storage
  4. Error: Collection "@nestjs/azure-storage" cannot be resolved.

After a while, when @nestjs/azure-storage is actually installed.
nest add @nestjs/azure-storage works as expected

Minimum reproduction code

nest new add-issue

Steps to reproduce

  1. nest new add-issue
  2. cd add-issue
  3. nest add @nestjs/azure-storage

Expected behavior

use library schematic after package install.

Package version

8.2.6

NestJS version

No response

Node.js version

No response

In which operating systems have you tested?

  • macOS
  • Windows
  • Linux

Other

I found out that variable collect here is alwasy true
It seems that the problem is run function here will resolve immediately.

image

Add support to instanciate Azure Storage Client via connection string

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

This lib only supports storage account name and storage account SAS Key, either through configuration, or through additional storage options in the interceptor declaration.

Expected behavior

We should be able to precise either an account name/storage account pair OR a connection string.

Minimal reproduction of the problem with instructions

Irrelevant :)

What is the motivation / use case for changing the behavior?

Having a connection string support will enable the following scenarios :

  • Support for Local Azure Storage Emulator (a must-do for developers)
  • Support for other storage emulators, like Azurite, used in integration testing via CD/CI pipelines
  • Support for non-global+public Azure Instance (Azure Gov, Azure Germany, Azure China, ...)

Environment


Nest version: 6.6.4

 
For Tooling issues:
- Node version: 10.16.1  
- Platform: Linux  

Others:

Validate the SAS key format

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

Users can provide a SAS key when requested to. They might enter a key/token that is not of a SAS format.

Expected behavior

If the user provides a key that is not a SAS, we need to guide them (via a link) to the correct docs so they can create and then use a valid SAS key.

What is the motivation / use case for changing the behavior?

The SAS key might be confusing for developers who are not familiar with Azure. This module uses a SAS key to access the user's storage instance on Azure. People might consider using other types of keys, such as a connection string. But we do require a SAS key as of this version of the module.

I can't install this package with npm@7

I'm submitting a...


[ ] Regression 
[X] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

I get errors installing this package using npm version 7

Expected behavior

Successful install

Minimal reproduction of the problem with instructions

install npm version 7 npm -g i [email protected]

install package npm i @nestjs/azure-storage

What is the motivation / use case for changing the behavior?

Environment


Nest version: 7.5.1

 
For Tooling issues:
- Node version: 15.0.1  
- Platform:  Windows 

Others:

Using AzureStorageModule in different modules overrides one another's config

I'm submitting a...


[ ] Regression 
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

I am using AzureStorageModule.withConfigAsync() in different modules with different config, which includes SAS, storage account and default container name and then injecting AzureStorageService in providers section. So all services (in my case 2) contains same config. And a small note, second module using config of first module in list order in app.module.ts, if I change that order, config will be changed as well

imagine two module's options as below, the only difference is config in useFactory field

@Module({
    imports: [
      AzureStorageModule.withConfigAsync({
          useFactory: firstAzureConfig, // <-- in second module another config
          inject: [ConfigService],
      }),
    ], 
    providers: [
      {
          provide: AzureImagesStorageClient,
          useFactory: (azureService: AzureStorageService) => {
            console.log(azureService); // <-- here I see same configuration
            return new AzureImagesStorageClient(azureService);
          },
          inject: [AzureStorageService],
        }
    ]

image

Expected behavior

I expect that each module should contain its configuration and one should not override another one

Environment


Nest version: 10.2.10

 
For Tooling issues:
- Node version: v18.13.0
- Platform:  Linux

Dependency issue

I'm submitting a...


[ ] Regression 
[*] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

When installing I get this error


npm ERR! Found: @nestjs/[email protected]
npm ERR! node_modules/@nestjs/common
npm ERR!   @nestjs/common@"^9.1.4" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer @nestjs/common@"^8.0.0" from @nestjs/[email protected]
npm ERR! node_modules/@nestjs/azure-storage
npm ERR!   @nestjs/azure-storage@"^3.0.0" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Expected behavior

It should install without any error. I think a pull request is already created to update this package's dependencies to nestjs latest version.

Environment


Nest version: 9.0.0

 
For Tooling issues:
- Node version: 16.18.0  
- Platform: Windows 

Add custom filename on upload

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

When uploading a file to Storage, we use the original filename.

Expected behavior

When uploading a file to Storage, we can provide a custom filename.

What is the motivation / use case for changing the behavior?

Users might want to have a custom logic when it comes to filename, or an internal pattern naming convention...etc

Notes

We should easily change this line so it can accept an external filename https://github.com/nestjs/azure-storage/blob/master/lib/azure-storage.service.ts#L116

Change UploadedFileMetaData interface's size property to number from string type.

Title^


[ ] Regression 
[] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

String type.

Expected behavior

Number type.

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

So i could use discriminating union in my storage module, because Express.Multer.File interface has it as a number property.

Environment


Nest version: 8.1.2
 
For Tooling issues:
- Node version: v14.18.1
- Platform:  MacOS Monterey 12.0.1

Others:
-

Could not resolve dependency

❯ npm i @nestjs/azure-storage
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: @nestjs/[email protected]
npm ERR! node_modules/@nestjs/common
npm ERR! @nestjs/common@"^8.0.0" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @nestjs/common@"^6.0.0" from @nestjs/[email protected]
npm ERR! node_modules/@nestjs/azure-storage
npm ERR! @nestjs/azure-storage@"*" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Upload: reject promise when Azure errors

I'm submitting a...


[x] Feature request

Current behavior

If there was an error during the upload process, we return Promise.resolve(false)

Expected behavior

We need to reject the promise and provide the error message.

Update dependencies

Can this package get some love? The dependencies are out of date at this point. The only way to install this on new nest versions is with --force.

I appreciate all the hard work, thanks!

Semi PR Request (upgrades in azure library)

I'm submitting a...


[ ] Regression 
[ ] Bug report
[X] Feature request
[X] Question and example answer
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

Nest version is old, @azure/storage-blob is old.
Cannot use key, connectionString nor any new authorization method.
Cannot use delete, download methods of library (they are nor written)

Expected behavior

Minimal reproduction of the problem with instructions

just download this version and all problems ocures.

What is the motivation / use case for changing the behavior?

Upgrade versions and add more functionality

##Example of fix
I rewrote some files - tests works - as i needs stated features asap. Only one thing i got (after upgrading) is error of:

TypeError: Cannot read property 'kind' of undefined

    |     const ms = node.moduleSpecifier;
    |     let modulePath;
>  |     switch (ms.kind) {
which im not able to even understand (i mean i never ever used such features of node).

So what im asking is to look at my repo and maybe incorporate what you thing is wrote ok or give me hints how to fix the issue i have now (which forbids building the library for npm).

Link to repo: https://github.com/Setitch/azure-storage

[RFC] Azure Storage for NestJs

Azure Storage for NestJs

This document is published and publicly accessible.

author@manekinekko
statusdraft | review | approved | implemented | obsolete
doc url#1
last update2019-06-19

Objective

This Azure Storage library for the NestJs framework adds support for deploying files and blobs to Azure Storage from a NestJS application. This library is a wrapper of the official @azure/storage-blob v10 SDK.

Approvals

We are seeking approvals from the following people:

Background

In order to upload Blobs or Files to Azure Storage, NestJs developers use the official @azure/storage-blob package and configure it.

However, we want to offer seamless integration with the NestJs framework by providing an off-the-shelf package ready to use with a minimum configuration.
This package should also implement NestJs best practices and paradigms.

Prior Art

List of existing solutions and their corresponding strong/weak points.

There is already a similar package angular-azure-blob-service that offers integration for Azure Storage with the Angular framework. However, this package doesn't seem to be maintained by its author.

Asim Hussain has also built a proof-of-concept for the Azure Storage integration with NestJs.

Prototype

Details on how to create a prototype with a small effort that proves that the proposed design will achieve the objective.

Package name

The Azure Storage for NestJs library lives under @nestjs/azure-storage.

Usage

Azure Storage credentials

The .env file at the root of a NestJs project will be used to provide the required Azure environment variables:

AZURE_STORAGE_SAS_KEY=
AZURE_STORAGE_ACCOUNT=

Importing and configuration the AzureStorageModule:

In the app.module.ts, we will import the AzureStorageModule module and provide the required configuration:

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import {
  AzureStorageModule,
  AZURE_STORAGE_MODULE_OPTIONS,
} from '@nestjs/azure-storage';

export const azureStorageOptions = {
  sasKey: process.env['AZURE_STORAGE_SAS_KEY'],
  accountName: process.env['AZURE_STORAGE_ACCOUNT'],
  containerName: 'nest-demo-container' // default container name
};

@Module({
  imports: [
    AzureStorageModule.withConfig(
      // Provide Azure Storafe credentials: option 1
      azureStorageOptions,
    ),
  ],
  controllers: [AppController],
  providers: [
    // Provide Azure Storafe credentials: option 2
    {provide: AZURE_STORAGE_MODULE_OPTIONS, useValue: azureStorageOptions}
  ],
})
export class AppModule {}

We may require other configuration if needed.

Use the Azure Storage Interceptor

In a NestJs Controller, on a Post() method that is responsible for the file upload process, we will use the AzureInterceptor as an interceptor, part of @UseInterceptors():

import { AzureBlobInterceptor } from '@nestjs/azure-storage';

@Controller('app')
export class AppController {

  @Post('upload')
  @UseInterceptors(AzureBlobInterceptor('file'))
  UploadedFilesUsingMulter( @UploadedFile() file: UploadedFileMetadata) {
    Logger.log(`Storage URL: ${file.storageUrl}`, 'AppController');
  }

}

The AzureBlobInterceptor() interceptor may accept additional configuration in order (ServiceClientOptions) to override some defaults params, or adds more (ie, Headers...etc):

AzureInterceptor('file', null, {
        containerName: 'different-container-name',
        clientOptions: { <ServiceClientOptions> }
})

We can also use the

import { AzureBlobStorageService } from '@nestjs/azure-storage';

@Controller('app')
export class AppController {

  constructor(private readonly azureBlobStorage: AzureBlobStorageService) {}

  @Post('azure/upload')
  @UseInterceptors(FileInterceptor('file'))
  async UploadedFilesUsingService(@UploadedFile() file: UploadedFileMetadata) {
    const storageUrl = await this.azureBlobStorage.upload(file, /* additional ServiceClientOptions options */);
    Logger.log(`Storage URL: ${storageUrl}`, 'AppController');
  }

}

Detailed Design

Details on how you’ll implement. Should be in sufficient detail for other engineers to materially comment on the structure to affect the end result.

API

Blob Storage

@todo

Documentation Plan

What types of documentation are required? [API | Concepts | Cookbooks | etc.]

The How-To will be documented in the README.md file.

The API will be documented using jsdoc-like syntax.

Security Considerations

How you’ll be secure. Considerations should include sanitization, escaping, existing security protocols and web standards, permission, user privacy, etc.

This library will be compliant with the security rules of both the NestJS and the official @azure/storage-blob SDK.

How can I delete an image from Azure Storage Container?

I'm submitting a...


[ ] Regression 
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

Expected behavior

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

Environment


Nest version: X.Y.Z

 
For Tooling issues:
- Node version: XX  
- Platform:  

Others:

Error encountered: 404

I did the exact code but I received this message? any idea?
{
"messages": "Error encountered: 404",
"statusCode": 500,
"path": "/entities/company/cg",
"timestamp": "2021-01-10T18:26:22.040Z"
}

Cannot set properties of undefined (setting 'start:azure')

I'm submitting a...


[ ] Regression 
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

When i ran nest add @nestjs/azure-func-http in my project. I received errors like below

ksnip_20221001-095427

Expected behavior

ran nest add @nestjs/azure-func-http successful

Minimal reproduction of the problem with instructions

  1. cd to project root
  2. Ran nest add @nestjs/azure-func-http

What is the motivation / use case for changing the behavior?

Environment


Nest version: 9.1.4

 
For Tooling issues:
- Node version: 16.17.1
- Platform:  PopOS (Linux)

Others:
VSCode

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • chore(deps): update dependency typescript to v4.9.5
  • chore(deps): update node.js to v20.13
  • fix(deps): update angular-cli monorepo to v14.2.13 (@angular-devkit/schematics, @schematics/angular)
  • chore(deps): update dependency @types/node to v20
  • chore(deps): update dependency husky to v9
  • chore(deps): update dependency lint-staged to v15
  • chore(deps): update dependency prettier to v3
  • chore(deps): update dependency ts-morph to v22
  • chore(deps): update dependency typescript to v5
  • chore(deps): update node.js to v22
  • 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

circleci
.circleci/config.yml
  • cimg/node 20.3
  • cimg/node 20.3
npm
package.json
  • @angular-devkit/schematics 14.0.5
  • @schematics/angular 14.0.5
  • @angular/cdk 8.2.3
  • @azure/storage-blob 12.11.0
  • @nestjs/common 8.4.7
  • @nestjs/core 8.4.7
  • @nestjs/platform-express 8.4.7
  • @types/jest ^28.1.4
  • @types/node 18.0.3
  • husky 8.0.1
  • jest ^28.1.2
  • lint-staged 13.0.3
  • mock-stdin ^1.0.0
  • prettier 2.7.1
  • ts-jest ^28.0.5
  • ts-morph 15.1.0
  • @angular/cdk ^14.0.4
  • reflect-metadata ^0.1.13
  • typescript 4.7.4
  • @nestjs/common ^8.0.0 || ^9.0.2
  • @nestjs/core ^8.0.0 || ^9.0.2
  • @nestjs/platform-express ^8.0.0 || ^9.0.2

  • Check this box to trigger a request for Renovate to run again on this repository

Accessing Azure SDK Method using this module

It can be a change request or on top of this if we can access azure blobService so we can access all different methods on azure blob provided by azure SDK

I'm submitting a...


[ ] Regression 
[ ] Bug report
[*] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

  • works fine when you just need to upload only

Expected behavior

I am just checking if we can access blobService in injectable service and do all rest operation on azure blob SDK
like shown in below examples

const storage = require("azure-storage");

const blobService = storage.createBlobService();

const listContainers = async () => {
  return new Promise((resolve, reject) => {
    blobService.listContainersSegmented(null, (err, data) => {
      if (err) {
        reject(err);
      } else {
        resolve({
          message: `${data.entries.length} containers`,
          containers: data.entries,
        });
      }
    });
  });
};

const createContainer = async (containerName) => {
  return new Promise((resolve, reject) => {
    blobService.createContainerIfNotExists(
      containerName,
      { publicAccessLevel: "blob" },
      (err) => {
        if (err) {
          reject(err);
        } else {
          resolve({ message: `Container '${containerName}' created` });
        }
      }
    );
  });
};

I am just wondering if i can do more then just simple file upload from this module

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

Environment


Nest version: X.Y.Z

 
For Tooling issues:
- Node version: XX  
- Platform:  

Others:

Add folder creation support

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

Currently, specifying a custom file name with a slash creates an actual file with that filename. So, specifying the filename "dir/file.txt" actually creates a "dir/file.txt" file instead of creating the folder "dir" with "file.txt" in it

Expected behavior

We should be able to create folders using the module

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

Folders are useful... 📦

Environment


Nest version: 6.7.2

 
For Tooling issues:
- Node version: 12.10  
- Platform: Windows 

Others:

NestJS 7 support

❯ npm i -S @nestjs/azure-storage
npm WARN @nestjs/[email protected] requires a peer of @nestjs/common@^6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN @nestjs/[email protected] requires a peer of @nestjs/core@^6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN @nestjs/[email protected] requires a peer of @nestjs/platform-express@^6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN @angular/[email protected] requires a peer of @angular/core@^8.0.0 || ^9.0.0-0 but none is installed. You must install peer dependencies yourself.
npm WARN @angular/[email protected] requires a peer of @angular/common@^8.0.0 || ^9.0.0-0 but none is installed. You must install peer dependencies yourself.

Add forRootAsync for configuration via configService

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

Currently, it's only possible to instantiate the module via the .withConfig method. It is not possible to inject a service into this method, so it is impossible to use a config service for the module instantiation.

Expected behavior

Enable both the .withConfig method and a factory-pattern method, that allows dependency injection.

What is the motivation / use case for changing the behavior?

I don't want to use process.env variables as all my config logic is inside a config module. It is possible to override this behavior by manually injecting all config values into process.env, but this seems like a bad choice.

Environment


Nest version: 6.6.0

 
For Tooling issues:
- Node version: 12  
- Platform: Windows 

Others:

Add SAS Key generation service

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

Currently, you pass the SAS key during the module registering with the withConfig method. This is undesirable, as they keys expire and you might want to change them or configure them better.

Expected behavior

There should be an option to generate the SAS key via the module, using an account name and key.

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

SAS keys expire and there should be more control over them.

Environment


Nest version: 6.7.2

 
For Tooling issues:
- Node version: 12.10  
- Platform: Windows 

Others:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.