sindresorhus / hasha Goto Github PK

View Code? Open in Web Editor NEW

945.0 12.0 41.0 105 KB

Hashing made simple. Get the hash of a buffer/string/stream/file.

License: MIT License

JavaScript 78.42% TypeScript 21.58%

hasha's Introduction

Hashing made simple. Get the hash of a buffer/string/stream/file.

Convenience wrapper around the core crypto Hash class with simpler API and better defaults.

Install

npm install hasha

Usage

import {hash} from 'hasha';

await hash('unicorn');
//=> 'e233b19aabc7d5e53826fb734d1222f1f0444c3a3fc67ff4af370a66e7cadd2cb24009f1bc86f0bed12ca5fcb226145ad10fc5f650f6ef0959f8aadc5a594b27'

API

See the Node.js crypto docs for more about hashing.

hash(input, options?)

The operation is executed using worker_threads. A thread is lazily spawned on the first operation and lives until the end of the program execution. It's unrefed, so it won't keep the process alive.

Returns a hash asynchronously.

hashSync(input, options?)

Returns a hash.

input

Type: Uint8Array | string | Array<Uint8Array | string> | NodeJS.ReadableStream (NodeJS.ReadableStream is not available in hashSync)

The value to hash.

While strings are supported you should prefer buffers as they're faster to hash. Although if you already have a string you should not convert it to a buffer.

Pass an array instead of concatenating strings and/or buffers. The output is the same, but arrays do not incur the overhead of concatenation.

options

Type: object

encoding

Type: string
Default: 'hex'
Values: 'hex' | 'base64' | 'buffer' | 'latin1'

The encoding of the returned hash.

algorithm

Type: string
Default: 'sha512'
Values: 'md5' | 'sha1' | 'sha256' | 'sha512' (Platform dependent)

The md5 algorithm is good for file revving, but you should never use md5 or sha1 for anything sensitive. They're insecure.

hashFile(filePath, options?)

The operation is executed using worker_threads. A thread is lazily spawned on the first operation and lives until the end of the program execution. It's unrefed, so it won't keep the process alive.

Returns a Promise for the calculated file hash.

import {hashFile} from 'hasha';

// Get the MD5 hash of an image
await hashFile('unicorn.png', {algorithm: 'md5'});
//=> '1abcb33beeb811dca15f0ac3e47b88d9'

hashFileSync(filePath, options?)

Returns the calculated file hash.

import {hashFileSync} from 'hasha';

// Get the MD5 hash of an image
hashFileSync('unicorn.png', {algorithm: 'md5'});
//=> '1abcb33beeb811dca15f0ac3e47b88d9'

hashingStream(options?)

Returns a hash transform stream.

import {hashingStream} from 'hasha';

// Hash the process input and output the hash sum
process.stdin.pipe(hashingStream()).pipe(process.stdout);

hasha-cli - CLI for this module
crypto-hash - Tiny hashing module that uses the native crypto API in Node.js and the browser
hash-object - Get the hash of an object
md5-hex - Create a MD5 hash with hex encoding

hasha's People

Contributors

Stargazers

Watchers

hasha's Issues

add support to calculate hash of first or last nkb of the file

how about adding support to calculate hash of specific part of the file?
Something like hash of first or last 64 KB of a file, can be useful when using API like SubDB where they require hash of first and last 64 KB of a file.

Add Salt?

Can you add salt on your hash so it's tasty?

Browser support

Using the Web Crypto API, we can generate sha1, sha256, sha384 and sha512 hashes with little code. For example:

const encoder = new TextEncoder("utf-8")

const algorithmMap = new Map([
	['sha1', 'SHA-1'],
	['sha256', 'SHA-256'],
	['sha384', 'SHA-384'],
	['sha512', 'SHA-512']
])

const arrayBufferToHex = (arrayBuffer, base) => [...new Uint8Array(arrayBuffer)].map(value => value.toString("hex").padStart(2, "0"))

module.exports = () => {
	throw new Error('Synchronous hashing is not supported in browsers!')
}

exports.async = (input, {algorithm = 'sha512', encoding = 'hex'} = {}) => {
	if (typeof input === "string") {
		input = encoder.encode(input)
	}

	// TODO: Support other encodings
	return arrayBufferToHex(await crypto.subtle.digest(algorithmMap.get(algorithm), input))
}

// @sindresorhus

Typescript import broken for `fromFile`?

I just tried to use the typescript definitions and they're not working for me:

import hasha from "hasha" -> the import works, but hasha is undefined

import { hasha } from "hasha" -> doesn't compile (as expected, hasha is the default export, not one of many)

import * as hasha from "hasha" -> works, but can't call fromFile (hasha is the hashing function (so input, options = {}) => ...), but fromFile isn't a property of it for some reason)

You can reproduce this with a new project and the following commands:

$ yarn init 
$ yarn add hasha
$ yarn add @types/hasha
$ yarn add ts-node 
$ ./node_modules/.bin/ts-node -e 'import hasha from "hasha"; console.log("hasha?", hasha);' # first case (undefined)
$ ./node_modules/.bin/ts-node -e 'import {hasha} from "hasha"; console.log("hasha?", hasha);' # second case (type error)
$ ./node_modules/.bin/ts-node -e 'import * as hasha from "hasha"; console.log("hasha?", hasha.fromFile);' # third case (`hasha` itself is defined but `hasha.fromFile` is not)

Am I forgetting something fundamental about imports or is this a real bug? The first case import hasha from "hasha" should have been all I needed.

Any chance for unhasha?

Just wondered if there would be any possibility of going the opposite direction as well? I love the simplicity of the hasha api and hate requiring yet another module with an inferior api just to get back to a string.

Thanks @sindresorhus! Just for the record, I actually bookmarked https://github.com/sindresorhus?tab=repositories, so whenever I'm looking for something, I can see if you've tackled it yet! 👍

Fatal error with Promise.all()

So i got this piece of code:

  console.log('RECEIVED ALL HASHES');
  const promises = [];


  data.paths.forEach(path => {
    console.log('RECEIVED PATH:', path)
    promises.push(hasha.fromFile(path, {
      algorithm: 'md5'
    }));
  });
  console.log('FINISHED LOOP');

  const results = [];
  for (let i = 0; i<promises.length; ++i) {
    const r = await promises[i];
    console.log(data.paths[i], ' got HASH: ', r);
    results.push(r);
  }

  console.log('FINISHED PROMISES');
  return results.map((r, i) => ({path: data.paths[i], hash: r}))

Running it like this it produces the right output as expected. It takes a while but creates all the hashes

But when using Promise.all like this:

  console.log('RECEIVED ALL HASHES');
  const promises = [];


  data.paths.forEach(path => {
    console.log('RECEIVED PATH:', path)
    promises.push(hasha.fromFile(path, {
      algorithm: 'md5'
    }));
  });
  console.log('FINISHED LOOP');

  const results = await Promise.all(promises);
  console.log('FINISHED PROMISES');
  return results.map((r, i) => ({path: data.paths[i], hash: r}))

I receive this fatal error:

#
# Fatal error in , line 0
# ignored
#
#
#
#FailureMessage Object: 0000005D5B5FC610
 1: 00007FF6D6B7B43F node::Buffer::New+127615
 2: 00007FF6D68C544A IsSandboxedProcess+653770
 3: 00007FF6D4D2F05B v8::Platform::SystemClockTimeMillis+413467
 4: 00007FF6D4A75BE3 v8::ArrayBuffer::GetContents+291
 5: 00007FF6D4A75AB0 v8::ArrayBuffer::Externalize+16
 6: 00007FF6D798AB74 uv_dlerror+338660
 7: 00007FF6D798CE41 uv_dlerror+347569
 8: 00007FF6D798D341 uv_dlerror+348849
 9: 00007FF6D4ABD62E v8::CFunction::CFunction+233422
10: 00007FF6D4ABCA8B v8::CFunction::CFunction+230443
11: 00007FF6D4ABC130 v8::CFunction::CFunction+228048
12: 00007FF6D4ABBDB7 v8::CFunction::CFunction+227159
13: 00007FF6D54D957C v8_inspector::V8StackTraceId::ToString+3453724
14: 00007FF6D546DE75 v8_inspector::V8StackTraceId::ToString+3013653
15: 00007FF6D549BC51 v8_inspector::V8StackTraceId::ToString+3201521
16: 00007FF6D55234D8 v8_inspector::V8StackTraceId::ToString+3756664
17: 00007FF6D548E4FA v8_inspector::V8StackTraceId::ToString+3146394
18: 00007FF6D546B82C v8_inspector::V8StackTraceId::ToString+3003852
19: 00007FF6D4B7D4D1 v8::Unwinder::PCIsInV8+2705
20: 00007FF6D4B7DF6F v8::Unwinder::PCIsInV8+5423
21: 00007FF6D4B7E076 v8::Unwinder::PCIsInV8+5686
22: 00007FF6D4BA2103 v8::Unwinder::PCIsInV8+153283
23: 00007FF6D4BA1EDF v8::Unwinder::PCIsInV8+152735
24: 00007FF6D546ED2E v8_inspector::V8StackTraceId::ToString+3017422
25: 00000067000D38F2
npm ERR! code ELIFECYCLE
npm ERR! errno 3
npm ERR! [email protected] electron:serve: `wait-on tcp:4200 && npm run electron:serve-tsc && npx electron . --serve`
npm ERR! Exit status 3
npm ERR!
npm ERR! Failed at the [email protected] electron:serve script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\danie\AppData\Roaming\npm-cache\_logs\2020-12-26T20_09_51_843Z-debug.log
ERROR: "electron:serve" exited with 3.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `npm-run-all -p electron:serve ng:serve`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\danie\AppData\Roaming\npm-cache\_logs\2020-12-26T20_09_51_947Z-debug.log

I have no idea what it's trying to tell me...

I have a 8 core 3700x and 32 GB of ram if this is any help.

I would like to run it in parallel since there are alot of files that get hashed somewhat often.

New version (5.1.0) hasha.fromFile makes Electron application crash (unallowed memory allocation for worker threads)

I'm using hasha in an Electron app that doesn't support Node 12 yet (v10.15.3).
Using hasha 5.0.0 there were no problems but since 5.1.0 my application crashes following an error that doesn't get caught in the fromFile promise.

The problem (I think) is the use of worker_threads in an environment below Node 12 resulting in an unallowed memory allocation windows error when built.

Locking my hasha version resolved the issue for me but I'm posting this as I don't know this is expected behaviour.

Image hash length

So your package is the fastest yet. other image hashing takes up to 500ms while yours is 5ms. Though there is something i can't figure out. i need to have my hashes around 64 characters and your hashing doesn't seem to have variable hashing length. is there a way to do this?

Add support for doing the hashing in a worker thread

Could maybe be useful for hashing large files, as the crypto Node.js module is synchronous, so it could block the main thread for some time.

https://nodejs.org/api/worker_threads.html

Thoughts?

IssueHunt Summary

stroncium has been rewarded.

Backers (Total: $80.00)

issuehunt ($80.00)

Submitted pull Requests

Tips

Checkout the Issuehunt explorer to discover more funded issues.
Need some help from other developers? Add your repositories on IssueHunt to raise funds.

IssueHunt has been backed by the following sponsors. Become a sponsor

hasha.fromFile throws error with message "The V8 platform used by this instance of Node does not support creating Workers" when updated to version 5.2.0

I am using hasha.Fromfile method for creating and comparing hash of two exe files. Recently I updated hasha to v 5.2.0 and I started getting this error. When I revert back to v 5.0.0 then this error is not there.

Some more details on machine environment:
Node version: 12.14.0
npm version: 6.13.0
OS: windows

However, my node and npm version is been same for quiet long. Is there any change in v 5.2.0 compared to v 5.0.0 which can cause this issue. I could not find any changelog document to verify the same.

create-hash instead crypto.createHash

if somebody will use this package with browserify bundle size will be awful
you can use createHash (part of crypto-browserify) for decrease bundle size without speed loss

Any interest in support for CRC?

True, CRC is less unique than MD5 - but it has its uses. If you're interested, I created the following Crypto-module-Hash-class-friendly wrapper a while back and could send a PR to add it: https://github.com/DavidAnson/crc-hash

convert to esm

I know you are converting everything to ESM in your one piece and you have a plan laid out. But i just wish to be pinged when it's done.
Thanks in advance ❤️

`fromFile` broken on Node 14

Because of this bug, hasha doesn't work with Node 14 right now: nodejs/node#33263

New version (5.1.0) throws webpack errors

WARNING Compiled with 1 warnings
warning in ./node_modules/hasha/index.js

Critical dependency: the request of a dependency is an expression

-

Hash multiple files?

Hey, Can I make a PR to hash multiple files? I could use combined-stream and reuse hasha.fromStream to add this functionality.

CLI: end with a new line

To improve the CLI, it would be nice if the output had a newline at the end of it. That way, the prompt will start on a new line. Instead of behind the hash.

Currently:

PROMPT > hasha foobar
0a50261eb...2425PROMPT>

Improved:

PROMPT > hasha foobar
0a50261eb...2425
PROMPT>

$ node -v
v10.16.3
$ npm -v
6.9.0
$ vue -V
@vue/cli 4.2.2

package.json

  "dependencies": {
    "hasha": "^5.2.0",
    "vue": "^2.6.11",
    "vue-i18n": "^8.15.3",
    "vue-router": "^3.1.5",
    "vuex": "^3.1.2",
  }

Support for AbortSignal

Especially fromFile would benefit from abort signal.