digitalbazaar / base58-universal Goto Github PK
View Code? Open in Web Editor NEWEncode/decode using "The Base58 Encoding Scheme".
License: Other
Encode/decode using "The Base58 Encoding Scheme".
License: Other
The encode and decode time per element increases as the number of input elements grows. Looking at the output from a benchmark, it looks like it's mostly linear or super linear growth up to 16k elements.
Possible optimizations should be explored for use cases that use larger input sizes. This library uses a generic base-n decoder, so perhaps there are specific base58 optimizations. It might be good to also document this issue in the README so there are not performance surprises compared to simpler base16 and base64 codecs.
Some references (thanks to @msporny):
https://stackoverflow.com/questions/28418332/computational-complexity-of-base-conversion
https://cs.stackexchange.com/questions/21736/time-complexity-of-base-conversion
Benchmark and output:
#5
t1-16k.csv
import {encode, decode} from 'base58-universal';
import {encode, decode} from 'base58-universal';
^^^^^^
SyntaxError: Named export 'decode' not found. The requested module 'base58-universal' is a CommonJS module, which may not support all module.exports as named exports.
CommonJS modules can always be imported via the default export, for example using:
import pkg from 'base58-universal';
const {encode, decode} = pkg;
import {encode} from 'base58-universal';
^^^^^^
SyntaxError: Named export 'encode' not found. The requested module 'base58-universal' is a CommonJS module, which may not support all module.exports as named exports.
CommonJS modules can always be imported via the default export, for example using:
import pkg from 'base58-universal';
const {encode} = pkg;
at ModuleJob._instantiate (node:internal/modules/esm/module_job:104:21)
at async ModuleJob.run (node:internal/modules/esm/module_job:149:5)
at async Loader.import (node:internal/modules/esm/loader:166:24)
at async Object.loadESM (node:internal/process/esm_loader:68:5)
import bs58 from 'base58-universal';
console.log({bs58, encode: bs58.encode, decode: bs58.decode});
{ bs58: {}, encode: undefined, decode: undefined }
import * as bs58 from 'base58-universal';
console.log({bs58, encode: bs58.encode, decode: bs58.decode});
{
bs58: [Module] { default: {} },
encode: undefined,
decode: undefined
}
node -v
v14.15.3
I'm curious why encoding a byte array of only a few hundred kilobytes results in an operation that takes 10+ minutes? I actually can't find any materials that explains why base58 encoding would struggle with a moderate amount of data. Maybe someone here can explain?
Below is some example code
/**
* nodejs script to test encrypting + encoding a base64 image data uri
*/
import fs from 'fs';
import * as url from 'url';
import path from 'path';
import { randomBytes, createCipheriv, createDecipheriv } from 'crypto';
import {encode, decode} from 'base58-universal';
import base58 from 'bs58';
const __dirname = url.fileURLToPath(new URL('.', import.meta.url));
const data = fs.readFileSync(path.join(__dirname, 'examples/docImageBase64String.txt'));
const key = randomBytes(32);
const iv = randomBytes(16);
const algorithm = 'aes-256-cbc';
const cipher = createCipheriv(algorithm, key, iv);
const startEncryption = new Date().getTime();
const encrypted1 = cipher.update(data);
const encrypted2 = cipher.final();
const encrypted = Buffer.concat([encrypted1, encrypted2]);
const endEncryption = new Date().getTime();
const timeToEncrypt = endEncryption - startEncryption;
console.log('time to encrypt', timeToEncrypt);
const beforeBase64Encoding = new Date().getTime();
const encodedBase64 = encrypted.toString('base64');
const afterBase64Encoding = new Date().getTime();
const timeToBase64Encode = afterBase64Encoding - beforeBase64Encoding;
console.log('time to encode base64 (using Buffer.toString())', timeToBase64Encode);
// write the encrypted data to a file so we can look at it without crashing the terminal lol
const outputDir = path.join(__dirname, 'output');
if (!fs.existsSync(outputDir)) {
fs.mkdirSync(outputDir);
}
fs.writeFileSync(path.join(__dirname, 'output/encrypted.txt'), encodedBase64);
// base58 encoding
// this is V E R Y slow
const beforeEncoding = new Date().getTime();
const encoded = encode(encrypted);
const afterEncoding = new Date().getTime();
const timeToEncode = afterEncoding - beforeEncoding;
console.log('time to encode base58', timeToEncode);
// write base58 encoded data to a file
fs.writeFileSync('base58encoded.txt', encoded);
// let's decrypt it and write back to a file as a further sanity check
const decipher = createDecipheriv(algorithm, key, iv);
const decrypted1 = decipher.update(encrypted);
const decrypted2 = decipher.final();
const decrypted = Buffer.concat([decrypted1, decrypted2]);
fs.writeFileSync(path.join(__dirname, 'output/decrypted.txt'), decrypted.toString());
You'll just need to have a base64 encoded image in the /examples
dir to run.
Running the above results in the output:
time to encrypt 2
time to encode base64 (using Buffer.toString()) 1
time to encode base58 1140551
However if one does run you'll see the base64 encoding taking a couple of ms while the base58 encoding takes tens of minutes. Why?
Thanks for your responses and for your efforts in open sourcing this project.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.