planttheidea / fast-copy Goto Github PK

tslint is a dead project, and has been replaced with typescript-eslint. In addition, the continued dependency on tslint causes installation issues with npm@8. It should be migrated.

I add fast-copy to my TS project and got this error. DesignElement.ts:125 Uncaught (in promise) TypeError: fast_copy_1.default is not a function. I've analyzed .js file and didn't find default import. But in declaration file it declares as default.

If a non-extensible object reference is used multiple times in the copied object, then the non-extensible object is still existing in the copied object. It's difficult to explain that - here is a reproduction case - https://codesandbox.io/s/fast-copy-non-extensible-bug-cv0vh

I checked the source code and looks like it is an issue with the cache. Maybe I will take a look at it.

When this function is executed multiple times and setState in react, it triggers a noticeable lag of up to a few seconds

I like the performance metrics you’re producing but wasn’t clear on what “relative margin of error” meant. I suspect other passers by will have this question so this is in the hope that you can update the README with a simple description of what this is and how it’s measured.

Instead of https://github.com/planttheidea/fast-copy/blob/master/src/utils.js#L223, wouldn't it be faster to just copy a set with new Set(oldSet)?

Hello! I code in a framework that builds applications for iOS and Android (via React Native) and the web (via React Native for Web). As part of server-side rendering for web, the framework has a small number of basic polyfills for the [Ww]indow object.

Recently when testing server-side network calls through the client function of an npm package that uses fast-copy, I saw the following error:

TypeError: Right-hand side of 'instanceof' is not an object

I traced this error back to my web application not including a Date or RegExp object in its polyfills for server-side rendering. As a result, if the FastCopy Realm is set as the Window object, and the Window object does not include Date or RegExp objects, then the handleCopy function throws the TypeError if checking instanceof against realm.Date or realm.Regexp.

Though I was able to add polyfills in my application to supplement those in the framework and solve for my particular use case, can we add existence checks for realm.Date and realm.RegExp in handleCopy similar to that of other realm objects? I would be glad to put together a PR if you agree with this suggestion.

How to use this package via CDN? I've tried https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/index.js but seems no global function exported to the window object.

const _ = require('lodash');
const Copy = require('fast-copy');

const hash = Crypto.createHash('sha256');
hash.update(sendPacketData);
//let newHash = _.cloneDeep(hash);      //success
let newHash = Copy(hash, {isStrict: true});  //failed
newHash.update(item.data);

at getObjectCloneStrict (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:114:36)
at handleCopy (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:286:16)
at copy (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:288:12)
at d:\src\node\WechatWrap\test\testPacketDecrypt.js:59:23
at Array.forEach (<anonymous>)
at Object.<anonymous> (d:\src\node\WechatWrap\test\testPacketDecrypt.js:54:9)
at Module._compile (internal/modules/cjs/loader.js:686:14)

Waiting for the debugger to disconnect...
Error: Digest method not supported
utils.ts:70
at getCleanClone (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:56:16)
at getObjectCloneStrict (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:105:17)
at handleCopy (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:286:16)
at getObjectCloneStrict (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:114:36)
at handleCopy (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:286:16)
at copy (d:\src\node_modules\fast-copy\dist\fast-copy.cjs.js:288:12)
at d:\src\node\WechatWrap\test\testPacketDecrypt.js:59:23
at Array.forEach ()
at Object. (d:\src\node\WechatWrap\test\testPacketDecrypt.js:54:9)
at Module._compile (internal/modules/cjs/loader.js:686:14)

I cloned the project and ran npm i. I am on Ubuntu-20.04 with most up-to-date npm

A codebase that works well using fast-copy 2.1.1 on node-v16.18.1-darwin-arm64 breaks if I switch to fast-copy 3.0.0.

/Users/JoaoGoncalves/.nvm/versions/node/v16.17.1/bin/node[49891]: ../src/tcp_wrap.cc:149:static void node::TCPWrap::New(const FunctionCallbackInfo<v8::Value> &): Assertion `args[0]->IsInt32()' failed.
 1: 0x1000b1918 node::Abort() [/Users/JoaoGoncalves/.nvm/versions/node/v16.17.1/bin/node]
 2: 0x1000b175c node::AppendExceptionLine(node::Environment*, v8::Local<v8::Value>, v8::Local<v8::Message>, node::ErrorHandlingMode) [/Users/JoaoGoncalves/.nvm/versions/node/v16.17.1/bin/node]
 3: 0x10015d248 node::TCPWrap::New(v8::FunctionCallbackInfo<v8::Value> const&) [/Users/JoaoGoncalves/.nvm/versions/node/v16.17.1/bin/node]
 ...

Here's the call stack from llnode:

  * thread #1: tid = 0x46403, 0x00000001a7fdb224 libsystem_kernel.dylib`__pthread_kill + 8, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00000001a7fdb224 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x00000001a8011cec libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x00000001a7f4b2c8 libsystem_c.dylib`abort + 180
    frame #3: 0x00000001000b1924 node`node::Abort() + 40
    frame #4: 0x00000001000b175c node`node::Assert(node::AssertionInfo const&) + 136
    frame #5: 0x000000010015d248 node`node::TCPWrap::New(v8::FunctionCallbackInfo<v8::Value> const&) + 240
    frame #6: 0x000000010026024c node`v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) + 276
    frame #7: 0x000000010025fa80 node`v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<true>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) + 504
    frame #8: 0x000000010025f5b0 node`v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) + 196
    frame #9: 0x000000010099ecec node`Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit + 108
    frame #10: 0x000000010092fa70 <constructor>
    frame #11: 0x0000000100a2db50 <stub>
    frame #12: 0x0000000100932838 getCleanClone(this=0x34a57a201599:<undefined>, 0xa891fac4741:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:41:23 fn=0x0000081060d86219
    frame #13: 0x0000000110e2b5d4 copyObjectLooseModern(this=0x34a57a201599:<undefined>, 0x345199629979:<Object: TCP>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:212:31 fn=0x0000081060d8e439
    frame #14: 0x0000000100932838 copier(this=0x21a0bacb8861:<Object: Object>, 0x345199629979:<Object: TCP>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:349:20 fn=0x000025dc78106e79
    frame #15: 0x0000000110e2b808 copyObjectLooseModern(this=0x34a57a201599:<undefined>, 0x1a351b18d801:<Object: Server>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:212:31 fn=0x0000081060d8e439
    frame #16: 0x0000000100932838 copier(this=0x21a0bacb8861:<Object: Object>, 0x1a351b18d801:<Object: Server>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:349:20 fn=0x000025dc78106e79
    frame #17: 0x0000000110e2b808 copyObjectLooseModern(this=0x34a57a201599:<undefined>, 0x7f3fbe4ff19:<Object: Socket>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:212:31 fn=0x0000081060d8e439
    frame #18: 0x0000000100932838 copier(this=0x21a0bacb8861:<Object: Object>, 0x7f3fbe4ff19:<Object: Socket>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:349:20 fn=0x000025dc78106e79
    frame #19: 0x0000000110e2b808 copyObjectLooseModern(this=0x34a57a201599:<undefined>, 0x3438d7cd8ae1:<Object: ServerResponse>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:212:31 fn=0x0000081060d8e439
    frame #20: 0x0000000100932838 copier(this=0x21a0bacb8861:<Object: Object>, 0x3438d7cd8ae1:<Object: ServerResponse>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:349:20 fn=0x000025dc78106e79
    frame #21: 0x0000000100932838 copyObjectLooseModern(this=0x34a57a201599:<undefined>, 0x21a0bacb7ed9:<Object: Object>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:212:31 fn=0x0000081060d8e439
    frame #22: 0x0000000100932838 copier(this=0x21a0bacb8861:<Object: Object>, 0x21a0bacb7ed9:<Object: Object>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:349:20 fn=0x000025dc78106e79
    frame #23: 0x0000000100932838 copyObjectLooseModern(this=0x34a57a201599:<undefined>, 0x21a0bacb7c59:<Object: Object>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:212:31 fn=0x0000081060d8e439
    frame #24: 0x0000000100932838 copier(this=0x34a57a201599:<undefined>, 0x21a0bacb7c59:<Object: Object>, 0x21a0bacb8861:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:349:20 fn=0x000025dc78106e79
    frame #25: 0x0000000100932838 copy(this=0x34a57a201599:<undefined>, 0x21a0bacb7c59:<Object: Object>) at REDACTED/node_modules/fast-copy/dist/cjs/index.cjs:373:25 fn=0x000025dc78106949
    frame #26: 0x0000000100932838 filterLog(this=0x34a57a201599:<undefined>, 0x21a0bacb8761:<Object: Object>) at REDACTED/node_modules/pino-pretty/lib/utils.js:591:20 fn=0x000025dc78106501
    frame #27: 0x0000000100932838 pretty(this=0x34a57a201599:<undefined>, 0x21a0bacb7c59:<Object: Object>) at REDACTED/node_modules/pino-pretty/index.js:119:19 fn=0x00000092c6417089
    frame #28: 0x0000000100932838 write(this=0x92c6414359:<Object: Object>, 0x21a0bacb65e1:<String: "{"level":"info",...">) at REDACTED/node_modules/pino/lib/tools.js:1:0 fn=0x00000092c6416699
    frame #29: 0x0000000100932838 write(this=0x21a0bacb3999:<Object: Object>, 0x21a0bacb5e69:<Object: Object>, 0x4de19f2b759:<String: "request complete">, <Smi: 30>) at REDACTED/node_modules/pino/lib/proto.js:1:0 fn=0x000025dc78109759
    frame #30: 0x0000000100932838 LOG(this=0x21a0bacb3999:<Object: Object>, 0x21a0bacb5e69:<Object: Object>) at REDACTED/node_modules/pino/lib/tools.js:1:0 fn=0x000021a0bacb5451
    frame #31: 0x0000000100932838 requestLog(this=0x34a57a201599:<undefined>, 0x3438d7cd8d01:<Object: IncomingMessage>, 0x3438d7cd8ae1:<Object: ServerResponse>, <Smi: 200>, 0x34a57a201599:<undefined>) at (no script):43:20 fn=0x000024bbc39ef3a9
...

I suspect this isn't an issue in x64 but could not confirm yet, will update as soon as I can.

npm i

ck Rolling back [email protected] failed (this is probably harmless): EPERM: operation not permitted, lstat 'C:\Users\sar\Documents\GitHub\fast-copy\node_modules\fsevents\node_modules'
npm WARN rollback Rolling back [email protected] failed (this is probably harmless): EPERM: operation not permitted, lstat 'C:\Users\sar\Documents\GitHub\fast-copy\node_modules\fsevents\node_modules'
npm WARN prepublish-on-install As of npm@5, `prepublish` scripts are deprecated.
npm WARN prepublish-on-install Use `prepare` for build steps and `prepublishOnly` for upload-only.
npm WARN prepublish-on-install See the deprecation note in `npm help scripts` for more information.

> [email protected] prepublish C:\Users\sar\Documents\GitHub\fast-copy
> if in-publish; then npm run prepublish:compile; fi

then was unexpected at this time.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] prepublish: `if in-publish; then npm run prepublish:compile; fi`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] prepublish script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\sar\AppData\Roaming\npm-cache\_logs\1-debug.log

When running NodeJS with --disable-proto option, fast-copy fails due to an access to the __proto__ property. Many people need/want to avoid this kind of access because it facilitates "prototype pollution" attacks.

• https://learn.snyk.io/lessons/prototype-pollution/javascript/

Is there any benchmark or something where this lib is compared to the spread operator?

I noticed you were trying to write a super fast object-related library, so I decided to poke around a little bit in the implementation. I created my own such library, too but for deep comparison instead. (BTW, I don't have benchmarks, but that's because I just forgot to save the Lodash/etc. benchmarks before actually publishing my initial commit. It's about 30-50% as fast as Lodash while doing about 4x as much.)

I did come across a few things that struck me as odd, so I have a few questions:

1. Is there a reason why you're using the boolean accessors directly instead of RegExp.prototype.flags here? I would normally use the boolean accessors, too, except if I'm immediately plugging it into new RegExp(re.source, re.flags). If you just inlined it as regExp.flags here, you should see some speed boost here.
2. In this particular case, is there a reason why you didn't fork it out into two separate functions? (Engines know how to inline that.)
   • Conversely, why not inline this in the only caller, guarded with HAS_PROPERTY_SYMBOL_SUPPORT? You can avoid some unnecessary work here by just iterating the original array as you go. It's a cheap test and the common case is that either all are enumerable or none are, so I'm not sure if anything here could be gained even by filtering in place first. And with the guard, engines will typically strip out the dead code when it goes to optimize it, and even if it doesn't, CPUs will predict it correctly.
3. In your map/set copying routine, have you considered just splitting the function into two? Engines will occasionally do loop-invariant code motion, but they don't typically in my experience do that through function barriers. Also, you would make it easier on the engine to discern how they're used, without hardly increasing code size much.
4. In your typed array copying routine, that's not actually copying the buffer. That's just copying the view itself. You need to copy the buffer, too, if you want to do a true deep copy.
5. Why aren't you lifting this out and adding the realm and cache as new parameters?

If this helps for other things, here's a dump of some what I found with performance when dealing with highly polymorphic and allocaton-heavy stuff:

1. I would say the best tool out there I've found for diagnosing low-level performance issues in Node is V8's Turbolizer, located in the tools/turbolizer/ subdirectory of its source code. The instructions do generally work if you substitute node for d8, and it's a lifesaver when you want to know what the heck TurboFan is doing to your code.
2. If you can convert a polymorphic call to a monomorphic one without adding much code independent of this, try it, even if it means duplicating a little bit of code. Every engine out there with tracing ICs loves monomorphic calls - if it's not inlined, the call is still specialized for the types it's being passed, and you're likely to see perf wins this way.
3. If it's megamorphic and recursive, it doesn't usually get inlined, and in my experience, it hits the slow path of almost everything. Also, sometimes in cases like this, when the engine attempts to inline, it inlines the wrong thing and completely breaks your performance altogether. I've been bit with this more than once with clean-match, linked above.
4. When something is very naturally allocation-heavy like this, it's easy for extraneous allocations to look like noise - I had to deal with that in Mithril quite a bit already (I'm a core committer there). It's in cases like these where the first major perf wins are just shedding the extra weight of objects that don't need to exist
5. Megamorphic calls are typically something to avoid, but you only run into most performance cliffs when you attempt to do something that requires checking an object's type map at runtime (like reading a property or writing a property). From doing my own experimentation, here's some of the things I found:
   • Identity checking (like a === b) and nullish checking (like x == null) are fully polymorphic, so it doesn't matter how polymorphic your types are. No types need checked here.
   • Things like typeof x === "object" perform a memory access and partial type map check (for heap objects) worst case scenario, which can thwart a few attempts to optimize, but they do not require a full property map check.
   • Prototype accesses via Object.getPrototypeOf(x) only require a partial type map check for heap objects, since they don't need to care about the underlying property map.
   • Property accesses and anything else of the sort will incur a full type map check, which involves sometimes a pretty expensive series of memory accesses from disparate memory locations that are almost never in cache.
   • If you guard a code path with x != null, an engine can assume property and prototype access is always possible, which enables a few optimizations.
   • If you guard a code path with x !== null && typeof x === "object", an engine can assume you not only have a heap object, but a proper reference type. This simplifies quite a few things, like Object.getPrototypeOf(x), which can likely be inlined in its most specific form. It also allows it to avoid also having to check the type of the heap object itself to know how to perform the property access. (Representations for things like doubles and strings are different than for normal JS objects, making this distinction pretty meaningful.)

dist/fast-copy.cjs.js exports

module.exports = copy

and dist/fast-copy.esm.js exports

export default copy

Request: Make dist/fast-copy.cjs.js export on default as well

module.exports = copy
copy.default = copy

When I import [email protected] as an ES module in Node.js I get the following error:

(node:6782) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.

/<my>/<absolute>/<path>/node_modules/fast-copy/dist/fast-copy.esm.js:346
export { copy as default };
^^^^^^

SyntaxError: Unexpected token 'export'
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1033:15)
    at Module._compile (node:internal/modules/cjs/loader:1069:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:170:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:198:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:385:24)

This is due to the fact that you added conditional exports key in package.json:

"exports": {
    ".": {
        "require": "./dist/fast-copy.cjs.js",
        "import": "./dist/fast-copy.esm.js",
        "default": "./dist/fast-copy.js"
    }
},

but you didn't add "type": "module", which is mandatory in order for the Node.js ESM Loader to correctly interpret your conditional exports. Because "type": "module" is missing, the following default policy is applied:

Within a package, the package.json "type" field defines how Node.js should interpret .js files. If a package.json file does not have a "type" field, .js files are treated as CommonJS.

as stated here https://nodejs.org/api/packages.html#packagejson-and-file-extensions

Here from pinojs/pino-pretty#477

Basically I get an error from trying to log null object that uses V8 optimization

TypeError: Function.prototype.toString requires that 'this' be a Function
    at toString (<anonymous>)
    at getCleanClone (/Users/osiyuk/Projects/github/pino-pretty-issue/node_modules/.pnpm/[email protected]/node_modules/fast-copy/dist/cjs/index.cjs:49:27)
    at copyObjectLooseModern (/Users/osiyuk/Projects/github/pino-pretty-issue/node_modules/.pnpm/[email protected]/node_modules/fast-copy/dist/cjs/index.cjs:213:17)
    at copier (/Users/osiyuk/Projects/github/pino-pretty-issue/node_modules/.pnpm/[email protected]/node_modules/fast-copy/dist/cjs/index.cjs:362:20)
    at copy (/Users/osiyuk/Projects/github/pino-pretty-issue/node_modules/.pnpm/[email protected]/node_modules/fast-copy/dist/cjs/index.cjs:375:16)
    at filterLog (/Users/osiyuk/Projects/github/pino-pretty-issue/node_modules/.pnpm/[email protected]/node_modules/pino-pretty/lib/utils/filter-log.js:30:19)
    at Object.pretty (/Users/osiyuk/Projects/github/pino-pretty-issue/node_modules/.pnpm/[email protected]/node_modules/pino-pretty/lib/pretty.js:81:11)
    at Object.<anonymous> (/Users/osiyuk/Projects/github/pino-pretty-issue/index.js:18:14)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)

Optimization looks like this and it used in fast-querystring package

// Optimization: Use new Empty() instead of Object.create(null) for performance
// v8 has a better optimization for initializing functions compared to Object
const Empty = function () {};
Empty.prototype = Object.create(null);

I figured out that internal function getCleanClone only check that prototype exist

  if (!prototype) {
    return create(null);
  }

I think it should handle case where var Constructor = prototype.constructor === undefined that would mean that this is null prototype and we should return create(null). I could make PR if you agree with it. What do you think?

Also it possible to use this optimization, but I'm not sure how much it would be breaking change and should be perf tested to understand if this is worth it. Separate issue ofc.

Currently i am using rfdc library for deep clone, would be interesting to see how it compares using your benchmarks, if i shoul switch. Thanks.

Hello! I want to contribute on your project with making logo design. What do you think about it?

Not an issue, not a PR. Just wanted to let you know there's a build-in structuredClone now.
I did the benchmarks on node v17 with Macbook 13" M1 2020. structuredClone does not seem to perform that well:
https://github.com/fratzinger/fast-copy/tree/structured-clone#benchmarks

Simple objects

Small number of properties, all values are primitives

structuredClone is supported by:

• Chrome 98
• Safari 137 (Technology Preview Release)
• Firefox 94
• Node.js 17.0
• Deno 1.14

Reference:

https://2ality.com/2022/01/structured-clone.html
https://nodejs.org/api/globals.html#structuredclonevalue-options
https://developer.mozilla.org/en-US/docs/Web/API/structuredClone

Just wanted to let you know. Maybe you also find it interesting. Thanks again for this great package! Excellent benchmark setup, that made the test of structuredClone a one liner!
Feel free to close this issue or whatever you think is necessary. Or do you want to have a PR for this?

this didn't work for me:

  let target, res
  target = { value: 42 }
  Object.defineProperty(target, 'id', {
    value: 1,
    writable: true,
    enumerable: false,
    configurable: true
  })
  res = copy(target)
  t.is(res.id, 1)

Hi, I'd love to test the benchmark links myself!

Is it possible to add the benchmark links to the README?

I also wrote my own copy function and am particularly interested in how it holds up against fast-copy! 😉