Verification does not catch tampered VC about jsonld-signatures

[lemoustachiste]

@dlongley I totally agree.

Jest provides a nice DevX so it's hard to just abandon, plus it would mean quite some rewrite so for now I am stuck with it. But it is quite painful to configure at times. I've seen that they've got experimental support for ESM with more recent versions of Node, but I tried that on Friday and it was a world of hell to untangle. For now I don't have the leeway to spend much time on that, but yeah, longer term it is something to consider.

For typescript I find the support for ESM more transparent and haven't experienced as many issues.

Anyhow I'll see what I can do for now

[dlongley]

@lemoustachiste,

Yes, I can reproduce the segfault -- which occurs when using libraries with esm.js and they are imported via different mechanisms (via both import and require either directly or indirectly from dependencies). This is a problem with v8 that people have been waiting to be fixed for 2 years. It's not specific to any JS library. Sometimes it can be avoided by doing early imports (at the start of your application) using:

import {createRequire} from 'node:module';
const require = createRequire(import.meta.url);
require('whatever-esm.js-lib');

However, the best way to avoid it is to just not use any esm.js-based libs.

I see that your repo is still using the old vc-revocation-list library (replaced by @digitalbazaar/vc-revocation-list) as well as the old esm.js-based version (2.x) of @digitalbazaar/did-method-key. I recommend updating those and seeing where that gets you.

[dlongley]

@lemoustachiste,

If I upgrade the libs as I mentioned above and fix the imports (i.e., do import * as foo instead of import foo) ... and I fix up one change with the did key driver, i.e.:

import * as didMethodKey from '@digitalbazaar/did-method-key';
//const { driver: didKeyDriver } = didMethodKey;
const didKeyDriver = didMethodKey.driver();

Then I can run generate and get a safe mode error! So that's progress, I think. Hopefully you can do the same.

[lemoustachiste]

Hey so I figured out the original error that I was reporting and it was my fault, the list didn't have a proper id set so the canonization didn't occur on a document, but rather a empty string (had the same issue with a non-prefixed uuid though). I set a URL as id and now it works as expected.

Which is also probably why I quickly saw the issue with 11.1.0

I'll keep in mind your notes here when I upgrade, thanks also for pointing me to the correct package versions I'll try to use them.

[dlongley]

@lemoustachiste,

Thank you for the sign and verify example code -- but we'd be able to test this more quickly if you had a full example for us to run. Would you be able to provide that? If not, we'll have to find time to build that out ourselves and paste / edit your code (to remove any TypeScriptisms) to see if we can replicate.

I will say that just from looking at the VC examples, I'm a little surprised no safe mode error was thrown (before any signing / verifying could even occur) given that there are relative URLs being used both for the credential ID and the credential subject ID. What library and version of the Ed25519Signature2020 suite are you using? That could be where a potential bug is. We would have to know which implementation and version you're using for that to investigate further anyway.

[davidlehn]

I'm guessing you are using @digitalcredentials/jsonld-signatures which is at 9.3.1. This project is published as jsonld-signatures, which is at 11.1.0. We've added some newer features like the "safe" mode Dave mentioned. That feature is able to fail on many JSON-LD constructs that can cause signing problems. The digitalcredentials group forked this project and a bunch of others to address some TypeScript and/or React Native issues. Ideally we could fix those here and deprecate the forks to avoid duplicated effort. Help wanted.

You can paste your JSON-LD into the playground and either turn on the safe option, or run the canonize algorithm, and you'll see safe mode violations. Fixing the relative URLs or adding a base would resolve those, depending on your use case.

And yes, it would be easier to debug this with a running example.

[davidlehn]

To clarify, you should put that data in the playground to see what's going on. Look at the N-Quads tab output in non-safe mode. With the relative ids, it just drops everything, as per the spec, and all you have is the proof quads data. I assume that's what the unchanging signing data you see is. If you change credentialSubject.id to something like ex:#list, you'll see those two properties quads show up. Similarly, if you change the root id to ex:foo, all the other quads show up. We added the safe mode in (our) newer jsonld and jsonld-signatures and related packages in order to be more strict with input data and detect these sorts of issues.

[lemoustachiste]

Hi @davidlehn @dlongley,

thanks for your answers, I tried upgrading jsonld-signatures to 11.1.0 but I entered a world of hell with Jest and couldn't fix it (Jest is not great at handling ESM). I'm back on 9.3.1, I haven't been able to explore more on the reported issue.

I'll try to work on an isolated repro to better target the issue.

I did go through the jsonld playground and nothing unusual jumped to me. Here is the n-quads output if you catch something I don't:

<https://fhircat.org/jsonld/playground/#list> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/vc/status-list#StatusList2021> .
<https://fhircat.org/jsonld/playground/#list> <https://w3id.org/vc/status-list#encodedList> "H4sIAAAAAAAAA-3OMQEAAAgDoJ32T2yL6QEJSAAAgJK5DgAAAAAAAAAAAAAAAAB_LUkhKJ4AQAAA" .
<https://fhircat.org/jsonld/playground/> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/vc/status-list#StatusList2021Credential> .
<https://fhircat.org/jsonld/playground/> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2018/credentials#VerifiableCredential> .
<https://fhircat.org/jsonld/playground/> <https://w3id.org/security#proof> _:b0 .
<https://fhircat.org/jsonld/playground/> <https://www.w3.org/2018/credentials#credentialSubject> <https://fhircat.org/jsonld/playground/#list> .
<https://fhircat.org/jsonld/playground/> <https://www.w3.org/2018/credentials#issuanceDate> "2023-03-22T15:01:32.117Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
<https://fhircat.org/jsonld/playground/> <https://www.w3.org/2018/credentials#issuer> <did:key:z6MkvmPwfKvALHbGPjTh1J8oMum6YTQaeGWsarAbik7P82J4> .
_:b1 <http://purl.org/dc/terms/created> "2023-03-22T15:01:44.256Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> _:b0 .
_:b1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/security#Ed25519Signature2020> _:b0 .
_:b1 <https://w3id.org/security#proofPurpose> <https://w3id.org/security#assertionMethod> _:b0 .
_:b1 <https://w3id.org/security#proofValue> "zwbQ5E1MazeFGsceFJ4yY6RMnTURQ7wkX7HbXAfwAgxM2EB2jgm8gUvwvYTyxCMLQQXMEWVNTZBuyYxHBZFCntVD"^^<https://w3id.org/security#multibase> _:b0 .
_:b1 <https://w3id.org/security#verificationMethod> <did:key:z6MkvmPwfKvALHbGPjTh1J8oMum6YTQaeGWsarAbik7P82J4#z6MkvmPwfKvALHbGPjTh1J8oMum6YTQaeGWsarAbik7P82J4> _:b0 .

Finally I'm using a forked version (3.0.1-0) of your Ed25519 library (https://github.com/lemoustachiste/ed25519-signature-2020/tree/convert-to-module), I'm aware it's not up to date, but I haven't had issues so far (I understand this might not be thing to hold to). That fork allows for CJS entry points. My main concern is the general lack of compatibility with Jest (which expects CJS) when relying on Digital Bazaar libs so I tend to not try to upgrade too much (as proved with the infructuous upgrade of jsonld-signatures 11.1.0).

[dlongley]

So I think it needs to be said that if a modern JavaScript testing framework does not support the standard module interface for JavaScript -- that's a serious flaw of that testing framework. I do not recommend anyone creating and maintaining forks of JavaScript libraries that use ESM (as all JavaScript libraries should aim to do moving forward) on account of the flaws of a testing framework. It would be much better for those in the Jest community to come together and support ESM -- rather than everyone else that uses Jest start to fork every other library they need.

I think there are considerable problems generated by the TypeScript and Jest communities (of which this is an example) that are starting to outweigh the benefits of using those technologies generally. Those technologies should focus on bringing themselves inline with the ECMAScript standard or admit that they are incompatible and constitute an entirely different ecosystem requiring the re-implementation of libraries written in standard JavaScript -- not unlike any other separate language, e.g., Rust, C++, Python, etc.

[lemoustachiste]

I have created a repro here: https://github.com/blockchain-certificates/rl-blockcerts-poc/tree/repro-esm.
I have removed all typescript and upgraded to the latest versions of the packages.
I'm running node v18.15.0.

However I'm seeing a segfault error that I haven't yet debugged, I'm not sure if it's only my machine or if you can see it too, I'm sharing in case it works for you and you can debug. Until that segfault issue modifying the list didn't throw any error on verification (although I was testing with [email protected]).

[lemoustachiste]

ok here is my bash history which shows some variations:

$ npm run generate

> [email protected] generate
> node --trace-warnings src/createVCRevocationList.js

(node:95823) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
    at emitExperimentalWarning (node:internal/util:238:11)
    at ESMLoader.jsonStrategy (node:internal/modules/esm/translators:267:3)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:468:14)
[1]    95810 segmentation fault  npm run generate



(base) ➜  rl-blockcerts-poc git:(repro-esm) ✗ npm i [email protected]
npm notice Beginning October 4, 2021, all connections to the npm registry - including for package installation - must use TLS 1.2 or higher. You are currently using plaintext http to connect. Please visit the GitHub blog for more information: https://github.blog/2021-08-23-npm-registry-deprecating-tls-1-0-tls-1-1/
npm notice Beginning October 4, 2021, all connections to the npm registry - including for package installation - must use TLS 1.2 or higher. You are currently using plaintext http to connect. Please visit the GitHub blog for more information: https://github.blog/2021-08-23-npm-registry-deprecating-tls-1-0-tls-1-1/

added 8 packages, removed 8 packages, and changed 1 package in 1s

19 packages are looking for funding
  run `npm fund` for details



(base) ➜  rl-blockcerts-poc git:(repro-esm) ✗ npm run generate

> [email protected] generate
> node --trace-warnings src/createVCRevocationList.js

(node:95869) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
    at emitExperimentalWarning (node:internal/util:238:11)
    at ESMLoader.jsonStrategy (node:internal/modules/esm/translators:267:3)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:468:14)
file:///Users/julien/work/rl-blockcerts-poc/src/helpers/generateList.js:7
  const list = await createList({ length: 131072 });
                     ^

TypeError: createList is not a function
    at generateList (file:///Users/julien/work/rl-blockcerts-poc/src/helpers/generateList.js:7:22)
    at generateEncodedList (file:///Users/julien/work/rl-blockcerts-poc/src/helpers/generateList.js:12:22)
    at file:///Users/julien/work/rl-blockcerts-poc/src/helpers/generateList.js:17:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

Node.js v18.15.0

So, it seems that potentially the SEGFAULT issue is coming from jsonld-signatures, when I downgrade to 9.3.1 it does not fail as such.

However I'm blocked in multiple ways.

If I use jsonld-sig@11 installed, then the imports look potentially good, but the SEGFAULT issue occurs.
If I downgrade to 9.3.1, then the imports seem to be faulty (I need to do a cjs style import), if I fix them, then it fails on the Bitstring subdependency import:

$ npm run generate

> [email protected] generate
> node --trace-warnings src/createVCRevocationList.js

(node:94582) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
    at emitExperimentalWarning (node:internal/util:238:11)
    at ESMLoader.jsonStrategy (node:internal/modules/esm/translators:267:3)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:468:14)
file:///Users/julien/work/rl-blockcerts-poc/node_modules/vc-revocation-list/RevocationList.js:4
import Bitstring from '@digitalbazaar/bitstring';
       ^^^^^^^^^
SyntaxError: The requested module '@digitalbazaar/bitstring' does not provide an export named 'default'
    at ModuleJob._instantiate (node:internal/modules/esm/module_job:124:21)
    at async ModuleJob.run (node:internal/modules/esm/module_job:190:5)

I'm a bit lost with all that.

[dlongley]

@lemoustachiste,

If I clone that repo -- I can install on node v18.15.0 and run npm run generate without issue. It outputs three JSON files upon completion. If I install [email protected] I get the same result.

Should I be running something else to see a problem? There are other commands for verify and revoke and the latter looks like it may take a command line argument, etc. Can you provide more instructions on what to do?

[dlongley]

I missed the repro-esm branch. Let me try again.