python-jsonschema / referencing Goto Github PK

View Code? Open in Web Editor NEW

31.0 3.0 8.0 429 KB

Cross-specification JSON referencing (JSON Schema, OpenAPI, and the one you just made up!)

Home Page: https://referencing.readthedocs.io/

License: MIT License

Python 100.00%

jsonschema asyncapi hypermedia json openapi api json-schema

referencing's People

Contributors

Stargazers

Watchers

Forkers

braingram alvistack eliahkagan vidminas sysfce2 wolfi-chainguard-demo jamescw19 michaelfeil

referencing's Issues

Consider adding `referencing.jsonschema.SchemaValidatingRegistry`

It might be a nice convenience to have an object with the same interface as referencing.Registry but which automatically validated schemas (Resources) on insertion.

Specifically an object where (in pseudocode):

import referencing.jsonschema

valid_schema = referencing.jsonschema.DRAFT7.create_resource({"type": "integer"})
invalid_schema = referencing.jsonschema.DRAFT7.create_resource({"type": 37})

registry = referencing.jsonschema.SchemaValidatingRegistry()
valid_schema @ registry  # fine
invalid_schema @ registry # -> big boom!

This would be an inverted dependency on jsonschema (the library) -- because we'd call DraftNValidator.check_schema each time a resource was added to the SchemaValidatingRegistry.

We should likely allow for parametrizing looking up the right object for validation (i.e. fetching which metaschema to use and what function to call with it), especially considering that this library (referencing) is meant to be JSON Schema implementation agnostic.

TypeError: field() got an unexpected keyword argument 'alias'

Hi,

I am using nbconvert to convert Jupyter notebooks to RST files in my project and getting the following jsonschema/referencing exception during the build process:

  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbconvert/filters/markdown.py", line 13, in <module>
    from .markdown_mistune import markdown2html_mistune
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbconvert/filters/markdown_mistune.py", line 22, in <module>
    from nbconvert.filters.strings import add_anchor
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbconvert/filters/strings.py", line 23, in <module>
    from nbconvert.preprocessors.sanitize import _get_default_css_sanitizer
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbconvert/preprocessors/__init__.py", line 3, in <module>
    from nbclient.exceptions import CellExecutionError
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbclient/__init__.py", line 5, in <module>
    from .client import NotebookClient, execute  # noqa: F401
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbclient/client.py", line 17, in <module>
    from nbformat import NotebookNode
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbformat/__init__.py", line 11, in <module>
    from . import v1, v2, v3, v4
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbformat/v4/__init__.py", line 23, in <module>
    from .convert import downgrade, upgrade
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbformat/v4/convert.py", line 11, in <module>
    from nbformat import v3, validator
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbformat/validator.py", line 16, in <module>
    from .json_compat import ValidationError, _validator_for_name, get_current_validator
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/nbformat/json_compat.py", line 11, in <module>
    import jsonschema
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/jsonschema/__init__.py", line 13, in <module>
    from jsonschema._format import FormatChecker
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/jsonschema/_format.py", line 11, in <module>
    from jsonschema.exceptions import FormatError
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/jsonschema/exceptions.py", line 14, in <module>
    from referencing.exceptions import Unresolvable as _Unresolvable
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/referencing/__init__.py", line 4, in <module>
    from referencing._core import Anchor, Registry, Resource, Specification
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/referencing/_core.py", line 29, in <module>
    class Specification(Generic[D]):
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/referencing/_core.py", line 55, in Specification
    ] = field(alias="anchors_in")
TypeError: field() got an unexpected keyword argument 'alias'
Exception occurred:
  File "/builds/group/myproject/.tox/docs/lib/python3.8/site-packages/referencing/_core.py", line 55, in Specification
    ] = field(alias="anchors_in")
TypeError: field() got an unexpected keyword argument 'alias'

I am clueless about why this might be happening and appreciate any help. Thank you.

Add hypothesis strategies for generating registries and resources

Provide a helper to fully resolve $refs in a schema

I want to generate some documentation/description (probably some input forms) from a json schema that contains $refs. Is there a way in jsonschema to retrieve a json schema with $refs resolved and inlined, so I can iterate through the schema?

So from a schema like this:

{
  "definitions": {
    "some_object": {
      "type": "object",
      "title": "My definition",
      "description": "Some description",
      "properties": {
        "name": {
          "type": "string"
        },
        "value": {
          "type": "number"
        }
      }
    }
  },
  "properties": {
    "list_of_objects": {
      "type": "array",
      "title": "Some stuff",
      "description": "List of stuff",
      "items": {"$ref": "#/definitions/some_object"}
    }
  }
}

I want to have this structure in my Python object (notice that the {$ref:...} section is replaced by the actual definition):

{
  "definitions": {
    "some_object": {
      "type": "object",
      "title": "My definition",
      "description": "Some description",
      "properties": {
        "name": {
          "type": "string"
        },
        "value": {
          "type": "number"
        }
      }
    }
  },
  "properties": {
    "list_of_objects": {
      "type": "array",
      "title": "Some stuff",
      "description": "List of stuff",
      "items": {
        "type": "object",
        "title": "My definition",
        "description": "Some description",
        "properties": {
          "name": {
            "type": "string"
          },
          "value": {
            "type": "number"
          }
        }
      }
    }
  }
}

I am aware that with self-referencing schemas this will lead to an infinite structure, but hopefully this is feasible with reasonable limitations (eg. excluding self-refs).

Give Registry a unicode tree output

This is useful for debugging what's in a registry -- show some unicode-box-drawing output which breaks down all the resources in the registry.

Probably this should print URLs in some sort of tree, e.g. (without the box drawing):

http://example.com/  – Resource(...)
  foo/ – Resource(...)
    bar/ – Resource(...)
    baz/ – Resource(...)
http://example.org/ – Resource(...)

Relative URL paths as $refs

I am trying to replace RefResolver with Registry in pystac (stac-utils/pystac#1215) and cannot get relative paths to work. I wrote a retrieve function that should make the relative schema URIs absolute with respect to the location of the base schema, but it seems like the ../.. are getting stripped off.

Here is an example of the type of schemas I am working with: https://raw.githubusercontent.com/radiantearth/stac-spec/v0.8.1/collection-spec/json-schema/collection.json.

"../../catalog-spec/json-schema/catalog.json" for instance is just "catalog-spec/json-schema/catalog.json" within my retrieve function.

This is potentially similar to #70, but these refs are not prefixed by file so I think this style of ref is supposed to be legit

Add the (simplest) example of usage to the doc index and/or README

Right now the page is pretty bare.

documentation uses `url` module which is no longer maintained

On building documentation url module is required

+ /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man
Running Sphinx v7.1.2

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 356, in eval_config_file
    exec(code, namespace)  # NoQA: S102
  File "/home/builder/rpmbuild/BUILD/referencing-0.31.1/docs/conf.py", line 3, in <module>
    from url import URL
ModuleNotFoundError: No module named 'url'

This module had last release in 2017 https://pypi.org/project/url/#history and is not maintained since 2020 https://github.com/seomoz/url-py

Implement a resource bundler

We should offer a resource bundler which implements the "official JSON Schema bundling process". Doing so shouldn't be too difficult on top of what's already here.

Here's one from json-schema-ref-parser, and another from @jderosiers.

It's probably a judgement call whether to do this here in this repository or in a separate one which is JSON Schema specific (even though here we have referencing.jsonschema, a module for JSON Schema-specific functionality). But this kind of functionality relies on functionality which is less likely to be cross-spec applicable -- or maybe not and it can be (would probably involve adding some methods to Specification regardless which specify how to perform the bundling). Thinking required. If it does live here, it's probably something that can be a method on Registry, and may tangentially relate to #16.

Add a persistent API for adding additional known keywords to those where subresources are searched

I.e. someone will want to write some new Specification where there's an additional foo keyword with subschemas in its values, and therefore it'd be nice to offer referencing.jsonschema.DRAFT202012.subresources_of + "foo" as a way to get "all the draft 2020 behavior plus this new extra keyword".

Import fails on typing.py: AttributeError: type object 'Mapping' has no attribute '__class_getitem__'

My project's unit tests are failing to pull in flask_restx, now that typing-extensions 4.7.0 has been released and pulls in referencing. The backtrace is this:

.tox/pypy3-test/lib/pypy3.8/site-packages/flask_restx/__init__.py:1: in <module>
    from . import fields, reqparse, apidoc, inputs, cors
.tox/pypy3-test/lib/pypy3.8/site-packages/flask_restx/reqparse.py:15: in <module>
    from .model import Model
.tox/pypy3-test/lib/pypy3.8/site-packages/flask_restx/model.py:13: in <module>
    from jsonschema import Draft4Validator
.tox/pypy3-test/lib/pypy3.8/site-packages/jsonschema/__init__.py:13: in <module>
    from jsonschema._format import FormatChecker
.tox/pypy3-test/lib/pypy3.8/site-packages/jsonschema/_format.py:11: in <module>
    from jsonschema.exceptions import FormatError
.tox/pypy3-test/lib/pypy3.8/site-packages/jsonschema/exceptions.py:15: in <module>
    from referencing.exceptions import Unresolvable as _Unresolvable
.tox/pypy3-test/lib/pypy3.8/site-packages/referencing/__init__.py:4: in <module>
    from referencing._core import Anchor, Registry, Resource, Specification
.tox/pypy3-test/lib/pypy3.8/site-packages/referencing/_core.py:10: in <module>
    from referencing import exceptions
.tox/pypy3-test/lib/pypy3.8/site-packages/referencing/exceptions.py:11: in <module>
    from referencing.typing import URI
.tox/pypy3-test/lib/pypy3.8/site-packages/referencing/typing.py:11: in <module>
    Mapping[str, str]
E   AttributeError: type object 'Mapping' has no attribute '__class_getitem__'

This is within the context of pytest, under PyPy 3.8. I'm having trouble reproducing this minimally, because any attempts to reproduce are giving me a TypeError, like your code expects.

If I run this directly in a console, it returns the expected TypeError.

from collections.abc import Mapping as Mapping
Mapping[str, str]

If I do this, I get the same AttributeError:

Mapping.__class_getitem__

Bump the classifier from Alpha to Beta

Once #28 is closed and we've added referencing.to_cached_resource, we'll change referencing's classifiers to mark it in Beta.

It will stay in this state until some or all of:

it has support beyond the JSON Schema specifications (i.e. OpenAPI)
another library beyond jsonschema uses it
jsonschema has used it for around a year

Provide a way to extend the specifications known by `Resource.from_contents` and `specification_with`

This is needed downstream by implementations (jsonschema) who allow new dialect creation.

Add a mechanism for evolving JSON Schema drafts to add keywords

We need some generic mechanism for saying "I have a new Specification, it's like DRAFT202012 but with a new keyword foo" and then allowing the new keyword foo to either have a schema value (a la additionalItems or whatever) or schema-containing values (properties) etc...

Obviously the most "generic" mechanism will be fully defining subresources_of (and now maybe_in_subresource) as a generic callable -- but a simple/fast interface which just allows adding the extra keyword will make things easier downstream.

Check all $refs are resolvable

I've added 5 json schemas which cross-reference each other and refer to $defs in the same file using $ref into a Registry. Is there a straightforward way to check if all of the $refs resolve successfully? I can of course use resolver.lookup() on individual paths - I'd like that, but automatically, for all $refs. I've spent some time looking at the docs and code for this library and haven't figured it out, but apologies if I've missed something obvious. (If not, I guess I need to use jsonref for this?)

Jsonschema validation 4 to 5 times slower when upgrading to referencing 0.31.0

Hello,

We are using intensively jsonschema to validate the schema of internal documents. jsonschema uses referencing to resolve some json references.

Our Python script is spending 99,99% of the time looping over a function like:
jsonschema.validate(dictionary, schema)

When running with referencing 0.30.2, we timed the Python script (with time command):

13.49user 0.07system 0:13.58elapsed 99%CPU (0avgtext+0avgdata 54936maxresident)k 0inputs+16outputs (0major+15239minor)pagefaults 0swaps

When upgrading to referencing 0.31.0, we timed:
56.42user 0.12system 0:56.60elapsed 99%CPU (0avgtext+0avgdata 55364maxresident)k 0inputs+16outputs (0major+17380minor)pagefaults 0swaps

Could you help to understand why the Python script is 4 to 5 times slower when moving to referencing 0.31.0 ?

Best regards,
Riadh

Cannot pip install on alpine - maybe prebuilt version missing?

The following fails on alpine: pip install referencing.

In fairness, the problem is with the rpds-py package, but I could not find a separate issue tracker for that, it seems to only exist for the purpose of this project, so I am posting it here instead.

I originally ran into this problem using Poetry in CI on Alpine (which depends on jsonschema, which depends on this).

I think maybe a "musllinux" build is missing?

In full it can be reproduced in a small example like this:

docker run --rm -it alpine:3.18
apk add python3 py-pip
pip install referencing

Which gives the output:

Collecting referencing
  Using cached referencing-0.29.1-py3-none-any.whl (25 kB)
Collecting attrs>=22.2.0 (from referencing)
  Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting rpds-py>=0.7.0 (from referencing)
  Using cached rpds_py-0.7.1.tar.gz (15 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]

      Cargo, the Rust package manager, is not installed or is not on PATH.
      This package requires Rust and Cargo to compile extensions. Install it through
      the system's package manager or via https://rustup.rs/

      Checking for Rust toolchain....
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

RefResolver (or its successor) should allow customizing deserialization, not only scheme / retrieval

I'm using currently handling schemas written in YAML (for human-readability) but validated by jsonschema. I parse the schema in YAML (and make sure I subset my YAML to stay JSON-compatible) and then run the parsed dictionary through jsonschema. It is working nicely with one issue: When I want to use file:// includes, the files are implicitly assumed to be json.

@Julian I would like to add support for these files being YAML and am wondering if this is something you would support. If so, what is your recommendation for the cleanest way to add it? If it's not something you support, what would you recommend as the cleanest way to add the support for my own code without hacking/forking jsonschema? I thought about adding custom resolver as is done in python-jsonschema/jsonschema#225, but it sounds like RefResolver is not intended to be API-stable, so subclassing is liable to break in the future. Is there another way?

Thanks,
Martin

Invalid `type: ignore` style when checking with mypy

The jsonschema module has 3 type: ignore comments that are not in PEP484 format. This results in the following error when running mypy where referencing has been imported:

$ mypy my_package
/usr/local/lib/python3.8/dist-packages/referencing/jsonschema.py:432: error: Invalid "type: ignore" comment
/usr/local/lib/python3.8/dist-packages/referencing/jsonschema.py:474: error: Invalid "type: ignore" comment
/usr/local/lib/python3.8/dist-packages/referencing/jsonschema.py:505: error: Invalid "type: ignore" comment
Found 3 errors in 1 file (errors prevented further checking)

The error produced is the same as reported here. However, newer versions of mypy do not fix the issue, as the style is not correct.

These can't be ignored with the mypy ignore_errors = True flag, because these are fatal errors that do not allow mypy to run over the rest of the codebase.

From what I can tell, the error messages just need an extra # after the error code to allow mypy to pass

Are relative JSON pointers supported?

Sorry it's me again 😄

I'm making some progress on my "schema walker" experiments, and wanted to see how relative references using relative JSON pointers were handled.

The docstring of the Resolver class states:

referencing/referencing/_core.py

Lines 615 to 621 in 7d6069a

    
           @frozen 
        
           class Resolver(Generic[D]): 
        
               """ 
        
               A reference resolver. 
        
               Resolvers help resolve references (including relative ones) by 
        
               pairing a fixed base URI with a `Registry`.

However, trying with the following example (and making use of the jsonschema library for demonstration purposes), I get the following:

from jsonschema.validators import Draft7Validator

schema = {
    "$id": "my_schema",
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "first_email": {
            "type": "string",
            "format": "email"
        },
        "second_email": {
            "$ref": "1/first_email"
        }
    }
}

Draft7Validator(alt).validate({"first_email": "[email protected]", "second_email": "[email protected]"})

I couldn't find anything in the codebase that would handle these relative JSON pointers. I'm also unsure when this was introduced in the json schema spec(s), I couldn't find anything in the changelogs. Thanks in advance!

Consider providing `retrieve` functions with an enclosing specification

Sometimes one wants to retrieve a resource but default to the specification of the document it is contained within.

In other words, to call Resource.from_contents(..., default_specification=enclosing_specification) from within a retrieval function.

This isn't completely trivial to implement however, because users can call .get_or_retrieve(uri) directly, meaning the argument would have to exist there too. Additionally, resources can be root resources of course, so enclosing_specification would anyways potentially be None for such cases.

Relative file path references not supported?

Hi,

I've inherited some code using the old RefResolver (4.17) version of jsonschema, I'm working on updating it to use the new referencing library.

Our JSON schemas contain references like this:

"network": {
  "$ref": "file:../networks/network_asset.json"
},

I've constructed a Registry object with a retrieve function, which is passed to the validator.
(For context, schema_base_path is the directory path to the primary schema, from which relative references are resolved.)

def retrieve_schema(uri: str):
    reference_path = urlparse(uri).path[1:]
    path = schema_base_path / reference_path
    contents = read_json_file(path)
    return Resource.from_contents(contents)

registry = Registry(retrieve=retrieve_schema)

Draft7Validator(schema_json, registry=registry).validate(json_to_validate)

However, referencing appears to ignore the relative file path ... In the above example, retrieve_schema receives the following URI: file:///networks/network_asset.json. In testing, I discovered that file:./networks/network_asset.json and file:networks/network_asset.json resolve to an identical URI. This prevents me from resolving the relative path to the reference...

I narrowed it down to the lookup function in referencing/_core.py, line 581:

if ref.startswith("#"):
    uri, fragment = self._base_uri, ref[1:]
else:
    uri, fragment = urldefrag(urljoin(self._base_uri, ref))

Does referencing support relative file path references? If so, how can I configure that?

RefResolutionError when using self signed certificate

jsonschema.exceptions.RefResolutionError: HTTPSConnectionPool(host='xxxxx.xxxx.xxxx', port=443): Max retries exceeded with url: /schema.json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)')))

https://github.com/python-jsonschema/jsonschema/blob/main/jsonschema/validators.py#L993
Could you please allow users to pass CA_BUNDLE path to verify parameter in requests.get() method?
e.g. result = requests.get(uri, verify='/path/to/certfile').json()

Question regarding usage of generics

Hello,

thanks for this library. I'm looking for a way to parse JSON Schema into something that can be used in Python but couldn't find anything suitable. jsonschema can validate JSON "instances" against a schema, but does not seem to provide such parsing abilities.

As I'm probably going to parse the schemas "by hand", referencing looks interesting for my task as I will not have to resolve references manually.

That said, I just wanted to raise something that feels inconsistent regarding typing. The Registry is currently defined as a Mapping[URI, Resource[D]], simplified as Mapping[str, D]. Does this mean a Registry should hold only one type of resources?

This D type variable seems to be kept when iterating over subresources and anchors as well, e.g. subresources returns an Iterable[Resource[D]].

My understanding is for JSON schemas, this D type variable should always resolve to Union[bool, ObjectSchema], even when iterating over the subresources. But I don't know if this is the case if not dealing with JSON schemas?

Build fails with ValueError: Unknown classifier in field `project.classifiers`: Topic :: File Formats :: JSON

Hi, it seems that the project metadata is wrong.

Full build log here:

Processing /build/referencing-0.29.1
  Running command Preparing metadata (pyproject.toml)
  Traceback (most recent call last):
    File "/nix/store/vwk3jva2a07pwvsa7qxvzh9znzk47z28-python3.10-pip-23.0.1/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_proc>
      main()
    File "/nix/store/vwk3jva2a07pwvsa7qxvzh9znzk47z28-python3.10-pip-23.0.1/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_proc>
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/nix/store/vwk3jva2a07pwvsa7qxvzh9znzk47z28-python3.10-pip-23.0.1/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_proc>
      whl_basename = backend.build_wheel(metadata_directory, config_settings)
    File "/nix/store/6kvyd5r4zp7prfwh1zm0k6d049di1wmw-python3.10-hatchling-1.13.0/lib/python3.10/site-packages/hatchling/build.py", line 56, in build_whe>
      return os.path.basename(next(builder.build(wheel_directory, ['standard'])))
    File "/nix/store/6kvyd5r4zp7prfwh1zm0k6d049di1wmw-python3.10-hatchling-1.13.0/lib/python3.10/site-packages/hatchling/builders/plugin/interface.py", l>
      self.metadata.validate_fields()
    File "/nix/store/6kvyd5r4zp7prfwh1zm0k6d049di1wmw-python3.10-hatchling-1.13.0/lib/python3.10/site-packages/hatchling/metadata/core.py", line 244, in >
      self.core.validate_fields()
    File "/nix/store/6kvyd5r4zp7prfwh1zm0k6d049di1wmw-python3.10-hatchling-1.13.0/lib/python3.10/site-packages/hatchling/metadata/core.py", line 1326, in>
      getattr(self, attribute)
    File "/nix/store/6kvyd5r4zp7prfwh1zm0k6d049di1wmw-python3.10-hatchling-1.13.0/lib/python3.10/site-packages/hatchling/metadata/core.py", line 979, in >
      raise ValueError(message)
  ValueError: Unknown classifier in field `project.classifiers`: Topic :: File Formats :: JSON
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /nix/store/1r6n7v2wam7gkr18gxccpg7p5ywgw551-python3-3.10.12/bin/python3.10 /nix/store/vwk3jva2a07pwvsa7qxvzh9znzk47z28-python3.10-pip-23.>
  cwd: /build/referencing-0.29.1
  Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Add support for `definitions` to newer JSON Schema drafts

The specification seems to use SHOULD language here, saying:

implementations SHOULD assume that "$defs" and "definitions" have the same behavior when that meta-schema is used.

Moving from `RefResolver` to the referencing library

I'm a bit confused as how to replace RefResolver with the referencing library.

In my test code, I create a RefResolver that points to the same folder as my top schema file is, and this works really well:

import json

from pathlib import Path

import pytest

import jsonschema

def test_foo():

    schema_file = Path("foo.schema.json")

    with open(schema_file) as f:
        schema = json.load(f)

    schema_dir = schema_file.parent.resolve().as_posix()
    print(schema_dir)

    resolver = jsonschema.RefResolver(base_uri = 'file://' + schema_dir + '/', referrer = schema_file.name)

    data = {
        "name"              : "FOO!",
        "sumthing"          : [{
            "name"  : "Groot"
        }]
    }

    jsonschema.validate(instance=data, schema=schema, resolver=resolver)

I've read Dynamically Retrieving Resources. My code for using the referencing library is:

import json

from pathlib import Path

import pytest

from referencing import Registry, Resource
import jsonschema

def test_foo():

    schema_file = Path("foo.schema.json")

    with open(schema_file) as f:
        resource = Resource.from_contents(json.load(f))

    schema_dir = schema_file.parent.resolve().as_posix()

    registry = resource @ Registry()  # Add the resource to a new registry

    resolver = registry.resolver(base_uri = 'file://' + schema_dir + '/')

    data = {
        "name"              : "FOO!",
        "sumthing"          : [{
            "name"  : "BAR!"
        }]
    }

    jsonschema.validate(instance=data, schema=resource.contents, resolver=resolver, registry=registry)

I get a AttributeError: 'Resolver' object has no attribute 'resolving' exception. Of course, because sumthing is optional, this works as expected:

    data = {
        "name"              : "FOO!"
    }

    jsonschema.validate(instance=data, schema=resource.contents)

foo.schema.json:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "file:foo.schema.json",
    "title": "The Foo metadata JSON schema file",
    "description": "Schema for validating the serialization of a Foo object",
    "type": "object",
    "properties": {
        "name": {
            "description": "The name of the object",
            "type": "string"
        },
        "sumthing": {
            "description": "The list of logical inputs for this module",
            "type": "array",
            "items": {
                "$ref": "bar.schema.json"
            },
            "minItems": 1
        }
    },
    "required": [
        "name"
    ]
}

bar.schema.json:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "file:bar.schema.json",
    "title": "The Bar metadata JSON schema file",
    "description": "Schema for validating the serialization of a Bar object",
    "type": "object",
    "properties": {
        "name": {
            "description": "The name of the object",
            "type": "string"
        }
    },
    "required": [
        "name"
    ]
}

Thanks!

Take yarl.URLs instead of strs

Should cut down on a tiny bit of code and generally be more ergonomic -- some minor benchmarking undoubtedly couldn't hurt.

RFE: merge python-jsonschema/referencing-suite into python-jsonschema/referencing?🤔

Looks like refferencing module uses external git submodule https://github.com/python-jsonschema/referencing-suite.

If that submodules is not used anywhere else (so far I was not able to find that is used by some other modules) probably it would be good to merge python-jsonschema/referencing-suite directly into python-jsonschema/referencing test suite.

Ref v2 Spec

This is some random notes on a new version of the ref resolution API. Feel free to leave comments, although some of the below might be overly terse chicken scratch

It should probably be easier to pre-resolve all refs (maybe that should even be the default) rather than lazy/forced resolution
Keep in mind async use cases
Too many methods bruh. Just 2? Resolver().resolve(ref) + Resolver().for_scope(url)?
Take yarl.URLs, not strs
What should be the thing for users who want to resolve relative to files on disk?
Move the resolver argument to extend

sdist is missing suite

The sdist package at PyPI is missing suite directory. Without the directory testing fails:

_________ ERROR collecting referencing/tests/test_referencing_suite.py _________
/usr/lib/python3.9/vendor-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
/usr/lib/python3.9/vendor-packages/_pytest/runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/usr/lib/python3.9/vendor-packages/_pytest/python.py:531: in collect
    self._inject_setup_module_fixture()
/usr/lib/python3.9/vendor-packages/_pytest/python.py:545: in _inject_setup_module_fixture
    self.obj, ("setUpModule", "setup_module")
/usr/lib/python3.9/vendor-packages/_pytest/python.py:310: in obj
    self._obj = obj = self._getobj()
/usr/lib/python3.9/vendor-packages/_pytest/python.py:528: in _getobj
    return self._importtestmodule()
/usr/lib/python3.9/vendor-packages/_pytest/python.py:617: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
/usr/lib/python3.9/vendor-packages/_pytest/pathlib.py:565: in import_path
    importlib.import_module(module_name)
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1030: in _gcd_import
    ???
<frozen importlib._bootstrap>:1007: in _find_and_load
    ???
<frozen importlib._bootstrap>:986: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:680: in _load_unlocked
    ???
/usr/lib/python3.9/vendor-packages/_pytest/assertion/rewrite.py:178: in exec_module
    exec(co, module.__dict__)
referencing/tests/test_referencing_suite.py:27: in <module>
    raise SuiteNotFound()
E   referencing.tests.test_referencing_suite.SuiteNotFound: Cannot find the referencing suite. Set the REFERENCING_SUITE environment variable to the path to the suite, or run the test suite from alongside a full checkout of the git repository.

Please add the suite directory to sdist. Thank you.

Please note the github tarball contains the suite directory, but it is empty.

0.32.0: pytest fails

You'v locked the ticket #138 before I was able add more details 😞

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-referencing-0.32.0-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-referencing-0.32.0-2.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/referencing-0.32.0
configfile: pyproject.toml
plugins: subtests-0.11.0
collected 802 items

referencing/tests/test_core.py ...........................................................................................................                                            [ 13%]
referencing/tests/test_exceptions.py ....................................                                                                                                             [ 17%]
referencing/tests/test_jsonschema.py .........................................................                                                                                        [ 24%]
referencing/tests/test_referencing_suite.py ,.,.,.,.,.,.,.,.,.,.,.,.,,.,.,.,.,,.,,,.,.,.,,.,,.,,.,.,.,.,,,.,.uu,uuu.uu,uuu.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,,,,.,.,.,.,.,,.,,.,.,.,.,,.,. [ 31%]
,.,,.,.,.,,.,,,,.,,.,,,.,,.,.,.,.,.,,.,,.,,.,.,.,.,,,.,.uu,uuu,u,uuu.uu,uuuuu,uuu.,,,,.,.,.,.,.,.,.,.,.,.,.,.,.,.,,,,.,.,.,,.,,.,.,.,.,.,,.,.,.,,.,.,.,.,,.,,,,.,,.,,,.,,.,.,.,.,.,.,. [ 39%]
,,.,,.,,.,.,.,.,,,.,.uu,uuu,u,uuu.uu,uuuuu,uuu.,,,,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,,,,.,.,.,,.,,.,.,.,.,.,.,.,,.,.,.,,.,.,.,.,.,,.,,,,.,,.,,,.,,.,.,.,.,.,.,.,,.,,.,,.,.,.,.,,,.,.uu,uuu,u,uuu. [ 47%]
uu,uuuuu,uuu.,,,,.,.,.,.,.,.,.,.,.,.,.,.,.,,,,.,.,.,,.,,.,.,.u.,.,.,.,.,,.,.,.,,.,.,.,.,.,.,.,,.,,,,,.,,.,,,.,,.,.,.,.,.,.,.,,.,,.,.,.,.,,,.,.uu,uuu,u,uuu.uu,uuuuu,uuu.,,,,.,.,.,.,. [ 55%]
,.,.,.,.,.,.,.,.,.,,,,.,.,,.,,.,.,.u.,.,.,.,.,.,.,,.,.,,.,.,.,.,.,.,,.,,,,,.,,.,,,.,,.,.,.,.,.,.,.,.,.,.,,,.,.uu,uuu,u,uuu.uu,uuuuu,uuu.,,,,.,.,.                                     [ 61%]
referencing/tests/test_retrieval.py ....                                                                                                                                              [ 62%]
suite/test_sanity.py ................................................................................................................................................................ [ 82%]
..............................................................................................................................................                                        [100%]

========================================================================================= FAILURES ==========================================================================================

[..]

------------------------------------------------------------------------------------- Captured log call -------------------------------------------------------------------------------------

================================================================================== short test summary info ==================================================================================
(test={'ref': 'http://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-scheme
(test={'ref': 'http://example.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%C2%B1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%C2%B1b
(test={'ref': 'http://example.com/unreserved/~foo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/unreserved/~foo
(test={'ref': 'http://example.com/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/default/port
(test={'ref': 'hTtP://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%c2%b1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%c2%b1b
(test={'ref': 'http://example.com/unreserved/%7Efoo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-03-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/default/port
(test={'ref': 'http://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-scheme
(test={'ref': 'http://example.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%C2%B1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%C2%B1b
(test={'ref': 'http://example.com/unreserved/~foo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/unreserved/~foo
(test={'ref': 'http://example.com/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/default/port
(test={'ref': 'http://example.com/id/case-insensitive-host', 'target': {'id': 'http://exAmpLe.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%C2%B1b', 'target': {'id': 'http://example.com/id/escapes/a%c2%b1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%C2%B1b
(test={'ref': 'http://example.com/id/unreserved/~foo', 'target': {'id': 'http://example.com/id/unreserved/%7Efoo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/~foo
(test={'ref': 'http://example.com/id/default/port', 'target': {'id': 'http://example.com:80/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/default/port
(test={'ref': 'hTtP://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%c2%b1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%c2%b1b
(test={'ref': 'http://example.com/unreserved/%7Efoo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/default/port
(test={'ref': 'hTtP://example.com/id/case-insensitive-scheme', 'target': {'id': 'http://example.com/id/case-insensitive-scheme', 'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/id/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/id/case-insensitive-host', 'target': {'id': 'http://example.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%c2%b1b', 'target': {'id': 'http://example.com/id/escapes/a%C2%B1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%c2%b1b
(test={'ref': 'http://example.com/id/unreserved/%7Efoo', 'target': {'id': 'http://example.com/id/unreserved/~foo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/id/default/port', 'target': {'id': 'http://example.com/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-04-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/id/default/port
(test={'ref': 'http://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-scheme
(test={'ref': 'http://example.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%C2%B1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%C2%B1b
(test={'ref': 'http://example.com/unreserved/~foo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/unreserved/~foo
(test={'ref': 'http://example.com/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/default/port
(test={'ref': 'http://example.com/id/case-insensitive-host', 'target': {'$id': 'http://exAmpLe.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%C2%B1b', 'target': {'$id': 'http://example.com/id/escapes/a%c2%b1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%C2%B1b
(test={'ref': 'http://example.com/id/unreserved/~foo', 'target': {'$id': 'http://example.com/id/unreserved/%7Efoo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/~foo
(test={'ref': 'http://example.com/id/default/port', 'target': {'$id': 'http://example.com:80/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/default/port
(test={'ref': 'hTtP://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%c2%b1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%c2%b1b
(test={'ref': 'http://example.com/unreserved/%7Efoo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/default/port
(test={'ref': 'hTtP://example.com/id/case-insensitive-scheme', 'target': {'$id': 'http://example.com/id/case-insensitive-scheme', 'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/id/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/id/case-insensitive-host', 'target': {'$id': 'http://example.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%c2%b1b', 'target': {'$id': 'http://example.com/id/escapes/a%C2%B1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%c2%b1b
(test={'ref': 'http://example.com/id/unreserved/%7Efoo', 'target': {'$id': 'http://example.com/id/unreserved/~foo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/id/default/port', 'target': {'$id': 'http://example.com/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-06-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/id/default/port
(test={'ref': 'http://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-scheme
(test={'ref': 'http://example.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%C2%B1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%C2%B1b
(test={'ref': 'http://example.com/unreserved/~foo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/unreserved/~foo
(test={'ref': 'http://example.com/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/default/port
(test={'ref': 'http://example.com/id/case-insensitive-host', 'target': {'$id': 'http://exAmpLe.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%C2%B1b', 'target': {'$id': 'http://example.com/id/escapes/a%c2%b1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%C2%B1b
(test={'ref': 'http://example.com/id/unreserved/~foo', 'target': {'$id': 'http://example.com/id/unreserved/%7Efoo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/~foo
(test={'ref': 'http://example.com/id/default/port', 'target': {'$id': 'http://example.com:80/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/default/port
(test={'ref': 'hTtP://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%c2%b1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%c2%b1b
(test={'ref': 'http://example.com/unreserved/%7Efoo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/default/port
(test={'ref': 'hTtP://example.com/id/case-insensitive-scheme', 'target': {'$id': 'http://example.com/id/case-insensitive-scheme', 'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/id/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/id/case-insensitive-host', 'target': {'$id': 'http://example.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%c2%b1b', 'target': {'$id': 'http://example.com/id/escapes/a%C2%B1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%c2%b1b
(test={'ref': 'http://example.com/id/unreserved/%7Efoo', 'target': {'$id': 'http://example.com/id/unreserved/~foo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/id/default/port', 'target': {'$id': 'http://example.com/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-07-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/id/default/port
(test={'ref': 'http://example.com/oh-hey-a-subschema', 'target': {'$id': 'http://example.com/oh-hey-a-subschema', 'abc': 123}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-keywords-definitions] - referencing.exceptions.Unresolvable: http://example.com/oh-hey-a-subschema
(test={'ref': 'http://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-scheme
(test={'ref': 'http://example.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%C2%B1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%C2%B1b
(test={'ref': 'http://example.com/unreserved/~foo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/unreserved/~foo
(test={'ref': 'http://example.com/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/default/port
(test={'ref': 'http://example.com/id/case-insensitive-host', 'target': {'$id': 'http://exAmpLe.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%C2%B1b', 'target': {'$id': 'http://example.com/id/escapes/a%c2%b1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%C2%B1b
(test={'ref': 'http://example.com/id/unreserved/~foo', 'target': {'$id': 'http://example.com/id/unreserved/%7Efoo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/~foo
(test={'ref': 'http://example.com/id/default/port', 'target': {'$id': 'http://example.com:80/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/default/port
(test={'ref': 'hTtP://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%c2%b1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%c2%b1b
(test={'ref': 'http://example.com/unreserved/%7Efoo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/default/port
(test={'ref': 'hTtP://example.com/id/case-insensitive-scheme', 'target': {'$id': 'http://example.com/id/case-insensitive-scheme', 'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/id/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/id/case-insensitive-host', 'target': {'$id': 'http://example.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%c2%b1b', 'target': {'$id': 'http://example.com/id/escapes/a%C2%B1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%c2%b1b
(test={'ref': 'http://example.com/id/unreserved/%7Efoo', 'target': {'$id': 'http://example.com/id/unreserved/~foo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/id/default/port', 'target': {'$id': 'http://example.com/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2019-09-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/id/default/port
(test={'ref': 'http://example.com/oh-hey-a-subschema', 'target': {'$id': 'http://example.com/oh-hey-a-subschema', 'abc': 123}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-keywords-definitions] - referencing.exceptions.Unresolvable: http://example.com/oh-hey-a-subschema
(test={'ref': 'http://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-scheme
(test={'ref': 'http://example.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%C2%B1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%C2%B1b
(test={'ref': 'http://example.com/unreserved/~foo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/unreserved/~foo
(test={'ref': 'http://example.com/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/default/port
(test={'ref': 'http://example.com/id/case-insensitive-host', 'target': {'$id': 'http://exAmpLe.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%C2%B1b', 'target': {'$id': 'http://example.com/id/escapes/a%c2%b1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%C2%B1b
(test={'ref': 'http://example.com/id/unreserved/~foo', 'target': {'$id': 'http://example.com/id/unreserved/%7Efoo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/~foo
(test={'ref': 'http://example.com/id/default/port', 'target': {'$id': 'http://example.com:80/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-insertion] - referencing.exceptions.Unresolvable: http://example.com/id/default/port
(test={'ref': 'hTtP://example.com/case-insensitive-scheme', 'target': {'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/case-insensitive-host', 'target': {'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/case-insensitive-host
(test={'ref': 'http://example.com/escapes/a%c2%b1b', 'target': {'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/escapes/a%c2%b1b
(test={'ref': 'http://example.com/unreserved/%7Efoo', 'target': {'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/default/port', 'target': {'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/default/port
(test={'ref': 'hTtP://example.com/id/case-insensitive-scheme', 'target': {'$id': 'http://example.com/id/case-insensitive-scheme', 'foo': 'bar'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: hTtP://example.com/id/case-insensitive-scheme
(test={'ref': 'http://exAmpLe.com/id/case-insensitive-host', 'target': {'$id': 'http://example.com/id/case-insensitive-host', 'baz': 'quux'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://exAmpLe.com/id/case-insensitive-host
(test={'ref': 'http://example.com/id/escapes/a%c2%b1b', 'target': {'$id': 'http://example.com/id/escapes/a%C2%B1b', 'spam': 'eggs'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/escapes/a%c2%b1b
(test={'ref': 'http://example.com/id/unreserved/%7Efoo', 'target': {'$id': 'http://example.com/id/unreserved/~foo', 'snap': 'crackle'}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com/id/unreserved/%7Efoo
(test={'ref': 'http://example.com:80/id/default/port', 'target': {'$id': 'http://example.com/id/default/port', 'pop': 37}}) SUBFAIL referencing/tests/test_referencing_suite.py::test_referencing_suite[json-schema-draft-2020-12-rfc3986-normalization-on-retrieval] - referencing.exceptions.Unresolvable: http://example.com:80/id/default/port
=================================================================== 107 failed, 802 passed, 431 subtests passed in 3.16s ====================================================================

Full pytest output in attachment python-referencing.FAIL.txt

Please let me know if you need more details or want me to perform some diagnostics.

Lookup of resources from a Registry that retrieve'd them (e.g. from the network) should crawl the retrieved resource

It may contain further resources, or else we need to be able to do lookup(...).resolver.lookup("#") at very least.

0.32.0: pytest fails

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

python3 -sBm build -w --no-isolation
because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
install .whl file in </install/prefix> using installer module
run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

List of installed modules in build env:

Package                   Version
------------------------- -----------
attrs                     23.2.0
build                     1.1.1
distro                    1.9.0
dnf                       4.19.0
editables                 0.5
exceptiongroup            1.1.3
gpg                       1.23.2
hatch-vcs                 0.4.0
hatchling                 1.21.1
idna                      3.6
importlib_metadata        7.0.1
iniconfig                 2.0.0
installer                 0.7.0
jsonschema                4.20.0
jsonschema-specifications 2023.12.1
libdnf                    0.73.0
multidict                 6.0.5
packaging                 24.0
pathspec                  0.12.1
pluggy                    1.4.0
pyproject_hooks           1.0.0
pytest                    8.1.1
pytest-subtests           0.11.0
python-dateutil           2.9.0.post0
referencing               0.32.0
rpds-py                   0.18.0
setuptools                69.1.1
setuptools-scm            8.0.4
tokenize_rt               5.2.0
tomli                     2.0.1
trove-classifiers         2024.3.13
typing_extensions         4.10.0
wheel                     0.43.0
yarl                      1.9.4
zipp                      3.17.0

Potential issue with attrs==22.1.0

(report originating here)

It seems that in some cases, using attrs==22.1.0 with referencing raises this on import:

>> import referencing
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python3.8/site-packages/referencing/__init__.py", line 4, in <module>
    from referencing._core import Anchor, Registry, Resource, Specification
  File ".../lib/python3.8/site-packages/referencing/_core.py", line 29, in <module>
    class Specification(Generic[D]):
  File ".../lib/python3.8/site-packages/referencing/_core.py", line 55, in Specification
    ] = field(alias="anchors_in")
TypeError: field() got an unexpected keyword argument 'alias'

No such error is raised when using attrs==22.2.0.

Looking at the attrs source code for the 22.1.0 tag, the alias argument is not available in all overrides for the field() function. Specifically, the attrs change log for version 22.2.0 states that "attrs.field() now supports an alias option for explicit __init__ argument names." which basically implies that there's no full support of that in previous versions.

I would suggest raising the attrs dependency to >=22.2.0

Complain when adding conflicting identified resources

Either by default or when using a stricter wrapper object, we should have a way to complain if a resource is being added in a conflicting way -- i.e. if two different resources have the same URI.

So:

registry.with_resource("http://example.com", Resource.opaque(12))
registry.with_resource("http://example.com", Resource.opaque("foo"))

should raise some conflicting resource error.

The same should happen when calling registry.combine on multiple registries -- though here we should be careful because downstream (e.g. in jsonschema) a use of combining registries is to add "known" schemas to some additional ones, and sometimes the combining is to allow users to override schemas for whatever purpose. Perhaps we could explicitly force such registries to first drop the known resources before combining.

Give Registries a way to serialize to/from JSON

This is probably a useful way to quickly save/load registries, usable by projects like jsonschema-specifications.

A minor thing to decide is whether to automatically call .crawl when serializing or not.

Add changelog to documentation

Resolving a `$ref` to a nested definition can fail to catch that a (bad) subschema is neither bool nor object, resulting in an `AttributeError`

This originates downstream in python-jsonschema/check-jsonschema#376 . I was just able to look at the offending schema and trace things out enough to find the true cause. I've stuck with using the CLI for my reproducers, but I could work up some python for this if it would be useful.

check-jsonschema is using jsonschema+referencing. It keeps the jsonschema behavior of checking a schema against its metaschema. Therefore, the following schema is caught as invalid:

invalid-caught-by-metaschema

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type": "object",
  "definitions": {
    "foo": "invalid"
  },
  "properties": {
    "foo": {
      "$ref": "#/definitions/foo"
    }
  }
}

(Note how the definition for "foo" is a string, rather than a schema.)

However, if the invalid definition is nested, as follows, the metaschema check is not sufficient:

invalid-not-caught-by-metaschema

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type": "object",
  "definitions": {
    "sub": {
      "foo": "invalid"
    }
  },
  "properties": {
    "foo": {
      "$ref": "#/definitions/sub/foo"
    }
  }
}

As a result, it is possible, with jsonschema+referencing, to be handling this in a validator. The next step is to try to use it.
With the check-jsonschema CLI, we get the following trace (trimmed to relevant parts):

full-cli-traceback

$ check-jsonschema --schemafile badrefschema.json <(echo '{"foo": "bar"}')  
Traceback (most recent call last):
  
  <<<<<check-jsonschema invocation path shows here>>>>>

  File "/home/sirosen/projects/jsonschema/check-jsonschema/src/check_jsonschema/checker.py", line 73, in _build_result
    for err in validator.iter_errors(data):
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/jsonschema/validators.py", line 368, in iter_errors
    for error in errors:
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/jsonschema/_keywords.py", line 295, in properties
    yield from validator.descend(
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/jsonschema/validators.py", line 416, in descend
    for error in errors:
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/jsonschema/_keywords.py", line 274, in ref
    yield from validator._validate_reference(ref=ref, instance=instance)
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/jsonschema/validators.py", line 410, in descend
    for k, v in applicable_validators(schema):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/jsonschema/_legacy_keywords.py", line 17, in ignore_ref_siblings
    ref = schema.get("$ref")
          ^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'

This shows an error in jsonschema (not quite at referencing yet!). I had to tinker around with things to find the right trigger conditions, but this seems to do the trick, as a nasty schema:

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type": "object",
  "definitions": {
    "sub": {
      "foo": {
        "type": "object",
        "properties": {
          "bar": "invalid"
        }
      }
    }
  },
  "properties": {
    "foo": {
      "$ref": "#/definitions/sub/foo"
    }
  }
}

Now, trying to use it triggers an attempt to call .get() on a string:

$ check-jsonschema --schemafile badrefschema.json <(echo '{"foo": {"bar": "baz"}}')

(trimmed trace)

  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/jsonschema/validators.py", line 408, in descend
    resolver = self._resolver.in_subresource(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/referencing/_core.py", line 689, in in_subresource
    id = subresource.id()
         ^^^^^^^^^^^^^^^^
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/referencing/_core.py", line 225, in id
    id = self._specification.id_of(self.contents)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sirosen/projects/jsonschema/check-jsonschema/.venv/lib/python3.11/site-packages/referencing/jsonschema.py", line 50, in _legacy_dollar_id
    id = contents.get("$id")
         ^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'

Proposed Resolution

Obviously, the best resolution is for such a schema to never get written, since it's invalid. 😁
And I'll be trying to advise the upstream provider of the schema against nesting things under definitions, since it weakens the kind of checking which is possible.

But should various referencing internals be more wary that an input may be a dict, a bool, or something unwanted/unexpected? Or should referencing check for bad inputs more aggressively when loading data?
The goal here is to have a better and clearer error in this case, not to ignore the malformed schema.

I think the best thing here is that one of the levels of document loading in Referencing checks if the value is dict | boolean. If we only consider JSON Schema, it could be a validator on Resource.contents. That makes the most sense to me, conceptually, for JSON Schema, but it ties in non-generic details of that spec (vs more general $ref resolution). Perhaps JSONSchemaResource(Resource) is the right thing, which adds said validation?

(NB: I'm reading a lot of the internals pretty quickly and for the first time, so my ideas here may not hold up.)

Type annotations for `Registry` usage are not well-supported by `mypy`, slightly tricky with `pyright`

I'm finally back to taking some time to work on updating check-jsonschema to use jsonschema>=4.18 with referencing!

The only issue I've really had so far is that the type annotations are not as straightforward to satisfy as I would like.
The attrs-derived initializers are not understood by mypy as well as they are for pyright, and pyright is being a bit "fussy" about type invariance.
There's a cluster of related problems to work out, but I think my main item is the failure under mypy. This seems like a deficiency of mypy, since pyright works, but it may be a deficiency which is easy enough to be worth accommodating; I don't know yet.

Here's a simplified version of my registry builder, resolver_simple.py:

from __future__ import annotations

import referencing
import requests
from referencing.jsonschema import DRAFT202012, Schema

def retrieve_reference(uri: str) -> referencing.Resource[Schema]:
    data = requests.get(uri, stream=True).json()
    return referencing.Resource.from_contents(data, default_specification=DRAFT202012)

def make_reference_registry(
    schema_uri: str | None, schema: dict
) -> referencing.Registry:
    schema_resource = referencing.Resource.from_contents(
        schema, default_specification=DRAFT202012
    )
    registry = referencing.Registry(retrieve=retrieve_reference)

    # aside: I'm not 100% sure these next parts are correct
    # schemas with no explicit `$id` are inherently ambiguous
    # but it would be nice to support using the URI from whence we retrieved them
    if schema_uri is not None:
        registry = registry.with_resource(uri=schema_uri, resource=schema_resource)

    id_attribute = schema.get("$id")
    if id_attribute is not None:
        registry = registry.with_resource(uri=id_attribute, resource=schema_resource)

    return registry

mypy doesn't understand that retrieve is an attrs-provided field:

$ mypy resolver_simple.py                                   
resolver_simple.py:19: error: Unexpected keyword argument "retrieve" for "Registry"  [call-arg]
resolver_simple.py:19: error: Need type annotation for "registry"  [var-annotated]
Found 2 errors in 1 file (checked 1 source file)

By contrast, pyright likes this file and gives it a pass. Which makes my next example a little bit of a headscratcher at first.

The real implementation makes parsing a bit more pluggable, for which I have a simple object, ParserSet. As a result, retrieve is built dynamically, and the resulting type somehow mismatches under pyright. It looks something like this:

def create_retrieve_callable(
    parsers: ParserSet,
) -> typing.Callable[[str], referencing.Resource[Schema]]:
    def retrieve_reference(uri: str) -> referencing.Resource[Schema]:
        data = requests.get(uri, stream=True)
        parsed_object = parsers.parse(data.raw, uri)

        return referencing.Resource.from_contents(
            parsed_object, default_specification=DRAFT202012
        )

    return retrieve_reference

which results in a pyright error like

error: Argument of type "(str) -> Resource[Schema]" cannot be assigned to parameter "retrieve" of type "Retrieve[D@Registry]" in function "__init__"
    Type "(str) -> Resource[Schema]" cannot be assigned to type "(uri: URI) -> Resource[Schema]"

After staring at the error output long enough, I realized that pyright is picky about this because that Callable type doesn't specifically have the signature def f(uri: str) -> referencing.Resource[Schema]: .... f here takes uri as a keyword-or-positional -- something which Callable is inadequate to express.

The fix is "easy" but weird! 😅

class _RetrieverFunc(typing.Protocol):
    def __call__(self, uri: str) -> referencing.Resource[Schema]:
        ...

def create_retrieve_callable(parsers: ParserSet) -> _RetrieverFunc:
    # body unchanged

Between the two, it feels difficult to setup what feel -- at first -- like relatively simple annotations to pass. As I said above, given that pyright can follow the _attrs module in referencing, I'm inclined to think that's a mypy bug or missing feature. And pyright is picky, but in a way that feels tolerable to use and correct.

The last issue I have is with pyright's type matching rules, so I might be wrong about how to solve this or whether or not it should be allowed, but I found that I had to import referencing.jsonschema.Schema to satisfy types. (You may have noticed this usage above.)
I tried setting up a recursive TypeAlias to describe JSON datatypes, and using that

JsonPrimitive: t.TypeAlias = t.Union[int, float, str, bool, None]
JsonElement: t.TypeAlias = t.Union[JsonPrimitive, "JsonList", "JsonDict"]
JsonList: t.TypeAlias = t.List[JsonElement]
JsonDict: t.TypeAlias = t.Dict[str, JsonElement]

JsonSchema: t.TypeAlias = JsonDict | bool

But pyright doesn't like Resource[JsonSchema] because it doesn't match against Resource[Schema].
I think it would work if I used the exact value in Schema of bool | Mapping[str, Any], but no other type will pass because Resource is invariant in its type variable.

So my final note (long issue report, sorry!) is that I'm probably going to be importing Schema on the assumption that it's safe to rely on that name and using it to ensure that my annotations always match referencing. If we can adjust the variance of Resource to be looser, that might make it easier to apply a narrower type.

FYI: Small breaking changes are coming to some (non-schema author related) APIs

This is an FYI to anyone who sees it that an upcoming release (possibly v0.35) will likely contain breaking changes to the retrieval API.

referencing of course is in pre-1.0 / stable status (for slightly more details there see #38). I still of course don't wish to unnecessarily cause churn for people who've begun to adopt it, but it's clear there's at least one, possibly two, things in need of fixing.

Specifically, though I knew it would happen at some point, referencing needs to use a URL implementation in order to properly perform URL normalization -- in other words, as a simple example, knowing that HTTP://example.com is the same as http://example.com. Of course the full gamut is more complicated.

This means it needs to internally use a library (which will be url.py) which properly implements RFC3986 (though technically that one implements WHATWG URLs). Some functions then will need to take URLs rather than strs. Yes it is possible to sprinkle isinstance checks everywhere and convert always between str and URL but this will leave a small performance penalty everywhere, and also just generally seems unwise to leave in place before referencing continues to mature.

In particular:

Expect retrieve callbacks to likely receive URL objects rather than strs in an upcoming release.

I still expect this to affect only a small number of users ultimately, because the relevant APIs do not relate to Resource + Registry assembly, they relate to "internal" (but public) APIs like Resolver.lookup and Registry retrieval but I'm putting this here anyways.

Further details to come shortly, I'm still working on a set of changes to support this. If you wish to follow along, #74 contains the tests I have added (upstream to the test suite) which "tease out" the underlying issue the above references.

(And as a second example, which I again am unsure how easy I can fix but which I suspect should happen before #38 -- see #65. Ideally I'll find a way to do both of these changes at the same time.)

As a third example (and a slight reminder to myself), we also may not have precisely the right behavior vis a vis re-setting the base URL after retrieval, which should likely take precedence over the initial base URL for a Retriever (i.e. the object you get back should have a new base URI using the retrieval URL)

Type check errors

Type check errors thrown by mypy when importing referencing

To reproduce this issue, create a test.py as below:

import referencing

Execute command:

mypy test.py

This issue is applicable on versions 0.31.0 and above.

Evaluate adding Registry.crawl(until=URI)

This is a potential performance enhancement / laziness enhancement -- we could stop crawling a registry when a particular URI of desire is found -- perhaps relatedly we could store crawled URIs in a trie or something so we go directly to the most likely location to find the resource (e.g. look in http://example.com/foo first if we're looking for http://example.com/foo/bar).

crawl(until=None) (the default) would crawl the entire set of uncrawled resources, i.e. the current behavior.

Doing this should first come with some benchmarking.

	@frozen
	class Resolver(Generic[D]):
	"""
	A reference resolver.

	Resolvers help resolve references (including relative ones) by
	pairing a fixed base URI with a `Registry`.