Provide a helper to fully resolve $refs in a schema

[somiandras]

I want to generate some documentation/description (probably some input forms) from a json schema that contains $refs. Is there a way in jsonschema to retrieve a json schema with $refs resolved and inlined, so I can iterate through the schema?

So from a schema like this:

{
  "definitions": {
    "some_object": {
      "type": "object",
      "title": "My definition",
      "description": "Some description",
      "properties": {
        "name": {
          "type": "string"
        },
        "value": {
          "type": "number"
        }
      }
    }
  },
  "properties": {
    "list_of_objects": {
      "type": "array",
      "title": "Some stuff",
      "description": "List of stuff",
      "items": {"$ref": "#/definitions/some_object"}
    }
  }
}

I want to have this structure in my Python object (notice that the {$ref:...} section is replaced by the actual definition):

{
  "definitions": {
    "some_object": {
      "type": "object",
      "title": "My definition",
      "description": "Some description",
      "properties": {
        "name": {
          "type": "string"
        },
        "value": {
          "type": "number"
        }
      }
    }
  },
  "properties": {
    "list_of_objects": {
      "type": "array",
      "title": "Some stuff",
      "description": "List of stuff",
      "items": {
        "type": "object",
        "title": "My definition",
        "description": "Some description",
        "properties": {
          "name": {
            "type": "string"
          },
          "value": {
            "type": "number"
          }
        }
      }
    }
  }
}

I am aware that with self-referencing schemas this will lead to an infinite structure, but hopefully this is feasible with reasonable limitations (eg. excluding self-refs).

[calvin]

jsonref https://pypi.org/project/jsonref/ resolves references so that you can have fully expanded schemas.

[jpmckinney]

I think @Julian previously decided that this belongs in a separate library: python-jsonschema/jsonschema#419

I would like a helper function in jsonschema, though, as https://github.com/gazpachoking/jsonref seems to not be actively maintained (all its tests are failing on CI, so it's not clear that it would even be easy to take maintenance of).

[Julian]

@jpmckinney that was my stance early on, but given nothing popped up, and folks still seem to want this, I'm willing to entertain a helper function (essentially for the reason you mentioned).

So yeah a PR is welcome to add this.

[jpmckinney]

Would python-jsonschema/jsonschema#419 be a starting point? e.g. if you re-open it, it'd be possible to see how bad the conflicts are (if any).

[Julian]

Probably it's a good one yeah -- though I can't reopen it because GH prevents reopening PRs that targeted the old master branch now that it targets main :( -- but if you or someone wants to take what's there and (while preserving authorship of the commits) push it as a new PR it certainly may be a good basis to start from.

[jpmckinney]

Is it possible to edit the PR to point it to a new base branch?

I think I commented here, when I ran into trouble with jsonref and PyPy open-contracting/ocdskit#178, and thought to check whether jsonschema could resolve $ref.

At any rate, the library I was working on has other issues with PyPy. So, I'm not sure I still have a need at the moment.

[Julian]

Is it possible to edit the PR to point it to a new base branch?

Not as far as I know :(

At any rate, the library I was working on has other issues with PyPy. So, I'm not sure I still have a need at the moment.

Fair enough.

[lucasrodes]

I think this would be very valuable!

To load a schema in path that contains references, we are using jsonref like this:

from pathlib import Path
from urllib.parse import urljoin

path = Path(path)

# pathlib does not append trailing slashes, but jsonref needs that.
base_dir_url = path.parent.absolute().as_uri() + "/"
base_file_url = urljoin(base_dir_url, path.name)
with path.open("r") as f:
    return jsonref.loads(f.read(), base_uri=base_file_url, lazy_load=False)

It works fine, but it has some limitations: see gazpachoking/jsonref#60

[Julian]

The thing is, this is actually both impossible to do in general (when considering $dynamicRef) and also explicitly now discouraged by the JSON Schema specification. It's true this has been here for quite awhile and is a lot closer to being implementable now, but can you (or anyone else still interested) perhaps elaborate a bit on what use you have for this functionality?

[lucasrodes]

Hi @Julian,
Thanks for the rapid response.

In our case, we are using this functionality for the same use case as the OP. We are centralising the documentation about some fields in a schema. This helps maintenance a lot, since we are surfacing field descriptions and other things in multiple places:

• On hover in VS Code
• On a simple web app we have to introduce some fields in a form-like style.
• And in the future, on our official documentation.

I am still working on it, and the code is quite complex, but I am loading the schema in this script, and later using it to render the help, title and other aspects of the form's input fields.

We happen to use #refs in our JSON schema files, to also simplify maintenance of some field definitions (they might be referenced in multiple places).