CLI#

Examples#

Running a Single Test Suite File#

To run the draft 7 type-keyword tests on the Lua jsonschema implementation, run:

$ bowtie suite -i lua-jsonschema https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft7/type.json | bowtie report

Running the Official Suite Across All Implementations#

The following will run all Draft 7 tests from the official test suite (which it will automatically retrieve) across all implementations supporting Draft 7, and generate an HTML report named bowtie-report.html in the current directory:

$ bowtie suite $(find /path/to/bowtie/checkout/implementations/ -mindepth 1 -maxdepth 1 -type d | sed 's/.*\/implementations\//-i /') https://github.com/json-schema-org/JSON-Schema-Test-Suite/tree/main/tests/draft7 | bowtie report

Running Test Suite Tests From Local Checkouts#

Providing a local path to the test suite can be used as well, which is useful if you have local changes:

$ bowtie suite $(find /path/to/bowtie/checkout/implementations/ -mindepth 1 -maxdepth 1 -type d | sed 's/.*\/implementations\//-i /') ~/path/to/json-schema-org/suite/tests/draft2020-12/ | bowtie report

Checking An Implementation Functions On Basic Input#

If you wish to verify that a particular implementation works on your machine (e.g. if you suspect a problem with the container image, or otherwise aren’t seeing results), you can run bowtie smoke. E.g., to verify the Golang jsonschema implementation is functioning, you can run:

$ bowtie smoke -i go-jsonschema

Reference#

bowtie#

A meta-validator for the JSON Schema specifications.

bowtie [OPTIONS] COMMAND [ARGS]...

Options

--version#: Show the version and exit.

info#

Retrieve a particular implementation (harness)’s metadata.

bowtie info [OPTIONS]

Options

-i, --implementation <image_names>#: A docker image which implements the bowtie IO protocol.

report#

Generate a Bowtie report from a previous run.

bowtie report [OPTIONS] [INPUT]

Options

-o, --out <output>#: Where to write the outputted report HTML.

Arguments

INPUT#: Optional argument

run#

Run a sequence of cases provided on standard input.

bowtie run [OPTIONS] [INPUT]

Options

-i, --implementation <image_names>#: A docker image which implements the bowtie IO protocol.

-D, --dialect <dialect>#: A URI or shortname identifying the dialect of each test case.Shortnames include: [‘2019’, ‘2019-09’, ‘201909’, ‘2020’, ‘2020-12’, ‘202012’, ‘3’, ‘4’, ‘6’, ‘7’, ‘draft2019-09’, ‘draft201909’, ‘draft2020-12’, ‘draft202012’, ‘draft3’, ‘draft4’, ‘draft6’, ‘draft7’].

-k <filter>#: Only run cases whose description match the given glob pattern.

-x, --fail-fast#: Fail immediately after the first error or disagreement.

-S, --set-schema, --no-set-schema#: Explicitly set $schema in all (non-boolean) case schemas sent to implementations. Note this of course means what is passed to implementations will differ from what is provided in the input.

-V, --validate-implementations#: When speaking to implementations (provided via -i), validate the requests and responses sent to them under Bowtie’s JSON Schema specification. Generally, this option protects against broken Bowtie implementations and can be left at its default (of off) unless you are developing a new implementation container.

Arguments

INPUT#: Optional argument

smoke#

Smoke test one or more implementations for basic correctness.

bowtie smoke [OPTIONS]

Options

-i, --implementation <image_names>#: A docker image which implements the bowtie IO protocol.

suite#

Run test cases from the official JSON Schema test suite.

Supports file or URL inputs like:

{ROOT}/tests/draft7 to run a version’s tests

{ROOT}/tests/draft7/foo.json to run just one file

https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft7/ to run a version directly from a branch which exists in GitHub

https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft7/foo.json to run a single file directly from a branch which exists in GitHub

bowtie suite [OPTIONS] INPUT

Options

-i, --implementation <image_names>#: A docker image which implements the bowtie IO protocol.

-k <filter>#: Only run cases whose description match the given glob pattern.

-x, --fail-fast#: Fail immediately after the first error or disagreement.

-S, --set-schema, --no-set-schema#: Explicitly set $schema in all (non-boolean) case schemas sent to implementations. Note this of course means what is passed to implementations will differ from what is provided in the input.

-V, --validate-implementations#: When speaking to implementations (provided via -i), validate the requests and responses sent to them under Bowtie’s JSON Schema specification. Generally, this option protects against broken Bowtie implementations and can be left at its default (of off) unless you are developing a new implementation container.

Arguments

INPUT#: Required argument

summary#

Generate an (in-terminal) summary of a Bowtie run.

bowtie summary [OPTIONS] [INPUT]

Options

-f, --format <format>#

What format to use for the output

Options:: json | pretty

Arguments

INPUT#: Optional argument

validate#

Validate a schema & one or more instances across implementations.

bowtie validate [OPTIONS] SCHEMA [INSTANCES]...

Options

-i, --implementation <image_names>#: A docker image which implements the bowtie IO protocol.

-D, --dialect <dialect>#: A URI or shortname identifying the dialect of each test case.Shortnames include: [‘2019’, ‘2019-09’, ‘201909’, ‘2020’, ‘2020-12’, ‘202012’, ‘3’, ‘4’, ‘6’, ‘7’, ‘draft2019-09’, ‘draft201909’, ‘draft2020-12’, ‘draft202012’, ‘draft3’, ‘draft4’, ‘draft6’, ‘draft7’].

-S, --set-schema, --no-set-schema#: Explicitly set $schema in all (non-boolean) case schemas sent to implementations. Note this of course means what is passed to implementations will differ from what is provided in the input.

-V, --validate-implementations#: When speaking to implementations (provided via -i), validate the requests and responses sent to them under Bowtie’s JSON Schema specification. Generally, this option protects against broken Bowtie implementations and can be left at its default (of off) unless you are developing a new implementation container.

Arguments

SCHEMA#: Required argument

INSTANCES#: Optional argument(s)