JSON-XT

JSON-XT compresses JSON documents into short URIs, using eXternal Templates. Depending on the application, these URIs are 25 - 35% of the original JSON document size.

This Open Source project is sponsored by Consensas

See jsonxt.io for more, as well as this paper for a lot more explanation.

Introduction

Basic Idea

A more formal description of the algorithm will come with traction and or time!

Code

Source on GitHub, package on npm. Note that the Javascript module has no external dependecies!

Use

Installation

npm install jsonxt

Setup

const jsonxt = require("jsonxt")

// sample data…
const original = {
  "@context": {
    "ical": "http://www.w3.org/2002/12/cal/ical#",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "ical:dtstart": {
      "@type": "xsd:dateTime"
    }
  },
  "ical:summary": "Lady Gaga Concert",
  "ical:location": "New Orleans Arena, New Orleans, Louisiana, USA",
  "ical:dtstart": "2011-04-09T20:00:00Z"
}
const templates = {
  "simple:1": {
    "columns": [
      {
        "path": "ical:summary",
        "type": "string"
      },
      {
        "path": "ical:location",
        "type": "string"
      },
      {
        "path": "ical:dtstart",
        "type": "isodatetime-epoch-base32"
      }
    ],
    "template": {
      "@context": {
        "ical": "http://www.w3.org/2002/12/cal/ical#",
        "ical:dtstart": {
          "@type": "xsd:dateTime"
        },
        "xsd": "http://www.w3.org/2001/XMLSchema#"
      }
    }
  }
}
const TYPE = "simple"
const VERSION = "1"

The templates can have multiple compression formats encoded, they will be selected by TYPE and VERSION.

Encoding

This will compress the original JSON payload into a URI

const packed = await jsonxt.pack(original, templates, TYPE, VERSION, "jsonxt.io")

Which yields:

jxt:jsonxt.io:simple:1:Lady~Gaga~Concert/New~Orleans~Arena%2C~New~Orleans%2C~Louisiana%2C~USA/16Q1EM0

This is a compression of 279 bytes to 101 bytes: 36% of the original size.

See test code for a more fully worked through example.

Decoding

This will decompress the packed URI into the original URI.

const unpacked = await jsonxt.unpack(packed, jsonxt.resolve)

See test code for a more fully worked through example.

Resolving

Resolving takes a "resolver name" like example.com embedded in the URI and a file name and retrieves a document.

The following rules are used:

Generally you can just use jsonxt.resolve.

Examples

Note: that if we (as a community) can come up with a proof JWS that is small - for example, the Base64 MD5 of an existing proof, we can get all these credentials to sub-160 bytes!

CCCC4 Vaccination Credential

See here for more details.

Mattr/W3C Vaccination Credential

See here for more details. My understand is the encoding they are using now (CBOR-LD, Base64) yields a size of about 700 bytes.