Skip to content

Adds support for @included in compaction, expansion and flattening. #349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jan 29, 2020
Merged

Adds support for @included in compaction, expansion and flattening. #349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
326cf30
Adds support for `@included` in compaction, expansion and flattening.
gkellogg Dec 31, 2019
43d6680
Skip things that look like keywords.
gkellogg Jan 3, 2020
c5ba474
In Node Map Generation, skip objects where `@id` is `null`.
gkellogg Jan 3, 2020
59159bf
Allow `@type` to be defined with implicit `@container: @set`.
gkellogg Jan 3, 2020
063ab32
Keywords may not be used as prefixes
gkellogg Jan 3, 2020
75bf623
Update CHANGELOG.
gkellogg Jan 3, 2020
bdba8d1
Move changes above 2,0.1
gkellogg Jan 6, 2020
7d48f60
Use a constant for KEYWORD_PATTERN.
gkellogg Jan 6, 2020
0c6501b
Fix changelog.
davidlehn Jan 26, 2020
7c566e5
If ignoring a keyword pattern, restore any previous mapping before re…
gkellogg Jan 26, 2020
462e1be
Enable tests.
davidlehn Jan 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
- Terms of the form of an IRI must map to the same IRI.
- Terms of the form of a relative IRI may not be used as prefixes.
- Match spec error code "invalid context entry" vs "invalid context member".
- Keywords may not be used as prefixes.

### Changed
- Keep term definitions mapping to null so they may be protected.
Expand All @@ -22,6 +23,8 @@

### Added
- Support for `"@import"`.
- Added support for `@included` blocks
- Skip things that have the form of a keyword, with warning.

## 2.0.2 - 2020-01-17

Expand Down
4 changes: 3 additions & 1 deletion lib/compact.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,8 +343,10 @@ api.compact = async ({
continue;
}

// skip array processing for keywords that aren't @graph or @list
// skip array processing for keywords that aren't
// @graph, @list, or @included
if(expandedProperty !== '@graph' && expandedProperty !== '@list' &&
expandedProperty !== '@included' &&
_isKeyword(expandedProperty)) {
// use keyword alias and add value as is
const alias = api.compactIri({
Expand Down
43 changes: 42 additions & 1 deletion lib/context.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ const {

const INITIAL_CONTEXT_CACHE = new Map();
const INITIAL_CONTEXT_CACHE_MAX_SIZE = 10000;
const KEYWORD_PATTERN = /^@[a-zA-Z]+$/;

const api = {};
module.exports = api;
Expand Down Expand Up @@ -382,7 +383,7 @@ api.createTermDefinition = ({

if(term === '@type' &&
_isObject(value) &&
value['@container'] === '@set' &&
(value['@container'] || '@set') === '@set' &&
api.processingMode(activeCtx, 1.1)) {

const validKeys = ['@container', '@id', '@protected'];
Expand All @@ -397,6 +398,11 @@ api.createTermDefinition = ({
'Invalid JSON-LD syntax; keywords cannot be overridden.',
'jsonld.SyntaxError',
{code: 'keyword redefinition', context: localCtx, term});
} else if(term.match(KEYWORD_PATTERN)) {
// FIXME: remove logging and use a handler
console.warn('WARNING: terms beginning with "@" are reserved' +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using console.warn here is the same issue that resulted in the protectedMode warn/error flag. That was supposed to eventually change a better handler, but that hasn't been designed yet. Might want another similar flag and we can fix them all at once. The machinery to pass around those options still exists so would be easy to add. I'm not sure what the best behavior even is.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave this for you to come up with a comprehensive solution. There are other areas where the spec says to generate warnings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Printing out warnings to a console can be invisible for backend code and UIs, hence the idea to have some of handler. I haven't put time into a design for that. A more strict mode that throws errors seems useful to me as well.

' for future use and ignored', {term});
return;
} else if(term === '') {
throw new JsonLdError(
'Invalid JSON-LD syntax; a term cannot be an empty string.',
Expand Down Expand Up @@ -484,6 +490,19 @@ api.createTermDefinition = ({
'absolute IRI or a blank node identifier.',
'jsonld.SyntaxError', {code: 'invalid IRI mapping', context: localCtx});
}

if(reverse.match(KEYWORD_PATTERN)) {
// FIXME: remove logging and use a handler
console.warn('WARNING: values beginning with "@" are reserved' +
' for future use and ignored', {reverse});
if(previousMapping) {
activeCtx.mappings.set(term, previousMapping);
} else {
activeCtx.mappings.delete(term);
}
return;
}

mapping['@id'] = id;
mapping.reverse = true;
} else if('@id' in value) {
Expand All @@ -497,6 +516,16 @@ api.createTermDefinition = ({
if(id === null) {
// reserve a null term, which may be protected
mapping['@id'] = null;
} else if(!api.isKeyword(id) && id.match(KEYWORD_PATTERN)) {
// FIXME: remove logging and use a handler
console.warn('WARNING: values beginning with "@" are reserved' +
' for future use and ignored', {id});
if(previousMapping) {
activeCtx.mappings.set(term, previousMapping);
} else {
activeCtx.mappings.delete(term);
}
return;
} else if(id !== term) {
// expand and add @id mapping
id = _expandIri(
Expand Down Expand Up @@ -750,6 +779,12 @@ api.createTermDefinition = ({
'jsonld.SyntaxError',
{code: 'invalid term definition', context: localCtx});
}
if(api.isKeyword(mapping['@id'])) {
throw new JsonLdError(
'Invalid JSON-LD syntax; keywords may not be used as prefixes',
'jsonld.SyntaxError',
{code: 'invalid term definition', context: localCtx});
}
if(typeof value['@prefix'] === 'boolean') {
mapping._prefix = value['@prefix'] === true;
} else {
Expand Down Expand Up @@ -850,6 +885,11 @@ function _expandIri(activeCtx, value, relativeTo, localCtx, defined, options) {
return value;
}

// ignore non-keyword things that look like a keyword
if(value.match(KEYWORD_PATTERN)) {
return null;
}

// define term dependency if not defined
if(localCtx && localCtx.hasOwnProperty(value) &&
defined.get(value) !== true) {
Expand Down Expand Up @@ -1230,6 +1270,7 @@ api.isKeyword = v => {
case '@explicit':
case '@graph':
case '@id':
case '@included':
case '@index':
case '@json':
case '@language':
Expand Down
39 changes: 34 additions & 5 deletions lib/expand.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ const {
const {
isList: _isList,
isValue: _isValue,
isGraph: _isGraph
isGraph: _isGraph,
isSubject: _isSubject
} = require('./graphTypes');

const {
Expand Down Expand Up @@ -251,8 +252,8 @@ api.expand = async ({
expandedParent: rval,
options,
insideList,
typeScopedContext,
typeKey,
typeScopedContext,
expansionMap});

// get property count on expanded output
Expand Down Expand Up @@ -392,6 +393,7 @@ api.expand = async ({
* @param expandedParent the expanded result into which to add values.
* @param options the expansion options.
* @param insideList true if the element is a list, false if not.
* @param typeKey first key found expanding to @type.
* @param typeScopedContext the context before reverting.
* @param expansionMap(info) a function that can be used to custom map
* unmappable values (or to throw an error when they are detected);
Expand All @@ -406,8 +408,8 @@ async function _expandObject({
expandedParent,
options = {},
insideList,
typeScopedContext,
typeKey,
typeScopedContext,
expansionMap
}) {
const keys = Object.keys(element).sort();
Expand Down Expand Up @@ -459,8 +461,9 @@ async function _expandObject({
{code: 'invalid reverse property map', value});
}
if(expandedProperty in expandedParent &&
expandedProperty !== '@included' &&
expandedProperty !== '@type') {
expandedProperty !== '@included' &&
expandedProperty !== '@type')
{
throw new JsonLdError(
'Invalid JSON-LD syntax; colliding keywords detected.',
'jsonld.SyntaxError',
Expand Down Expand Up @@ -519,6 +522,31 @@ async function _expandObject({
continue;
}

// Included blocks are treated as an array of separate object nodes sharing
// the same referencing active_property.
// For 1.0, it is skipped as are other unknown keywords
if(expandedProperty === '@included' && _processingMode(activeCtx, 1.1)) {
const includedResult = _asArray(await api.expand({
activeCtx,
activeProperty,
element: value,
options,
expansionMap
}));

// Expanded values must be node objects
if(!includedResult.every(v => _isSubject(v))) {
throw new JsonLdError(
'Invalid JSON-LD syntax; ' +
'values of @included must expand to node objects.',
'jsonld.SyntaxError', {code: 'invalid @included value', value});
}

_addValue(
expandedParent, '@included', includedResult, {propertyIsArray: true});
continue;
}

// @graph must be an array or an object
if(expandedProperty === '@graph' &&
!(_isObject(value) || _isArray(value))) {
Expand Down Expand Up @@ -825,6 +853,7 @@ async function _expandObject({
insideList,
typeScopedContext,
typeKey,
typeScopedContext,
expansionMap});
}
}
Expand Down
7 changes: 7 additions & 0 deletions lib/frame.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,13 @@ api.frame = (state, subjects, frame, parent, property = null) => {
}
}

// if frame has @included, recurse over its sub-frame
if('@included' in frame) {
api.frame(
state,
subjects, frame['@included'], output, '@included');
}

// iterate over subject properties
for(const prop of Object.keys(subject).sort()) {
// copy keywords to output
Expand Down
11 changes: 11 additions & 0 deletions lib/nodeMap.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,12 @@ api.createNodeMap = (input, graphs, graph, issuer, name, list) => {
continue;
}

// recurse into included
if(property === '@included') {
api.createNodeMap(input[property], graphs, graph, issuer);
continue;
}

// copy non-@type keywords
if(property !== '@type' && isKeyword(property)) {
if(property === '@index' && property in subject &&
Expand Down Expand Up @@ -180,6 +186,11 @@ api.createNodeMap = (input, graphs, graph, issuer, name, list) => {

// handle embedded subject or subject reference
if(graphTypes.isSubject(o) || graphTypes.isSubjectReference(o)) {
// skip null @id
if('@id' in o && !o['@id']) {
continue;
}

// relabel blank node @id
const id = graphTypes.isBlankNode(o) ?
issuer.getId(o['@id']) : o['@id'];
Expand Down
62 changes: 0 additions & 62 deletions tests/test-common.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,6 @@ const TEST_TYPES = {
specVersion: ['json-ld-1.0'],
// FIXME
idRegex: [
// included
/compact-manifest.jsonld#tin01$/,
/compact-manifest.jsonld#tin02$/,
/compact-manifest.jsonld#tin03$/,
/compact-manifest.jsonld#tin04$/,
/compact-manifest.jsonld#tin05$/,
// direction
/compact-manifest.jsonld#tdi01$/,
/compact-manifest.jsonld#tdi02$/,
Expand Down Expand Up @@ -69,10 +63,6 @@ const TEST_TYPES = {
specVersion: ['json-ld-1.0'],
// FIXME
idRegex: [
// terms having form of keyword
/expand-manifest.jsonld#t0119$/,
/expand-manifest.jsonld#t0120$/,
/expand-manifest.jsonld#t0122$/,
// html
/html-manifest.jsonld#te001$/,
/html-manifest.jsonld#te002$/,
Expand Down Expand Up @@ -103,28 +93,6 @@ const TEST_TYPES = {
/expand-manifest.jsonld#thc05$/,
// remote
/remote-doc-manifest.jsonld#t0013$/, // HTML
// colliding keywords
/expand-manifest.jsonld#t0114$/,
// included
/expand-manifest.jsonld#tin01$/,
/expand-manifest.jsonld#tin02$/,
/expand-manifest.jsonld#tin03$/,
/expand-manifest.jsonld#tin04$/,
/expand-manifest.jsonld#tin05$/,
/expand-manifest.jsonld#tin06$/,
/expand-manifest.jsonld#tin07$/,
/expand-manifest.jsonld#tin08$/,
/expand-manifest.jsonld#tin09$/,
// keywords
/expand-manifest.jsonld#tpr30$/,
/expand-manifest.jsonld#tpr31$/,
/expand-manifest.jsonld#tpr32$/,
/expand-manifest.jsonld#tpr33$/,
/expand-manifest.jsonld#tpr34$/,
/expand-manifest.jsonld#tpr35$/,
/expand-manifest.jsonld#tpr36$/,
/expand-manifest.jsonld#tpr37$/,
/expand-manifest.jsonld#tpr39$/,
// direction
/expand-manifest.jsonld#tdi01$/,
/expand-manifest.jsonld#tdi02$/,
Expand Down Expand Up @@ -159,13 +127,6 @@ const TEST_TYPES = {
/html-manifest.jsonld#tf002$/,
/html-manifest.jsonld#tf003$/,
/html-manifest.jsonld#tf004$/,
// included
/flatten-manifest.jsonld#tin01$/,
/flatten-manifest.jsonld#tin02$/,
/flatten-manifest.jsonld#tin03$/,
/flatten-manifest.jsonld#tin04$/,
/flatten-manifest.jsonld#tin05$/,
/flatten-manifest.jsonld#tin06$/,
]
},
fn: 'flatten',
Expand Down Expand Up @@ -298,10 +259,6 @@ const TEST_TYPES = {
specVersion: ['json-ld-1.0'],
// FIXME
idRegex: [
// terms having form of keyword
/toRdf-manifest.jsonld#te119$/,
/toRdf-manifest.jsonld#te120$/,
/toRdf-manifest.jsonld#te122$/,
// well formed
/toRdf-manifest.jsonld#twf05$/,
// html
Expand Down Expand Up @@ -329,25 +286,6 @@ const TEST_TYPES = {
/toRdf-manifest.jsonld#te075$/,
/toRdf-manifest.jsonld#te111$/,
/toRdf-manifest.jsonld#te112$/,
// colliding keyword
/toRdf-manifest.jsonld#te114$/,
// included
/toRdf-manifest.jsonld#tin01$/,
/toRdf-manifest.jsonld#tin02$/,
/toRdf-manifest.jsonld#tin03$/,
/toRdf-manifest.jsonld#tin04$/,
/toRdf-manifest.jsonld#tin05$/,
/toRdf-manifest.jsonld#tin06$/,
// keywords
/toRdf-manifest.jsonld#tpr30$/,
/toRdf-manifest.jsonld#tpr31$/,
/toRdf-manifest.jsonld#tpr32$/,
/toRdf-manifest.jsonld#tpr33$/,
/toRdf-manifest.jsonld#tpr34$/,
/toRdf-manifest.jsonld#tpr35$/,
/toRdf-manifest.jsonld#tpr36$/,
/toRdf-manifest.jsonld#tpr37$/,
/toRdf-manifest.jsonld#tpr39$/,
// direction
/toRdf-manifest.jsonld#tdi01$/,
/toRdf-manifest.jsonld#tdi02$/,
Expand Down