Our blog post Introducing ConceptEngine gives an overview of what ConceptEngine is all about, and also lays out a simple example.

ConceptML

ConceptML is a minimalistic language to express connected data. It includes syntax for defining relationships and meta-relationships, as well as composing related concepts with its expansion syntax.

What are Concepts?

In ConceptEngine, data is represented in labeled hierarchical structures we call concepts. Here's one way to vizualize them:

Atoms

The simplest kind of concept is an atom, which is just a single token or a block of text enclosed with << and >>.

These are examples of atoms:

javascript
typescript
ProgrammingLanguage
42
<<December 30, 1985>>
john-smith
#some_hashtag

Compounds

A compound is a list of two or more concepts. You can embed compound concepts by enclosing them with [ and ].

These are examples of compounds:

javascript ProgrammingLanguage
typescript superset-of javascript
[john-smith knows javascript] since 1999

The ability to include an arbitrary number of concepts, along with nesting them as deeply as you'd like, gives concepts extra expressiveness.

Here's one example of deeply nested concepts:

[mary-mullens disagrees [[jon-smith knows javascript] since 1999]] at <<March 7, 2022 10:35:00>>

Here we're using nested concepts to represent metadata, which can be useful for higher-level tools that manage the data, such as content management systems. However, nesting concepts isn't just for metadata--it gives you the freedom to design your own grammars.

There are many different ways you can represent information using concepts. Here are four ways of expressing once piece of information:

john-smith knows javascript
john-smith [knows javascript]
[john-smith javascript] Knowledge
knows john-smith javascript

As long as there are distinct patterns between different kinds of information, it will work.

Expansion Syntax

Let's say we want to compose a set of related concepts. ConceptML provides expansion syntax to branch off multiple concepts within the flow of a composition.

There are three kinds of expansion, which each involve surrounding concept fragments with brackets and separating each branch with a comma or newline:

• Nested: []
• Inline: {}
• Parenthetical: ()

Here's an example that involves all three:

john-smith [
  Person
  knows {
    {javascript, typescript} (ProgrammingLanguage)
    {ms-word, ms-excel} (App, [created-by microsoft])
  }
]

This expands into these concepts:

john-smith Person
john-smith [knows javascript]
john-smith [knows typescript]
john-smith [knows ms-word]
john-smith [knows ms-excel]
javascript ProgrammingLanguage
typescript ProgrammingLanguage
ms-word App
ms-excel App
ms-word [created-by microsoft]
ms-excel [created-by microsoft]

Below are the three types in detail.

Nested Expansion

Creates branches within a nested compound block.

For example:

mary wrote [
  javascript ProgrammingLanguage
  javascript #awesome
]

This would expand to:

mary wrote [javascript ProgrammingLanguage]
mary wrote [javascript #awesome]

Inline Expansion

Creates branches within the flow of a concept definition.

For example:

john {knows, likes} javascript

would expand to:

john knows javascript
john likes javascript

Parenthetical Expansion

Creates concepts outside the flow of concept definition, with the immediately preceding concept acting as the head of the expanded concepts.

For example:

john knows {javascript, typescript} (ProgrammingLanguage)

This would expand to:

john knows javascript
john knows typescript
javascript ProgrammingLanguage
typescript ProgrammingLanguage

Space API

The Space API uses GraphQL to read, write, and match concepts using pattern matching. On top of pattern matching, it provides methods to project queried concepts into JSON structures.

You can also store arbitrary strings (up to 25MB) against concepts.

Because you can use concepts to represent just about any kind of information, and you can use the Space API's GraphQL fields to query and project them into custom payloads, ConceptEngine can act as an instant backend for all sorts of apps.

Access

URL

Each space has its own GraphQL API. URLs follow this pattern:

https://api.coeng.workers.dev/spaces/:spaceId/graphql

You can find your space's URL by navigating to its API Access tab in the dashboard.

Authorization

In order to use the API, you must include a Bearer token in the Authorization header of your requests.

First, create an access token in the API Access tab. You can then use that token to make requests. They follow this pattern:

fetch('https://api.coeng.workers.dev/spaces/:spaceId/graphql', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: 'Bearer <token>',
  },
  body: JSON.stringify({
    query: 'query { concepts { key } }',
    variables: {}
  })
});

Concepts

These are all of the queries, mutations, and fields related to the Concept type, which we'll cover in more depth in the sections below:

type Query {
  concept(key: ID!): Concept
  concepts(query: ConceptsQuery = {}): [Concept!]!
}

type Mutation {
  addConcepts(input: AddConceptsInput!): [Concept!]!
  removeConcepts(keys: [String!]!): Int!
  updateConceptData(input: UpdateConceptDataInput!): Concept!
}

type Concept {
  key: ID!
  text: String!
  data: String
  parts: [Concept!]!
  contexts: [Concept!]!
}

List Concepts

Type Definitions

type Query {
  concepts(query: ConceptsQuery = {}): [Concept!]!
}

input ConceptsQuery {
  pagination: Pagination = {}
}

input Pagination {
  limit: Int = 100
  startKey: String
  endKey: String
}

Example

Query:

query {
  concepts(query: {
    pagination: {
      startKey: "docs"
      limit: 3
    }
  }) {
    key
  }
}

Result:

{
  "data": {
    "concepts": [
      {
        "key": "docs-overview-page Page"
      },
      {
        "key": "docs-overview-page [named <<ConceptEngine Documentation>>]"
      },
      {
        "key": "docs-overview-page"
      }
    ]
  }
}

Get a Concept

Type Definitions

type Query {
  concept(key: ID!): Concept
}

Example

Query:

query {
  concept(key: "Post") {
    key
    contexts {
      key
    }
  }
}

Result:

{
  "data": {
    "concept": {
      "key": "Post",
      "contexts": [
        {
          "key": "Post Type"
        },
        {
          "key": "intro-post Post"
        }
      ]
    }
  }
}

Get Concept Text

Returns the key, but stripped of any surrounding << and >> brackets.

Type Definitions

type Concept {
  text: String!
}

Example

Query:

query {
  concept(key: "<<Introducing Concept Engine>>") {
    text
  }
}

Result:

{
  "data": {
    "concept": {
      "text": "Introducing Concept Engine"
    }
  }
}

Get Concept Data

Concepts also act as key-value pairs. You can access the stored value by using the Concept.data field.

Type Definitions

type Concept {
  data: String
}

Example

Query:

query {
  concept(key: "intro-post excerpt") {
    data
  }
}

Result:

{
  "data": {
    "concept": {
      "data": "ConceptEngine is a blazing-fast data store for nested hypergraphs, a data model that goes even further than traditional graphs to capture highly connected data."
    }
  }
}

List Parts of a Concept

Get all concepts that a concept is composed of.

Type Definitions

type Concept {
  parts: [Concept!]!
}

Example

Query:

query {
  concept(key: "intro-post [named <<Introducing ConceptEngine>>]") {
    parts {
      key
      parts {
        key
      }
    }
  }
}

Result:

{
  "data": {
    "concept": {
      "parts": [
        {
          "key": "intro-post",
          "parts": []
        },
        {
          "key": "named <<Introducing ConceptEngine>>",
          "parts": [
            {
              "key": "named"
            },
            {
              "key": "<<Introducing ConceptEngine>>"
            }
          ]
        }
      ]
    }
  }
}

List Contexts of a Concept

Get all concepts that a concept is a part of.

Type Definitions

type Concept {
  contexts: [Concept!]!
}

Example

Query:

query {
  concept(key: "intro-post") {
    contexts {
      key
    }
  }
}

Result:

{
  "data": {
    "concept": {
      "contexts": [
        {
          "key": "intro-post Post"
        },
        {
          "key": "intro-post excerpt"
        },
        {
          "key": "intro-post [named <<Introducing ConceptEngine>>]"
        },
        {
          "key": "intro-post [published-on <<February 2, 2022>>]"
        },
        {
          "key": "intro-post [slug <<introducing-concept-engine>>]"
        },
        {
          "key": "intro-post [written-by eric-weber]"
        }
      ]
    }
  }
}

Add Concepts

Type Definitions

type Mutation {
  addConcepts(input: AddConceptsInput!): [Concept!]!
}

input AddConceptsInput {
  source: [String!]!
  interpolate: [Interpolation!] = []
}

input Interpolation {
  key: String!
  value: [String!]!
}

Example

Mutation:

mutation AddPost($post: String!, $slug: String!, $name: String!) {
  newConcepts: addConcepts(input: {
    source: "$post [Post, slug $slug, named $name]"
    interpolate: [
      { key: "$post", value: [$post] }
      { key: "$slug", value: [$slug] }
      { key: "$name", value: [$name] }
    ]
  }) {
    key
  }
}

Result:

{
  "data": {
    "newConcepts": [
      {
        "key": "hello-world-post Post"
      },
      {
        "key": "hello-world-post [slug <<hello-world>>]"
      },
      {
        "key": "hello-world-post [named <<Hello World>>]"
      },
      {
        "key": "hello-world-post"
      },
      {
        "key": "slug <<hello-world>>"
      },
      {
        "key": "<<hello-world>>"
      },
      {
        "key": "named <<Hello World>>"
      },
      {
        "key": "<<Hello World>>"
      }
    ]
  }
}

Remove Concepts

Type Definitions

type Mutation {
  removeConcepts(keys: [String!]!): Int!
}

Example

Mutation:

mutation {
  removeConcepts(keys: ["test-post"])
}

Result:

{
  "data": {
    "removeConcepts": 0
  }
}

Update Concept Data

Type Definitions

type Mutation {
  updateConceptData(input: UpdateConceptDataInput!): Concept!
}

input UpdateConceptDataInput {
  key: ID!
  data: String!
}

Example

Mutation:

mutation {
  concept: updateConceptData(input: {
    key: "test-post excerpt"
    data: "Just a test."
  }) {
    data
  }
}

Result:

{
  "data": {
    "concept": {
      "data": "Just a test."
    }
  }
}

Pattern Matching

ConceptEngine uses pattern matching in a similar way to SPARQL, in that it attempts to complete a set of patterns, which are composed of concepts and variables.

For example:

$person [Person, knows $topic (ProgrammingLanguage)]

Find All Matches

Type Definitions

type Query {
  matches(query: MatchesQuery!): [RuleSetMatch!]!
}

input MatchesQuery {
  rules: [String!]!
  interpolate: [Interpolation!] = []
  pagination: Pagination = {}
}

Example

Query:

query {
  matches(
    query: {
      rules: [
        "$post [Post, named $name, written-by $author]"
      ]
      interpolate: [
        { key: "$author", value: "eric-weber" }
      ]
    }
  ) {
    post: keyOf(name: "$post")
  }
}

Result:

{
  "data": {
    "matches": [
      {
        "post": "intro-post"
      }
    ]
  }
}

Find First Match

Type Definitions

type Query {
  match(query: MatchQuery!): RuleSetMatch
}

input MatchQuery {
  rules: [String!]!
  interpolate: [Interpolation!] = []
}

Example

Query:

query {
  match(
    query: {
      rules: [
        "$post [Post, named $name, written-by $author]"
      ]
      interpolate: [
        { key: "$author", value: "eric-weber" }
      ]
    }
  ) {
    post: keyOf(name: "$post")
  }
}

Result:

{
  "data": {
    "match": {
      "post": "intro-post"
    }
  }
}

Get Match Variables

Type Definitions

type RuleSetMatch {
  variables: [VariableMatch!]!
}

type VariableMatch {
  name: ID!
  match: Concept!
}

Example

Query:

query {
  match(query: {
    rules: "$person Person"
  }) {
    variables {
      name
      match {
        key
      }
    }
  }
}

Result:

{
  "data": {
    "match": {
      "variables": [
        {
          "name": "$person",
          "match": {
            "key": "eric"
          }
        }
      ]
    }
  }
}

Get Specific Variable

Type Definitions

type RuleSetMatch {
  valueOf(name: ID!): Concept!
}

Example

Query:

query {
  match(query: {
    rules: "$person Person"
  }) {
    person: valueOf(name: "$person") {
      key
    }
  }
}

Result:

{
  "data": {
    "match": {
      "person": {
        "key": "eric"
      }
    }
  }
}

Get Key of Variable

Type Definitions

type RuleSetMatch {
  keyOf(name: ID!): String
}

Example

Query:

query {
  match(query: {
    rules: "$person Person"
  }) {
    personKey: keyOf(name: "$person")
  }
}

Result:

{
  "data": {
    "match": {
      "personKey": "eric"
    }
  }
}

Get Text Value of Variable

Type Definitions

type RuleSetMatch {
  textOf(name: ID!): String
}

Example

Query:

query {
  match(query: {
    rules: "$person [Person, named $personName]"
  }) {
    name: textOf(name: "$personName")
  }
}

Result:

{
  "data": {
    "match": {
      "name": "Eric"
    }
  }
}

Get Data of Variable

Type Definitions

type RuleSetMatch {
  dataOf(name: ID!): String
}

Example

Query:

query {
  posts: matches(query: {
    rules: "$post Post"
  }) {
    key: keyOf(name: "$post")
    content: dataOf(name: "$post")
  }
}

Result:

{
  "data": {
    "posts": [
      {
        "key": "test-post",
        "content": "Just a test."
      }
    ]
  }
}

Subquery Matches

Type Definitions

type RuleSetMatch {
  matches(rules: [String!]!): [RuleSetMatch!]!
}

Example

Query:

query {
  posts: matches(query: {
    rules: "$post [Post]"
  }) {
    key: keyOf(name: "$post")
    tags: matches(rules: "$post $tag (Tag)") {
      key: keyOf(name: "$tag")
    }
  }
}

Result:

{
  "data": {
    "posts": [
      {
        "key": "test-post",
        "tags": [
          {
            "key": "#coeng"
          },
          {
            "key": "#test"
          }
        ]
      }
    ]
  }
}

Subquery Single Match

Type Definitions

type RuleSetMatch {
  match(rules: [String!]!): RuleSetMatch
}

Example

Query:

query {
  posts: matches(query: {
    rules: "$post [Post]"
  }) {
    key: keyOf(name: "$post")
    author: match(rules: "$post [written-by $author]") {
      key: keyOf(name: "$author")
    }
  }
}

Result:

{
  "data": {
    "posts": [
      {
        "key": "test-post",
        "author": {
          "key": "eric"
        }
      }
    ]
  }
}

Subquery Concepts

Type Definitions

type RuleSetMatch {
  concepts(rule: String!): [Concept!]!
}

Example

Query:

query {
  posts: matches(query: {
    rules: "$post Post"
  }) {
    tags: concept(rule: "$post $tag (Tag)") {
      key
    }
  }
}

Result:

{
  "data": {
    "posts": [
      {
        "tags": {
          "key": "test-post #coeng"
        }
      }
    ]
  }
}

Subquery Single Concept

Type Definitions

type RuleSetMatch {
  concept(rule: String!): Concept
}

Example

Query:

query {
  posts: matches(query: {
    rules: "$post Post"
  }) {
    excerpt: concept(rule: "$post excerpt") {
      data
    }
  }
}

Result:

{
  "data": {
    "posts": [
      {
        "excerpt": {
          "data": "Just a test."
        }
      }
    ]
  }
}