ConceptEngine is a blazing-fast data store for nested hypergraphs, a data model that goes even further than traditional graphs to capture highly connected data.

Though ConceptEngine is built from the ground up to handle complexity, interacting with it is simple. Data is represented in a language called ConceptML, which uses sentence-like expressions to describe entities, relationships, and meta-relationships. We call these expressions concepts.

It's schema-less, and there's zero friction to spin it up. New spaces are up and running in milliseconds, come with dedicated computing resources, and are distributed across the planet. You can have a powerful backend serving connected data in a matter of minutes.

Wikis, content management systems, knowledge graphs, social networks, talent marketplaces, recommendation systems, research repositories—all of these are right in ConceptEngine’s wheelhouse.

We can't wait to see what you create!

How it Works

Under the hood, ConceptEngine translates between ConceptML and the underlying nested hypergraph structure, providing a GraphQL API to perform pattern-based queries against it and project the data into JSON.

This blog post itself was created with this ConceptML source:

intro-post [
    Post
    named <<Introducing Concept Engine>>
    written-by eric-weber
    published-on <<March 2, 2022>>
    slug introduction-to-concept-engine
]

eric-weber [
    Person
    Author
    named <<Eric Weber>>
]

{Post, Person, Author} Type

And we fetch this post's data with this GraphQL query:

query PostBySlug($slug: String!) {
    post: match(query: {
        rules: "$post [Post, named $name, slug $slug, published-on $publishedOn, written-by $author]"
        interpolate: [{key: "$slug", value: [$slug]}]
    }) {
        name: textOf(name: "$name")
        publishedOn: keyOf(name: "$publishedOn")
        author: match(rules: "$author [named $authorName]") {
            key: keyOf(name: "$author")
            name: textOf(name: "$authorName")
        }
        content: valueOf(name: "$post") {
            mdx: data
        }
    }
}

Essentially, ConceptML hides the complexity of the nested hypergraph and makes it intuitive to read and write data, while GraphQL grabs subsets of the data and maps them into usable objects for our frontends.

Example: Talent Marketplace

Today, companies such as Pepsi, Unilever, and Schneider Electric are using talent marketplaces to match open needs with available expertise.

Below is a mock-up of what this might look like in ConceptEngine. To follow along, go to https://dash.coeng.io, create an account, then create a new knowledge space. Let’s call it “Talent Marketplace Example”. During this example we’ll be using the Compose, Query, and optionally, GraphQL Console tab.

Data Patterns

First, let’s think about what kinds of information we will be working with, and the patterns we will use to represent them. We don’t have to create a schema in ConceptEngine, but it is worth thinking through a little bit.

Expertise

People can be experts in topics, which we’ll represent with $person [knows $topic].

Notice the square brackets—this is how we nest concepts within one another. Our convention is to structure it this way because we’ve come to think of things like knows $topic as their own building block of data that we can tag entities with. It also comes with some conveniences when browsing the knowledge space.

You could also represent this as $person knows $topic, knowledge $person $topic, [$person $topic] Knowledge, and so on. It’s up to you!

Project Requirements

Projects can have requirements for areas of expertise. We’ll represent this with $project [requires $topic].

Entity Types

As we can see, the knowledge space will have three kinds of entities: people, projects, and topics of expertise.

Our own convention is to use the pattern of $entity $Type to categorize entities. For example, python ProgrammingLanguage and john-smith Person follow this pattern.

We’ll also give types a type called, well, Type, which can be useful for certain queries (i.e. list all types, or list the types of a particular entity). An example concept would be Person Type.

Mock Data

Enter this in the Compose tab:

person-1 [
  Person
  knows {
    python
    html
    css
  }
]

person-2 [
  Person
  knows {
    rust
  }
]

project-1 [
  Project
  requires {
    html
    css
    rust
  }
]

{python, rust, html, css} [
  ProgrammingLanguage
]

{Person, Project, ProgrammingLanguage} [
  Type
]

This is equivalent to entering this in your space’s GraphQL console:

mutation {
  addConcepts(
    input: {
      source: [
        "person-1 [Person, knows {python, html, css}]"
        "person-2 [Person, knows rust]"
        "project-1 [Project, requires {html, css, rust}]"
        "{python, rust, html, css} ProgrammingLanguage"
        "{Person, Project, ProgrammingLanguage} Type"
      ]
    }
  ) {
    key
  }
}

Perhaps you noticed the commas, newlines, and curly brackets, which are part of ConceptML’s expansion syntax. Newlines and commas are equivalent, and when you place them within a bracket, it permutes the concepts within those brackets.

The above ConceptML markup expands to these concepts:

person-1 Person
person-1 [knows python]
person-1 [knows html]
person-1 [knows css]
project-1 Project
project-1 [requires html]
project-1 [requires css]
project-1 [requires rust]
person-2 Person
person-2 [knows rust]
python ProgrammingLanguage
rust ProgrammingLanguage
html ProgrammingLanguage
css ProgrammingLanguage
Person Type
Project Type
ProgrammingLanguage Type

Queries

It’s a simple knowledge space, but it’s capable of answering some interesting questions. Try entering these into the Query tab of the ConceptEngine Dashboard:

$project Project
$person Person
$project [Project, requires $topic (ProgrammingLanguage)]
person-1 [knows $topic], $project [requires $topic]

Here’s how the last one can be written in the GraphQL API:

query {
  matches(
    query: {
      rules: ["person-1 [knows $topic]", "$project [Project, requires $topic]"]
    }
  ) {
    topic: keyOf(name: "$topic")
  }
}

Which would return:

{
  "data": {
    "matches": [
      {
        "topic": "css"
      },
      {
        "topic": "html"
      }
    ]
  }
}

Subqueries

We can also use pattern-based matching to create subqueries. In this example, we’ll ask our knowledge space for each project, its requirements, and people that can fulfill them:

query {
  matches(query: { rules: "$project Project" }) {
    project: keyOf(name: "$project")
    requirements: matches(rules: "$project [requires $topic]") {
      topic: keyOf(name: "$topic")
      experts: matches(rules: "$person [knows $topic]") {
        person: keyOf(name: "$person")
      }
    }
  }
}

Which would produce:

{
  "data": {
    "matches": [
      {
        "project": "project-1",
        "requirements": [
          {
            "topic": "css",
            "experts": [
              {
                "person": "person-1"
              }
            ]
          },
          {
            "topic": "html",
            "experts": [
              {
                "person": "person-1"
              }
            ]
          },
          {
            "topic": "rust",
            "experts": [
              {
                "person": "person-2"
              }
            ]
          }
        ]
      }
    ]
  }
}

Taking it Further

Naming and Labeling

How about assigning labels to entities, so that we can display them in apps? In ConceptML, we can escape text with << and >>, the whole of which is treated as a concept. Here’s an example:

eric-weber [named <<Eric Weber>>]

We can then query people along with their names like this:

$person [named $name]

In GraphQL, you can use Concept.text to get the text value without the << >> brackets. You can also use RuleSetMatch.textOf(name) to get the text of a particular variable within a pattern match.

Storing Data Against Keys

ConceptEngine also acts as a kind of key-value store. Let’s store a markdown description for project-1.

You have the option to use the Dashboard to save data against a concept key. Go to the Browse, then search for “project-1”. Click on project-1 in the results, which will bring you to a detail view for that concept. On the right size, you can put any arbitrary data you please.

Write this, then press the Save button:

# Project 1

An awesome project. Come work with us!

Alternatively, you can use the GraphQL Console tab:

mutation {
  concept: updateConceptData(
    input: {
      key: "project-1"
      data: "# Project 1\n\nAn awesome project. Come work with us!"
    }
  ) {
    key
    data
  }
}

Parenthetical Expansion

One piece of syntax we breezed past in the above example is parenthetical expansion, which allows us to define concepts inline. It should be used judiciously, as it can sometimes make compositions more difficult to read.

Here’s an example:

person-1 [
  Person
  knows {
    python
    html
    css
  } (ProgrammingLanguage (Type))
]

The parenthetical expansion would create these concepts:

python ProgrammingLanguage
html ProgrammingLanguage
css ProgrammingLanguage
ProgrammingLanguage Type

Conclusion

ConceptEngine makes it easy to create knowledge spaces, populate them with richly connected data, then write GraphQL queries that project them into JSON objects.

In the above examples, we can quickly see our data taking shape. Our own little grammar emerged from the way we chose to structure our concepts. You can use completely different conventions, and that would work as well. The key is finding grammar rules that you can easily reuse.

You may be thinking that this could lead to a messy, inconsistent database. However, there is a feature that helps tame this: you can define rules to create and remove concepts in reaction to patterns, as well as to notify external webhooks.

This not only allows for the creation of helper functions, but aids in maintaining data consistency and setup ConceptEngine to act as an automation hub. We look forward to covering this in an upcoming blog post!