feat(core): Add a new string index: n-gram

[ghost]

This PR adds ngram (aka shingles) indexing to string predicates.

Background

Ngram (also known as shingles) indexing is a technique for indexing and searching text that allows for flexible, partial-text matching. It works by breaking down a string into a sequence of n overlapping tokens (words).

Implementation

The implementation supports basic text analysis (lower-casing and normalization), stop word removal, language-specific work stemming and adds support in both DQL and GraphQL.

DQL:

We introduce the ngram search directive and search function.

Schema

description: string @index(ngram) .

Query syntax

{
    me(func: ngram(description, "quick brown fox")) {
	uid
	description
    }
}

GraphQL

Schema

description: String @search(by: [ngram])

Query syntax

queryFoo(filter: { description: { ngram: $title } }) {
    title
}

Checklist

• Code compiles correctly and linting passes locally
• For all code changes, an entry added to the CHANGELOG.md file describing and linking to
  this PR
• Tests added for new functionality, or regression tests for bug fixes added as applicable

[trunk-io]

     

Failed Test	Failure Summary	Logs
TestBackupMinio	The backup test failed because it could not establish a connection to retrieve predicate information.	Logs ↗︎

View Full Report ↗︎ ⋅ Docs

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[mangalaman93]

We should add a description stating why we felt the need to add this new index and keep it when we merge the PR.