Time to move on: why RushDB took the hard path off Neo4j

For two major versions RushDB was an API on top of Neo4j. You pushed JSON, we turned it into linked records and kept track of every field, type and value, so you (and later, your agents) could see what's in the data before querying it. RushDB 3 keeps the same promise, but it runs on our own storage engine, written in Rust.

Writing your own database engine is the hard path. Everybody tells you not to do it, and IMO usually they're right. So I want to explain why I did it anyway, in order, the way it actually happened.

One thing first. This isn't a "why Neo4j is bad" post. Neo4j carried RushDB for years. Their team spent real time with us, we had good talks, and RushDB was accepted into their startup program. Most of what I know about running a graph in production I learned on Neo4j. It's a mature, reliable database. It just stopped being the right foundation for what RushDB became, and at some point it was time to move on.

The promise, and how we emulated it

RushDB is built around a model I call LMPG, the Labeled Meta Property Graph. In a normal property graph, properties are attributes sitting on nodes. In LMPG, properties are first-class: each property is a typed definition shared by every record that uses it. That's what makes the first questions cheap. Which fields does this label have? What types? Which values exist, and how many records carry each one? An agent should be able to ask that before it writes a single query: question -> schema -> filter -> schema of what's left -> precise query.

On Neo4j we emulated it. Each record was a node with its values as normal properties. Each (name, type) pair in a project got its own property node, linked to every record that carries the field. And because a property node can't tell you the type of a field on one specific record, every record also kept a JSON string with its own types map. This is the heart of the 2.x write path, slightly trimmed:

SET record.__RUSHDB__KEY__PROPERTIES__META__ = apoc.convert.toJson(typesMap)
SET record += valuesMap
WITH record
UNWIND $properties AS prop
MERGE (p:__RUSHDB__LABEL__PROPERTY__ {
  name: prop.name, type: coalesce(prop.type, 'string'), projectId: $projectId
})
MERGE (p)-[:__RUSHDB__RELATION__VALUE__]->(record)

For some questions this shape was great. Counting how many records have a property is just counting relationships, and Neo4j does that without touching values. There's literally a comment in our 2.x code about it: "Neo4j expands and counts, never touching value storage."

For the questions that mattered most, it was O(n) at best. Fields of the records a filter matches? Unwind the keys of every matching record. Distinct values of a field? Walk every VALUE edge, read the value from each record, dedupe, and only then cut the page. Rename a field? Rewrite the JSON types map on every record that has it. Search over values? A case-insensitive regex, which no index can help with.

On small projects nobody noticed. On the largest projects it was the main thing everybody noticed, because it scaled linearly with the data, and that undermines the core promise of LMPG. The schema is supposed to be the cheap part.

Our 2.x changelog reads like a diary of fighting this: imports that "could hit the 30s transaction ceiling and fail outright" (so we doubled the budget to 60 s), a schema rebuild that was "one monolithic full-graph scan inside a single transaction", AI schema endpoints that would "block or 408 on large projects" (so we started sampling instead of reading), vector search that could "stall behind background indexing work".

Every single fix was reasonable. Every single fix was also a workaround for the same thing, and each one added a bit more entropy, because we were storing a meta-property graph inside a property graph and paying for the translation on every read.

Tenancy by convention

In RushDB 2.x, every project lived in one graph. The isolation between them was a hidden property on every node, __RUSHDB__KEY__PROJECT__ID__, and every query had to match on it. A hidden property on every node. Ugh. We had a helper for that:

// 2.x: isolation was a predicate every query had to remember
export const projectIdInline = () => '__RUSHDB__KEY__PROJECT__ID__: $projectId'

Dozens of hand-written and generated Cypher fragments called it. The code was honest about the risk; one comment says: "projectId scoping here is the tenant-isolation boundary … an unscoped match would let any token reach another project's property." That's not a boundary. That's a convention, and a convention holds exactly as long as every code path remembers it.

Physical isolation was possible, but only one way: a separate Neo4j instance. 2.x let a project bring its own Neo4j, with its own driver pool, and for a customer who needs their data on separate hardware that's a fine answer. For a platform with many small projects it's not, because every isolated tenant is one more database server to run, watch, back up and pay for.

Neo4j Enterprise has more to offer for this kind of setup, but an Enterprise licence was never something we considered. RushDB is something people self-host, and its core can't depend on a licence our users would have to buy separately. So we stayed on Community Edition (GPLv3), with one instance and one property filter.

In RushDB 3 tenancy is built different. A tenant isn't a property on the record. It's part of the key. Every row a tenant owns lives under a five-byte prefix: a namespace byte plus the tenant's 32-bit id. A scan can't cross that prefix, deleting a tenant is a range delete, and on a multi-tenant server a request without a tenant scope simply doesn't run. There's no filter left to remember.

Indexes we couldn't own

Indexes had the same problem. In 2.x we created a fixed set of global constraints and indexes at startup (record id, project id, property name), shared by every project on the instance. There was no way to give one project an index on its own customer_id without giving it to everyone. In practice, a filter on any user field meant an index lookup on the project id and then a scan through the project's records.

With one shared Neo4j instance and a tenancy requirement, there was no proper way to manage this. In 3.x, index definitions and index entries live under the tenant prefix like everything else, so each tenant declares its own indexes and they never mix with anyone else's.

Vectors: rank first, filter later

Agents almost never want "the most similar documents in the database". Who cares about the whole database? They want the most similar documents among the ones this user may see, in this project, where lang = "en". Filter -> candidates -> rank, in that order.

The way we built vector search in 2.x did it the other way around. Embeddings lived on the VALUE relationships, in one vector index per embedding type, similarity and dimension, shared across all projects. A search asked that index for the global top-K (somewhere between 1,000 and 10,000 candidates) and only then kept the ones from the caller's project:

CALL db.index.vector.queryRelationships($vectorIndexName, $candidateLimit, $queryVector)
YIELD relationship AS rel, score
WHERE rel.__projectId = $projectId AND rel.__propKey = $propKey

If other projects dominated that neighbourhood, you got fewer results than you asked for. And the moment a request had a filter, we skipped the index completely and scored every candidate exactly. Correct, but it grows with the data. To be fair to Neo4j, this was our design on their vector index, not a limit of their vector search. But it's the design a shared multi-tenant graph pushed us into.

In 3.x the filter becomes a candidate set before ranking, and exact or approximate search only looks inside it. A filtered search never returns a record outside the filter, and never comes back short because good matches were thrown away after ranking:

{
  "kind": "vector",
  "query": {
    "labels": ["Doc"],
    "property": "embedding",
    "index": "doc-embedding",
    "vector": [0.10, 0.95, 0.00],
    "predicate": { "lang": "en" },
    "limit": 5
  }
}

The vectors sit inside the tenant's keyspace too, so there's no global index to post-filter in the first place.

One door, and it was Cypher

On Neo4j, Cypher is the language, so for RushDB it was the only language. Our own query format was translated into Cypher strings, roughly 2,700 lines of parser gluing queries together with a string builder. Raw Cypher was only allowed for projects on their own Neo4j, because on the shared graph a raw query could skip the project filter. So tenancy was also limiting which languages we could expose at all.

And customers kept asking. For a year, the most common reaction after a demo was some version of: "Good stuff, Artemiy, but why Neo4j? Why not Postgres, or X, or Y?" Their BI tools speak SQL. Their agent frameworks expect Postgres. Their graph code is Gremlin or Cypher, their app code thinks in Mongo-style documents. Nobody wanted to adopt Neo4j just to adopt RushDB, and every new integration started with the same tension, because you first had to learn our database's database.

In RushDB 3 the same data answers all of them:

SELECT name FROM FACTION ORDER BY name;

MATCH (n:FACTION) RETURN n.name AS name ORDER BY name

g.V().hasLabel('FACTION').values('name')

{ "collection": "FACTION", "filter": {}, "projection": { "name": 1 }, "sort": { "name": 1 } }

SQL and our native query format are stable; openCypher and ISO GQL are release candidates, and we say so openly in the docs. For this story the bigger change is where tenancy sits now. Every language runs on a tenant-scoped view of one snapshot. In 2.x, a new language would have meant another translator that has to remember the project filter. In 3.x it's just another frontend on the same keyspace, and the Rust engine guarantees the isolation underneath it.

Middleman

There's a less technical reason, and honestly it weighed as much as the rest.

RushDB 2.x was a middleman between Neo4j and a handy API. The engine was Neo4j, the useful layer on top was ours, and the API and the dashboard lived together under one Elastic License 2.0 repo. The ownership was clumsy, both for the business moat and for the tech moat, because a lot of our value was plumbing around somebody else's engine. If somebody asked "so what is the database?", the honest answer was "Neo4j, plus our model on top".

RushDB 3 splits this properly. The engine is ours, open source under Apache-2.0, one binary you can run anywhere. Cloud is a harness on top of it. RushDB Cloud runs the same engine and adds what a managed service should: dashboards, managed embeddings, natural-language search, relationship suggestions, hosted MCP. The database people depend on is open, as it should be. The service we run is ours to build, so we can ship LLM-powered features fast without worrying about whose engine they sit on.

Same market, different roads

When RushDB started, it was an API for developers who wanted graphs without modelling them. As we built more and more for AI agents, we landed right in agent memory, and that's a space Neo4j cares about too.

While we were in their startup program, the Neo4j team supported us, including on the promotional side: reposting our posts, giving us visibility. Later that promotional support ended. That's fair anyway, nobody owes a startup visibility. But it made one thing clear for me. Building on a platform whose owner is in your market means depending on goodwill, and that's not the play for me.

What the hard path costs

I don't want to pretend this was free.

Neo4j has years of production hardening, clustering, a big algorithm library and an ecosystem we don't have. RushDB 3 runs as a single node today. Replication to followers is asynchronous with manual promotion, and Raft-based high availability is still cooking (preview). openCypher is a release candidate.

The engine has measured limits too. On a laptop-class machine (measured on storage format 5; 3.0 writes format 6) one tenant takes about 1,015 single-record commits per second with 32 writers (114 with one), because every write goes through one writer, which is on purpose . In our own capacity runs at 30M records, on a build before 3.0.1, reads that have to visit every candidate take about 15 s at p50 with 32 clients. Some of that will get better. Some of it is the price of the design.

What we got for it is a system whose shape matches its promise. Properties are first-class in storage. The catalog of labels, fields, types and values is updated on every write, in the same commit, so it's exact, not sampled. Tenants are key ranges. Filters come before vector ranking. Every major query language reads the same records. And the line between the open-source engine and the managed service is finally where it should be.

Thank you, Neo4j

RushDB wouldn't exist without Neo4j. It let a very small team ship a graph product to real customers years before we could have written a storage engine ourselves, and the habits we picked up on it (thinking in relationships, respecting transactions, reading query plans) are in every crate of the new engine. If you need graph analytics at scale today, Neo4j is still where I'd start looking.

But for what RushDB set out to be, a database that tells AI agents exactly what it holds before they ask, we had to own the layer where that promise is kept. So it was time to move on. The release notes cover what shipped, the upgrade guide covers moving from 2.x, and the architecture deep dive shows how the engine stores one key per value.