Back to skills
SkillHub ClubDesign ProductFull StackDevOpsData / AI
surrealdb
Expert SurrealDB 3 architect and developer skill. SurrealQL mastery, multi-model data modeling (document, graph, vector, time-series, geospatial), schema design, security, deployment, performance tuning, SDK integration (JS, Python, Go, Rust), Surrealism WASM extensions, and full ecosystem (Surrealist, Surreal-Sync, SurrealFS). Universal skill for 30+ AI agents.
Packaged view
This page reorganizes the original catalog entry around fit, installability, and workflow context first. The original raw source lives below.
Stars
3,114
Hot score
99
Updated
March 20, 2026
Overall rating
C0.0
Composite score
0.0
Best-practice grade
C64.8
Install command
npx @skill-hub/cli install openclaw-skills-surrealdb
Repository
openclaw/skills
Skill path: skills/24601/surrealdb
Expert SurrealDB 3 architect and developer skill. SurrealQL mastery, multi-model data modeling (document, graph, vector, time-series, geospatial), schema design, security, deployment, performance tuning, SDK integration (JS, Python, Go, Rust), Surrealism WASM extensions, and full ecosystem (Surrealist, Surreal-Sync, SurrealFS). Universal skill for 30+ AI agents.
Open repositoryBest for
Primary workflow: Design Product.
Technical facets: Full Stack, DevOps, Data / AI, Designer, Security, Integration.
Target audience: everyone.
License: MIT.
Original source
Catalog source: SkillHub Club.
Repository owner: openclaw.
This is still a mirrored public skill entry. Review the repository before installing into production workflows.
What it helps with
- Install surrealdb into Claude Code, Codex CLI, Gemini CLI, or OpenCode workflows
- Review https://github.com/openclaw/skills before adding surrealdb to shared team environments
- Use surrealdb for development workflows
Works across
Claude CodeCodex CLIGemini CLIOpenCode
Favorites: 0.
Sub-skills: 0.
Aggregator: No.
Original source / Raw SKILL.md
---
name: surrealdb
description: "Expert SurrealDB 3 architect and developer skill. SurrealQL mastery, multi-model data modeling (document, graph, vector, time-series, geospatial), schema design, security, deployment, performance tuning, SDK integration (JS, Python, Go, Rust), Surrealism WASM extensions, and full ecosystem (Surrealist, Surreal-Sync, SurrealFS). Universal skill for 30+ AI agents."
license: MIT
metadata:
version: "1.2.1"
author: "24601"
snapshot_date: "2026-03-13"
repository: "https://github.com/24601/surreal-skills"
requires:
binaries:
- name: surreal
install: "brew install surrealdb/tap/surreal"
purpose: "SurrealDB CLI for server management, SQL REPL, import/export"
optional: false
- name: python3
version: ">=3.10"
purpose: "Required for skill scripts (doctor.py, schema.py, onboard.py)"
optional: false
- name: uv
install: "brew install uv"
purpose: "PEP 723 script runner -- installs script deps automatically"
optional: false
- name: docker
purpose: "Containerized SurrealDB instances"
optional: true
- name: gh
install: "brew install gh"
purpose: "GitHub CLI -- used only by check_upstream.py for comparing upstream repo SHAs"
optional: true
env_vars:
- name: SURREAL_ENDPOINT
purpose: "SurrealDB server URL"
default: "http://localhost:8000"
sensitive: false
- name: SURREAL_USER
purpose: "Authentication username"
default: "root"
sensitive: true
- name: SURREAL_PASS
purpose: "Authentication password"
default: "root"
sensitive: true
- name: SURREAL_NS
purpose: "Default namespace"
default: "test"
sensitive: false
- name: SURREAL_DB
purpose: "Default database"
default: "test"
sensitive: false
security:
no_network: false
no_network_note: "doctor.py and schema.py connect to a user-specified SurrealDB endpoint (WebSocket) for health checks and schema introspection. check_upstream.py calls GitHub API via gh CLI to compare upstream repo SHAs. No other third-party network calls."
no_credentials: false
no_credentials_note: "Scripts accept SURREAL_USER/SURREAL_PASS for DB authentication. No credentials are stored in the skill itself."
no_env_write: true
no_file_write: true
no_shell_exec: false
no_shell_exec_note: "Scripts invoke surreal CLI and gh for health checks."
scripts_auditable: true
scripts_use_pep723: true
no_obfuscated_code: true
no_binary_blobs: true
no_minified_scripts: true
no_curl_pipe_sh: false
no_curl_pipe_sh_note: "Documentation mentions curl|sh as ONE install option alongside safer alternatives (brew, Docker, package managers). The skill itself never executes curl|sh."
---
# SurrealDB 3 Skill
Expert-level SurrealDB 3 architecture, development, and operations. Covers SurrealQL, multi-model data modeling, graph traversal, vector search, security, deployment, performance tuning, SDK integration, and the full SurrealDB ecosystem.
## For AI Agents
Get a full capabilities manifest, decision trees, and output contracts:
```bash
uv run {baseDir}/scripts/onboard.py --agent
```
See [AGENTS.md]({baseDir}/AGENTS.md) for the complete structured briefing.
| Command | What It Does |
|---------|-------------|
| `uv run {baseDir}/scripts/doctor.py` | Health check: verify surreal CLI, connectivity, versions |
| `uv run {baseDir}/scripts/doctor.py --check` | Quick pass/fail check (exit code only) |
| `uv run {baseDir}/scripts/schema.py introspect` | Dump full schema of a running SurrealDB instance |
| `uv run {baseDir}/scripts/schema.py tables` | List all tables with field counts and indexes |
| `uv run {baseDir}/scripts/onboard.py --agent` | JSON capabilities manifest for agent integration |
## Prerequisites
- **surreal CLI** -- `brew install surrealdb/tap/surreal` (macOS) or see [install docs](https://surrealdb.com/docs/surrealdb/installation)
- **Python 3.10+** -- Required for skill scripts
- **uv** -- `brew install uv` (macOS) or `pip install uv` or see [uv docs](https://docs.astral.sh/uv/getting-started/installation/)
Optional:
- **Docker** -- For containerized SurrealDB instances (`docker run surrealdb/surrealdb:v3`)
- **SDK of choice** -- JavaScript, Python, Go, Rust, Java, .NET, C, PHP, or Dart
> **Security note**: This skill's documentation references package manager installs
> (brew, pip, cargo, npm, Docker) as the recommended install method. If you
> encounter `curl | sh` examples in the rules files, prefer your OS package
> manager or download-and-review workflow instead.
## Quick Start
> **Credential warning**: Examples below use `root/root` for **local development
> only**. Never use default credentials against production or shared instances.
> Create scoped, least-privilege users for non-local environments.
```bash
# Start SurrealDB in-memory for LOCAL DEVELOPMENT ONLY
surreal start memory --user root --pass root --bind 127.0.0.1:8000
# Start with persistent RocksDB storage (local dev)
surreal start rocksdb://data/mydb.db --user root --pass root
# Start with SurrealKV (time-travel queries supported, local dev)
surreal start surrealkv://data/mydb --user root --pass root
# Connect via CLI REPL (local dev)
surreal sql --endpoint http://localhost:8000 --user root --pass root --ns test --db test
# Import a SurrealQL file
surreal import --endpoint http://localhost:8000 --user root --pass root --ns test --db test schema.surql
# Export the database
surreal export --endpoint http://localhost:8000 --user root --pass root --ns test --db test backup.surql
# Check version
surreal version
# Run the skill health check
uv run {baseDir}/scripts/doctor.py
```
## Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `SURREAL_ENDPOINT` | SurrealDB server URL | `http://localhost:8000` |
| `SURREAL_USER` | Root or namespace username | `root` |
| `SURREAL_PASS` | Root or namespace password | `root` |
| `SURREAL_NS` | Default namespace | `test` |
| `SURREAL_DB` | Default database | `test` |
These map directly to the `surreal sql` CLI flags (`--endpoint`, `--user`, `--pass`, `--ns`, `--db`) and are recognized by official SurrealDB SDKs.
## Core Capabilities
### SurrealQL Mastery
Full coverage of the SurrealQL query language: `CREATE`, `SELECT`, `UPDATE`, `UPSERT`, `DELETE`, `RELATE`, `INSERT`, `LIVE SELECT`, `DEFINE`, `REMOVE`, `INFO`, subqueries, transactions, futures, and all built-in functions (array, crypto, duration, geo, math, meta, object, parse, rand, string, time, type, vector).
See: `rules/surrealql.md`
### Multi-Model Data Modeling
Design schemas that leverage SurrealDB's multi-model capabilities -- document collections, graph edges, relational references, vector embeddings, time-series data, and geospatial coordinates -- all in a single database with a single query language.
See: `rules/data-modeling.md`
### Graph Queries
First-class graph traversal without JOINs. `RELATE` creates typed edges between records. Traverse with `->` (outgoing), `<-` (incoming), and `<->` (bidirectional) operators. Filter, aggregate, and recurse at any depth.
See: `rules/graph-queries.md`
### Vector Search
Built-in vector similarity search using HNSW and brute-force indexes. Define vector fields, create indexes with configurable distance metrics (cosine, euclidean, manhattan, minkowski), and query with `vector::similarity::*` functions. Build RAG pipelines and semantic search directly in SurrealQL.
See: `rules/vector-search.md`
### Security and Permissions
Row-level security via `DEFINE TABLE ... PERMISSIONS`, namespace/database/record-level access control, `DEFINE ACCESS` for JWT/token-based auth, `DEFINE USER` for system users, and `$auth`/`$session` runtime variables for permission predicates.
See: `rules/security.md`
### Deployment and Operations
Single-binary deployment, Docker, Kubernetes (Helm charts), storage engine selection (memory, RocksDB, SurrealKV, TiKV for distributed), backup/restore, monitoring, and production hardening.
See: `rules/deployment.md`
### Performance Tuning
Index strategies (unique, search, vector HNSW, MTree), query optimization with `EXPLAIN`, connection pooling, storage engine trade-offs, batch operations, and resource limits.
See: `rules/performance.md`
### SDK Integration
Official SDKs for JavaScript/TypeScript (Node.js, Deno, Bun, browser), Python, Go, Rust, Java, .NET, C, PHP, and Dart. Connection protocols (HTTP, WebSocket), authentication flows, live query subscriptions, and typed record handling.
See: `rules/sdks.md`
### Surrealism WASM Extensions
New in SurrealDB 3: extend the database with custom functions, analyzers, and logic written in Rust and compiled to WASM. Define, deploy, and manage Surrealism modules.
See: `rules/surrealism.md`
### Ecosystem Tools
- **Surrealist** -- Official IDE and GUI for SurrealDB (schema designer, query editor, graph visualizer)
- **Surreal-Sync** -- Change Data Capture (CDC) for migrations from other databases
- **SurrealFS** -- AI agent filesystem built on SurrealDB
- **SurrealML** -- Machine learning model management and inference within SurrealDB
See: `rules/surrealist.md`, `rules/surreal-sync.md`, `rules/surrealfs.md`
## Doctor / Health Check
```bash
# Full diagnostic (Rich output on stderr, JSON on stdout)
uv run {baseDir}/scripts/doctor.py
# Quick check (exit code 0 = healthy, 1 = issues found)
uv run {baseDir}/scripts/doctor.py --check
# Check a specific endpoint
uv run {baseDir}/scripts/doctor.py --endpoint http://my-server:8000
```
The doctor script verifies: surreal CLI installed and on PATH, server reachable, authentication succeeds, namespace and database exist, version compatibility, and storage engine status.
## Schema Introspection
```bash
# Full schema dump (all tables, fields, indexes, events, accesses)
uv run {baseDir}/scripts/schema.py introspect
# List tables with summary
uv run {baseDir}/scripts/schema.py tables
# Inspect a specific table
uv run {baseDir}/scripts/schema.py table <table_name>
# Export schema as SurrealQL (reproducible DEFINE statements)
uv run {baseDir}/scripts/schema.py export --format surql
# Export schema as JSON
uv run {baseDir}/scripts/schema.py export --format json
```
Introspection uses `INFO FOR DB`, `INFO FOR TABLE`, and `INFO FOR NS` to reconstruct the full schema.
## Rules Reference
| Rule File | Coverage |
|-----------|----------|
| `rules/surrealql.md` | SurrealQL syntax, statements, functions, operators, idioms |
| `rules/data-modeling.md` | Schema design, record IDs, field types, relations, normalization |
| `rules/graph-queries.md` | RELATE, graph traversal operators, path expressions, recursive queries |
| `rules/vector-search.md` | Vector fields, HNSW/brute-force indexes, similarity functions, RAG patterns |
| `rules/security.md` | Permissions, access control, authentication, JWT, row-level security |
| `rules/deployment.md` | Installation, storage engines, Docker, Kubernetes, production config |
| `rules/performance.md` | Indexes, EXPLAIN, query optimization, batch ops, resource tuning |
| `rules/sdks.md` | JavaScript, Python, Go, Rust SDK usage, connection patterns, live queries |
| `rules/surrealism.md` | WASM extensions, custom functions, Surrealism module authoring |
| `rules/surrealist.md` | Surrealist IDE/GUI usage, schema designer, query editor |
| `rules/surreal-sync.md` | CDC migration tool, source/target connectors, migration workflows |
| `rules/surrealfs.md` | AI agent filesystem, file storage, metadata, retrieval patterns |
## Workflow Examples
> **All workflow examples use `root/root` for local development only.**
> For production, use `DEFINE USER` with scoped, least-privilege credentials.
### New Project Setup
```bash
# 1. Verify environment
uv run {baseDir}/scripts/doctor.py
# 2. Start SurrealDB
surreal start rocksdb://data/myproject.db --user root --pass root
# 3. Design schema (use rules/data-modeling.md for guidance)
# 4. Import initial schema
surreal import --endpoint http://localhost:8000 --user root --pass root \
--ns myapp --db production schema.surql
# 5. Introspect to verify
uv run {baseDir}/scripts/schema.py introspect
```
### Migration from SurrealDB v2
```bash
# 1. Export v2 data
surreal export --endpoint http://old-server:8000 --user root --pass root \
--ns myapp --db production v2-backup.surql
# 2. Review breaking changes (see rules/surrealql.md v2->v3 migration section)
# Key changes: range syntax 1..4 is now exclusive of end, new WASM extension system
# 3. Import into v3
surreal import --endpoint http://localhost:8000 --user root --pass root \
--ns myapp --db production v2-backup.surql
# 4. Verify schema
uv run {baseDir}/scripts/schema.py introspect
```
### Data Modeling for a New Domain
```bash
# 1. Read rules/data-modeling.md for schema design patterns
# 2. Read rules/graph-queries.md if your domain has relationships
# 3. Read rules/vector-search.md if you need semantic search
# 4. Draft schema.surql with DEFINE TABLE, DEFINE FIELD, DEFINE INDEX
# 5. Import and test
surreal import --endpoint http://localhost:8000 --user root --pass root \
--ns dev --db test schema.surql
uv run {baseDir}/scripts/schema.py introspect
```
### Deploying to Production
```bash
# 1. Read rules/deployment.md for storage engine selection and hardening
# 2. Read rules/security.md for access control setup
# 3. Read rules/performance.md for index strategy
# 4. Run doctor against production endpoint
uv run {baseDir}/scripts/doctor.py --endpoint https://prod-surreal:8000
# 5. Verify schema matches expectations
uv run {baseDir}/scripts/schema.py introspect --endpoint https://prod-surreal:8000
```
## Upstream Source Check
```bash
# Check if upstream SurrealDB repos have changed since this skill was built
uv run {baseDir}/scripts/check_upstream.py
# JSON-only output for agents
uv run {baseDir}/scripts/check_upstream.py --json
# Only show repos that have new commits
uv run {baseDir}/scripts/check_upstream.py --stale
```
Compares current HEAD SHAs and release tags of all tracked repos against the
baselines in `SOURCES.json`. Use this to plan incremental skill updates.
## Source Provenance
This skill was built on **2026-02-19** from these upstream sources:
| Repository | Release | Snapshot Date |
|------------|---------|---------------|
| [surrealdb/surrealdb](https://github.com/surrealdb/surrealdb) | v3.0.0 | 2026-02-19 |
| [surrealdb/surrealist](https://github.com/surrealdb/surrealist) | v3.7.2 | 2026-02-21 |
| [surrealdb/surrealdb.js](https://github.com/surrealdb/surrealdb.js) | v1.3.2 | 2026-02-20 |
| [surrealdb/surrealdb.js](https://github.com/surrealdb/surrealdb.js) (v2 beta) | v2.0.0-beta.1 | 2026-02-20 |
| [surrealdb/surrealdb.py](https://github.com/surrealdb/surrealdb.py) | v1.0.8 | 2026-02-03 |
| [surrealdb/surrealdb.go](https://github.com/surrealdb/surrealdb.go) | v1.3.0 | 2026-02-12 |
| [surrealdb/surreal-sync](https://github.com/surrealdb/surreal-sync) | v0.3.4 | 2026-02-12 |
| [surrealdb/surrealfs](https://github.com/surrealdb/surrealfs) | -- | 2026-01-29 |
Documentation: [surrealdb.com/docs](https://surrealdb.com/docs) snapshot 2026-02-22.
Machine-readable provenance: `SOURCES.json`.
## Output Convention
All Python scripts in this skill follow a dual-output pattern:
- **stderr**: Rich-formatted human-readable output (tables, panels, status indicators)
- **stdout**: Machine-readable JSON for programmatic consumption by AI agents
This means `2>/dev/null` hides the human output, and piping stdout gives clean JSON for downstream processing.
---
## Referenced Files
> The following files are referenced in this skill and included for context.
### rules/data-modeling.md
```markdown
# SurrealDB Data Modeling Guide
This guide covers data modeling patterns, schema design, and best practices for SurrealDB v3. SurrealDB is a multi-model database, meaning a single deployment can serve document, graph, relational, vector, time-series, geospatial, and full-text search use cases simultaneously.
---
## Core Concepts
### Namespace / Database / Table / Record Hierarchy
SurrealDB organizes data in a four-level hierarchy:
```
Root
└── Namespace (organizational boundary, e.g., per-tenant)
└── Database (application boundary)
└── Table (collection of records)
└── Record (individual document, identified by table:id)
```
```surql
-- Define the hierarchy
DEFINE NAMESPACE mycompany;
USE NS mycompany;
DEFINE DATABASE production;
USE DB production;
DEFINE TABLE person SCHEMAFULL;
-- Create a record (table:id is the record's unique identity)
CREATE person:tobie SET name = 'Tobie', age = 33;
```
Namespaces are useful for multi-tenancy. Databases isolate application concerns within a namespace. Tables hold records. Every record has a globally unique identifier in the form `table:id`.
### Record IDs as First-Class Citizens
In SurrealDB, every record is uniquely identified by its record ID, which combines the table name and the record's identifier. Record IDs are not opaque surrogate keys; they are meaningful, typeable, and can be used directly in queries.
```surql
-- Various record ID forms
person:tobie -- string-based
person:100 -- integer-based
person:uuid() -- auto-generated UUID v7
person:ulid() -- auto-generated ULID (time-sortable)
person:rand() -- auto-generated random
-- Compound record IDs (encode composite keys directly in the ID)
temperature:['London', d'2026-02-19T10:00:00Z']
route:{ from: 'LAX', to: 'JFK' }
-- Record IDs ARE the foreign key system
-- No separate FK columns needed; just reference the record ID
CREATE article SET title = 'Hello', author = person:tobie;
-- Traverse the link directly
SELECT author.name FROM article;
```
Record links are not strings. They are typed references resolved by the engine. You can traverse them with dot notation, and they participate in graph queries.
### Schema Modes
SurrealDB supports three schema enforcement modes:
| Mode | Description | Use When |
|------|-------------|----------|
| `SCHEMAFULL` | Only explicitly defined fields are allowed. Attempts to set undefined fields are silently ignored. | Data integrity is critical. Well-known stable schema. |
| `SCHEMALESS` | Any field can be added without prior definition. Defined fields still enforce their types. | Rapid prototyping. Flexible or evolving schemas. |
| `TYPE ANY` | Table can hold both normal documents and serve as graph edge endpoints. | Rare. Multi-purpose tables. |
| `TYPE NORMAL` | Explicit marker for standard document tables. | Clarity in schema definitions. Default behavior. |
| `TYPE RELATION` | Dedicated graph edge table. Requires `IN` and `OUT` (or `FROM`/`TO`). | Graph relationships with enforced types. |
```surql
-- Schemafull: strict, safe, predictable
DEFINE TABLE person SCHEMAFULL;
DEFINE FIELD name ON TABLE person TYPE string;
DEFINE FIELD age ON TABLE person TYPE int;
-- Inserting an undefined field like 'foo' is silently dropped.
-- Schemaless: flexible, fast iteration
DEFINE TABLE event SCHEMALESS;
-- Any field can appear on any record, no definition needed.
-- You can still define fields for validation:
DEFINE FIELD timestamp ON TABLE event TYPE datetime DEFAULT time::now();
-- Type Relation: graph edge table with enforcement
DEFINE TABLE purchased TYPE RELATION IN person OUT product ENFORCED;
-- ENFORCED means RELATE x->purchased->y will fail
-- unless x is a person record and y is a product record.
```
---
## Document Model Patterns
### Flat Documents
The simplest pattern. Each record is a single-level key-value document.
```surql
DEFINE TABLE product SCHEMAFULL;
DEFINE FIELD name ON TABLE product TYPE string;
DEFINE FIELD sku ON TABLE product TYPE string;
DEFINE FIELD price ON TABLE product TYPE decimal;
DEFINE FIELD stock ON TABLE product TYPE int DEFAULT 0;
DEFINE FIELD active ON TABLE product TYPE bool DEFAULT true;
CREATE product:widget SET
name = 'Widget Pro',
sku = 'WDG-PRO-001',
price = 29.99dec,
stock = 150,
active = true;
```
### Nested Objects
SurrealDB supports arbitrary nesting. Define sub-fields using dot notation.
```surql
DEFINE TABLE person SCHEMAFULL;
DEFINE FIELD name ON TABLE person TYPE object;
DEFINE FIELD name.first ON TABLE person TYPE string;
DEFINE FIELD name.last ON TABLE person TYPE string;
DEFINE FIELD address ON TABLE person TYPE object;
DEFINE FIELD address.street ON TABLE person TYPE string;
DEFINE FIELD address.city ON TABLE person TYPE string;
DEFINE FIELD address.state ON TABLE person TYPE string;
DEFINE FIELD address.zip ON TABLE person TYPE string;
DEFINE FIELD address.country ON TABLE person TYPE string DEFAULT 'US';
CREATE person:tobie CONTENT {
name: { first: 'Tobie', last: 'Morgan Hitchcock' },
address: {
street: '123 Main St',
city: 'London',
state: 'England',
zip: 'EC1A 1BB',
country: 'UK'
}
};
-- Query nested fields directly
SELECT name.first, address.city FROM person;
```
### Arrays and Sets
```surql
DEFINE TABLE article SCHEMAFULL;
DEFINE FIELD title ON TABLE article TYPE string;
DEFINE FIELD tags ON TABLE article TYPE array<string>;
DEFINE FIELD categories ON TABLE article TYPE set<string>; -- unique elements only
DEFINE FIELD scores ON TABLE article TYPE array<int>;
CREATE article:intro SET
title = 'Getting Started',
tags = ['tutorial', 'beginner', 'surrealdb'],
categories = ['docs', 'tutorial'],
scores = [95, 87, 92];
-- Filter within arrays
SELECT tags[WHERE $value LIKE 'surreal%'] FROM article;
-- Array aggregation
SELECT math::mean(scores) AS avg_score FROM article;
```
### Optional Fields
Fields that may or may not be present use `option<T>`.
```surql
DEFINE TABLE person SCHEMAFULL;
DEFINE FIELD name ON TABLE person TYPE string;
DEFINE FIELD email ON TABLE person TYPE string;
DEFINE FIELD phone ON TABLE person TYPE option<string>; -- may be absent
DEFINE FIELD nickname ON TABLE person TYPE option<string>;
DEFINE FIELD bio ON TABLE person TYPE option<string>;
-- phone, nickname, bio can be omitted entirely
CREATE person:alice SET name = 'Alice', email = '[email protected]';
-- Or explicitly set to NONE
CREATE person:bob SET
name = 'Bob',
email = '[email protected]',
phone = NONE;
```
### Default Values
```surql
DEFINE TABLE order SCHEMAFULL;
DEFINE FIELD status ON TABLE order TYPE string DEFAULT 'pending';
DEFINE FIELD created_at ON TABLE order TYPE datetime DEFAULT time::now();
DEFINE FIELD priority ON TABLE order TYPE int DEFAULT 0;
DEFINE FIELD currency ON TABLE order TYPE string DEFAULT 'USD';
-- Defaults are applied automatically
CREATE order:1 SET total = 99.99;
-- Result: { id: order:1, total: 99.99, status: 'pending', created_at: ..., priority: 0, currency: 'USD' }
```
### Computed Fields
Fields derived from other fields. These are recalculated on every read or write.
```surql
DEFINE TABLE person SCHEMAFULL;
DEFINE FIELD name ON TABLE person TYPE object;
DEFINE FIELD name.first ON TABLE person TYPE string;
DEFINE FIELD name.last ON TABLE person TYPE string;
DEFINE FIELD full_name ON TABLE person VALUE string::concat(name.first, ' ', name.last);
DEFINE FIELD initials ON TABLE person VALUE string::concat(
string::slice(name.first, 0, 1),
string::slice(name.last, 0, 1)
);
DEFINE TABLE product SCHEMAFULL;
DEFINE FIELD price ON TABLE product TYPE decimal;
DEFINE FIELD tax_rate ON TABLE product TYPE decimal DEFAULT 0.08dec;
DEFINE FIELD total ON TABLE product VALUE price + (price * tax_rate);
-- READONLY computed field: cannot be set manually
DEFINE FIELD updated_at ON TABLE product VALUE time::now() READONLY;
```
### Schema Evolution Strategies
SurrealDB handles schema evolution differently than traditional RDBMS:
```surql
-- Strategy 1: Add new fields with defaults (non-breaking)
DEFINE FIELD new_field ON TABLE person TYPE option<string>;
-- Existing records automatically have new_field = NONE
-- Strategy 2: Rename via computed field (bridge period)
DEFINE FIELD display_name ON TABLE person VALUE
IF name.full != NONE { name.full }
ELSE { string::concat(name.first, ' ', name.last) };
-- Strategy 3: Backfill existing records
UPDATE person SET new_status = IF status = 'active' { 'enabled' } ELSE { 'disabled' };
-- Strategy 4: Use SCHEMALESS during migration, then switch to SCHEMAFULL
DEFINE TABLE OVERWRITE person SCHEMALESS;
-- ... migrate data ...
DEFINE TABLE OVERWRITE person SCHEMAFULL;
-- Strategy 5: Remove old field definition
REMOVE FIELD old_field ON TABLE person;
-- Existing data is not deleted but the field is no longer enforced
```
---
## Graph Model Patterns
### When to Use Graph Relationships vs Record Links
SurrealDB offers two ways to connect records:
**Record Links** (simple references):
- No metadata on the relationship itself
- One-directional by default
- Simpler queries with dot notation
- Use when the relationship is purely structural (e.g., `article.author`)
**Graph Edges** (via RELATE):
- Relationships are records themselves, stored in edge tables
- Can carry properties (timestamp, weight, role, etc.)
- Support bidirectional traversal with `->`, `<-`, `<->`
- Use when the relationship has meaning, attributes, or multiplicity
```surql
-- Record Link: simple, no relationship metadata
CREATE article SET title = 'Hello', author = person:tobie;
SELECT author.name FROM article;
-- Graph Edge: relationship has properties
RELATE person:tobie->wrote->article:hello SET
published_at = time::now(),
role = 'primary_author';
SELECT ->wrote->article FROM person:tobie;
-- Access edge properties
SELECT ->wrote.role, ->wrote->article.title FROM person:tobie;
```
### RELATE Syntax and Edge Tables
```surql
-- Define a typed edge table
DEFINE TABLE wrote TYPE RELATION IN person OUT article ENFORCED;
DEFINE FIELD published_at ON TABLE wrote TYPE datetime DEFAULT time::now();
DEFINE FIELD role ON TABLE wrote TYPE string DEFAULT 'author';
-- Create edges
RELATE person:tobie->wrote->article:surreal SET role = 'primary_author';
RELATE person:jaime->wrote->article:surreal SET role = 'co_author';
-- Edge records can be queried directly like any table
SELECT * FROM wrote;
SELECT * FROM wrote WHERE role = 'primary_author';
-- SET syntax for edge properties
RELATE person:alice->purchased->product:laptop SET
quantity = 1,
price = 1299.99,
purchased_at = time::now();
-- CONTENT syntax for edge properties
RELATE person:bob->reviewed->product:laptop CONTENT {
rating: 5,
text: 'Excellent product',
helpful_votes: 0
};
```
### Graph Traversal Patterns
```surql
-- Forward traversal: who did person:tobie write articles for?
SELECT ->wrote->article FROM person:tobie;
-- Backward traversal: who wrote article:surreal?
SELECT <-wrote<-person FROM article:surreal;
-- Multi-hop: find articles written by people that tobie knows
SELECT ->knows->person->wrote->article FROM person:tobie;
-- Bidirectional: all people connected to tobie via 'knows' (either direction)
SELECT <->knows<->person FROM person:tobie;
-- Filter on traversal
SELECT ->purchased->product WHERE price > 100 FROM person:tobie;
-- Select specific fields from traversed records
SELECT ->wrote->article.title AS articles FROM person:tobie;
-- Access edge properties during traversal
SELECT ->reviewed.rating, ->reviewed->product.name FROM person;
-- Count relationships
SELECT count(->wrote->article) AS article_count FROM person;
-- Traverse with conditions on intermediate edges
SELECT ->purchased[WHERE quantity > 1]->product FROM person:tobie;
```
### Recursive Graph Queries
```surql
-- Setup: family tree
CREATE person:alice, person:bob, person:charlie, person:diana;
RELATE person:bob->parent_of->person:alice;
RELATE person:charlie->parent_of->person:bob;
RELATE person:diana->parent_of->person:bob;
-- Parents (1 hop)
SELECT <-parent_of<-person AS parents FROM person:alice;
-- Grandparents (2 hops)
SELECT <-parent_of<-person<-parent_of<-person AS grandparents FROM person:alice;
-- All ancestors (chain traversals)
SELECT
<-parent_of<-person AS parents,
<-parent_of<-person<-parent_of<-person AS grandparents
FROM person:alice;
-- Descendants
SELECT ->parent_of->person AS children FROM person:charlie;
SELECT ->parent_of->person->parent_of->person AS grandchildren FROM person:charlie;
```
### Social Network Pattern
```surql
-- Tables
DEFINE TABLE user SCHEMAFULL;
DEFINE FIELD username ON TABLE user TYPE string;
DEFINE FIELD display_name ON TABLE user TYPE string;
DEFINE FIELD bio ON TABLE user TYPE option<string>;
DEFINE FIELD joined_at ON TABLE user TYPE datetime DEFAULT time::now();
DEFINE INDEX username_idx ON TABLE user COLUMNS username UNIQUE;
DEFINE TABLE follows TYPE RELATION IN user OUT user ENFORCED;
DEFINE FIELD created_at ON TABLE follows TYPE datetime DEFAULT time::now();
DEFINE TABLE post SCHEMAFULL;
DEFINE FIELD author ON TABLE post TYPE record<user>;
DEFINE FIELD content ON TABLE post TYPE string;
DEFINE FIELD created_at ON TABLE post TYPE datetime DEFAULT time::now();
DEFINE TABLE likes TYPE RELATION IN user OUT post ENFORCED;
DEFINE FIELD created_at ON TABLE likes TYPE datetime DEFAULT time::now();
-- Create users and relationships
CREATE user:alice SET username = 'alice', display_name = 'Alice';
CREATE user:bob SET username = 'bob', display_name = 'Bob';
RELATE user:alice->follows->user:bob;
-- Create a post
CREATE post:1 SET author = user:alice, content = 'Hello SurrealDB!';
RELATE user:bob->likes->post:1;
-- Feed query: posts from followed users, ordered by time
SELECT
->follows->user->wrote->post.* AS feed
FROM user:alice
ORDER BY feed.created_at DESC;
-- Alternative feed using record links
SELECT * FROM post
WHERE author IN (SELECT VALUE ->follows->user FROM user:alice)
ORDER BY created_at DESC
LIMIT 20;
-- Mutual follows
SELECT ->follows->user INTERSECT <-follows<-user AS mutual FROM user:alice;
-- Followers count
SELECT count(<-follows<-user) AS follower_count FROM user:alice;
-- Recommendations: friends of friends not already followed
SELECT ->follows->user->follows->user AS fof FROM user:alice
WHERE fof NOT IN (SELECT VALUE ->follows->user FROM user:alice)
AND fof != user:alice;
```
### Knowledge Graph Pattern
```surql
-- Entity tables
DEFINE TABLE concept SCHEMAFULL;
DEFINE FIELD name ON TABLE concept TYPE string;
DEFINE FIELD description ON TABLE concept TYPE option<string>;
DEFINE FIELD embedding ON TABLE concept TYPE array<float>;
-- Relationship edge tables
DEFINE TABLE is_a TYPE RELATION IN concept OUT concept;
DEFINE TABLE part_of TYPE RELATION IN concept OUT concept;
DEFINE TABLE related_to TYPE RELATION IN concept OUT concept;
DEFINE FIELD weight ON TABLE related_to TYPE float DEFAULT 1.0;
-- Vector index for semantic search
DEFINE INDEX concept_embedding ON TABLE concept FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- Build the knowledge graph
CREATE concept:database SET name = 'Database', description = 'A system for storing data';
CREATE concept:nosql SET name = 'NoSQL', description = 'Non-relational database';
CREATE concept:surrealdb SET name = 'SurrealDB', description = 'Multi-model database';
RELATE concept:nosql->is_a->concept:database;
RELATE concept:surrealdb->is_a->concept:nosql;
-- Traverse the ontology
SELECT ->is_a->concept.name AS parent_concepts FROM concept:surrealdb;
SELECT <-is_a<-concept.name AS child_concepts FROM concept:database;
```
### Supply Chain / Logistics Pattern
```surql
DEFINE TABLE facility SCHEMAFULL;
DEFINE FIELD name ON TABLE facility TYPE string;
DEFINE FIELD type ON TABLE facility TYPE string; -- 'warehouse', 'factory', 'store'
DEFINE FIELD location ON TABLE facility TYPE geometry<point>;
DEFINE FIELD capacity ON TABLE facility TYPE int;
DEFINE TABLE ships_to TYPE RELATION IN facility OUT facility ENFORCED;
DEFINE FIELD transit_days ON TABLE ships_to TYPE int;
DEFINE FIELD cost_per_unit ON TABLE ships_to TYPE decimal;
DEFINE FIELD active ON TABLE ships_to TYPE bool DEFAULT true;
-- Build supply chain network
CREATE facility:factory_1 SET name = 'Main Factory', type = 'factory',
location = (-73.9857, 40.7484), capacity = 10000;
CREATE facility:warehouse_east SET name = 'East Warehouse', type = 'warehouse',
location = (-71.0589, 42.3601), capacity = 5000;
CREATE facility:store_nyc SET name = 'NYC Store', type = 'store',
location = (-73.9857, 40.7484), capacity = 500;
RELATE facility:factory_1->ships_to->facility:warehouse_east SET
transit_days = 2, cost_per_unit = 0.50dec;
RELATE facility:warehouse_east->ships_to->facility:store_nyc SET
transit_days = 1, cost_per_unit = 0.25dec;
-- Find all facilities reachable from factory
SELECT ->ships_to->facility AS direct,
->ships_to->facility->ships_to->facility AS two_hop
FROM facility:factory_1;
-- Find cheapest path (manual approach)
SELECT ->ships_to.cost_per_unit AS leg1_cost,
->ships_to->facility->ships_to.cost_per_unit AS leg2_cost
FROM facility:factory_1;
-- Geospatial: find facilities within radius
SELECT * FROM facility
WHERE geo::distance(location, (-73.9857, 40.7484)) < 500000;
```
---
## Relational Patterns
### Foreign Key Equivalents (Record Links)
SurrealDB replaces traditional foreign keys with record links. A record link is a field whose value is a record ID, resolved automatically by the engine.
```surql
DEFINE TABLE author SCHEMAFULL;
DEFINE FIELD name ON TABLE author TYPE string;
DEFINE TABLE book SCHEMAFULL;
DEFINE FIELD title ON TABLE book TYPE string;
DEFINE FIELD author ON TABLE book TYPE record<author>; -- typed record link
CREATE author:tolkien SET name = 'J.R.R. Tolkien';
CREATE book:lotr SET title = 'The Lord of the Rings', author = author:tolkien;
-- Automatic resolution via dot notation
SELECT title, author.name FROM book;
-- Returns: [{ title: 'The Lord of the Rings', author: { name: 'J.R.R. Tolkien' } }]
-- Or use FETCH for explicit resolution
SELECT * FROM book FETCH author;
```
### One-to-One Relationships
```surql
DEFINE TABLE user SCHEMAFULL;
DEFINE FIELD username ON TABLE user TYPE string;
DEFINE TABLE profile SCHEMAFULL;
DEFINE FIELD user ON TABLE profile TYPE record<user>;
DEFINE FIELD bio ON TABLE profile TYPE string;
DEFINE FIELD avatar_url ON TABLE profile TYPE option<string>;
DEFINE INDEX user_idx ON TABLE profile COLUMNS user UNIQUE; -- enforces 1:1
CREATE user:alice SET username = 'alice';
CREATE profile:alice_profile SET user = user:alice, bio = 'Developer';
-- Query from user to profile
SELECT *, (SELECT * FROM profile WHERE user = $parent.id LIMIT 1) AS profile FROM user;
-- Or embed the profile directly in the user (simpler for 1:1)
DEFINE TABLE user SCHEMAFULL;
DEFINE FIELD username ON TABLE user TYPE string;
DEFINE FIELD profile ON TABLE user TYPE object;
DEFINE FIELD profile.bio ON TABLE user TYPE string;
DEFINE FIELD profile.avatar_url ON TABLE user TYPE option<string>;
```
### One-to-Many Relationships
```surql
-- Approach 1: Record link from child to parent (standard relational)
DEFINE TABLE department SCHEMAFULL;
DEFINE FIELD name ON TABLE department TYPE string;
DEFINE TABLE employee SCHEMAFULL;
DEFINE FIELD name ON TABLE employee TYPE string;
DEFINE FIELD department ON TABLE employee TYPE record<department>;
CREATE department:engineering SET name = 'Engineering';
CREATE employee:alice SET name = 'Alice', department = department:engineering;
CREATE employee:bob SET name = 'Bob', department = department:engineering;
-- Get employees in a department
SELECT * FROM employee WHERE department = department:engineering;
-- Get department with employees (subquery)
SELECT *, (SELECT * FROM employee WHERE department = $parent.id) AS employees
FROM department:engineering;
-- Approach 2: Array of record links from parent (embedded references)
DEFINE TABLE team SCHEMAFULL;
DEFINE FIELD name ON TABLE team TYPE string;
DEFINE FIELD members ON TABLE team TYPE array<record<employee>>;
CREATE team:alpha SET name = 'Alpha', members = [employee:alice, employee:bob];
SELECT members.*.name FROM team:alpha;
```
### Many-to-Many Relationships
```surql
-- Approach 1: Graph edges (preferred when relationship has properties)
DEFINE TABLE student SCHEMAFULL;
DEFINE FIELD name ON TABLE student TYPE string;
DEFINE TABLE course SCHEMAFULL;
DEFINE FIELD title ON TABLE course TYPE string;
DEFINE TABLE enrolled_in TYPE RELATION IN student OUT course ENFORCED;
DEFINE FIELD enrolled_at ON TABLE enrolled_in TYPE datetime DEFAULT time::now();
DEFINE FIELD grade ON TABLE enrolled_in TYPE option<string>;
RELATE student:alice->enrolled_in->course:math SET enrolled_at = time::now();
RELATE student:alice->enrolled_in->course:physics;
RELATE student:bob->enrolled_in->course:math;
-- Students in a course
SELECT <-enrolled_in<-student.name FROM course:math;
-- Courses for a student
SELECT ->enrolled_in->course.title FROM student:alice;
-- With edge properties
SELECT ->enrolled_in.grade, ->enrolled_in->course.title FROM student:alice;
-- Approach 2: Array of record links (simpler, no edge properties)
DEFINE TABLE article SCHEMAFULL;
DEFINE FIELD title ON TABLE article TYPE string;
DEFINE FIELD tags ON TABLE article TYPE array<record<tag>>;
DEFINE TABLE tag SCHEMAFULL;
DEFINE FIELD name ON TABLE tag TYPE string;
CREATE tag:rust SET name = 'Rust';
CREATE tag:database SET name = 'Database';
CREATE article:1 SET title = 'SurrealDB Internals', tags = [tag:rust, tag:database];
-- Articles with a specific tag
SELECT * FROM article WHERE tags CONTAINS tag:rust;
```
### Normalization vs Denormalization Trade-offs
**When to normalize (use record links / separate tables):**
- Data is updated independently (e.g., user profile changes should not require updating every order)
- Data integrity is critical (single source of truth)
- Many-to-many relationships exist
- Storage efficiency matters (avoid duplicating large objects)
**When to denormalize (embed data):**
- Data is always read together (e.g., order with line items)
- Read performance is critical and writes are infrequent
- The embedded data rarely changes
- The embedded data is specific to the parent (not shared)
```surql
-- Normalized: separate tables with record links
DEFINE TABLE order SCHEMAFULL;
DEFINE FIELD customer ON TABLE order TYPE record<customer>;
DEFINE FIELD total ON TABLE order TYPE decimal;
DEFINE TABLE order_item SCHEMAFULL;
DEFINE FIELD order ON TABLE order_item TYPE record<order>;
DEFINE FIELD product ON TABLE order_item TYPE record<product>;
DEFINE FIELD quantity ON TABLE order_item TYPE int;
DEFINE FIELD price ON TABLE order_item TYPE decimal;
-- Denormalized: embedded line items
DEFINE TABLE order SCHEMAFULL;
DEFINE FIELD customer ON TABLE order TYPE record<customer>;
DEFINE FIELD total ON TABLE order TYPE decimal;
DEFINE FIELD items ON TABLE order TYPE array<object>;
-- Each item: { product: record<product>, quantity: int, price: decimal, name: string }
CREATE order:1 CONTENT {
customer: customer:alice,
total: 149.97dec,
items: [
{ product: product:widget, quantity: 3, price: 49.99dec, name: 'Widget' }
]
};
```
### When to Embed vs Link
| Criterion | Embed | Link |
|-----------|-------|------|
| Data read together? | Yes | Sometimes |
| Data changes independently? | No | Yes |
| Data shared across records? | No | Yes |
| Relationship has metadata? | No | Use RELATE |
| Unbounded growth? | No (keep arrays small) | Yes (separate table) |
| Needs independent queries? | No | Yes |
---
## Vector Data Patterns
### Storing Embeddings
```surql
DEFINE TABLE document SCHEMAFULL;
DEFINE FIELD title ON TABLE document TYPE string;
DEFINE FIELD content ON TABLE document TYPE string;
DEFINE FIELD embedding ON TABLE document TYPE array<float>;
DEFINE FIELD source ON TABLE document TYPE option<string>;
DEFINE FIELD created_at ON TABLE document TYPE datetime DEFAULT time::now();
-- Store a document with its embedding (e.g., from OpenAI text-embedding-3-large)
CREATE document:doc1 SET
title = 'Introduction to SurrealDB',
content = 'SurrealDB is a multi-model database...',
embedding = [0.0123, -0.0456, 0.0789, ...]; -- 3072-dimensional vector
```
### HNSW Index Configuration
HNSW (Hierarchical Navigable Small World) is the recommended index for approximate nearest-neighbor vector search. It provides fast queries with tunable accuracy.
```surql
-- Basic HNSW index
DEFINE INDEX doc_embedding ON TABLE document FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- HNSW with full configuration
DEFINE INDEX doc_embedding ON TABLE document FIELDS embedding
HNSW DIMENSION 3072 -- must match your embedding model's output dimension
DIST COSINE -- distance metric
TYPE F32 -- element type (F32, F64, I16, I32, I64)
EFC 200 -- ef_construction: higher = better recall, slower build
M 16; -- max connections per layer: higher = better recall, more memory
-- Query using vector search with the <|K,EF|> operator
SELECT id, title,
vector::similarity::cosine(embedding, $query_embedding) AS similarity
FROM document
WHERE embedding <|10,100|> $query_embedding
ORDER BY similarity DESC;
-- <|10,100|> means: return 10 nearest neighbors, using ef_search=100
```
**HNSW parameter guidance:**
| Parameter | Default | Low (fast, less accurate) | High (slower, more accurate) |
|-----------|---------|---------------------------|------------------------------|
| `EFC` | 150 | 100 | 300-500 |
| `M` | 12 | 8 | 24-48 |
| ef_search (in query) | 40 | 20 | 200-500 |
**Distance metrics**: `COSINE` (most common for text/image embeddings), `EUCLIDEAN` (L2 distance), `MANHATTAN` (L1 distance).
### MTREE Index
MTREE provides exact nearest-neighbor results (no approximation) but is slower than HNSW for large datasets.
```surql
DEFINE INDEX doc_embedding ON TABLE document FIELDS embedding
MTREE DIMENSION 1536 DIST COSINE;
-- MTREE with capacity tuning
DEFINE INDEX doc_embedding ON TABLE document FIELDS embedding
MTREE DIMENSION 1536 DIST EUCLIDEAN CAPACITY 40;
-- Query MTREE indexes similarly
SELECT id, title,
vector::similarity::cosine(embedding, $query_embedding) AS similarity
FROM document
WHERE embedding <|10|> $query_embedding
ORDER BY similarity DESC;
```
### RAG (Retrieval-Augmented Generation) Pattern
```surql
-- Schema for a RAG system
DEFINE TABLE knowledge_chunk SCHEMAFULL;
DEFINE FIELD content ON TABLE knowledge_chunk TYPE string;
DEFINE FIELD embedding ON TABLE knowledge_chunk TYPE array<float>;
DEFINE FIELD source_doc ON TABLE knowledge_chunk TYPE record<source_document>;
DEFINE FIELD chunk_index ON TABLE knowledge_chunk TYPE int;
DEFINE FIELD token_count ON TABLE knowledge_chunk TYPE int;
DEFINE FIELD metadata ON TABLE knowledge_chunk FLEXIBLE TYPE object;
DEFINE TABLE source_document SCHEMAFULL;
DEFINE FIELD title ON TABLE source_document TYPE string;
DEFINE FIELD url ON TABLE source_document TYPE option<string>;
DEFINE FIELD ingested_at ON TABLE source_document TYPE datetime DEFAULT time::now();
-- Vector index for similarity search
DEFINE INDEX chunk_embedding ON TABLE knowledge_chunk FIELDS embedding
HNSW DIMENSION 3072 DIST COSINE EFC 200 M 16;
-- Full-text index for keyword fallback
DEFINE ANALYZER english_analyzer TOKENIZERS blank, class
FILTERS ascii, lowercase, snowball(english);
DEFINE INDEX chunk_content_ft ON TABLE knowledge_chunk COLUMNS content
SEARCH ANALYZER english_analyzer BM25(1.2, 0.75) HIGHLIGHTS;
-- Ingest a chunk
CREATE knowledge_chunk SET
content = 'SurrealDB supports HNSW indexes for vector similarity search...',
embedding = $embedding_vector,
source_doc = source_document:doc1,
chunk_index = 0,
token_count = 45,
metadata = { section: 'indexes', heading: 'Vector Indexes' };
-- Semantic retrieval (vector search)
LET $query_emb = $query_embedding;
SELECT content, source_doc.title,
vector::similarity::cosine(embedding, $query_emb) AS score
FROM knowledge_chunk
WHERE embedding <|5,150|> $query_emb
ORDER BY score DESC;
```
### Hybrid Search (Vector + Metadata Filtering)
Combine vector similarity with structured metadata filters for precise retrieval.
```surql
-- Hybrid query: semantic similarity + metadata filter
LET $query_emb = $query_embedding;
SELECT content, source_doc.title,
vector::similarity::cosine(embedding, $query_emb) AS score
FROM knowledge_chunk
WHERE embedding <|20,200|> $query_emb
AND metadata.section = 'security'
AND token_count < 500
ORDER BY score DESC
LIMIT 5;
-- Hybrid: vector search + full-text search scoring
SELECT content,
vector::similarity::cosine(embedding, $query_emb) AS vec_score,
search::score(1) AS text_score
FROM knowledge_chunk
WHERE embedding <|20|> $query_emb
OR content @1@ 'authentication security'
ORDER BY (vec_score * 0.7 + text_score * 0.3) DESC
LIMIT 10;
```
### Embedding Dimension Choices
| Model | Dimensions | Use Case |
|-------|-----------|----------|
| OpenAI text-embedding-3-small | 1536 | Cost-effective general purpose |
| OpenAI text-embedding-3-large | 3072 | High-accuracy retrieval |
| Cohere embed-v3 | 1024 | Multilingual search |
| BGE-large | 1024 | Open-source alternative |
| Nomic embed-text | 768 | Lightweight open-source |
Always set `DIMENSION` in the index to match your embedding model's output dimension exactly.
---
## Time-Series Patterns
### Timestamp-Ordered Records
```surql
-- Use compound record IDs for natural time-ordering
DEFINE TABLE sensor_reading SCHEMAFULL;
DEFINE FIELD sensor_id ON TABLE sensor_reading TYPE string;
DEFINE FIELD temperature ON TABLE sensor_reading TYPE float;
DEFINE FIELD humidity ON TABLE sensor_reading TYPE float;
DEFINE FIELD recorded_at ON TABLE sensor_reading TYPE datetime DEFAULT time::now();
DEFINE INDEX time_idx ON TABLE sensor_reading COLUMNS recorded_at;
DEFINE INDEX sensor_time_idx ON TABLE sensor_reading COLUMNS sensor_id, recorded_at;
-- Use compound IDs to encode time naturally
CREATE sensor_reading:['sensor_1', d'2026-02-19T10:00:00Z'] SET
sensor_id = 'sensor_1',
temperature = 22.5,
humidity = 45.0;
-- Range query by time
SELECT * FROM sensor_reading
WHERE recorded_at >= d'2026-02-19T00:00:00Z'
AND recorded_at < d'2026-02-20T00:00:00Z'
ORDER BY recorded_at ASC;
-- Latest reading per sensor
SELECT * FROM sensor_reading
WHERE sensor_id = 'sensor_1'
ORDER BY recorded_at DESC
LIMIT 1;
```
### Aggregated Views
```surql
-- Auto-aggregated hourly view
DEFINE TABLE hourly_sensor_avg AS
SELECT
sensor_id,
time::group(recorded_at, 'hour') AS hour,
math::mean(temperature) AS avg_temp,
math::min(temperature) AS min_temp,
math::max(temperature) AS max_temp,
math::mean(humidity) AS avg_humidity,
count() AS reading_count
FROM sensor_reading
GROUP BY sensor_id, time::group(recorded_at, 'hour');
-- Query the aggregated view
SELECT * FROM hourly_sensor_avg
WHERE sensor_id = 'sensor_1'
ORDER BY hour DESC
LIMIT 24;
-- Daily aggregation view
DEFINE TABLE daily_sensor_stats AS
SELECT
sensor_id,
time::group(recorded_at, 'day') AS day,
math::mean(temperature) AS avg_temp,
math::stddev(temperature) AS temp_stddev,
count() AS reading_count
FROM sensor_reading
GROUP BY sensor_id, time::group(recorded_at, 'day');
```
### Retention Policies
SurrealDB does not have built-in TTL. Use scheduled cleanup or the DROP table option:
```surql
-- Approach 1: Periodic cleanup
DELETE sensor_reading WHERE recorded_at < time::now() - 90d;
-- Approach 2: DROP table (records are deleted as soon as written -- write-only audit)
DEFINE TABLE ephemeral_log DROP;
-- Records vanish after being processed by events
-- Approach 3: Event-driven cleanup
DEFINE EVENT cleanup ON TABLE sensor_reading
WHEN $event = "CREATE"
THEN {
DELETE sensor_reading WHERE recorded_at < time::now() - 30d LIMIT 100;
};
```
### IoT Data Pattern
```surql
DEFINE TABLE device SCHEMAFULL;
DEFINE FIELD name ON TABLE device TYPE string;
DEFINE FIELD type ON TABLE device TYPE string;
DEFINE FIELD location ON TABLE device TYPE geometry<point>;
DEFINE FIELD status ON TABLE device TYPE string DEFAULT 'active';
DEFINE FIELD last_heartbeat ON TABLE device TYPE option<datetime>;
DEFINE TABLE telemetry SCHEMAFULL;
DEFINE FIELD device ON TABLE telemetry TYPE record<device>;
DEFINE FIELD payload ON TABLE telemetry FLEXIBLE TYPE object;
DEFINE FIELD recorded_at ON TABLE telemetry TYPE datetime DEFAULT time::now();
DEFINE INDEX telemetry_device_time ON TABLE telemetry COLUMNS device, recorded_at;
-- Event: update device heartbeat on new telemetry
DEFINE EVENT heartbeat ON TABLE telemetry WHEN $event = "CREATE" THEN {
UPDATE $after.device SET last_heartbeat = time::now();
};
-- Ingest telemetry
CREATE telemetry SET
device = device:sensor_42,
payload = { temperature: 22.5, battery: 85, signal: -67 };
-- Find offline devices (no heartbeat in 5 minutes)
SELECT * FROM device
WHERE last_heartbeat < time::now() - 5m
OR last_heartbeat IS NONE;
```
### Event Sourcing Pattern
```surql
-- Immutable event log
DEFINE TABLE account_event SCHEMAFULL;
DEFINE FIELD account ON TABLE account_event TYPE record<account>;
DEFINE FIELD type ON TABLE account_event TYPE string;
DEFINE FIELD amount ON TABLE account_event TYPE decimal;
DEFINE FIELD balance_after ON TABLE account_event TYPE decimal;
DEFINE FIELD metadata ON TABLE account_event FLEXIBLE TYPE object;
DEFINE FIELD occurred_at ON TABLE account_event TYPE datetime DEFAULT time::now();
DEFINE INDEX event_account_time ON TABLE account_event COLUMNS account, occurred_at;
-- Materialized current state
DEFINE TABLE account SCHEMAFULL;
DEFINE FIELD name ON TABLE account TYPE string;
DEFINE FIELD balance ON TABLE account TYPE decimal DEFAULT 0dec;
-- Record an event and update state in a transaction
BEGIN TRANSACTION;
LET $acct = (SELECT * FROM ONLY account:alice);
LET $new_balance = $acct.balance + 100dec;
UPDATE account:alice SET balance = $new_balance;
CREATE account_event SET
account = account:alice,
type = 'deposit',
amount = 100dec,
balance_after = $new_balance;
COMMIT TRANSACTION;
-- Replay events to reconstruct state
SELECT type, amount, balance_after, occurred_at
FROM account_event
WHERE account = account:alice
ORDER BY occurred_at ASC;
```
---
## Geospatial Patterns
### GeoJSON Storage
SurrealDB stores geospatial data as GeoJSON-compatible geometry types.
```surql
DEFINE TABLE place SCHEMAFULL;
DEFINE FIELD name ON TABLE place TYPE string;
DEFINE FIELD location ON TABLE place TYPE geometry<point>;
DEFINE FIELD boundary ON TABLE place TYPE option<geometry<polygon>>;
-- Point (longitude, latitude) -- NOTE: longitude first, per GeoJSON spec
CREATE place:nyc SET
name = 'New York City',
location = (-73.935242, 40.730610);
-- Full GeoJSON syntax
CREATE place:london SET
name = 'London',
location = {
type: 'Point',
coordinates: [-0.1278, 51.5074]
};
-- Polygon
CREATE place:central_park SET
name = 'Central Park',
boundary = {
type: 'Polygon',
coordinates: [[
[-73.981, 40.768], [-73.958, 40.800],
[-73.949, 40.797], [-73.973, 40.764],
[-73.981, 40.768]
]]
};
```
### Proximity Queries
```surql
-- Find places within 10km of a point
SELECT * FROM place
WHERE geo::distance(location, (-73.935242, 40.730610)) < 10000;
-- Find nearest places, ordered by distance
SELECT *, geo::distance(location, (-73.935242, 40.730610)) AS distance
FROM place
ORDER BY distance ASC
LIMIT 10;
-- Find places within a bounding polygon
SELECT * FROM place
WHERE location INSIDE {
type: 'Polygon',
coordinates: [[
[-74.0, 40.7], [-73.9, 40.7],
[-73.9, 40.8], [-74.0, 40.8],
[-74.0, 40.7]
]]
};
```
### Geofencing Pattern
```surql
DEFINE TABLE geofence SCHEMAFULL;
DEFINE FIELD name ON TABLE geofence TYPE string;
DEFINE FIELD boundary ON TABLE geofence TYPE geometry<polygon>;
DEFINE FIELD alert_type ON TABLE geofence TYPE string; -- 'enter', 'exit', 'both'
DEFINE TABLE device_location SCHEMAFULL;
DEFINE FIELD device ON TABLE device_location TYPE record<device>;
DEFINE FIELD position ON TABLE device_location TYPE geometry<point>;
DEFINE FIELD timestamp ON TABLE device_location TYPE datetime DEFAULT time::now();
-- Check if a device is inside any geofence
SELECT * FROM geofence
WHERE position INSIDE boundary;
-- Event-based geofence checking
DEFINE EVENT geofence_check ON TABLE device_location
WHEN $event = "CREATE"
THEN {
LET $inside = (
SELECT id, name FROM geofence
WHERE $after.position INSIDE boundary
);
IF array::len($inside) > 0 {
CREATE geofence_alert SET
device = $after.device,
geofences = $inside,
position = $after.position,
triggered_at = time::now();
};
};
```
### Location-Based Service Pattern
```surql
DEFINE TABLE restaurant SCHEMAFULL;
DEFINE FIELD name ON TABLE restaurant TYPE string;
DEFINE FIELD cuisine ON TABLE restaurant TYPE string;
DEFINE FIELD location ON TABLE restaurant TYPE geometry<point>;
DEFINE FIELD rating ON TABLE restaurant TYPE float;
DEFINE FIELD price_level ON TABLE restaurant TYPE int; -- 1-4
-- Find nearby restaurants of a specific cuisine
SELECT name, cuisine, rating,
geo::distance(location, $user_location) AS distance
FROM restaurant
WHERE cuisine = 'Italian'
AND geo::distance(location, $user_location) < 5000
ORDER BY rating DESC
LIMIT 20;
-- Bearing and distance for navigation
SELECT name,
geo::distance(location, $user_location) AS distance_m,
geo::bearing($user_location, location) AS bearing_deg
FROM restaurant
ORDER BY distance_m ASC
LIMIT 5;
```
---
## Full-Text Search Patterns
### Analyzer Configuration
Analyzers define how text is tokenized and filtered for search indexes.
```surql
-- ASCII + lowercase analyzer (good default for English)
DEFINE ANALYZER ascii_lower TOKENIZERS blank, class FILTERS ascii, lowercase;
-- English stemming analyzer (reduces words to root form)
DEFINE ANALYZER english TOKENIZERS blank, class FILTERS ascii, lowercase, snowball(english);
-- N-gram analyzer (substring matching, good for autocomplete)
DEFINE ANALYZER autocomplete TOKENIZERS blank FILTERS lowercase, ngram(2, 8);
-- Edge n-gram (prefix matching)
DEFINE ANALYZER prefix TOKENIZERS blank FILTERS lowercase, edgengram(1, 10);
-- Code search analyzer (splits camelCase and snake_case)
DEFINE ANALYZER code TOKENIZERS camel, blank, class FILTERS lowercase;
-- Minimal analyzer (full word match only)
DEFINE ANALYZER exact TOKENIZERS blank FILTERS lowercase;
```
### Search Index Creation
```surql
-- Basic search index
DEFINE INDEX article_search ON TABLE article COLUMNS title, content
SEARCH ANALYZER ascii_lower BM25;
-- Search index with custom BM25 parameters
DEFINE INDEX article_search ON TABLE article COLUMNS content
SEARCH ANALYZER english BM25(1.2, 0.75);
-- Search index with highlights support
DEFINE INDEX article_search ON TABLE article COLUMNS content
SEARCH ANALYZER english BM25(1.2, 0.75) HIGHLIGHTS;
-- Separate indexes for different fields (independent scoring)
DEFINE INDEX title_search ON TABLE article COLUMNS title
SEARCH ANALYZER english BM25;
DEFINE INDEX content_search ON TABLE article COLUMNS content
SEARCH ANALYZER english BM25 HIGHLIGHTS;
```
### Scoring and Ranking
```surql
-- Basic search with BM25 scoring
SELECT id, title,
search::score(1) AS relevance
FROM article
WHERE content @1@ 'SurrealDB multi-model database'
ORDER BY relevance DESC
LIMIT 10;
-- Multi-field scoring with weighted combination
SELECT id, title,
search::score(1) AS title_score,
search::score(2) AS content_score,
(search::score(1) * 2.0 + search::score(2)) AS combined_score
FROM article
WHERE title @1@ 'SurrealDB' OR content @2@ 'SurrealDB'
ORDER BY combined_score DESC;
-- Highlighted results
SELECT id, title,
search::highlight('<mark>', '</mark>', 1) AS highlighted_content,
search::score(1) AS score
FROM article
WHERE content @1@ 'vector search'
ORDER BY score DESC;
```
### Faceted Search Pattern
```surql
-- Product search with faceted filtering
DEFINE TABLE product SCHEMAFULL;
DEFINE FIELD name ON TABLE product TYPE string;
DEFINE FIELD description ON TABLE product TYPE string;
DEFINE FIELD category ON TABLE product TYPE string;
DEFINE FIELD brand ON TABLE product TYPE string;
DEFINE FIELD price ON TABLE product TYPE decimal;
DEFINE FIELD in_stock ON TABLE product TYPE bool;
DEFINE ANALYZER product_search TOKENIZERS blank, class FILTERS ascii, lowercase, snowball(english);
DEFINE INDEX product_name_ft ON TABLE product COLUMNS name SEARCH ANALYZER product_search BM25;
DEFINE INDEX product_desc_ft ON TABLE product COLUMNS description SEARCH ANALYZER product_search BM25 HIGHLIGHTS;
-- Faceted search: text search + structured filters
SELECT id, name, price, category, brand,
search::score(1) AS relevance
FROM product
WHERE description @1@ 'wireless bluetooth headphones'
AND category = 'electronics'
AND price >= 50 AND price <= 200
AND in_stock = true
ORDER BY relevance DESC
LIMIT 20;
-- Category facet counts after search
SELECT category, count() AS count
FROM product
WHERE description @1@ 'wireless bluetooth headphones'
GROUP BY category
ORDER BY count DESC;
```
### Multilingual Search
```surql
-- Per-language analyzers
DEFINE ANALYZER english_search TOKENIZERS blank, class FILTERS ascii, lowercase, snowball(english);
DEFINE ANALYZER french_search TOKENIZERS blank, class FILTERS ascii, lowercase, snowball(french);
DEFINE ANALYZER german_search TOKENIZERS blank, class FILTERS ascii, lowercase, snowball(german);
-- Multi-language content table
DEFINE TABLE content SCHEMAFULL;
DEFINE FIELD title ON TABLE content TYPE string;
DEFINE FIELD body_en ON TABLE content TYPE option<string>;
DEFINE FIELD body_fr ON TABLE content TYPE option<string>;
DEFINE FIELD body_de ON TABLE content TYPE option<string>;
DEFINE FIELD language ON TABLE content TYPE string;
-- Per-language search indexes
DEFINE INDEX content_en_ft ON TABLE content COLUMNS body_en SEARCH ANALYZER english_search BM25;
DEFINE INDEX content_fr_ft ON TABLE content COLUMNS body_fr SEARCH ANALYZER french_search BM25;
DEFINE INDEX content_de_ft ON TABLE content COLUMNS body_de SEARCH ANALYZER german_search BM25;
-- Search in a specific language
SELECT * FROM content WHERE body_en @1@ 'database performance' ORDER BY search::score(1) DESC;
SELECT * FROM content WHERE body_fr @1@ 'performance de base de donnees' ORDER BY search::score(1) DESC;
```
---
## Multi-Model Combinations
### Document + Graph (Social Network with Rich Profiles)
```surql
-- Rich user profiles (document model)
DEFINE TABLE user SCHEMAFULL;
DEFINE FIELD username ON TABLE user TYPE string;
DEFINE FIELD profile ON TABLE user TYPE object;
DEFINE FIELD profile.display_name ON TABLE user TYPE string;
DEFINE FIELD profile.bio ON TABLE user TYPE option<string>;
DEFINE FIELD profile.interests ON TABLE user TYPE array<string>;
DEFINE FIELD profile.location ON TABLE user TYPE option<geometry<point>>;
DEFINE FIELD joined_at ON TABLE user TYPE datetime DEFAULT time::now();
-- Graph relationships
DEFINE TABLE follows TYPE RELATION IN user OUT user ENFORCED;
DEFINE TABLE blocked TYPE RELATION IN user OUT user ENFORCED;
DEFINE TABLE member_of TYPE RELATION IN user OUT group ENFORCED;
DEFINE FIELD role ON TABLE member_of TYPE string DEFAULT 'member';
-- Query: nearby users with shared interests, excluding blocked
LET $me = user:alice;
LET $my_interests = (SELECT VALUE profile.interests FROM ONLY $me);
LET $my_blocked = (SELECT VALUE ->blocked->user FROM ONLY $me);
SELECT username, profile.display_name, profile.interests,
geo::distance(profile.location, $my_location) AS distance
FROM user
WHERE id != $me
AND id NOT IN $my_blocked
AND profile.interests CONTAINSANY $my_interests
AND geo::distance(profile.location, $my_location) < 50000
ORDER BY distance ASC
LIMIT 20;
```
### Vector + Document (Semantic Search Over Structured Data)
```surql
-- Products with embeddings for semantic search
DEFINE TABLE product SCHEMAFULL;
DEFINE FIELD name ON TABLE product TYPE string;
DEFINE FIELD description ON TABLE product TYPE string;
DEFINE FIELD category ON TABLE product TYPE string;
DEFINE FIELD price ON TABLE product TYPE decimal;
DEFINE FIELD embedding ON TABLE product TYPE array<float>;
DEFINE INDEX product_vec ON TABLE product FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
DEFINE INDEX product_category ON TABLE product COLUMNS category;
-- Semantic search: "comfortable office chair under $500"
LET $query_emb = $embedding_for_query;
SELECT name, description, price, category,
vector::similarity::cosine(embedding, $query_emb) AS relevance
FROM product
WHERE embedding <|20|> $query_emb
AND category = 'furniture'
AND price <= 500
ORDER BY relevance DESC
LIMIT 10;
```
### Graph + Vector (Knowledge Graph with Embeddings)
```surql
-- Concepts with embeddings and graph relationships
DEFINE TABLE concept SCHEMAFULL;
DEFINE FIELD name ON TABLE concept TYPE string;
DEFINE FIELD description ON TABLE concept TYPE string;
DEFINE FIELD embedding ON TABLE concept TYPE array<float>;
DEFINE INDEX concept_vec ON TABLE concept FIELDS embedding
HNSW DIMENSION 768 DIST COSINE;
DEFINE TABLE related_to TYPE RELATION IN concept OUT concept;
DEFINE FIELD weight ON TABLE related_to TYPE float DEFAULT 1.0;
DEFINE FIELD relation_type ON TABLE related_to TYPE string;
-- Find semantically similar concepts, then explore graph neighbors
LET $similar = (
SELECT id, name,
vector::similarity::cosine(embedding, $query_emb) AS score
FROM concept
WHERE embedding <|5|> $query_emb
ORDER BY score DESC
);
-- For each similar concept, get graph neighbors
SELECT name,
->related_to->concept.name AS related,
<-related_to<-concept.name AS referenced_by
FROM $similar;
```
### Time-Series + Geospatial (Fleet Tracking)
```surql
DEFINE TABLE vehicle SCHEMAFULL;
DEFINE FIELD name ON TABLE vehicle TYPE string;
DEFINE FIELD type ON TABLE vehicle TYPE string;
DEFINE FIELD status ON TABLE vehicle TYPE string DEFAULT 'active';
DEFINE TABLE position SCHEMAFULL;
DEFINE FIELD vehicle ON TABLE position TYPE record<vehicle>;
DEFINE FIELD location ON TABLE position TYPE geometry<point>;
DEFINE FIELD speed ON TABLE position TYPE float;
DEFINE FIELD heading ON TABLE position TYPE float;
DEFINE FIELD recorded_at ON TABLE position TYPE datetime DEFAULT time::now();
DEFINE INDEX pos_vehicle_time ON TABLE position COLUMNS vehicle, recorded_at;
-- Live tracking: event to update vehicle's current location
DEFINE EVENT update_vehicle_pos ON TABLE position WHEN $event = "CREATE" THEN {
UPDATE $after.vehicle SET
current_location = $after.location,
current_speed = $after.speed,
last_seen = $after.recorded_at;
};
-- Find vehicles near a location
SELECT *, geo::distance(current_location, $center) AS distance
FROM vehicle
WHERE status = 'active'
AND geo::distance(current_location, $center) < 10000
ORDER BY distance ASC;
-- Historical route for a vehicle
SELECT location, speed, recorded_at
FROM position
WHERE vehicle = vehicle:truck_1
AND recorded_at >= d'2026-02-19T00:00:00Z'
AND recorded_at < d'2026-02-20T00:00:00Z'
ORDER BY recorded_at ASC;
```
### Full-Text + Document (Content Management)
```surql
DEFINE TABLE article SCHEMAFULL;
DEFINE FIELD title ON TABLE article TYPE string;
DEFINE FIELD slug ON TABLE article TYPE string;
DEFINE FIELD content ON TABLE article TYPE string;
DEFINE FIELD author ON TABLE article TYPE record<user>;
DEFINE FIELD tags ON TABLE article TYPE array<string>;
DEFINE FIELD status ON TABLE article TYPE string DEFAULT 'draft';
DEFINE FIELD published_at ON TABLE article TYPE option<datetime>;
DEFINE FIELD views ON TABLE article TYPE int DEFAULT 0;
DEFINE ANALYZER content_analyzer TOKENIZERS blank, class
FILTERS ascii, lowercase, snowball(english);
DEFINE INDEX article_title_ft ON TABLE article COLUMNS title
SEARCH ANALYZER content_analyzer BM25;
DEFINE INDEX article_content_ft ON TABLE article COLUMNS content
SEARCH ANALYZER content_analyzer BM25 HIGHLIGHTS;
DEFINE INDEX article_slug ON TABLE article COLUMNS slug UNIQUE;
DEFINE INDEX article_status ON TABLE article COLUMNS status;
-- Full-text search on published articles
SELECT id, title, slug,
search::highlight('<b>', '</b>', 1) AS excerpt,
search::score(1) AS relevance,
author.name AS author_name,
tags, published_at, views
FROM article
WHERE content @1@ 'vector database performance'
AND status = 'published'
ORDER BY relevance DESC
LIMIT 10;
-- Related articles by shared tags
SELECT * FROM article
WHERE tags CONTAINSANY (SELECT VALUE tags FROM ONLY article:current)
AND id != article:current
AND status = 'published'
ORDER BY published_at DESC
LIMIT 5;
```
---
## Schema Design Best Practices
### Naming Conventions
- **Tables**: lowercase singular nouns (`person`, `article`, `order`)
- **Edge tables**: lowercase verb phrases (`wrote`, `purchased`, `follows`, `enrolled_in`)
- **Fields**: lowercase snake_case (`first_name`, `created_at`, `order_total`)
- **Indexes**: descriptive names with suffix pattern (`email_idx`, `name_search_ft`, `embedding_vec`)
- **Analyzers**: descriptive names (`english`, `autocomplete`, `code_search`)
- **Functions**: namespaced with `fn::` prefix (`fn::calculate_tax`, `fn::full_name`)
- **Parameters**: `$` prefix, lowercase snake_case (`$max_results`, `$app_name`)
### Record ID Strategies
| Strategy | Example | Use When |
|----------|---------|----------|
| Auto UUID | `person:uuid()` | General purpose, globally unique, no coordination needed |
| Auto ULID | `person:ulid()` | Need time-sortable IDs, useful for time-ordered data |
| Auto random | `person:rand()` | Simple random IDs, no time ordering needed |
| Meaningful string | `person:tobie` | Natural keys exist (usernames, slugs, codes) |
| Integer | `person:100` | Sequential data, legacy system compatibility |
| Compound array | `reading:['sensor_1', d'2026-02-19']` | Composite natural keys (sensor+time, route segments) |
| Compound object | `config:{ env: 'prod', key: 'db_url' }` | Named composite keys |
Best practice: prefer `uuid()` or `ulid()` for most tables. Use meaningful string IDs only when a natural key exists and is stable.
### Access Control Design
```surql
-- Layered access control pattern
-- 1. System users for service-to-service auth
DEFINE USER api_service ON DATABASE PASSWORD 'strong-service-password' ROLES EDITOR;
-- 2. Record access for end-user authentication
DEFINE ACCESS user_auth ON DATABASE TYPE RECORD
SIGNUP (
CREATE user SET
email = $email,
pass = crypto::argon2::generate($pass),
role = 'user',
created_at = time::now()
)
SIGNIN (
SELECT * FROM user
WHERE email = $email
AND crypto::argon2::compare(pass, $pass)
)
DURATION FOR TOKEN 15m, FOR SESSION 24h;
-- 3. Table-level permissions based on auth context
DEFINE TABLE post SCHEMALESS
PERMISSIONS
FOR select FULL
FOR create WHERE $auth.id != NONE
FOR update WHERE author = $auth.id
FOR delete WHERE author = $auth.id OR $auth.role = 'admin';
DEFINE TABLE user SCHEMAFULL
PERMISSIONS
FOR select WHERE id = $auth.id OR $auth.role = 'admin'
FOR update WHERE id = $auth.id
FOR delete WHERE $auth.role = 'admin';
-- 4. Field-level permissions for sensitive data
DEFINE FIELD email ON TABLE user TYPE string
PERMISSIONS
FOR select WHERE id = $auth.id OR $auth.role = 'admin';
DEFINE FIELD pass ON TABLE user TYPE string
PERMISSIONS
FOR select NONE; -- never expose password hash
```
### Anti-Patterns to Avoid
**1. Storing record IDs as strings instead of record links**
```surql
-- BAD: string reference
DEFINE FIELD author ON TABLE article TYPE string;
CREATE article SET author = 'person:tobie'; -- string, not a link
-- GOOD: typed record link
DEFINE FIELD author ON TABLE article TYPE record<person>;
CREATE article SET author = person:tobie; -- actual record reference
```
**2. Over-nesting instead of using separate tables**
```surql
-- BAD: deeply nested orders within user records
CREATE user SET orders = [
{ items: [...], total: 99.99, ... },
{ items: [...], total: 149.99, ... }
];
-- Cannot query orders independently, unbounded array growth
-- GOOD: separate order table with record link
CREATE order SET customer = user:alice, total = 99.99;
```
**3. Using RELATE for simple references**
```surql
-- BAD: graph edge for a simple belongs-to relationship
RELATE article:1->written_by->person:tobie;
-- GOOD: record link (no edge metadata needed)
CREATE article:1 SET author = person:tobie;
-- GOOD: graph edge when the relationship has properties
RELATE person:tobie->wrote->article:1 SET role = 'primary', drafted_at = time::now();
```
**4. Missing indexes on filtered fields**
```surql
-- BAD: no index on frequently filtered field
SELECT * FROM user WHERE email = '[email protected]'; -- full table scan
-- GOOD: index on the field
DEFINE INDEX email_idx ON TABLE user COLUMNS email UNIQUE;
```
**5. Using CONTENT when you mean MERGE**
```surql
-- BAD: CONTENT replaces the entire record
UPDATE person:tobie CONTENT { age: 34 };
-- Result: all other fields are gone
-- GOOD: SET or MERGE for partial updates
UPDATE person:tobie SET age = 34;
UPDATE person:tobie MERGE { age: 34 };
```
**6. Unbounded arrays**
```surql
-- BAD: comments stored in article array (can grow to millions)
DEFINE FIELD comments ON TABLE article TYPE array<object>;
-- GOOD: separate table with record link
DEFINE TABLE comment SCHEMAFULL;
DEFINE FIELD article ON TABLE comment TYPE record<article>;
DEFINE FIELD text ON TABLE comment TYPE string;
```
---
## Migration from Other Databases
### From PostgreSQL (Relational Mapping)
| PostgreSQL | SurrealDB |
|------------|-----------|
| Schema | Namespace + Database |
| Table | Table (SCHEMAFULL) |
| Row | Record |
| Primary key | Record ID (`table:id`) |
| Foreign key | Record link (`TYPE record<table>`) |
| JOIN | Record link traversal (dot notation) or subquery |
| Serial/sequence | `uuid()`, `ulid()`, or DEFINE SEQUENCE |
| ENUM | `ASSERT $value IN [...]` |
| JSON/JSONB column | Nested object fields or FLEXIBLE TYPE object |
| INDEX | DEFINE INDEX |
| VIEW | DEFINE TABLE ... AS SELECT |
| TRIGGER | DEFINE EVENT |
| FUNCTION | DEFINE FUNCTION |
| Row-level security | Table/field PERMISSIONS |
```surql
-- PostgreSQL:
-- CREATE TABLE users (
-- id SERIAL PRIMARY KEY,
-- email VARCHAR(255) UNIQUE NOT NULL,
-- name VARCHAR(100) NOT NULL,
-- created_at TIMESTAMP DEFAULT NOW()
-- );
-- CREATE TABLE posts (
-- id SERIAL PRIMARY KEY,
-- author_id INTEGER REFERENCES users(id),
-- title VARCHAR(255) NOT NULL,
-- body TEXT
-- );
-- SurrealDB equivalent:
DEFINE TABLE user SCHEMAFULL;
DEFINE FIELD email ON TABLE user TYPE string ASSERT string::is::email($value);
DEFINE FIELD name ON TABLE user TYPE string;
DEFINE FIELD created_at ON TABLE user TYPE datetime DEFAULT time::now();
DEFINE INDEX email_idx ON TABLE user COLUMNS email UNIQUE;
DEFINE TABLE post SCHEMAFULL;
DEFINE FIELD author ON TABLE post TYPE record<user>;
DEFINE FIELD title ON TABLE post TYPE string;
DEFINE FIELD body ON TABLE post TYPE option<string>;
-- PostgreSQL JOIN:
-- SELECT p.title, u.name FROM posts p JOIN users u ON p.author_id = u.id;
-- SurrealDB:
SELECT title, author.name FROM post;
```
### From MongoDB (Document Mapping)
| MongoDB | SurrealDB |
|---------|-----------|
| Database | Database |
| Collection | Table (SCHEMALESS) |
| Document | Record |
| `_id` (ObjectId) | Record ID (`table:id`) |
| Embedded document | Nested object |
| DBRef / manual reference | Record link (`TYPE record<table>`) |
| Aggregation pipeline | SELECT with GROUP BY, subqueries |
| Change streams | LIVE SELECT, CHANGEFEED |
| Atlas Search | DEFINE INDEX ... SEARCH ANALYZER |
| Atlas Vector Search | DEFINE INDEX ... HNSW |
```surql
-- MongoDB:
-- db.users.insertOne({
-- name: { first: "Tobie", last: "Morgan" },
-- email: "[email protected]",
-- tags: ["admin", "developer"],
-- address: { city: "London", country: "UK" }
-- });
-- SurrealDB (schemaless, closest to MongoDB):
DEFINE TABLE user SCHEMALESS;
CREATE user SET
name = { first: 'Tobie', last: 'Morgan' },
email = '[email protected]',
tags = ['admin', 'developer'],
address = { city: 'London', country: 'UK' };
-- MongoDB $lookup (JOIN equivalent):
-- db.orders.aggregate([{ $lookup: { from: "users", localField: "userId", ... } }])
-- SurrealDB: just use record links
SELECT *, customer.name FROM order FETCH customer;
```
### From Neo4j (Graph Mapping)
| Neo4j | SurrealDB |
|-------|-----------|
| Node | Record (in a table) |
| Node label | Table name |
| Relationship | Edge record (via RELATE) |
| Relationship type | Edge table name |
| `MATCH (n:Person)` | `SELECT * FROM person` |
| `CREATE (n:Person)` | `CREATE person SET ...` |
| `MATCH (a)-[:KNOWS]->(b)` | `SELECT ->knows->person FROM person:a` |
| `(a)-[:KNOWS*1..3]->(b)` | Chain traversals: `->knows->person->knows->person...` |
| Cypher | SurrealQL |
| APOC | DEFINE FUNCTION + built-in functions |
```surql
-- Neo4j Cypher:
-- CREATE (alice:Person {name: 'Alice'})-[:KNOWS {since: 2020}]->(bob:Person {name: 'Bob'})
-- SurrealDB:
CREATE person:alice SET name = 'Alice';
CREATE person:bob SET name = 'Bob';
RELATE person:alice->knows->person:bob SET since = 2020;
-- Neo4j:
-- MATCH (p:Person {name: 'Alice'})-[:KNOWS]->(friend)-[:KNOWS]->(fof)
-- RETURN fof.name
-- SurrealDB:
SELECT ->knows->person->knows->person.name AS friends_of_friends
FROM person:alice;
-- Neo4j:
-- MATCH (p:Person)-[r:KNOWS]->(other) WHERE r.since > 2019 RETURN p, other
-- SurrealDB:
SELECT <-knows[WHERE since > 2019]<-person AS recent_connections
FROM person;
```
### From Redis (Caching Patterns)
| Redis | SurrealDB |
|-------|-----------|
| SET key value | `CREATE cache:key SET value = ...` |
| GET key | `SELECT value FROM ONLY cache:key` |
| HSET | Nested object fields |
| LPUSH/RPUSH | Array fields with `array::push` |
| EXPIRE | No native TTL; use DELETE with time conditions |
| PUB/SUB | LIVE SELECT |
```surql
-- Key-value cache pattern
DEFINE TABLE cache SCHEMALESS;
DEFINE FIELD value ON TABLE cache FLEXIBLE TYPE any;
DEFINE FIELD expires_at ON TABLE cache TYPE option<datetime>;
-- Set a cache entry
UPSERT cache:session_abc123 SET
value = { user_id: 'alice', role: 'admin' },
expires_at = time::now() + 1h;
-- Get a cache entry (with expiration check)
SELECT value FROM ONLY cache:session_abc123
WHERE expires_at > time::now() OR expires_at IS NONE;
-- Periodic cleanup
DELETE cache WHERE expires_at < time::now();
-- Pub/Sub via LIVE SELECT
LIVE SELECT * FROM cache;
```
### From Elasticsearch (Search Mapping)
| Elasticsearch | SurrealDB |
|---------------|-----------|
| Index | Table |
| Document | Record |
| Mapping | DEFINE FIELD + DEFINE INDEX |
| Analyzer | DEFINE ANALYZER |
| Full-text query | `WHERE field @N@ 'query'` |
| Highlighting | `search::highlight()` |
| Score | `search::score()` |
| Aggregation | SELECT ... GROUP BY |
```surql
-- Elasticsearch-style search schema
DEFINE TABLE article SCHEMAFULL;
DEFINE FIELD title ON TABLE article TYPE string;
DEFINE FIELD body ON TABLE article TYPE string;
DEFINE FIELD author ON TABLE article TYPE string;
DEFINE FIELD published_at ON TABLE article TYPE datetime;
DEFINE FIELD tags ON TABLE article TYPE array<string>;
DEFINE ANALYZER article_analyzer TOKENIZERS blank, class
FILTERS ascii, lowercase, snowball(english);
DEFINE INDEX title_ft ON TABLE article COLUMNS title SEARCH ANALYZER article_analyzer BM25;
DEFINE INDEX body_ft ON TABLE article COLUMNS body SEARCH ANALYZER article_analyzer BM25 HIGHLIGHTS;
-- Elasticsearch: { "query": { "match": { "body": "vector search" } } }
-- SurrealDB:
SELECT title,
search::highlight('<em>', '</em>', 1) AS snippet,
search::score(1) AS score
FROM article
WHERE body @1@ 'vector search'
ORDER BY score DESC
LIMIT 10;
-- Elasticsearch aggregation equivalent
SELECT tags, count() AS doc_count
FROM article
WHERE body @1@ 'vector search'
SPLIT tags
GROUP BY tags
ORDER BY doc_count DESC;
```
### From Pinecone/Weaviate (Vector Mapping)
| Vector DB | SurrealDB |
|-----------|-----------|
| Collection/Index | Table + HNSW/MTREE index |
| Vector + metadata | Record with embedding field + other fields |
| Upsert | UPSERT |
| Query (top-K) | `WHERE embedding <\|K\|> $query_vec` |
| Metadata filter | Standard WHERE clauses |
| Namespace | Namespace or Database |
```surql
-- Pinecone-style vector store
DEFINE TABLE vectors SCHEMAFULL;
DEFINE FIELD embedding ON TABLE vectors TYPE array<float>;
DEFINE FIELD metadata ON TABLE vectors FLEXIBLE TYPE object;
DEFINE FIELD text ON TABLE vectors TYPE option<string>;
DEFINE INDEX vec_idx ON TABLE vectors FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE EFC 200 M 16;
-- Pinecone: index.upsert([("id1", [0.1, 0.2, ...], {"category": "tech"})])
-- SurrealDB:
UPSERT vectors:id1 SET
embedding = [0.1, 0.2, ...],
metadata = { category: 'tech' },
text = 'Original text content';
-- Pinecone: index.query(vector=[0.1, ...], top_k=5, filter={"category": "tech"})
-- SurrealDB:
SELECT id, text, metadata,
vector::similarity::cosine(embedding, $query_vec) AS score
FROM vectors
WHERE embedding <|5|> $query_vec
AND metadata.category = 'tech'
ORDER BY score DESC;
```
---
## Summary
SurrealDB's multi-model architecture means you choose the right model for each part of your application rather than forcing everything into a single paradigm. The key guidelines are:
1. **Start with your access patterns.** Design your schema around how you query data, not how you store it.
2. **Use record links for structural references** and graph edges for relationships with properties.
3. **Use SCHEMAFULL for core business data** and SCHEMALESS for flexible or evolving data.
4. **Combine models freely.** A single table can have vector indexes, full-text search, and participate in graph traversals.
5. **Index strategically.** Create indexes for fields in WHERE clauses, but avoid over-indexing.
6. **Leverage computed table views** for frequently-accessed aggregations instead of running expensive queries repeatedly.
7. **Use transactions** for multi-step mutations that must be atomic.
8. **Use compound record IDs** to encode natural composite keys directly in the record identifier.
```
### rules/surrealql.md
```markdown
# SurrealQL Reference
This is the comprehensive SurrealQL language reference for SurrealDB v3. SurrealQL is a SQL-like query language designed for SurrealDB's multi-model architecture, supporting document, graph, relational, vector, time-series, geospatial, and full-text search operations in a single language.
---
## Statements
### CREATE
Creates one or more records in a table.
```surql
-- Create a record with a random ID
CREATE person CONTENT {
name: 'Tobie',
age: 33,
email: '[email protected]'
};
-- Create a record with a specific ID
CREATE person:tobie SET
name = 'Tobie',
age = 33;
-- Create with a UUID-based ID
CREATE person:uuid() SET name = 'Jane';
-- Create with a ULID-based ID
CREATE person:ulid() SET name = 'John';
-- Create multiple records
CREATE person CONTENT [
{ name: 'Alice', age: 28 },
{ name: 'Bob', age: 35 }
];
-- Create with RETURN clause
CREATE person SET name = 'Eve' RETURN id, name;
-- Create with RETURN NONE (no output)
CREATE person SET name = 'Frank' RETURN NONE;
-- Create with RETURN BEFORE / AFTER / DIFF
CREATE person SET name = 'Grace' RETURN AFTER;
```
### SELECT
Retrieves records from one or more tables.
```surql
-- Select all fields from a table
SELECT * FROM person;
-- Select specific fields
SELECT name, age FROM person;
-- Select with alias
SELECT name AS full_name, math::floor(age) AS years FROM person;
-- Conditional filtering
SELECT * FROM person WHERE age > 30 AND name != 'Tobie';
-- Ordering results
SELECT * FROM person ORDER BY age DESC;
-- Limit and pagination
SELECT * FROM person LIMIT 10 START 20;
-- Grouping with aggregation
SELECT country, count() AS total FROM person GROUP BY country;
-- Nested field access
SELECT name.first, address.city FROM person;
-- Array filtering within records
SELECT emails[WHERE active = true] FROM person;
-- Select with value keyword (returns flat array)
SELECT VALUE name FROM person;
-- Select from specific record
SELECT * FROM person:tobie;
-- Select with FETCH to resolve record links
SELECT *, author.name FROM article FETCH author;
-- Select with SPLIT to unnest arrays
SELECT * FROM person SPLIT emails;
-- Select with TIMEOUT
SELECT * FROM person TIMEOUT 5s;
-- Select with PARALLEL execution
SELECT * FROM person PARALLEL;
-- Select with EXPLAIN to analyze query plan
SELECT * FROM person WHERE age > 30 EXPLAIN;
SELECT * FROM person WHERE age > 30 EXPLAIN FULL;
-- Subquery in SELECT
SELECT *, (SELECT count() FROM ->wrote->article GROUP ALL) AS article_count FROM person;
```
### UPDATE
Modifies existing records.
```surql
-- Update all records in a table
UPDATE person SET active = true;
-- Update a specific record
UPDATE person:tobie SET age = 34;
-- Merge data into a record
UPDATE person:tobie MERGE {
settings: { theme: 'dark', lang: 'en' }
};
-- Update with CONTENT (replaces entire record)
UPDATE person:tobie CONTENT {
name: 'Tobie',
age: 34,
active: true
};
-- Conditional update
UPDATE person SET verified = true WHERE age >= 18;
-- Update with RETURN clause
UPDATE person:tobie SET age = 35 RETURN DIFF;
UPDATE person:tobie SET age = 36 RETURN BEFORE;
UPDATE person:tobie SET age = 37 RETURN AFTER;
-- Increment / decrement numeric fields
UPDATE person:tobie SET age += 1;
UPDATE product:widget SET stock -= 5;
-- Append to an array
UPDATE person:tobie SET tags += 'admin';
-- Remove from an array
UPDATE person:tobie SET tags -= 'guest';
```
### DELETE
Removes records from tables.
```surql
-- Delete all records in a table
DELETE person;
-- Delete a specific record
DELETE person:tobie;
-- Conditional delete
DELETE person WHERE active = false;
-- Delete with RETURN
DELETE person:tobie RETURN BEFORE;
-- Delete with TIMEOUT
DELETE person WHERE last_login < time::now() - 1y TIMEOUT 30s;
```
### UPSERT
Creates a record if it does not exist, or updates it if it does.
```surql
-- Upsert a specific record
UPSERT person:tobie SET
name = 'Tobie',
age = 34,
updated_at = time::now();
-- Upsert with CONTENT
UPSERT person:tobie CONTENT {
name: 'Tobie',
age: 34,
company: 'SurrealDB'
};
-- Upsert with MERGE
UPSERT person:tobie MERGE {
last_seen: time::now()
};
```
### INSERT
Inserts records, supporting bulk operations and ON DUPLICATE KEY UPDATE.
```surql
-- Insert a single record
INSERT INTO person {
id: person:tobie,
name: 'Tobie',
age: 33
};
-- Bulk insert
INSERT INTO person [
{ name: 'Alice', age: 28 },
{ name: 'Bob', age: 35 },
{ name: 'Charlie', age: 42 }
];
-- Insert with ON DUPLICATE KEY UPDATE (upsert behavior)
INSERT INTO person {
id: person:tobie,
name: 'Tobie',
age: 34
} ON DUPLICATE KEY UPDATE age = $input.age;
-- INSERT IGNORE: skip on conflict instead of error
-- (Silently ignores records that violate unique constraints)
```
### RELATE
Creates graph edges (relationships) between records.
```surql
-- Create a basic relationship
RELATE person:tobie->wrote->article:surreal;
-- Create a relationship with properties (SET syntax)
RELATE person:tobie->bought->product:laptop SET
quantity = 1,
price = 1299.99,
purchased_at = time::now();
-- Create a relationship with CONTENT
RELATE person:alice->follows->person:bob CONTENT {
since: time::now(),
notifications: true
};
-- Relate multiple records at once
RELATE person:tobie->knows->[person:alice, person:bob, person:charlie];
-- Relate with a specific edge ID
RELATE person:tobie->wrote->article:surreal SET
id = wrote:first_article;
-- Return the created edge
RELATE person:tobie->likes->post:123 RETURN AFTER;
```
### DEFINE NAMESPACE
Defines a namespace, the top-level organizational unit.
```surql
DEFINE NAMESPACE myapp;
-- With OVERWRITE
DEFINE NAMESPACE OVERWRITE myapp;
-- With IF NOT EXISTS
DEFINE NAMESPACE IF NOT EXISTS myapp;
-- With COMMENT
DEFINE NAMESPACE myapp COMMENT 'Production namespace';
```
### DEFINE DATABASE
Defines a database within a namespace.
```surql
DEFINE DATABASE mydb;
DEFINE DATABASE OVERWRITE mydb;
DEFINE DATABASE IF NOT EXISTS mydb;
DEFINE DATABASE mydb COMMENT 'Main application database';
```
### DEFINE TABLE
Defines a table with schema enforcement, type, permissions, and other options.
```surql
-- Schemaless table (default: any fields allowed)
DEFINE TABLE article SCHEMALESS;
-- Schemafull table (only defined fields allowed)
DEFINE TABLE person SCHEMAFULL;
-- Table with TYPE NORMAL (standard document table)
DEFINE TABLE person TYPE NORMAL SCHEMAFULL;
-- Table with TYPE ANY (can hold documents and be used as graph edges)
DEFINE TABLE flexible TYPE ANY SCHEMALESS;
-- Table with TYPE RELATION (graph edge table)
DEFINE TABLE wrote TYPE RELATION IN person OUT article;
-- Relation table with ENFORCED (strict in/out types)
DEFINE TABLE purchased TYPE RELATION IN person OUT product ENFORCED;
-- Relation table with FROM/TO syntax (aliases for IN/OUT)
DEFINE TABLE likes TYPE RELATION FROM person TO post;
-- Drop table: deletes records immediately upon write, useful for write-only audit logs
DEFINE TABLE events DROP;
-- Computed table view (auto-updated projection)
DEFINE TABLE person_by_age AS
SELECT age, count() AS total
FROM person
GROUP BY age;
-- Table with changefeed
DEFINE TABLE orders CHANGEFEED 7d;
-- Table with changefeed including original data
DEFINE TABLE orders CHANGEFEED 30d INCLUDE ORIGINAL;
-- Table with permissions
DEFINE TABLE post SCHEMALESS
PERMISSIONS
FOR select FULL
FOR create WHERE $auth.id != NONE
FOR update WHERE author = $auth.id
FOR delete WHERE author = $auth.id OR $auth.role = 'admin';
-- Table with COMMENT
DEFINE TABLE person SCHEMAFULL COMMENT 'Stores user profiles';
```
### DEFINE FIELD
Defines a field on a table with type constraints, defaults, assertions, and permissions.
```surql
-- Basic typed field
DEFINE FIELD name ON TABLE person TYPE string;
-- Numeric field
DEFINE FIELD age ON TABLE person TYPE int;
-- Optional field (can be null)
DEFINE FIELD nickname ON TABLE person TYPE option<string>;
-- Field with default value
DEFINE FIELD created_at ON TABLE person TYPE datetime DEFAULT time::now();
-- Field with VALUE (set on every create/update)
DEFINE FIELD updated_at ON TABLE person VALUE time::now();
-- Computed field (read-only, derived from other fields)
DEFINE FIELD full_name ON TABLE person VALUE string::concat(name.first, ' ', name.last);
-- READONLY field (cannot be changed after creation)
DEFINE FIELD created_at ON TABLE person TYPE datetime VALUE time::now() READONLY;
-- Field with ASSERT (validation constraint)
DEFINE FIELD email ON TABLE person TYPE string
ASSERT string::is::email($value);
-- Field with range assertion
DEFINE FIELD age ON TABLE person TYPE int
ASSERT $value >= 0 AND $value <= 150;
-- Record link field
DEFINE FIELD author ON TABLE article TYPE record<person>;
-- Array field with inner type
DEFINE FIELD tags ON TABLE article TYPE array<string>;
-- Set field (unique elements)
DEFINE FIELD categories ON TABLE article TYPE set<string>;
-- Nested object field
DEFINE FIELD address ON TABLE person TYPE object;
DEFINE FIELD address.street ON TABLE person TYPE string;
DEFINE FIELD address.city ON TABLE person TYPE string;
DEFINE FIELD address.zip ON TABLE person TYPE string;
-- Array of records
DEFINE FIELD reviewers ON TABLE article TYPE array<record<person>>;
-- Field with FLEXIBLE type (accepts any type, stores as-is)
DEFINE FIELD metadata ON TABLE article FLEXIBLE TYPE object;
-- Field with permissions
DEFINE FIELD email ON TABLE person TYPE string
PERMISSIONS
FOR select WHERE $auth.id = id OR $auth.role = 'admin'
FOR update WHERE $auth.id = id;
-- Overwrite existing field definition
DEFINE FIELD OVERWRITE name ON TABLE person TYPE string;
-- IF NOT EXISTS
DEFINE FIELD IF NOT EXISTS name ON TABLE person TYPE string;
-- Geometry field
DEFINE FIELD location ON TABLE place TYPE geometry<point>;
-- Vector embedding field
DEFINE FIELD embedding ON TABLE document TYPE array<float> DEFAULT [];
-- Duration field
DEFINE FIELD duration ON TABLE event TYPE duration;
-- Decimal field (precise numeric)
DEFINE FIELD price ON TABLE product TYPE decimal;
-- Bytes field
DEFINE FIELD payload ON TABLE message TYPE bytes;
-- UUID field
DEFINE FIELD session_id ON TABLE session TYPE uuid;
-- Enum-like pattern using ASSERT
DEFINE FIELD status ON TABLE order TYPE string
ASSERT $value IN ['pending', 'processing', 'shipped', 'delivered', 'cancelled'];
```
### DEFINE INDEX
Creates indexes for query optimization, uniqueness, full-text search, and vector similarity search.
```surql
-- Standard index
DEFINE INDEX age_idx ON TABLE person COLUMNS age;
-- Unique index
DEFINE INDEX email_idx ON TABLE person COLUMNS email UNIQUE;
-- Composite index
DEFINE INDEX name_age_idx ON TABLE person COLUMNS name, age;
-- Full-text search index with analyzer
DEFINE INDEX content_search ON TABLE article COLUMNS content
SEARCH ANALYZER ascii BM25;
-- Full-text search with BM25 tuning
DEFINE INDEX content_search ON TABLE article COLUMNS content
SEARCH ANALYZER ascii BM25(1.2, 0.75);
-- Full-text search with highlights enabled
DEFINE INDEX content_search ON TABLE article COLUMNS content
SEARCH ANALYZER ascii BM25 HIGHLIGHTS;
-- HNSW vector index (for approximate nearest neighbor search)
DEFINE INDEX embedding_idx ON TABLE document FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- HNSW with tuning parameters
DEFINE INDEX embedding_idx ON TABLE document FIELDS embedding
HNSW DIMENSION 3072
DIST COSINE
TYPE F32
EFC 150
M 12;
-- MTREE vector index (for exact metric space search)
DEFINE INDEX embedding_idx ON TABLE document FIELDS embedding
MTREE DIMENSION 1536 DIST EUCLIDEAN;
-- MTREE with capacity tuning
DEFINE INDEX embedding_idx ON TABLE document FIELDS embedding
MTREE DIMENSION 1536 DIST COSINE CAPACITY 40;
-- Overwrite existing index
DEFINE INDEX OVERWRITE email_idx ON TABLE person COLUMNS email UNIQUE;
-- Rebuild an index
REBUILD INDEX email_idx ON TABLE person;
-- Rebuild all indexes on a table
REBUILD INDEX ON TABLE person;
```
**HNSW distance metrics**: `COSINE`, `EUCLIDEAN`, `MANHATTAN`, `CHEBYSHEV`, `HAMMING`, `JACCARD`, `MINKOWSKI`, `PEARSON`.
**HNSW parameters**:
- `DIMENSION` -- Number of dimensions in the vector (required)
- `DIST` -- Distance metric (default: `COSINE`)
- `TYPE` -- Element type: `F32`, `F64`, `I16`, `I32`, `I64` (default: `F32`)
- `EFC` -- Size of dynamic candidate list during construction (default: 150)
- `M` -- Max number of connections per node per layer (default: 12)
### DEFINE ACCESS
Defines authentication and authorization access methods.
```surql
-- Record-based access (signup/signin for end users)
DEFINE ACCESS account ON DATABASE TYPE RECORD
SIGNUP ( CREATE user SET email = $email, pass = crypto::argon2::generate($pass) )
SIGNIN ( SELECT * FROM user WHERE email = $email AND crypto::argon2::compare(pass, $pass) )
DURATION FOR TOKEN 15m, FOR SESSION 12h;
-- JWT access (external identity provider)
DEFINE ACCESS token_auth ON DATABASE TYPE JWT
ALGORITHM HS256 KEY 'your-secret-key-here'
DURATION FOR TOKEN 1h;
-- JWT with JWKS URL (for OAuth/OIDC providers)
DEFINE ACCESS oauth ON DATABASE TYPE JWT
URL 'https://auth.example.com/.well-known/jwks.json'
DURATION FOR TOKEN 1h, FOR SESSION 24h;
-- JWT with issuer key for token verification
DEFINE ACCESS api_auth ON NAMESPACE TYPE JWT
ALGORITHM RS256 KEY '-----BEGIN PUBLIC KEY-----...'
WITH ISSUER KEY '-----BEGIN PRIVATE KEY-----...';
-- Overwrite and IF NOT EXISTS
DEFINE ACCESS OVERWRITE account ON DATABASE TYPE RECORD
SIGNUP ( CREATE user SET email = $email, pass = crypto::argon2::generate($pass) )
SIGNIN ( SELECT * FROM user WHERE email = $email AND crypto::argon2::compare(pass, $pass) );
DEFINE ACCESS IF NOT EXISTS account ON DATABASE TYPE RECORD
SIGNUP ( CREATE user SET email = $email, pass = crypto::argon2::generate($pass) )
SIGNIN ( SELECT * FROM user WHERE email = $email AND crypto::argon2::compare(pass, $pass) );
```
### DEFINE ANALYZER
Defines text analyzers for full-text search indexes.
```surql
-- Basic analyzer with tokenizer and filters
DEFINE ANALYZER ascii TOKENIZERS blank, class FILTERS ascii, lowercase;
-- Snowball stemming analyzer for English
DEFINE ANALYZER english TOKENIZERS blank, class FILTERS ascii, snowball(english);
-- N-gram analyzer for autocomplete
DEFINE ANALYZER autocomplete TOKENIZERS blank FILTERS lowercase, ngram(2, 10);
-- Edge N-gram analyzer (prefix matching)
DEFINE ANALYZER prefix_search TOKENIZERS blank FILTERS lowercase, edgengram(1, 15);
-- Camel case tokenizer (splits camelCase words)
DEFINE ANALYZER code_search TOKENIZERS camel, blank FILTERS lowercase;
-- Custom multilingual analyzer
DEFINE ANALYZER multilingual TOKENIZERS blank, class FILTERS lowercase;
```
**Tokenizers**: `blank` (whitespace), `class` (character class boundaries), `camel` (camelCase split), `punct` (punctuation).
**Filters**: `ascii` (ASCII folding), `lowercase`, `uppercase`, `snowball(language)` (stemming), `ngram(min, max)`, `edgengram(min, max)`.
### DEFINE EVENT
Defines table events that trigger on record changes.
```surql
-- Event that fires on creation
DEFINE EVENT new_user ON TABLE user WHEN $event = "CREATE" THEN {
CREATE log SET
action = 'user_created',
user = $after.id,
timestamp = time::now();
};
-- Event that fires on update
DEFINE EVENT profile_change ON TABLE user WHEN $event = "UPDATE" THEN {
CREATE audit_log SET
table = 'user',
record = $after.id,
before = $before,
after = $after,
changed_at = time::now();
};
-- Event that fires on delete
DEFINE EVENT user_deleted ON TABLE user WHEN $event = "DELETE" THEN {
-- Clean up related data
DELETE session WHERE user = $before.id;
};
-- Event with conditional trigger
DEFINE EVENT stock_alert ON TABLE product
WHEN $event = "UPDATE" AND $after.stock < 10
THEN {
CREATE notification SET
type = 'low_stock',
product = $after.id,
stock = $after.stock;
};
-- Event that sends HTTP webhook
DEFINE EVENT webhook ON TABLE order WHEN $event = "CREATE" THEN {
http::post('https://hooks.example.com/orders', {
order_id: $after.id,
total: $after.total
});
};
```
**Event variables**: `$event` (CREATE, UPDATE, DELETE), `$before` (record state before change), `$after` (record state after change).
### DEFINE FUNCTION
Defines reusable custom functions.
```surql
-- Simple function
DEFINE FUNCTION fn::greet($name: string) {
RETURN string::concat('Hello, ', $name, '!');
};
-- Function with multiple parameters and return type
DEFINE FUNCTION fn::calculate_tax($amount: decimal, $rate: decimal) {
RETURN $amount * $rate;
};
-- Function that queries the database
DEFINE FUNCTION fn::get_user_orders($user_id: record<person>) {
RETURN SELECT * FROM order WHERE customer = $user_id ORDER BY created_at DESC;
};
-- Function with complex logic
DEFINE FUNCTION fn::full_name($person: record<person>) {
LET $p = (SELECT name FROM ONLY $person);
RETURN string::concat($p.name.first, ' ', $p.name.last);
};
-- Recursive-capable function
DEFINE FUNCTION fn::factorial($n: int) {
IF $n <= 1 {
RETURN 1;
};
RETURN $n * fn::factorial($n - 1);
};
-- Overwrite existing function
DEFINE FUNCTION OVERWRITE fn::greet($name: string) {
RETURN string::concat('Hi, ', $name, '!');
};
```
### DEFINE MODULE
Defines WASM (WebAssembly) extension modules. New in SurrealDB v3. Allows extending SurrealDB with custom logic compiled to WASM.
```surql
-- Define a WASM module from a file
DEFINE MODULE my_module;
-- Modules provide custom functions that become available
-- as module::function_name() after loading
```
### DEFINE BUCKET
Defines file/object storage buckets. New in SurrealDB v3. Provides built-in file storage capabilities within SurrealDB.
```surql
-- Define a file storage bucket
DEFINE BUCKET images;
-- Define a bucket with configuration
DEFINE BUCKET documents COMMENT 'Document storage for user uploads';
```
### DEFINE USER
Defines system users with scoped access.
```surql
-- Root-level user (full system access)
DEFINE USER root_admin ON ROOT PASSWORD 'strong-password-here' ROLES OWNER;
-- Namespace-level user
DEFINE USER ns_admin ON NAMESPACE PASSWORD 'ns-password' ROLES OWNER;
-- Database-level user
DEFINE USER db_editor ON DATABASE PASSWORD 'db-password' ROLES EDITOR;
-- Database viewer
DEFINE USER db_viewer ON DATABASE PASSWORD 'viewer-password' ROLES VIEWER;
-- User with password hash (pre-hashed)
DEFINE USER admin ON ROOT PASSHASH '$argon2id$...' ROLES OWNER;
-- User with COMMENT
DEFINE USER admin ON DATABASE PASSWORD 'secret' ROLES OWNER
COMMENT 'Primary database administrator';
```
**Roles**: `OWNER` (full access), `EDITOR` (read/write), `VIEWER` (read-only).
### DEFINE PARAM
Defines global parameters accessible across queries.
```surql
-- Define a parameter
DEFINE PARAM $app_name VALUE 'My Application';
-- Define a numeric parameter
DEFINE PARAM $max_results VALUE 100;
-- Define an object parameter
DEFINE PARAM $config VALUE {
theme: 'dark',
lang: 'en',
version: 3
};
-- Use a defined parameter in queries
SELECT * FROM person LIMIT $max_results;
```
### DEFINE SEQUENCE
Defines an auto-incrementing sequence for generating sequential numeric IDs.
```surql
-- Define a sequence with defaults
DEFINE SEQUENCE order_seq;
-- Define with custom start and batch size
DEFINE SEQUENCE invoice_seq START 1000 BATCH 50;
-- Use OVERWRITE to redefine
DEFINE SEQUENCE OVERWRITE order_seq START 1 BATCH 100;
-- Use IF NOT EXISTS
DEFINE SEQUENCE IF NOT EXISTS order_seq;
```
Syntax: `DEFINE SEQUENCE [ OVERWRITE | IF NOT EXISTS ] @name [ BATCH @batch ] [ START @start ]`
### USE
Switches the active namespace and/or database.
```surql
-- Switch namespace
USE NS myapp;
-- Switch database
USE DB production;
-- Switch both
USE NS myapp DB production;
```
### INFO FOR
Returns schema information about the system, namespace, database, or table.
```surql
-- System-level info
INFO FOR ROOT;
-- Namespace-level info
INFO FOR NAMESPACE;
-- or
INFO FOR NS;
-- Database-level info
INFO FOR DATABASE;
-- or
INFO FOR DB;
-- Table-level info
INFO FOR TABLE person;
-- or
INFO FOR TABLE person STRUCTURE;
```
### LET
Binds values to variables for use in subsequent statements.
```surql
-- Bind a simple value
LET $name = 'Tobie';
-- Bind a query result
LET $adults = (SELECT * FROM person WHERE age >= 18);
-- Bind a computed value
LET $now = time::now();
-- Use variables in subsequent queries
LET $user = (CREATE person SET name = $name);
RELATE $user->wrote->article:first;
```
### BEGIN / COMMIT / CANCEL (Transactions)
Groups multiple statements into atomic transactions.
```surql
-- Basic transaction
BEGIN TRANSACTION;
CREATE account:alice SET balance = 1000;
CREATE account:bob SET balance = 500;
COMMIT TRANSACTION;
-- Transaction with transfer logic
BEGIN TRANSACTION;
UPDATE account:alice SET balance -= 100;
UPDATE account:bob SET balance += 100;
CREATE transfer SET
from = account:alice,
to = account:bob,
amount = 100,
timestamp = time::now();
COMMIT TRANSACTION;
-- Cancel a transaction (rollback)
BEGIN TRANSACTION;
UPDATE account:alice SET balance -= 10000;
-- Oops, insufficient funds -- rollback
CANCEL TRANSACTION;
```
### RETURN
Returns a value from a block or function.
```surql
-- Return from a block
{
LET $x = 10;
LET $y = 20;
RETURN $x + $y;
};
-- Return in function context
DEFINE FUNCTION fn::add($a: int, $b: int) {
RETURN $a + $b;
};
```
### THROW
Throws a custom error, halting execution.
```surql
-- Throw a string error
THROW 'Something went wrong';
-- Throw conditionally
IF $balance < 0 {
THROW 'Insufficient funds';
};
-- Throw with dynamic message
THROW string::concat('User ', $id, ' not found');
```
### SLEEP
Pauses execution for a specified duration. Primarily useful for testing.
```surql
SLEEP 1s;
SLEEP 500ms;
SLEEP 2m;
```
### IF / ELSE
Conditional branching.
```surql
-- Basic if/else
IF $age >= 18 {
RETURN 'adult';
} ELSE {
RETURN 'minor';
};
-- If/else if/else
IF $score >= 90 {
RETURN 'A';
} ELSE IF $score >= 80 {
RETURN 'B';
} ELSE IF $score >= 70 {
RETURN 'C';
} ELSE {
RETURN 'F';
};
-- If as an expression (inline)
LET $label = IF $active { 'Active' } ELSE { 'Inactive' };
-- If in UPDATE
UPDATE person SET status = IF age >= 18 { 'adult' } ELSE { 'minor' };
```
### FOR
Iterates over arrays or query results.
```surql
-- Iterate over an array
FOR $name IN ['Alice', 'Bob', 'Charlie'] {
CREATE person SET name = $name;
};
-- Iterate over query results
FOR $user IN (SELECT * FROM person WHERE active = true) {
UPDATE $user.id SET last_check = time::now();
};
-- Nested loops
FOR $i IN [1, 2, 3] {
FOR $j IN ['a', 'b'] {
CREATE item SET num = $i, letter = $j;
};
};
```
### BREAK / CONTINUE
Controls loop execution flow.
```surql
-- Break out of a loop
FOR $item IN (SELECT * FROM product ORDER BY price ASC) {
IF $item.price > 100 {
BREAK;
};
UPDATE $item.id SET featured = true;
};
-- Skip iteration with CONTINUE
FOR $user IN (SELECT * FROM person) {
IF $user.role = 'bot' {
CONTINUE;
};
CREATE notification SET user = $user.id, message = 'System update';
};
```
### REMOVE
Removes schema definitions and data.
```surql
-- Remove a table and all its data
REMOVE TABLE person;
-- Remove a field definition
REMOVE FIELD email ON TABLE person;
-- Remove an index
REMOVE INDEX email_idx ON TABLE person;
-- Remove a namespace
REMOVE NAMESPACE myapp;
-- Remove a database
REMOVE DATABASE mydb;
-- Remove an event
REMOVE EVENT new_user ON TABLE user;
-- Remove a function
REMOVE FUNCTION fn::greet;
-- Remove a param
REMOVE PARAM $max_results;
-- Remove an analyzer
REMOVE ANALYZER english;
-- Remove an access method
REMOVE ACCESS account ON DATABASE;
-- Remove a user
REMOVE USER admin ON DATABASE;
-- Remove a module
REMOVE MODULE my_module;
-- Remove a bucket
REMOVE BUCKET images;
```
### REBUILD INDEX
Rebuilds indexes, useful after bulk data operations.
```surql
-- Rebuild a specific index
REBUILD INDEX email_idx ON TABLE person;
-- Rebuild all indexes on a table
REBUILD INDEX ON TABLE person;
```
### LIVE SELECT
Creates real-time subscriptions that push changes as they happen.
```surql
-- Live query on an entire table
LIVE SELECT * FROM person;
-- Live query with filtering
LIVE SELECT * FROM person WHERE age > 18;
-- Live query with DIFF (returns only changed fields)
LIVE SELECT DIFF FROM person;
-- Live query on specific fields
LIVE SELECT name, email FROM person;
-- Live query on a specific record
LIVE SELECT * FROM person:tobie;
```
Live queries return a UUID that can be used to cancel the subscription with `KILL`.
### KILL
Cancels an active live query.
```surql
-- Kill a live query by its UUID
KILL '1a2b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d';
-- Typically used with the UUID returned by LIVE SELECT
LET $live_id = (LIVE SELECT * FROM person);
-- ... later ...
KILL $live_id;
```
### SHOW CHANGES FOR TABLE
Retrieves the change feed for a table (requires CHANGEFEED to be enabled on the table).
```surql
-- Show all changes since a timestamp
SHOW CHANGES FOR TABLE orders SINCE '2026-01-01T00:00:00Z';
-- Show limited changes
SHOW CHANGES FOR TABLE orders SINCE '2026-01-01T00:00:00Z' LIMIT 100;
```
### VERSION Clause (Time-Travel Queries)
When running on the SurrealKV storage engine, you can query historical data at a specific point in time.
```surql
-- Query data as it existed at a specific time
SELECT * FROM person VERSION d'2026-01-15T12:00:00Z';
-- Time-travel with filtering
SELECT * FROM person WHERE active = true VERSION d'2025-12-01T00:00:00Z';
```
---
## Data Types
### Primitive Types
| Type | Description | Example |
|------|-------------|---------|
| `string` | UTF-8 text | `'hello'`, `"world"` |
| `int` | 64-bit signed integer | `42`, `-7` |
| `float` | 64-bit IEEE 754 floating point | `3.14`, `-0.5` |
| `decimal` | Arbitrary-precision decimal | `19.99dec`, `<decimal> 19.99` |
| `bool` | Boolean | `true`, `false` |
| `datetime` | ISO 8601 date and time | `d'2026-02-19T10:30:00Z'` |
| `duration` | Time duration | `1h30m`, `7d`, `500ms` |
| `bytes` | Binary data | `<bytes> "base64data"` |
| `uuid` | UUID value | `u'550e8400-e29b-41d4-a716-446655440000'` |
| `null` | Explicit null value | `null` |
| `none` | Absence of a value | `NONE` |
| `any` | Any type (no constraint) | -- |
### Complex Types
| Type | Description | Example |
|------|-------------|---------|
| `object` | Key-value map | `{ name: 'Tobie', age: 33 }` |
| `array` | Ordered list | `[1, 2, 3]` |
| `array<T>` | Typed array | `array<string>`, `array<int>` |
| `set` | Unique ordered list | -- |
| `set<T>` | Typed unique set | `set<string>` |
| `option<T>` | Nullable typed field | `option<string>` |
| `record` | Record link (any table) | `person:tobie` |
| `record<T>` | Record link (specific table) | `record<person>` |
### Geometry Types
| Type | Description |
|------|-------------|
| `geometry<point>` | GeoJSON Point |
| `geometry<line>` | GeoJSON LineString |
| `geometry<polygon>` | GeoJSON Polygon |
| `geometry<multipoint>` | GeoJSON MultiPoint |
| `geometry<multiline>` | GeoJSON MultiLineString |
| `geometry<multipolygon>` | GeoJSON MultiPolygon |
| `geometry<collection>` | GeoJSON GeometryCollection |
```surql
-- Geometry point (longitude, latitude)
CREATE place SET location = (-73.935242, 40.730610);
-- GeoJSON format
CREATE place SET location = {
type: 'Point',
coordinates: [-73.935242, 40.730610]
};
-- Polygon
CREATE zone SET area = {
type: 'Polygon',
coordinates: [[
[-73.98, 40.75],
[-73.97, 40.75],
[-73.97, 40.76],
[-73.98, 40.76],
[-73.98, 40.75]
]]
};
```
### Record IDs
Record IDs are first-class citizens in SurrealDB, uniquely identifying every record.
```surql
-- String-based ID
person:tobie
-- Integer-based ID
person:100
-- UUID-based ID (auto-generated)
person:uuid()
-- ULID-based ID (time-sortable, auto-generated)
person:ulid()
-- Random ID
person:rand()
-- Complex/compound ID (using arrays or objects)
temperature:['London', d'2026-02-19T10:00:00Z']
city:[36.775, -122.4194]
-- Object-based compound ID
person:{ first: 'Tobie', last: 'Morgan' }
```
### Duration Literals
```surql
-- Duration components
1ns -- nanoseconds
1us -- microseconds
1ms -- milliseconds
1s -- seconds
1m -- minutes
1h -- hours
1d -- days
1w -- weeks
1y -- years
-- Compound durations
1h30m
2d12h
1y6m3d
```
### Casting
Explicit type conversion using angle bracket syntax.
```surql
-- Cast to int
<int> '42'
<int> 3.14
-- Cast to float
<float> 42
<float> '3.14'
-- Cast to string
<string> 42
<string> true
-- Cast to bool
<bool> 'true'
<bool> 1
-- Cast to datetime
<datetime> '2026-02-19T10:00:00Z'
-- Cast to decimal
<decimal> 19.99
<decimal> '19.99'
-- Cast to duration
<duration> '1h30m'
-- Cast to record
<record> 'person:tobie'
```
---
## Operators
### Arithmetic Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `+` | Addition | `1 + 2` returns `3` |
| `-` | Subtraction | `5 - 3` returns `2` |
| `*` | Multiplication | `4 * 3` returns `12` |
| `/` | Division | `10 / 3` returns `3` |
| `**` | Exponentiation | `2 ** 8` returns `256` |
| `%` | Modulo | `10 % 3` returns `1` |
### Comparison Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `=` | Equals (loosely) | `1 = 1` |
| `!=` | Not equals | `1 != 2` |
| `==` | Exact equals (strict type) | `1 == 1` |
| `?=` | Any equals (for arrays) | `[1,2,3] ?= 2` |
| `*=` | All equals (for arrays) | `[1,1,1] *= 1` |
| `~` | Fuzzy match | `'hello' ~ 'helo'` |
| `!~` | Not fuzzy match | `'hello' !~ 'world'` |
| `?~` | Any fuzzy match | -- |
| `*~` | All fuzzy match | -- |
| `<` | Less than | `1 < 2` |
| `>` | Greater than | `2 > 1` |
| `<=` | Less than or equal | `1 <= 1` |
| `>=` | Greater than or equal | `2 >= 2` |
### Logical Operators
| Operator | Description |
|----------|-------------|
| `AND` / `&&` | Logical AND |
| `OR` / `\|\|` | Logical OR |
| `NOT` / `!` | Logical NOT |
### Containment Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `CONTAINS` | Value contains | `[1,2,3] CONTAINS 2` |
| `CONTAINSNOT` | Value does not contain | `[1,2,3] CONTAINSNOT 4` |
| `CONTAINSALL` | Contains all values | `[1,2,3] CONTAINSALL [1,2]` |
| `CONTAINSANY` | Contains any value | `[1,2,3] CONTAINSANY [2,4]` |
| `CONTAINSNONE` | Contains none of | `[1,2,3] CONTAINSNONE [4,5]` |
### Membership Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `IN` | Value is in | `2 IN [1,2,3]` |
| `NOT IN` | Value is not in | `4 NOT IN [1,2,3]` |
| `INSIDE` | Same as IN | `2 INSIDE [1,2,3]` |
| `NOTINSIDE` | Same as NOT IN | `4 NOTINSIDE [1,2,3]` |
| `ALLINSIDE` | All values are in | `[1,2] ALLINSIDE [1,2,3]` |
| `ANYINSIDE` | Any value is in | `[2,4] ANYINSIDE [1,2,3]` |
| `NONEINSIDE` | None of the values are in | `[4,5] NONEINSIDE [1,2,3]` |
### Pattern Matching
| Operator | Description | Example |
|----------|-------------|---------|
| `LIKE` | Wildcard pattern match | `name LIKE 'Tob%'` |
| `NOT LIKE` | Negative wildcard match | `name NOT LIKE '%bot%'` |
### Other Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `??` | Null coalescing | `$val ?? 'default'` |
| `?:` | Ternary | `$active ? 'yes' : 'no'` |
| `?.` | Optional chaining | `$user?.address?.city` |
| `..` | Range | `1..10` |
---
## Functions
### String Functions
```surql
string::concat('hello', ' ', 'world') -- 'hello world'
string::contains('SurrealDB', 'real') -- true
string::starts_with('SurrealDB', 'Surreal') -- true
string::ends_with('SurrealDB', 'DB') -- true
string::len('hello') -- 5
string::lowercase('HELLO') -- 'hello'
string::uppercase('hello') -- 'HELLO'
string::trim(' hello ') -- 'hello'
string::trim::start(' hello ') -- 'hello '
string::trim::end(' hello ') -- ' hello'
string::split('a,b,c', ',') -- ['a', 'b', 'c']
string::join(', ', 'a', 'b', 'c') -- 'a, b, c'
string::slug('Hello World!') -- 'hello-world'
string::replace('hello world', 'world', 'DB') -- 'hello DB'
string::reverse('hello') -- 'olleh'
string::repeat('ab', 3) -- 'ababab'
string::slice('SurrealDB', 0, 7) -- 'Surreal'
-- Validation functions
string::is::alphanum('abc123') -- true
string::is::alpha('abc') -- true
string::is::ascii('hello') -- true
string::is::datetime('2026-01-01T00:00:00Z') -- true
string::is::domain('surrealdb.com') -- true
string::is::email('[email protected]') -- true
string::is::hexadecimal('ff00ab') -- true
string::is::ip('192.168.1.1') -- true
string::is::ipv4('192.168.1.1') -- true
string::is::ipv6('::1') -- true
string::is::latitude('51.5074') -- true
string::is::longitude('-0.1278') -- true
string::is::numeric('12345') -- true
string::is::semver('1.2.3') -- true
string::is::url('https://surrealdb.com') -- true
string::is::uuid('550e8400-e29b-41d4-a716-446655440000') -- true
-- Method syntax (on string values)
'hello world'.uppercase() -- 'HELLO WORLD'
'a,b,c'.split(',') -- ['a', 'b', 'c']
```
### Array Functions
```surql
array::add([1, 2], 3) -- [1, 2, 3] (no duplicates)
array::all([true, true, true]) -- true
array::any([false, true, false]) -- true
array::append([1, 2], 3) -- [1, 2, 3]
array::at([1, 2, 3], 1) -- 2
array::boolean_and([true, false], [true, true]) -- [true, false]
array::boolean_or([true, false], [false, true]) -- [true, true]
array::boolean_not([true, false]) -- [false, true]
array::boolean_xor([true, false], [false, true]) -- [true, true]
array::combine([1, 2], [3, 4]) -- [[1,3],[1,4],[2,3],[2,4]]
array::complement([1,2,3,4], [2,4]) -- [1, 3]
array::concat([1, 2], [3, 4]) -- [1, 2, 3, 4]
array::clump([1,2,3,4,5], 2) -- [[1,2],[3,4],[5]]
array::difference([1,2,3], [2,3,4]) -- [1]
array::distinct([1, 2, 2, 3, 3]) -- [1, 2, 3]
array::find([1, 2, 3], 2) -- 2
array::find_index([1, 2, 3], 2) -- 1
array::first([1, 2, 3]) -- 1
array::flatten([[1, 2], [3, 4]]) -- [1, 2, 3, 4]
array::group([1,2,3,1,2]) -- [1, 2, 3]
array::insert([1, 3], 2, 1) -- [1, 2, 3]
array::intersect([1,2,3], [2,3,4]) -- [2, 3]
array::join([1, 2, 3], ', ') -- '1, 2, 3'
array::last([1, 2, 3]) -- 3
array::len([1, 2, 3]) -- 3
array::logical_and([1, 0], [0, 1]) -- [0, 0]
array::logical_or([1, 0], [0, 1]) -- [1, 1]
array::logical_xor([1, 0], [0, 1]) -- [1, 1]
array::max([3, 1, 2]) -- 3
array::min([3, 1, 2]) -- 1
array::pop([1, 2, 3]) -- [1, 2]
array::push([1, 2], 3) -- [1, 2, 3]
array::remove([1, 2, 3], 1) -- [1, 3]
array::reverse([1, 2, 3]) -- [3, 2, 1]
array::shuffle([1, 2, 3]) -- randomly shuffled
array::slice([1, 2, 3, 4], 1, 2) -- [2, 3]
array::sort([3, 1, 2]) -- [1, 2, 3]
array::sort::asc([3, 1, 2]) -- [1, 2, 3]
array::sort::desc([3, 1, 2]) -- [3, 2, 1]
array::transpose([[1,2],[3,4]]) -- [[1,3],[2,4]]
array::union([1, 2], [2, 3]) -- [1, 2, 3]
array::windows([1,2,3,4], 2) -- [[1,2],[2,3],[3,4]]
-- Method syntax
[1, 2, 3].len() -- 3
[1, 2, 2, 3].distinct() -- [1, 2, 3]
[[1, 2], [3, 4]].flatten() -- [1, 2, 3, 4]
[3, 1, 2].sort() -- [1, 2, 3]
```
### Math Functions
```surql
math::abs(-42) -- 42
math::ceil(3.2) -- 4
math::floor(3.8) -- 3
math::round(3.5) -- 4
math::sqrt(16) -- 4.0
math::pow(2, 10) -- 1024
math::log(100, 10) -- 2.0
math::log2(8) -- 3.0
math::log10(1000) -- 3.0
math::max([1, 5, 3]) -- 5
math::min([1, 5, 3]) -- 1
math::mean([1, 2, 3, 4, 5]) -- 3
math::median([1, 2, 3, 4, 5]) -- 3
math::sum([1, 2, 3, 4, 5]) -- 15
math::product([2, 3, 4]) -- 24
math::fixed(3.14159, 2) -- 3.14
math::clamp(15, 0, 10) -- 10
math::lerp(0, 10, 0.5) -- 5.0
math::spread([1, 5, 3]) -- 4
math::variance([1, 2, 3, 4, 5]) -- 2.0
math::stddev([1, 2, 3, 4, 5]) -- ~1.414
math::nearestrank([1, 2, 3, 4, 5], 75) -- 4
math::percentile([1, 2, 3, 4, 5], 50) -- 3.0
math::interquartile([1, 2, 3, 4, 5]) -- 2.0
math::midhinge([1, 2, 3, 4, 5]) -- 3.0
math::trimean([1, 2, 3, 4, 5]) -- 3.0
math::mode([1, 2, 2, 3]) -- 2
math::bottom([5, 1, 3, 2, 4], 3) -- [1, 2, 3]
math::top([5, 1, 3, 2, 4], 3) -- [3, 4, 5]
-- Constants
math::PI -- 3.14159...
math::E -- 2.71828...
math::TAU -- 6.28318...
math::INF -- Infinity
math::NEG_INF -- -Infinity
```
### Time Functions
```surql
time::now() -- current UTC datetime
time::day(d'2026-02-19T10:00:00Z') -- 19
time::hour(d'2026-02-19T10:30:00Z') -- 10
time::minute(d'2026-02-19T10:30:00Z') -- 30
time::second(d'2026-02-19T10:30:45Z') -- 45
time::month(d'2026-02-19T10:00:00Z') -- 2
time::year(d'2026-02-19T10:00:00Z') -- 2026
time::wday(d'2026-02-19T10:00:00Z') -- day of week (0=Sunday)
time::yday(d'2026-02-19T10:00:00Z') -- day of year
time::week(d'2026-02-19T10:00:00Z') -- ISO week number
time::unix(d'2026-02-19T10:00:00Z') -- Unix timestamp (seconds)
-- Formatting
time::format(time::now(), '%Y-%m-%d') -- '2026-02-19'
-- Grouping (truncate to period)
time::group(d'2026-02-19T10:30:45Z', 'hour') -- d'2026-02-19T10:00:00Z'
time::group(d'2026-02-19T10:30:45Z', 'day') -- d'2026-02-19T00:00:00Z'
time::group(d'2026-02-19T10:30:45Z', 'month') -- d'2026-02-01T00:00:00Z'
-- Rounding
time::floor(d'2026-02-19T10:30:45Z', 1h) -- d'2026-02-19T10:00:00Z'
time::ceil(d'2026-02-19T10:30:45Z', 1h) -- d'2026-02-19T11:00:00Z'
time::round(d'2026-02-19T10:30:45Z', 1h) -- d'2026-02-19T11:00:00Z'
-- From Unix timestamp
time::from::micros(1708344000000000)
time::from::millis(1708344000000)
time::from::nanos(1708344000000000000)
time::from::secs(1708344000)
time::from::unix(1708344000)
-- Timezone
time::timezone() -- server timezone
```
### Duration Functions
```surql
duration::days(90h) -- 3 (number of complete days)
duration::hours(2d12h) -- 60 (total hours)
duration::micros(1s) -- 1000000
duration::millis(1s) -- 1000
duration::mins(2h30m) -- 150
duration::nanos(1ms) -- 1000000
duration::secs(1h30m) -- 5400
-- From components
duration::from::days(7) -- 7d
duration::from::hours(24) -- 1d
duration::from::micros(1000000) -- 1s
duration::from::millis(1000) -- 1s
duration::from::mins(60) -- 1h
duration::from::nanos(1000000000) -- 1s
duration::from::secs(3600) -- 1h
```
### Type Functions
```surql
-- Type checking
type::is::array([1, 2]) -- true
type::is::bool(true) -- true
type::is::bytes(<bytes> 'data') -- true
type::is::datetime(time::now()) -- true
type::is::decimal(19.99dec) -- true
type::is::duration(1h) -- true
type::is::float(3.14) -- true
type::is::geometry((-73.9, 40.7)) -- true
type::is::int(42) -- true
type::is::null(null) -- true
type::is::none(NONE) -- true
type::is::number(42) -- true
type::is::object({ a: 1 }) -- true
type::is::point((-73.9, 40.7)) -- true
type::is::record(person:tobie) -- true
type::is::string('hello') -- true
type::is::uuid(rand::uuid()) -- true
-- Type construction
type::thing('person', 'tobie') -- person:tobie
type::field('name') -- field reference
type::fields(['name', 'age']) -- field references
type::record('person', 'tobie') -- person:tobie
```
### Crypto Functions
```surql
-- Hashing
crypto::md5('hello')
crypto::sha1('hello')
crypto::sha256('hello')
crypto::sha512('hello')
-- Password hashing (use for auth)
crypto::argon2::generate('MyPassword')
crypto::argon2::compare($hash, 'MyPassword')
crypto::bcrypt::generate('MyPassword')
crypto::bcrypt::compare($hash, 'MyPassword')
crypto::scrypt::generate('MyPassword')
crypto::scrypt::compare($hash, 'MyPassword')
```
### Geo Functions
```surql
-- Distance between two points (meters)
geo::distance((-0.04, 51.55), (30.46, -17.86))
-- Area of a polygon (square meters)
geo::area({
type: 'Polygon',
coordinates: [[
[-73.98, 40.75], [-73.97, 40.75],
[-73.97, 40.76], [-73.98, 40.76],
[-73.98, 40.75]
]]
})
-- Bearing between two points (degrees)
geo::bearing((-0.04, 51.55), (30.46, -17.86))
-- Centroid of a geometry
geo::centroid({
type: 'Polygon',
coordinates: [[
[0, 0], [10, 0], [10, 10], [0, 10], [0, 0]
]]
})
-- Geohash encoding/decoding
geo::hash::encode((-0.04, 51.55)) -- geohash string
geo::hash::encode((-0.04, 51.55), 6) -- with precision
geo::hash::decode('gcpuuz') -- geometry point
```
### HTTP Functions
Make outbound HTTP requests (requires network capability to be enabled).
```surql
-- GET request
http::get('https://api.example.com/data')
-- GET with headers
http::get('https://api.example.com/data', {
'Authorization': 'Bearer token123'
})
-- POST request with body
http::post('https://api.example.com/data', {
name: 'Tobie',
email: '[email protected]'
})
-- POST with custom headers
http::post('https://api.example.com/data', { name: 'Tobie' }, {
'Content-Type': 'application/json',
'Authorization': 'Bearer token123'
})
-- PUT request
http::put('https://api.example.com/data/1', { name: 'Updated' })
-- PATCH request
http::patch('https://api.example.com/data/1', { age: 34 })
-- DELETE request
http::delete('https://api.example.com/data/1')
-- HEAD request
http::head('https://api.example.com/data')
```
### Meta / Record Functions
```surql
-- Extract the ID portion of a record ID
meta::id(person:tobie) -- 'tobie'
record::id(person:tobie) -- 'tobie'
-- Extract the table name from a record ID
meta::tb(person:tobie) -- 'person'
record::tb(person:tobie) -- 'person'
record::table(person:tobie) -- 'person'
```
### Parse Functions
```surql
-- Email parsing
parse::email::host('[email protected]') -- 'surrealdb.com'
parse::email::user('[email protected]') -- 'tobie'
-- URL parsing
parse::url::domain('https://surrealdb.com/docs') -- 'surrealdb.com'
parse::url::host('https://surrealdb.com:8000') -- 'surrealdb.com'
parse::url::path('https://surrealdb.com/docs') -- '/docs'
parse::url::port('https://surrealdb.com:8000') -- 8000
parse::url::query('https://example.com?a=1&b=2') -- 'a=1&b=2'
parse::url::scheme('https://surrealdb.com') -- 'https'
parse::url::fragment('https://example.com#section') -- 'section'
```
### Rand Functions
```surql
rand() -- random float between 0 and 1
rand::bool() -- random boolean
rand::enum('one', 'two', 'three') -- random choice from values
rand::float() -- random float
rand::float(1.0, 100.0) -- random float in range
rand::guid() -- random GUID string
rand::guid(20) -- random GUID of specific length
rand::int() -- random integer
rand::int(1, 100) -- random integer in range
rand::string() -- random string
rand::string(10) -- random string of length
rand::string(5, 15) -- random string of length range
rand::time() -- random datetime
rand::time(d'2020-01-01', d'2026-12-31') -- random datetime in range
rand::uuid() -- random UUID v7
rand::uuid::v4() -- random UUID v4
rand::uuid::v7() -- random UUID v7
rand::ulid() -- random ULID
```
### Session Functions
```surql
session::db() -- current database name
session::id() -- current session ID
session::ip() -- client IP address
session::ns() -- current namespace name
session::origin() -- request origin
session::token() -- current auth token claims
```
### Object Functions
```surql
object::entries({ a: 1, b: 2 }) -- [['a', 1], ['b', 2]]
object::from_entries([['a', 1], ['b', 2]]) -- { a: 1, b: 2 }
object::keys({ a: 1, b: 2 }) -- ['a', 'b']
object::len({ a: 1, b: 2 }) -- 2
object::values({ a: 1, b: 2 }) -- [1, 2]
```
### Count Function
```surql
-- Count all records
SELECT count() FROM person GROUP ALL;
-- Count with condition
SELECT count() AS total FROM person WHERE active = true GROUP ALL;
-- Count values (non-null)
count([1, null, 2, null, 3]) -- 3
```
### Vector Functions
```surql
-- Arithmetic operations
vector::add([1, 2, 3], [4, 5, 6]) -- [5, 7, 9]
vector::subtract([4, 5, 6], [1, 2, 3]) -- [3, 3, 3]
vector::multiply([1, 2, 3], [4, 5, 6]) -- [4, 10, 18]
vector::divide([4, 10, 18], [4, 5, 6]) -- [1, 2, 3]
-- Geometric operations
vector::angle([1, 0], [0, 1]) -- angle in radians
vector::cross([1, 0, 0], [0, 1, 0]) -- [0, 0, 1]
vector::dot([1, 2, 3], [4, 5, 6]) -- 32
vector::magnitude([3, 4]) -- 5.0
vector::normalize([3, 4]) -- [0.6, 0.8]
vector::project([3, 4], [1, 0]) -- projection vector
-- Distance functions
vector::distance::chebyshev([1, 2], [4, 6]) -- 4
vector::distance::cosine([1, 2], [3, 4]) -- cosine distance
vector::distance::euclidean([1, 2], [4, 6]) -- 5.0
vector::distance::hamming([1, 0, 1], [1, 1, 0]) -- 2
vector::distance::manhattan([1, 2], [4, 6]) -- 7
vector::distance::jaccard([1, 2, 3], [2, 3, 4]) -- jaccard distance
vector::distance::minkowski([1, 2], [4, 6], 3) -- minkowski with p=3
vector::distance::pearson([1, 2, 3], [4, 5, 6]) -- pearson distance
-- Similarity functions (1 - distance, higher = more similar)
vector::similarity::cosine([1, 2], [3, 4]) -- cosine similarity
vector::similarity::jaccard([1, 2, 3], [2, 3, 4]) -- jaccard similarity
vector::similarity::pearson([1, 2, 3], [4, 5, 6]) -- pearson similarity
```
### Search Functions
Used in full-text search queries with `SEARCH ANALYZER` indexes.
```surql
-- Highlight matching terms
SELECT search::highlight('<b>', '</b>', 1) AS highlighted
FROM article
WHERE content @1@ 'SurrealDB';
-- Get offsets of matching terms
SELECT search::offsets(1) AS offsets
FROM article
WHERE content @1@ 'SurrealDB';
-- Get BM25 score
SELECT search::score(1) AS score
FROM article
WHERE content @1@ 'SurrealDB'
ORDER BY score DESC;
```
The `@N@` operator is the match operator for full-text search, where `N` is the index reference number used with `search::score()`, `search::highlight()`, and `search::offsets()`.
### Value Functions
```surql
-- Compute JSON Merge Patch diff between two values
value::diff({ a: 1, b: 2 }, { a: 1, b: 3 }) -- returns diff
-- Apply a JSON Merge Patch to a value
value::patch({ a: 1, b: 2 }, [{ op: 'replace', path: '/b', value: 3 }])
```
---
## Subqueries and Expressions
### Subqueries
Any SurrealQL query can be used as a subquery within another query.
```surql
-- Subquery in WHERE clause
SELECT * FROM article
WHERE author IN (SELECT VALUE id FROM person WHERE role = 'editor');
-- Subquery in field projection
SELECT *,
(SELECT VALUE count() FROM ->wrote->article GROUP ALL) AS article_count
FROM person;
-- Subquery in LET
LET $recent_articles = (
SELECT * FROM article
WHERE created_at > time::now() - 7d
ORDER BY created_at DESC
LIMIT 10
);
```
### Record Links
Records can directly link to other records using record IDs.
```surql
-- Create a record with a link
CREATE article SET
title = 'Introduction to SurrealDB',
author = person:tobie;
-- Query through the link
SELECT title, author.name FROM article;
-- Deep link traversal
SELECT title, author.company.name FROM article;
```
### Graph Traversal
Navigate graph relationships using arrow operators.
```surql
-- Forward traversal (outgoing edges)
SELECT ->wrote->article FROM person:tobie;
-- Backward traversal (incoming edges)
SELECT <-wrote<-person FROM article:surreal;
-- Multi-hop traversal
SELECT ->knows->person->wrote->article FROM person:tobie;
-- Bidirectional traversal
SELECT <->knows<->person FROM person:tobie;
-- Traversal with filtering
SELECT ->bought->product WHERE price > 100 FROM person:tobie;
-- Traversal with field selection
SELECT ->wrote->article.title FROM person:tobie;
-- Access edge properties during traversal
SELECT ->bought.quantity, ->bought->product.name FROM person:tobie;
-- Recursive traversal (ancestry)
-- Get parents
SELECT ->child_of->person FROM person:1;
-- Get grandparents
SELECT ->child_of->person->child_of->person FROM person:1;
-- All ancestors (variable depth)
SELECT ->child_of->person.* FROM person:1;
```
### Futures
Deferred computations that execute when queried.
```surql
-- Future value (recomputed on each read)
CREATE person SET
name = 'Tobie',
created = time::now(),
age_display = <future> { string::concat(<string> age, ' years old') };
```
### Parameters
Variables prefixed with `$` used in queries.
```surql
-- User-defined parameters (via LET or API)
LET $name = 'Tobie';
SELECT * FROM person WHERE name = $name;
-- System parameters
$auth -- Current authenticated user record
$session -- Current session data
$token -- Current JWT token claims
$before -- Record state before event (in events/live queries)
$after -- Record state after event (in events/live queries)
$value -- Current field value (in ASSERT/VALUE expressions)
$this -- Current record (in field expressions)
$parent -- Parent record (in subqueries)
$event -- Event type string: 'CREATE', 'UPDATE', 'DELETE' (in events)
$input -- Input data (in ON DUPLICATE KEY UPDATE)
```
### Embedded JavaScript
SurrealDB supports inline JavaScript functions for complex logic.
```surql
-- Inline JavaScript function
CREATE person SET
name = 'Tobie',
name_slug = function() {
return arguments[0].name.toLowerCase().replace(/\s+/g, '-');
};
-- JavaScript in function definitions
DEFINE FUNCTION fn::slugify($text: string) {
RETURN function($text) {
return arguments[0].toLowerCase()
.replace(/[^\w\s-]/g, '')
.replace(/\s+/g, '-');
};
};
```
---
## Idioms and Patterns
### Record ID Syntax
```surql
-- Table:ID is the universal record identifier
person:tobie -- string ID
person:100 -- numeric ID
person:uuid() -- auto UUID
person:ulid() -- auto ULID
person:rand() -- auto random
-- Compound IDs
temperature:['London', '2026-02-19'] -- array compound key
user_session:{user: 'tobie', device: 'laptop'} -- object compound key
```
### Destructuring and Nested Access
```surql
-- Access nested fields
SELECT name.first, name.last FROM person;
-- Access array elements
SELECT tags[0] FROM article;
-- Filter within arrays
SELECT emails[WHERE verified = true] FROM person;
-- Optional chaining for nullable fields
SELECT address?.city FROM person;
```
### Computed Table Views
```surql
-- Auto-updated aggregate view
DEFINE TABLE monthly_sales AS
SELECT
time::group(created_at, 'month') AS month,
count() AS order_count,
math::sum(total) AS revenue
FROM order
GROUP BY time::group(created_at, 'month');
-- Query the view like a regular table
SELECT * FROM monthly_sales ORDER BY month DESC;
```
### Changefeeds
```surql
-- Enable changefeed on a table
DEFINE TABLE orders CHANGEFEED 7d;
-- Read changes since a timestamp
SHOW CHANGES FOR TABLE orders SINCE '2026-02-01T00:00:00Z';
-- Changefeed with original data (for CDC patterns)
DEFINE TABLE orders CHANGEFEED 30d INCLUDE ORIGINAL;
```
---
## Best Practices
### SCHEMAFULL vs SCHEMALESS
- Use **SCHEMAFULL** when data integrity is paramount, for tables with well-known structures, user-facing data, and financial records. Every field must be defined before use. Provides compile-time-like safety for your data.
- Use **SCHEMALESS** for rapid prototyping, flexible metadata, log/event data, and when the schema is evolving frequently. Fields can be added without prior definition.
- Use **TYPE ANY** when a table may serve as both a normal document table and a graph edge table. Uncommon but useful in flexible schemas.
- Use **TYPE RELATION** for dedicated graph edge tables. Always specify `IN` and `OUT` types, and use `ENFORCED` to prevent edges from connecting incorrect record types.
### Transaction Patterns
```surql
-- Always wrap multi-step mutations in transactions
BEGIN TRANSACTION;
LET $order = (CREATE order SET
customer = $customer_id,
total = $total,
status = 'pending'
);
FOR $item IN $items {
RELATE $order->contains->$item.product SET
quantity = $item.qty,
price = $item.price;
};
UPDATE customer:$customer_id SET last_order = time::now();
COMMIT TRANSACTION;
```
### Index Strategy
- Create indexes for fields used in `WHERE` clauses
- Use `UNIQUE` indexes for natural keys (email, username)
- Use full-text search indexes (`SEARCH ANALYZER`) for text search rather than `LIKE '%term%'`
- Use HNSW indexes for vector similarity search (faster, approximate)
- Use MTREE indexes when exact nearest-neighbor results are required
- Composite indexes should list the most selective column first
- Avoid over-indexing: each index adds write overhead
- Use `EXPLAIN` to verify index usage in queries
### Query Optimization
- Use `SELECT VALUE` when you need a flat array of single values
- Use `FETCH` to resolve record links in a single query instead of multiple round trips
- Use `LIMIT` and `START` for pagination
- Prefer `PARALLEL` for large table scans
- Use `TIMEOUT` to prevent runaway queries
- Use computed table views for frequently-accessed aggregations
- Use record links instead of JOIN-style subqueries when possible
- Pre-filter with indexes, then apply complex logic in application code when needed
### Common Pitfalls
- Record IDs are case-sensitive: `person:Tobie` and `person:tobie` are different records
- `=` is a loose comparison; use `==` for strict type-matching comparison
- `CONTENT` replaces the entire record; use `MERGE` or `SET` for partial updates
- `array::add` prevents duplicates; `array::append` does not
- Datetime literals require the `d'...'` prefix: `d'2026-01-01T00:00:00Z'`
- Duration values do not use quotes: `1h30m` not `'1h30m'`
- `NONE` and `null` are distinct: `NONE` means "field absent", `null` means "field present with null value"
- `option<T>` allows `NONE` (field absent); it does not allow arbitrary types
- `RETURN NONE` suppresses output; omitting `RETURN` returns the affected records by default
- `DELETE table` deletes all records; `REMOVE TABLE table` removes the table definition entirely
- Graph edges created with `RELATE` are themselves records in a table; they can be queried directly
- Indexes cannot be created on computed fields (enforced since v3.0.1)
- Durations can be multiplied/divided by numbers and incremented/decremented (since v3.0.1)
---
## v3.0.1 Patch Notes (2026-02-24)
Key fixes and changes in SurrealDB v3.0.1:
- **Duration arithmetic**: Durations can now be multiplied and divided by numbers, and incremented/decremented like numbers (`1h * 2` = `2h`, `30m + 15m` = `45m`)
- **Computed field index prevention**: Creating indexes on computed fields is now correctly rejected (prevents silent index corruption)
- **Record ID dereference fix**: Record IDs are now properly dereferenced when a field is computed on them
- **Error serialization fix**: Error objects are correctly serialized across all protocols
- **GraphQL string enum fix**: String enum literals now work correctly in GraphQL queries
- **Root user permission fix**: Permission check conditions for root users are now evaluated correctly
- **Parallel index compaction**: Index compaction now runs in parallel across distinct indexes (performance improvement)
- **WASM compatibility**: Improved compatibility for embedded WASM deployments
- **RouterFactory trait**: New `RouterFactory` trait exposed for embedders to compose custom HTTP routers (advanced)
## v3.0.2 Patch Notes (2026-03-03)
Key fixes and changes in SurrealDB v3.0.2:
- **Non-existent record returns None** (#6978): `SELECT` on a record that does not exist now returns `NONE` instead of a confusing error. Code that catches errors on missing records should be updated to check for `NONE` instead.
- **Bind parameter resolution in MATCHES** (#6961): Bind parameters now correctly resolve in the `MATCHES (@N@)` operator and `search::score()` function
- **Datetime setter functions** (#6981): New functions to set individual parts of datetimes (year, month, day, hour, etc.)
- **Configurable CORS allow list** (#6998): `--allow-origins` flag now supports multiple origins for production CORS configuration
- **`--tables-exclude` flag** (#6999): New `surreal export --tables-exclude` flag to exclude specific tables from exports
- **Compound unique index fix** (#7002): Fixed compound unique indexing for multi-field unique constraints
- **DELETE live event permission fix** (#6992): Permission checks now correctly apply to DELETE events in live queries
- **DEFINE FUNCTION parsing fix** (#6987): Fixed parsing of `DEFINE FUNCTION` when loading from disk
- **Transaction timeout enforcement** (#6975): Transaction timeout is now correctly enforced for all queries
- **RecordIdKeyType::Object serialization** (#6977): Fixed serialization error for object-typed record ID keys
- **IndexAppending write-write conflicts** (#6982): Fixed write-write conflicts on `ip` keys during index appending
- **Executor optimizations** (#6995): New executor bug fixes and performance optimizations
- **SurrealValue for LinkedList/HashSet** (#6968): SDK embedders can now use `SurrealValue` with `LinkedList` and `HashSet` types
## v3.0.4 Patch Notes (2026-03-13)
Key fixes and changes in SurrealDB v3.0.3 and v3.0.4:
- **GraphQL Subscriptions** (#7027): New real-time GraphQL subscription support via WebSocket
- **BM25 search::score() fix** (#7057): Fixed `search::score()` returning 0 after index compaction (critical for full-text search ranking)
- **HNSW index compaction fix** (#7077): Fixed write conflicts during HNSW vector index compaction
- **UPSERT conditional count fix** (#7056): `UPSERT SET count = IF count THEN count + 1 ELSE 1 END` no longer always evaluates to 1 on existing records
- **LIMIT with incomplete WHERE fix** (#7063): `LIMIT` with incomplete `WHERE` clauses no longer produces fewer rows than expected
- **Subquery nested AS fix** (#7053): Subqueries now correctly respect nested fields with `AS key.key`
- **`+=`/`-=` operator fix** (#7048): Fixed discrepancies between `+=`/`-=` and `+`/`-` operators
- **Time formatting panic fix** (#7043): Invalid time formatting strings no longer cause a panic
- **START pushdown fix** (#7047): Fixed `START` issue with pushdown KV skipping records
- **Concurrent startup retry** (#7055): Added retry logic for initial datastore transactions to prevent conflicts on concurrent startup
- **Distributed task lease race fix** (#6501): Fixed race condition in distributed task lease acquisition
- **Index compaction stability** (#7065): Fixed `KeyAlreadyExists` and `TransactionConflict` errors during index compaction
- **Connection error propagation** (#7044): Propagates actual query errors instead of misleading 'Connection uninitialised'
- **Performance improvements** (#7018): General performance optimizations
- **Set increment/decrement** (#7079): More types supported for `TryAdd`/`Sub` operations
- **SurrealKV 0.21.0** (#7042): Updated embedded storage engine
- **GraphQL root field comments** (#7032): Comments on root-level GraphQL fields now supported
- **v2 subcommand** (#7058): New `surreal v2` subcommand to run the old v2 binary for migration assistance
- **NaiveDate SurrealValue** (#7040): `NaiveDate` now implements `SurrealValue` for SDK embedders
### v3.1.0-alpha (in progress on main)
The main branch tracks toward v3.1.0 with ongoing work on:
- Error chaining infrastructure (#6969)
- SurrealValue derive convenience (#6970)
- Timestamp code refactoring (#6892)
- Import overhead reduction and benchmarks (#7069)
```
### rules/graph-queries.md
```markdown
# SurrealDB Graph Queries
SurrealDB is a multi-model database with first-class graph capabilities. Unlike bolt-on graph layers, SurrealDB treats records as nodes and edge tables as typed, queryable relationships. Graph traversal uses arrow syntax (`->`, `<-`, `<->`) directly in SurrealQL, enabling complex relationship queries without separate graph query languages.
---
## RELATE Statement
The `RELATE` statement creates graph edges (relationships) between records. Each relationship is stored in an edge table with automatic `in` and `out` fields pointing to the source and target records.
### Basic Syntax
```surrealql
-- General form
RELATE @from->@edge->@to [SET | CONTENT | MERGE ...];
-- Create a simple relationship
RELATE person:alice->knows->person:bob;
-- The edge record is stored in the 'knows' table with:
-- in: person:alice
-- out: person:bob
SELECT * FROM knows;
-- Returns: [{ id: knows:..., in: person:alice, out: person:bob }]
```
### Setting Properties on Edges
Edge tables are full SurrealDB tables -- you can store arbitrary data on them.
```surrealql
-- Using SET for individual fields
RELATE person:alice->knows->person:bob SET
since = d'2023-06-15',
strength = 0.85,
context = 'work';
-- Using CONTENT for full object replacement
RELATE person:alice->follows->person:charlie CONTENT {
since: time::now(),
notifications: true,
tags: ['tech', 'surrealdb']
};
-- Using MERGE to add to existing properties
RELATE person:alice->knows->person:bob MERGE {
last_interaction: time::now()
};
```
### Creating Multiple Relationships at Once
```surrealql
-- Relate multiple sources to one target
RELATE [person:alice, person:bob, person:charlie]->likes->post:123;
-- Relate one source to multiple targets
RELATE person:alice->follows->[person:bob, person:charlie, person:dave];
-- Relate multiple to multiple (creates cartesian product)
RELATE [person:alice, person:bob]->likes->[post:1, post:2];
-- Creates 4 edges: alice->post:1, alice->post:2, bob->post:1, bob->post:2
```
### Typed Edge Tables with Schema Enforcement
```surrealql
-- Define the edge table with typed in/out fields
DEFINE TABLE wrote SCHEMAFULL;
DEFINE FIELD in ON TABLE wrote TYPE record<person>;
DEFINE FIELD out ON TABLE wrote TYPE record<article>;
DEFINE FIELD written_at ON TABLE wrote TYPE datetime DEFAULT time::now();
DEFINE FIELD word_count ON TABLE wrote TYPE int;
-- Enforce unique relationships (one person can write an article only once)
DEFINE INDEX unique_author_article ON TABLE wrote COLUMNS in, out UNIQUE;
-- This succeeds
RELATE person:aristotle->wrote->article:metaphysics SET word_count = 45000;
-- This fails because of the UNIQUE index
RELATE person:aristotle->wrote->article:metaphysics SET word_count = 50000;
-- This also fails because 'in' must be a person record
RELATE article:foo->wrote->article:bar;
-- Error: Expected a record<person>, but found record<article>
```
### Bidirectional Edges
```surrealql
-- Some relationships are inherently bidirectional
-- Model friendship as a single edge, query from either direction
DEFINE TABLE friends_with SCHEMAFULL;
DEFINE FIELD in ON TABLE friends_with TYPE record<person>;
DEFINE FIELD out ON TABLE friends_with TYPE record<person>;
DEFINE FIELD since ON TABLE friends_with TYPE datetime;
RELATE person:alice->friends_with->person:bob SET since = d'2023-01-01';
-- Query from either side using <-> operator
SELECT *, <->friends_with<->person AS friends FROM person:alice;
SELECT *, <->friends_with<->person AS friends FROM person:bob;
-- Exclude self-references from results
SELECT *,
array::complement(<->friends_with<->person, [id]) AS friends
FROM person;
```
### Self-Referential Edges
```surrealql
-- A person can manage themselves (e.g., solo founder)
DEFINE TABLE manages SCHEMAFULL;
DEFINE FIELD in ON TABLE manages TYPE record<person>;
DEFINE FIELD out ON TABLE manages TYPE record<person>;
DEFINE FIELD role ON TABLE manages TYPE string;
-- Hierarchical management
RELATE person:ceo->manages->person:vp_eng SET role = 'direct_report';
RELATE person:vp_eng->manages->person:lead_1 SET role = 'direct_report';
RELATE person:vp_eng->manages->person:lead_2 SET role = 'direct_report';
RELATE person:lead_1->manages->person:dev_1 SET role = 'direct_report';
RELATE person:lead_1->manages->person:dev_2 SET role = 'direct_report';
-- Self-referential: person references themselves
RELATE person:freelancer->manages->person:freelancer SET role = 'self';
```
---
## Graph Traversal
SurrealDB uses arrow operators for graph traversal directly within `SELECT` statements or as standalone expressions.
### Forward Traversal (`->`)
Follow edges from a record outward through the `out` direction.
```surrealql
-- Find all articles written by Aristotle
SELECT ->wrote->article FROM person:aristotle;
-- Returns: [{ "->wrote->article": [article:metaphysics, article:on_sleep] }]
-- Get specific fields from traversed records
SELECT ->wrote->article.title FROM person:aristotle;
-- Standalone expression (no SELECT needed)
RETURN person:aristotle->wrote->article;
-- Destructured form with aliases
person:aristotle.{ name, articles: ->wrote->article.title };
```
### Reverse Traversal (`<-`)
Follow edges backward through the `in` direction.
```surrealql
-- Find who wrote a specific article
SELECT <-wrote<-person FROM article:metaphysics;
-- Find all users who liked a post
SELECT <-likes<-person.name AS liked_by FROM post:123;
-- Standalone expression
RETURN article:metaphysics<-wrote<-person;
-- Find all followers of a person
SELECT <-follows<-person.name AS followers FROM person:alice;
```
### Bidirectional Traversal (`<->`)
Follow edges in both directions simultaneously.
```surrealql
-- Find all friends (regardless of who initiated the relationship)
SELECT <->friends_with<->person AS friends FROM person:alice;
-- Exclude self from results
SELECT
array::complement(<->friends_with<->person, [id]) AS friends
FROM person:alice;
-- Works across any relationship
SELECT <->knows<->person AS connections FROM person:bob;
```
### Multi-Hop Traversal
Chain arrow operators to traverse multiple relationship levels.
```surrealql
-- Friends of friends
SELECT ->knows->person->knows->person AS friends_of_friends
FROM person:alice;
-- Author -> articles -> topics
SELECT ->wrote->article->tagged_with->topic AS interests
FROM person:aristotle;
-- User -> orders -> products -> categories
SELECT ->placed->order->contains->product->belongs_to->category
FROM user:customer_1;
-- Multi-hop with field selection at each level
SELECT
->manages->person.name AS direct_reports,
->manages->person->manages->person.name AS skip_level_reports
FROM person:ceo;
-- Traversal through different edge types
SELECT
->likes->post<-wrote<-person AS liked_same_posts
FROM person:alice;
```
### Filtered Traversal
Apply WHERE conditions during traversal to filter intermediate results.
```surrealql
-- Only traverse 'knows' edges created after 2020
SELECT ->(knows WHERE since > d'2020-01-01')->person.name AS recent_contacts
FROM person:alice;
-- Filter on edge properties
SELECT ->(rated WHERE score >= 4.0)->movie.title AS highly_rated
FROM user:alice;
-- Filter on target node properties
SELECT ->knows->(person WHERE age > 30).name AS older_contacts
FROM person:alice;
-- Combine edge and node filters
SELECT
->(knows WHERE strength > 0.7)->(person WHERE active = true).name
AS strong_active_contacts
FROM person:alice;
-- Multi-hop with filters at each level
SELECT
->(manages WHERE role = 'direct_report')
->person
->(manages WHERE role = 'direct_report')
->person.name AS skip_level_reports
FROM person:ceo;
```
### Aliased Traversal
Use `AS` to alias intermediate results for later reference.
```surrealql
-- Alias the intermediate edge
SELECT
->(knows WHERE since > d'2023-01-01' AS recent_connections)->person.name
FROM person:alice;
-- Alias nodes at different traversal levels
SELECT
->wrote->(article AS authored_articles)->tagged_with->topic.name AS topics
FROM person:aristotle;
```
### Recursive Traversal Patterns
SurrealDB does not have a built-in recursive traversal keyword, but you can implement recursive patterns using subqueries and multi-hop chains.
```surrealql
-- Fixed-depth hierarchy (3 levels of management)
SELECT
name,
->manages->person.name AS level_1,
->manages->person->manages->person.name AS level_2,
->manages->person->manages->person->manages->person.name AS level_3
FROM person:ceo;
-- Collect all descendants up to N levels using array functions
SELECT
name,
array::flatten([
->manages->person,
->manages->person->manages->person,
->manages->person->manages->person->manages->person
]) AS all_reports
FROM person:ceo;
-- Recursive-like pattern using a subquery approach
-- Find all ancestors (who manages my manager?)
SELECT
name,
<-manages<-person AS manager,
<-manages<-person<-manages<-person AS skip_manager,
<-manages<-person<-manages<-person<-manages<-person AS exec
FROM person:dev_1;
```
---
## Advanced Graph Patterns
### Shortest Path Queries
SurrealDB does not have a native shortest-path function, but you can implement BFS-like patterns.
```surrealql
-- Check if a direct connection exists (depth 1)
SELECT ->knows->person
FROM person:alice
WHERE person:dave IN ->knows->person;
-- Check depth 2
SELECT ->knows->person->knows->person
FROM person:alice
WHERE person:dave IN ->knows->person->knows->person;
-- Find shortest path by testing increasing depths
-- Depth 1
LET $depth1 = SELECT VALUE ->knows->person FROM person:alice;
-- Depth 2
LET $depth2 = SELECT VALUE ->knows->person->knows->person FROM person:alice;
-- Depth 3
LET $depth3 = SELECT VALUE ->knows->person->knows->person->knows->person FROM person:alice;
-- Check which depth first contains the target
RETURN {
depth_1: person:dave IN array::flatten($depth1),
depth_2: person:dave IN array::flatten($depth2),
depth_3: person:dave IN array::flatten($depth3)
};
-- BFS-like approach using a SurrealDB function
DEFINE FUNCTION fn::find_path($from: record, $to: record, $edge: string) {
-- Check direct connection
LET $d1 = SELECT VALUE out FROM type::table($edge) WHERE in = $from;
IF $to IN $d1 { RETURN { depth: 1, found: true } };
-- Check depth 2
LET $d2 = SELECT VALUE out FROM type::table($edge) WHERE in IN $d1;
IF $to IN $d2 { RETURN { depth: 2, found: true } };
-- Check depth 3
LET $d3 = SELECT VALUE out FROM type::table($edge) WHERE in IN $d2;
IF $to IN $d3 { RETURN { depth: 3, found: true } };
RETURN { depth: -1, found: false };
};
RETURN fn::find_path(person:alice, person:dave, 'knows');
```
### Degree Centrality Calculations
```surrealql
-- Out-degree: number of outgoing relationships
SELECT
id,
name,
count(->knows->person) AS out_degree
FROM person
ORDER BY out_degree DESC;
-- In-degree: number of incoming relationships
SELECT
id,
name,
count(<-knows<-person) AS in_degree
FROM person
ORDER BY in_degree DESC;
-- Total degree (bidirectional)
SELECT
id,
name,
count(->knows->person) AS out_degree,
count(<-knows<-person) AS in_degree,
count(->knows->person) + count(<-knows<-person) AS total_degree
FROM person
ORDER BY total_degree DESC;
-- Weighted centrality using edge properties
SELECT
id,
name,
math::sum(->knows.strength) AS weighted_out_degree,
math::sum(<-knows.strength) AS weighted_in_degree
FROM person
ORDER BY weighted_out_degree DESC;
```
### Community Detection Patterns
```surrealql
-- Find clusters by shared connections (common neighbors)
SELECT
p1.name AS person_a,
p2.name AS person_b,
array::intersect(
p1->knows->person,
p2->knows->person
) AS common_friends,
count(array::intersect(
p1->knows->person,
p2->knows->person
)) AS overlap_count
FROM person AS p1, person AS p2
WHERE p1.id != p2.id
ORDER BY overlap_count DESC;
-- Find tightly connected subgroups
-- People who all know each other (triads)
SELECT
a.name AS person_a,
b.name AS person_b,
c.name AS person_c
FROM person AS a, person AS b, person AS c
WHERE
a.id != b.id AND b.id != c.id AND a.id != c.id
AND b IN a->knows->person
AND c IN a->knows->person
AND c IN b->knows->person;
-- Neighborhood overlap for community strength
DEFINE FUNCTION fn::jaccard_similarity($a: record<person>, $b: record<person>) {
LET $neighbors_a = SELECT VALUE ->knows->person FROM ONLY $a;
LET $neighbors_b = SELECT VALUE ->knows->person FROM ONLY $b;
LET $intersection = array::intersect($neighbors_a, $neighbors_b);
LET $union = array::union($neighbors_a, $neighbors_b);
RETURN IF count($union) > 0 {
count($intersection) / count($union)
} ELSE {
0.0
};
};
```
### Recommendation Engine Using Graph Traversal
```surrealql
-- Collaborative filtering: "users who liked X also liked Y"
SELECT
->likes->product AS also_liked,
count() AS frequency
FROM person
WHERE id IN (
SELECT VALUE <-likes<-person FROM product:target_product
)
AND ->likes->product != product:target_product
GROUP BY also_liked
ORDER BY frequency DESC
LIMIT 10;
-- Content-based with graph enrichment:
-- Find products in categories the user has shown interest in
SELECT
p.id,
p.name,
p.price
FROM product AS p
WHERE p->belongs_to->category IN (
SELECT VALUE ->purchased->product->belongs_to->category
FROM user:current_user
)
AND p.id NOT IN (
SELECT VALUE ->purchased->product FROM user:current_user
)
ORDER BY p.rating DESC
LIMIT 20;
-- Hybrid: People with similar taste who liked other things
LET $my_likes = SELECT VALUE ->likes->product FROM ONLY user:alice;
LET $similar_users = SELECT
id,
count(->likes->product INTERSECT $my_likes) AS overlap
FROM user
WHERE id != user:alice
ORDER BY overlap DESC
LIMIT 10;
SELECT
->likes->product AS recommended,
count() AS score
FROM $similar_users
WHERE ->likes->product NOT IN $my_likes
GROUP BY recommended
ORDER BY score DESC
LIMIT 10;
```
### Access Control Graphs
```surrealql
-- Model RBAC as a graph
DEFINE TABLE role SCHEMAFULL;
DEFINE FIELD name ON TABLE role TYPE string;
DEFINE TABLE permission SCHEMAFULL;
DEFINE FIELD resource ON TABLE permission TYPE string;
DEFINE FIELD action ON TABLE permission TYPE string;
DEFINE TABLE has_role SCHEMAFULL;
DEFINE FIELD in ON TABLE has_role TYPE record<user>;
DEFINE FIELD out ON TABLE has_role TYPE record<role>;
DEFINE TABLE grants SCHEMAFULL;
DEFINE FIELD in ON TABLE grants TYPE record<role>;
DEFINE FIELD out ON TABLE grants TYPE record<permission>;
DEFINE TABLE inherits SCHEMAFULL;
DEFINE FIELD in ON TABLE inherits TYPE record<role>;
DEFINE FIELD out ON TABLE inherits TYPE record<role>;
-- Setup hierarchy
CREATE role:admin SET name = 'Admin';
CREATE role:editor SET name = 'Editor';
CREATE role:viewer SET name = 'Viewer';
-- Role inheritance: admin inherits editor inherits viewer
RELATE role:admin->inherits->role:editor;
RELATE role:editor->inherits->role:viewer;
-- Assign permissions
CREATE permission:read_posts SET resource = 'posts', action = 'read';
CREATE permission:write_posts SET resource = 'posts', action = 'write';
CREATE permission:delete_posts SET resource = 'posts', action = 'delete';
RELATE role:viewer->grants->permission:read_posts;
RELATE role:editor->grants->permission:write_posts;
RELATE role:admin->grants->permission:delete_posts;
-- Assign user to role
RELATE user:alice->has_role->role:editor;
-- Check all permissions for a user (including inherited via role hierarchy)
SELECT
->has_role->role AS direct_roles,
->has_role->role->grants->permission AS direct_permissions,
->has_role->role->inherits->role AS inherited_roles,
->has_role->role->inherits->role->grants->permission AS inherited_permissions,
array::flatten([
->has_role->role->grants->permission,
->has_role->role->inherits->role->grants->permission,
->has_role->role->inherits->role->inherits->role->grants->permission
]) AS all_permissions
FROM user:alice;
-- Check if user has a specific permission
DEFINE FUNCTION fn::has_permission($user: record<user>, $resource: string, $action: string) {
LET $all_perms = array::flatten([
SELECT VALUE ->has_role->role->grants->permission FROM ONLY $user,
SELECT VALUE ->has_role->role->inherits->role->grants->permission FROM ONLY $user,
SELECT VALUE ->has_role->role->inherits->role->inherits->role->grants->permission FROM ONLY $user
]);
LET $matching = SELECT * FROM array::flatten($all_perms) WHERE resource = $resource AND action = $action;
RETURN count($matching) > 0;
};
```
### Hierarchical Data (Org Charts, Categories)
```surrealql
-- Category tree
DEFINE TABLE category SCHEMAFULL;
DEFINE FIELD name ON TABLE category TYPE string;
DEFINE FIELD description ON TABLE category TYPE option<string>;
DEFINE TABLE subcategory_of SCHEMAFULL;
DEFINE FIELD in ON TABLE subcategory_of TYPE record<category>;
DEFINE FIELD out ON TABLE subcategory_of TYPE record<category>;
-- Build category hierarchy
CREATE category:electronics SET name = 'Electronics';
CREATE category:computers SET name = 'Computers';
CREATE category:laptops SET name = 'Laptops';
CREATE category:desktops SET name = 'Desktops';
CREATE category:gaming_laptops SET name = 'Gaming Laptops';
RELATE category:computers->subcategory_of->category:electronics;
RELATE category:laptops->subcategory_of->category:computers;
RELATE category:desktops->subcategory_of->category:computers;
RELATE category:gaming_laptops->subcategory_of->category:laptops;
-- Find all children of a category
SELECT <-subcategory_of<-category.name AS children FROM category:electronics;
-- Find parent chain (breadcrumb)
SELECT
name,
->subcategory_of->category.name AS parent,
->subcategory_of->category->subcategory_of->category.name AS grandparent
FROM category:gaming_laptops;
-- Find all descendants (up to 3 levels)
SELECT
name,
array::flatten([
<-subcategory_of<-category,
<-subcategory_of<-category<-subcategory_of<-category,
<-subcategory_of<-category<-subcategory_of<-category<-subcategory_of<-category
]) AS all_descendants
FROM category:electronics;
-- Org chart example
DEFINE TABLE employee SCHEMAFULL;
DEFINE FIELD name ON TABLE employee TYPE string;
DEFINE FIELD title ON TABLE employee TYPE string;
DEFINE FIELD department ON TABLE employee TYPE string;
DEFINE TABLE reports_to SCHEMAFULL;
DEFINE FIELD in ON TABLE reports_to TYPE record<employee>;
DEFINE FIELD out ON TABLE reports_to TYPE record<employee>;
-- Build org chart
CREATE employee:ceo SET name = 'Jane', title = 'CEO', department = 'Executive';
CREATE employee:cto SET name = 'John', title = 'CTO', department = 'Engineering';
CREATE employee:lead SET name = 'Sam', title = 'Tech Lead', department = 'Engineering';
CREATE employee:dev SET name = 'Alex', title = 'Developer', department = 'Engineering';
RELATE employee:cto->reports_to->employee:ceo;
RELATE employee:lead->reports_to->employee:cto;
RELATE employee:dev->reports_to->employee:lead;
-- Full reporting chain upward
SELECT
name, title,
->reports_to->employee.name AS manager,
->reports_to->employee->reports_to->employee.name AS skip_manager,
->reports_to->employee->reports_to->employee->reports_to->employee.name AS exec
FROM employee:dev;
```
### Dependency Graphs
```surrealql
-- Package dependency management
DEFINE TABLE package SCHEMAFULL;
DEFINE FIELD name ON TABLE package TYPE string;
DEFINE FIELD version ON TABLE package TYPE string;
DEFINE TABLE depends_on SCHEMAFULL;
DEFINE FIELD in ON TABLE depends_on TYPE record<package>;
DEFINE FIELD out ON TABLE depends_on TYPE record<package>;
DEFINE FIELD version_constraint ON TABLE depends_on TYPE string;
-- Create packages and dependencies
CREATE package:app SET name = 'my-app', version = '1.0.0';
CREATE package:framework SET name = 'web-framework', version = '3.2.1';
CREATE package:orm SET name = 'orm-lib', version = '2.1.0';
CREATE package:db_driver SET name = 'db-driver', version = '1.5.0';
CREATE package:logger SET name = 'logger', version = '0.8.0';
RELATE package:app->depends_on->package:framework SET version_constraint = '^3.0.0';
RELATE package:app->depends_on->package:orm SET version_constraint = '^2.0.0';
RELATE package:orm->depends_on->package:db_driver SET version_constraint = '^1.0.0';
RELATE package:framework->depends_on->package:logger SET version_constraint = '^0.5.0';
RELATE package:orm->depends_on->package:logger SET version_constraint = '^0.7.0';
-- Find all transitive dependencies of a package
SELECT
name,
->depends_on->package.name AS direct_deps,
->depends_on->package->depends_on->package.name AS transitive_deps,
array::distinct(array::flatten([
->depends_on->package,
->depends_on->package->depends_on->package,
->depends_on->package->depends_on->package->depends_on->package
])) AS all_deps
FROM package:app;
-- Find reverse dependencies (what depends on this package?)
SELECT
name,
<-depends_on<-package.name AS depended_on_by,
<-depends_on<-package<-depends_on<-package.name AS transitive_dependents
FROM package:logger;
-- Shows that both framework and orm (and transitively, app) depend on logger
```
### Workflow and State Machine Patterns
```surrealql
-- State machine for order processing
DEFINE TABLE state SCHEMAFULL;
DEFINE FIELD name ON TABLE state TYPE string;
DEFINE FIELD description ON TABLE state TYPE string;
DEFINE TABLE transition SCHEMAFULL;
DEFINE FIELD in ON TABLE transition TYPE record<state>;
DEFINE FIELD out ON TABLE transition TYPE record<state>;
DEFINE FIELD action ON TABLE transition TYPE string;
DEFINE FIELD guard ON TABLE transition TYPE option<string>;
-- Define states
CREATE state:draft SET name = 'Draft', description = 'Order not yet submitted';
CREATE state:pending SET name = 'Pending', description = 'Awaiting approval';
CREATE state:approved SET name = 'Approved', description = 'Order approved';
CREATE state:shipped SET name = 'Shipped', description = 'Order in transit';
CREATE state:delivered SET name = 'Delivered', description = 'Order received';
CREATE state:cancelled SET name = 'Cancelled', description = 'Order cancelled';
-- Define transitions
RELATE state:draft->transition->state:pending SET action = 'submit';
RELATE state:pending->transition->state:approved SET action = 'approve', guard = 'role:manager';
RELATE state:pending->transition->state:cancelled SET action = 'cancel';
RELATE state:approved->transition->state:shipped SET action = 'ship';
RELATE state:shipped->transition->state:delivered SET action = 'deliver';
RELATE state:approved->transition->state:cancelled SET action = 'cancel';
-- Find valid transitions from current state
SELECT
->transition->state.name AS next_states,
->transition.action AS available_actions
FROM state:pending;
-- Check if a transition is valid
DEFINE FUNCTION fn::can_transition($current: record<state>, $action: string) {
LET $valid = SELECT * FROM transition
WHERE in = $current AND action = $action;
RETURN count($valid) > 0;
};
-- Apply a state transition to an order
DEFINE FUNCTION fn::apply_transition($order: record<order>, $action: string) {
LET $current_state = (SELECT VALUE current_state FROM ONLY $order);
LET $next = SELECT VALUE out FROM transition
WHERE in = $current_state AND action = $action
LIMIT 1;
IF count($next) = 0 {
THROW "Invalid transition: cannot " + $action + " from current state";
};
UPDATE $order SET
current_state = $next[0],
updated_at = time::now();
};
```
---
## Performance Considerations
### Graph Index Strategies
```surrealql
-- Index edge table fields for faster traversal
DEFINE INDEX idx_knows_in ON TABLE knows COLUMNS in;
DEFINE INDEX idx_knows_out ON TABLE knows COLUMNS out;
DEFINE INDEX idx_knows_in_out ON TABLE knows COLUMNS in, out UNIQUE;
-- Index edge properties used in filtered traversals
DEFINE INDEX idx_knows_since ON TABLE knows COLUMNS since;
DEFINE INDEX idx_knows_strength ON TABLE knows COLUMNS strength;
-- Composite index for common filter patterns
DEFINE INDEX idx_manages_role ON TABLE manages COLUMNS in, role;
```
### Traversal Depth Limits
Deep traversals can be expensive. Limit depth and result sets.
```surrealql
-- Limit results at each hop
SELECT ->(knows LIMIT 10)->person FROM person:alice;
-- Use LIMIT on the outer query
SELECT ->knows->person->knows->person AS fof
FROM person:alice
LIMIT 50;
-- Avoid unbounded multi-hop chains in production
-- BAD: unlimited depth chain
-- SELECT ->knows->person->knows->person->knows->person->knows->person->... FROM person:alice;
-- GOOD: bounded depth with explicit limits
SELECT
->knows->(person LIMIT 20) AS depth_1
FROM person:alice;
```
### Caching Patterns for Frequent Traversals
```surrealql
-- Precompute common graph aggregates
DEFINE EVENT update_friend_count ON TABLE knows WHEN $event = "CREATE" THEN {
UPDATE $after.in SET friend_count += 1;
};
DEFINE EVENT decrement_friend_count ON TABLE knows WHEN $event = "DELETE" THEN {
UPDATE $before.in SET friend_count -= 1;
};
-- Materialized view pattern: store computed traversal results
DEFINE TABLE user_stats SCHEMAFULL;
DEFINE FIELD user ON TABLE user_stats TYPE record<person>;
DEFINE FIELD friend_count ON TABLE user_stats TYPE int;
DEFINE FIELD follower_count ON TABLE user_stats TYPE int;
DEFINE FIELD following_count ON TABLE user_stats TYPE int;
-- Periodically refresh with a function
DEFINE FUNCTION fn::refresh_user_stats($user: record<person>) {
UPSERT user_stats SET
user = $user,
friend_count = count((SELECT ->friends_with->person FROM ONLY $user)),
follower_count = count((SELECT <-follows<-person FROM ONLY $user)),
following_count = count((SELECT ->follows->person FROM ONLY $user));
};
```
### General Tips
- Index the `in` and `out` columns on edge tables for faster traversal lookups.
- For large graphs, prefer shallow traversals (1-2 hops) and use application logic or stored functions for deeper searches.
- Use `LIMIT` within filtered traversals to cap intermediate result sets.
- Avoid cartesian explosions: chaining multiple multi-target traversals can produce exponentially large intermediate results.
- Use SCHEMAFULL edge tables with typed `in`/`out` fields to prevent invalid relationships and improve query planning.
- Consider precomputing and caching graph metrics (degree, centrality) on the node records themselves if they are queried frequently.
- Use `EXPLAIN` (covered in the performance rules) to understand traversal query plans.
```
### rules/vector-search.md
```markdown
# SurrealDB Vector Search
SurrealDB provides native vector storage and similarity search capabilities, making it suitable for AI/ML applications including RAG (Retrieval-Augmented Generation), semantic search, recommendations, and classification. Vectors are stored as array fields and searched using HNSW indexes with KNN operators.
---
## Vector Storage
### Storing Embeddings as Arrays
Vectors in SurrealDB are stored as standard array fields. Use typed array fields with float elements for embeddings.
```surrealql
-- Basic vector storage
CREATE document:1 SET
title = 'Introduction to SurrealDB',
content = 'SurrealDB is a multi-model database...',
embedding = [0.123, -0.456, 0.789, ...];
-- With explicit type definitions
DEFINE TABLE document SCHEMAFULL;
DEFINE FIELD title ON TABLE document TYPE string;
DEFINE FIELD content ON TABLE document TYPE string;
DEFINE FIELD embedding ON TABLE document TYPE array<float>;
DEFINE FIELD metadata ON TABLE document TYPE object;
DEFINE FIELD created_at ON TABLE document TYPE datetime DEFAULT time::now();
-- Store a high-dimensional embedding (e.g., OpenAI text-embedding-3-large)
CREATE document:2 SET
title = 'Vector Search Guide',
content = 'This guide covers vector similarity...',
embedding = [0.0012, -0.0034, 0.0056, ...], -- 3072 dimensions
metadata = { source: 'docs', chunk_index: 0 };
```
### Dimension Specifications
The embedding dimension must match the HNSW index definition exactly. Common dimensions from popular embedding models:
| Model | Dimensions |
|---|---|
| OpenAI text-embedding-3-small | 1536 |
| OpenAI text-embedding-3-large | 3072 |
| Cohere embed-v4 | 1024 |
| Mistral Embed | 1024 |
| Google text-embedding-005 | 768 |
| BAAI/bge-large-en-v1.5 | 1024 |
| all-MiniLM-L6-v2 | 384 |
```surrealql
-- Dimension must match between data and index
-- If your embedding model outputs 1536 dimensions:
DEFINE INDEX idx_embed ON TABLE document FIELDS embedding HNSW DIMENSION 1536;
-- Inserting a vector of wrong dimension will cause an index error
-- BAD: 768-dim vector into 1536-dim index
-- GOOD: ensure all vectors match the declared dimension
```
### Supported Data Types for Vectors
```surrealql
-- Float arrays (most common, default)
DEFINE FIELD embedding ON TABLE document TYPE array<float>;
-- The HNSW TYPE parameter controls storage precision in the index:
-- F32 (default) - 32-bit floating point, best accuracy
-- F64 - 64-bit floating point, highest precision, more memory
-- I16 - 16-bit integer, quantized, less memory
-- I32 - 32-bit integer
-- Example with explicit index type
DEFINE INDEX idx_embed ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE TYPE F32;
```
---
## Index Types
### HNSW (Hierarchical Navigable Small World)
HNSW is the primary (and as of SurrealDB 3.x, the only) vector index type. It builds a multi-layered graph structure for approximate nearest neighbor search with sub-linear query time.
Note: MTREE was deprecated in SurrealDB 2.x and removed in 3.x. All vector indexes should use HNSW.
#### Full Configuration Options
```surrealql
DEFINE INDEX index_name ON TABLE table_name FIELDS field_name
HNSW DIMENSION dim
[DIST distance_function]
[TYPE storage_type]
[EFC construction_ef]
[M max_connections]
[LM max_connections_layer0]
[EXTEND_CANDIDATES]
[KEEP_PRUNED_CONNECTIONS];
```
| Parameter | Description | Default | Guidance |
|---|---|---|---|
| DIMENSION | Vector dimensionality (must match data) | Required | Match your embedding model |
| DIST | Distance function: COSINE, EUCLIDEAN, MANHATTAN, MINKOWSKI, CHEBYSHEV, HAMMING, JACCARD, PEARSON | EUCLIDEAN | COSINE for normalized embeddings (most common) |
| TYPE | Storage type: F32, F64, I16, I32, I64 | F32 | F32 balances precision and memory |
| EFC | ef_construction: neighbors explored during build | 150 | Higher = better recall, slower build |
| M | Max connections per node (upper layers) | 12 | Higher = better recall, more memory |
| LM | Max connections at layer 0 | 2*M | Usually leave as default |
| EXTEND_CANDIDATES | Extend candidate list during construction | Off | Enable for higher recall |
| KEEP_PRUNED_CONNECTIONS | Keep pruned connections | Off | Enable for higher recall |
#### Common Index Configurations
```surrealql
-- Standard cosine similarity index (most common for text embeddings)
DEFINE INDEX idx_doc_embedding ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- High-recall configuration for critical search
DEFINE INDEX idx_high_recall ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536
DIST COSINE
TYPE F32
EFC 300
M 32
EXTEND_CANDIDATES
KEEP_PRUNED_CONNECTIONS;
-- Memory-efficient configuration for large datasets
DEFINE INDEX idx_compact ON TABLE document
FIELDS embedding
HNSW DIMENSION 384
DIST COSINE
TYPE I16
EFC 100
M 8;
-- Euclidean distance for spatial/geometric data
DEFINE INDEX idx_spatial ON TABLE location
FIELDS coordinates
HNSW DIMENSION 3 DIST EUCLIDEAN;
-- Manhattan distance for grid-based or feature-counting data
DEFINE INDEX idx_features ON TABLE item
FIELDS feature_vector
HNSW DIMENSION 128 DIST MANHATTAN;
```
#### When to Use HNSW
HNSW is the right choice for all vector search workloads in SurrealDB 3.x. It provides:
- Sub-linear query time (logarithmic in dataset size)
- High recall rates (typically 95-99% with good parameters)
- Good performance on high-dimensional data
- Support for incremental inserts without full rebuild
Trade-offs to be aware of:
- Memory usage scales with M and dimension count
- Build time increases with EFC and dataset size
- Approximate results (not exact KNN) -- tune EFC and M for desired recall
### MTREE (Removed in 3.x)
MTREE was deprecated in SurrealDB 2.x and fully removed in SurrealDB 3.x. If migrating from 2.x, replace all MTREE index definitions with HNSW.
```surrealql
-- OLD (SurrealDB 2.x) - no longer works
-- DEFINE INDEX idx ON TABLE doc FIELDS embedding MTREE DIMENSION 1536;
-- NEW (SurrealDB 3.x) - use HNSW instead
DEFINE INDEX idx ON TABLE doc FIELDS embedding HNSW DIMENSION 1536 DIST COSINE;
```
---
## Search Patterns
### Basic KNN Search
The `<|K|>` operator performs K-nearest-neighbor search using the defined HNSW index.
```surrealql
-- Create index
DEFINE INDEX idx_embedding ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- Insert documents with embeddings
CREATE document:1 SET title = 'SurrealDB Intro', embedding = [...];
CREATE document:2 SET title = 'Graph Databases', embedding = [...];
CREATE document:3 SET title = 'Vector Search', embedding = [...];
-- Find 10 nearest neighbors
LET $query_vector = [0.012, -0.034, 0.056, ...]; -- 1536 dimensions
SELECT
id,
title,
vector::distance::knn() AS distance
FROM document
WHERE embedding <|10|> $query_vector
ORDER BY distance;
```
### KNN with Distance Threshold
```surrealql
-- Find 10 nearest neighbors within a maximum distance of 40
LET $query_vector = [0.012, -0.034, 0.056, ...];
SELECT
id,
title,
vector::distance::knn() AS distance
FROM document
WHERE embedding <|10, 40|> $query_vector
ORDER BY distance;
```
### Vector Similarity Functions
SurrealDB provides built-in vector distance and similarity functions.
```surrealql
-- Cosine similarity (1 = identical, 0 = orthogonal, -1 = opposite)
SELECT
id, title,
vector::similarity::cosine(embedding, $query_vector) AS similarity
FROM document
ORDER BY similarity DESC
LIMIT 10;
-- Euclidean distance (0 = identical, higher = more different)
SELECT
id, title,
vector::distance::euclidean(embedding, $query_vector) AS distance
FROM document
ORDER BY distance ASC
LIMIT 10;
-- Manhattan distance
SELECT
id, title,
vector::distance::manhattan(embedding, $query_vector) AS distance
FROM document
ORDER BY distance ASC
LIMIT 10;
-- Chebyshev distance
SELECT
id, title,
vector::distance::chebyshev(embedding, $query_vector) AS distance
FROM document
ORDER BY distance ASC;
-- Hamming distance (for binary/integer vectors)
SELECT
id,
vector::distance::hamming(binary_features, $query_features) AS distance
FROM item
ORDER BY distance ASC;
```
### Combining KNN Index Search with Computed Similarity
```surrealql
-- Use the HNSW index for fast candidate retrieval,
-- then compute exact similarity for ranking
LET $query_vector = [0.012, -0.034, 0.056, ...];
SELECT
id,
title,
vector::distance::knn() AS approx_distance,
vector::similarity::cosine(embedding, $query_vector) AS exact_similarity
FROM document
WHERE embedding <|20|> $query_vector
ORDER BY exact_similarity DESC
LIMIT 10;
```
---
## Hybrid Search Patterns
### Vector + Full-Text Search
Combine vector similarity with SurrealDB's full-text search for hybrid retrieval.
```surrealql
-- Define both indexes
DEFINE ANALYZER doc_analyzer TOKENIZERS blank, class FILTERS lowercase, snowball(english);
DEFINE INDEX idx_ft_content ON TABLE document
FIELDS content
SEARCH ANALYZER doc_analyzer BM25;
DEFINE INDEX idx_vec_embedding ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- Full-text search only
SELECT id, title, search::score(1) AS text_score
FROM document
WHERE content @1@ 'vector database performance'
ORDER BY text_score DESC
LIMIT 10;
-- Vector search only
SELECT id, title, vector::distance::knn() AS vec_distance
FROM document
WHERE embedding <|10|> $query_vector
ORDER BY vec_distance;
-- Hybrid approach: union results from both methods
-- Step 1: Get text search candidates
LET $text_results = SELECT id, title, search::score(1) AS score, 'text' AS source
FROM document
WHERE content @1@ 'vector database performance'
ORDER BY score DESC
LIMIT 20;
-- Step 2: Get vector search candidates
LET $vec_results = SELECT id, title, vector::distance::knn() AS score, 'vector' AS source
FROM document
WHERE embedding <|20|> $query_vector
ORDER BY score;
-- Step 3: Merge and deduplicate
-- Application-level fusion is recommended for proper score normalization
```
### Vector + Metadata Filtering
```surrealql
-- Pre-filter by metadata, then vector search
-- Note: The metadata filter narrows the candidate set before KNN
LET $query_vector = [0.012, -0.034, 0.056, ...];
-- Filter by category and date, then find nearest vectors
SELECT
id, title,
vector::distance::knn() AS distance
FROM document
WHERE
category = 'engineering'
AND created_at > d'2025-01-01'
AND embedding <|10|> $query_vector
ORDER BY distance;
-- Filter by tags
SELECT
id, title,
vector::distance::knn() AS distance
FROM document
WHERE
'surrealdb' IN tags
AND status = 'published'
AND embedding <|10|> $query_vector
ORDER BY distance;
-- Multi-tenant vector search
SELECT
id, title,
vector::distance::knn() AS distance
FROM document
WHERE
tenant_id = $auth.tenant
AND embedding <|10|> $query_vector
ORDER BY distance;
```
---
## RAG (Retrieval-Augmented Generation) Patterns
### Document Chunking and Embedding Storage
```surrealql
-- Schema for chunked documents
DEFINE TABLE source_document SCHEMAFULL;
DEFINE FIELD title ON TABLE source_document TYPE string;
DEFINE FIELD url ON TABLE source_document TYPE option<string>;
DEFINE FIELD content ON TABLE source_document TYPE string;
DEFINE FIELD doc_type ON TABLE source_document TYPE string;
DEFINE FIELD created_at ON TABLE source_document TYPE datetime DEFAULT time::now();
DEFINE TABLE chunk SCHEMAFULL;
DEFINE FIELD source ON TABLE chunk TYPE record<source_document>;
DEFINE FIELD content ON TABLE chunk TYPE string;
DEFINE FIELD embedding ON TABLE chunk TYPE array<float>;
DEFINE FIELD chunk_index ON TABLE chunk TYPE int;
DEFINE FIELD token_count ON TABLE chunk TYPE int;
DEFINE FIELD metadata ON TABLE chunk TYPE object DEFAULT {};
DEFINE FIELD created_at ON TABLE chunk TYPE datetime DEFAULT time::now();
-- HNSW index on chunk embeddings
DEFINE INDEX idx_chunk_embedding ON TABLE chunk
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- Index for filtering by source
DEFINE INDEX idx_chunk_source ON TABLE chunk COLUMNS source;
-- Store a document and its chunks
CREATE source_document:doc1 SET
title = 'SurrealDB Documentation',
url = 'https://surrealdb.com/docs',
content = 'Full document content...',
doc_type = 'documentation';
-- Store chunks with embeddings (generated externally)
CREATE chunk SET
source = source_document:doc1,
content = 'SurrealDB is a multi-model database...',
embedding = [0.012, -0.034, ...],
chunk_index = 0,
token_count = 256,
metadata = { section: 'introduction' };
CREATE chunk SET
source = source_document:doc1,
content = 'Vector search in SurrealDB uses HNSW...',
embedding = [0.045, -0.067, ...],
chunk_index = 1,
token_count = 312,
metadata = { section: 'vector-search' };
```
### Context Window Assembly for RAG
```surrealql
-- Retrieve relevant chunks for a query
LET $query_embedding = [0.012, -0.034, 0.056, ...];
-- Step 1: Find most relevant chunks
LET $relevant_chunks = SELECT
id,
content,
source,
chunk_index,
token_count,
vector::distance::knn() AS distance
FROM chunk
WHERE embedding <|10|> $query_embedding
ORDER BY distance;
-- Step 2: Get surrounding chunks for context (context window expansion)
LET $expanded = SELECT
c.id,
c.content,
c.chunk_index,
c.source,
c.token_count
FROM chunk AS c
WHERE c.source IN (SELECT VALUE source FROM $relevant_chunks)
AND c.chunk_index >= (
SELECT VALUE chunk_index FROM $relevant_chunks WHERE source = c.source LIMIT 1
) - 1
AND c.chunk_index <= (
SELECT VALUE chunk_index FROM $relevant_chunks WHERE source = c.source LIMIT 1
) + 1
ORDER BY c.source, c.chunk_index;
-- Step 3: Assemble context with source attribution
SELECT
content,
source.title AS source_title,
source.url AS source_url,
chunk_index
FROM $expanded
ORDER BY chunk_index;
```
### Multi-Collection RAG Search
```surrealql
-- Search across multiple document types
LET $query_embedding = [0.012, -0.034, 0.056, ...];
-- Search documentation chunks
LET $docs = SELECT
id, content, 'documentation' AS type,
vector::distance::knn() AS distance
FROM doc_chunk
WHERE embedding <|5|> $query_embedding
ORDER BY distance;
-- Search FAQ entries
LET $faqs = SELECT
id, answer AS content, 'faq' AS type,
vector::distance::knn() AS distance
FROM faq_entry
WHERE embedding <|5|> $query_embedding
ORDER BY distance;
-- Search knowledge base
LET $kb = SELECT
id, content, 'knowledge_base' AS type,
vector::distance::knn() AS distance
FROM kb_article_chunk
WHERE embedding <|5|> $query_embedding
ORDER BY distance;
```
---
## AI Integration Patterns
### Embedding Generation Pipeline
```surrealql
-- Track embedding generation status
DEFINE TABLE embedding_job SCHEMAFULL;
DEFINE FIELD source_table ON TABLE embedding_job TYPE string;
DEFINE FIELD source_id ON TABLE embedding_job TYPE record;
DEFINE FIELD status ON TABLE embedding_job TYPE string
ASSERT $value IN ['pending', 'processing', 'completed', 'failed'];
DEFINE FIELD error ON TABLE embedding_job TYPE option<string>;
DEFINE FIELD created_at ON TABLE embedding_job TYPE datetime DEFAULT time::now();
DEFINE FIELD completed_at ON TABLE embedding_job TYPE option<datetime>;
-- Event-driven embedding generation
-- When a document is created, queue an embedding job
DEFINE EVENT on_document_create ON TABLE document WHEN $event = "CREATE" THEN {
CREATE embedding_job SET
source_table = 'document',
source_id = $after.id,
status = 'pending';
};
-- When a document is updated, re-queue embedding
DEFINE EVENT on_document_update ON TABLE document WHEN $event = "UPDATE"
AND $before.content != $after.content THEN {
CREATE embedding_job SET
source_table = 'document',
source_id = $after.id,
status = 'pending';
};
```
### Semantic Similarity Search
```surrealql
-- Find semantically similar documents
LET $source_embedding = (SELECT VALUE embedding FROM ONLY document:target);
SELECT
id,
title,
vector::similarity::cosine(embedding, $source_embedding) AS similarity
FROM document
WHERE id != document:target
AND embedding <|20|> $source_embedding
ORDER BY similarity DESC
LIMIT 10;
-- Semantic deduplication: find near-duplicates
SELECT
d1.id AS doc_a,
d2.id AS doc_b,
vector::similarity::cosine(d1.embedding, d2.embedding) AS similarity
FROM document AS d1, document AS d2
WHERE d1.id < d2.id
AND vector::similarity::cosine(d1.embedding, d2.embedding) > 0.95
ORDER BY similarity DESC;
```
### Clustering with Vector Data
```surrealql
-- Assign cluster labels based on centroid similarity
-- (Centroids computed externally, stored in SurrealDB)
DEFINE TABLE cluster_centroid SCHEMAFULL;
DEFINE FIELD label ON TABLE cluster_centroid TYPE string;
DEFINE FIELD centroid ON TABLE cluster_centroid TYPE array<float>;
DEFINE FIELD description ON TABLE cluster_centroid TYPE string;
DEFINE INDEX idx_centroid ON TABLE cluster_centroid
FIELDS centroid
HNSW DIMENSION 1536 DIST COSINE;
-- Assign documents to nearest cluster
LET $doc = SELECT * FROM ONLY document:target;
SELECT
label,
description,
vector::similarity::cosine(centroid, $doc.embedding) AS similarity
FROM cluster_centroid
WHERE centroid <|1|> $doc.embedding;
-- Batch assignment: label all unassigned documents
UPDATE document SET cluster = (
SELECT VALUE label FROM cluster_centroid
WHERE centroid <|1|> $parent.embedding
LIMIT 1
) WHERE cluster IS NONE;
```
### Classification Using Stored Vectors
```surrealql
-- KNN classification: classify by majority vote of nearest labeled examples
DEFINE TABLE labeled_example SCHEMAFULL;
DEFINE FIELD text ON TABLE labeled_example TYPE string;
DEFINE FIELD embedding ON TABLE labeled_example TYPE array<float>;
DEFINE FIELD label ON TABLE labeled_example TYPE string;
DEFINE INDEX idx_labeled ON TABLE labeled_example
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- Classify a new item by finding K nearest labeled examples
LET $new_embedding = [0.012, -0.034, ...];
SELECT
label,
count() AS votes,
math::mean(vector::similarity::cosine(embedding, $new_embedding)) AS avg_similarity
FROM labeled_example
WHERE embedding <|5|> $new_embedding
GROUP BY label
ORDER BY votes DESC, avg_similarity DESC
LIMIT 1;
```
---
## Production Considerations
### Index Build Time and Memory
```surrealql
-- Check index status
INFO FOR TABLE document;
-- Shows index definitions and their status
-- For large datasets, index building happens in the background
-- Monitor by checking if queries use the index:
-- If the index is still building, queries fall back to brute-force scan
-- Tune build parameters for your hardware:
-- More RAM available -> higher EFC and M for better recall
-- Limited RAM -> lower M and use I16 type for compression
-- Small dataset (<10K vectors): defaults are fine
DEFINE INDEX idx_small ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- Medium dataset (10K-1M vectors): tune M and EFC
DEFINE INDEX idx_medium ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE
EFC 200
M 16;
-- Large dataset (>1M vectors): optimize for memory
DEFINE INDEX idx_large ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE
TYPE I16
EFC 150
M 12;
```
### Query Latency Optimization
```surrealql
-- Return only the fields you need (avoid SELECT *)
SELECT id, title, vector::distance::knn() AS distance
FROM document
WHERE embedding <|10|> $query_vector
ORDER BY distance;
-- Pre-filter to reduce the candidate set
SELECT id, title, vector::distance::knn() AS distance
FROM document
WHERE category = 'engineering' -- metadata filter first
AND embedding <|10|> $query_vector -- then vector search
ORDER BY distance;
-- Use appropriate K value -- don't over-fetch
-- BAD: fetching 1000 when you only need 10
-- WHERE embedding <|1000|> $query_vector
-- GOOD: fetch slightly more than needed, then post-filter
-- WHERE embedding <|20|> $query_vector -- fetch 20, return top 10
```
### Embedding Dimension Trade-offs
Higher dimensions provide more representational capacity but cost more in storage and query time.
| Dimension | Storage/vector | Index Memory | Query Speed | Use Case |
|---|---|---|---|---|
| 384 | 1.5 KB | Low | Fast | Simple similarity, prototyping |
| 768 | 3 KB | Medium | Good | General purpose |
| 1024 | 4 KB | Medium | Good | Balanced accuracy/speed |
| 1536 | 6 KB | High | Moderate | High accuracy text search |
| 3072 | 12 KB | Very High | Slower | Maximum accuracy |
```surrealql
-- If your embedding model supports dimension reduction (e.g., Matryoshka),
-- use the smallest dimension that meets your accuracy requirements.
-- Test with your actual data to find the sweet spot.
```
### Batch Insertion Patterns
```surrealql
-- Insert vectors in batches for better throughput
-- Single batch insert (preferred for bulk loading)
INSERT INTO document [
{ id: document:1, title: 'Doc 1', embedding: [0.1, 0.2, ...], content: '...' },
{ id: document:2, title: 'Doc 2', embedding: [0.3, 0.4, ...], content: '...' },
{ id: document:3, title: 'Doc 3', embedding: [0.5, 0.6, ...], content: '...' }
];
-- For very large imports, batch in groups of 100-500 records
-- to balance throughput with memory usage.
-- The HNSW index updates incrementally with each insert.
```
### Index Rebuild Strategies
```surrealql
-- If index quality degrades after many updates/deletes,
-- rebuild by removing and recreating
-- Remove existing index
REMOVE INDEX idx_embedding ON TABLE document;
-- Recreate with potentially tuned parameters
DEFINE INDEX idx_embedding ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE
EFC 200
M 16;
-- The index will rebuild in the background from existing data.
-- Queries will work during rebuild but may be slower (brute-force fallback).
```
### Distance Function Selection Guide
| Function | Best For | Normalized Vectors | Range |
|---|---|---|---|
| COSINE | Text embeddings, semantic similarity | Yes (recommended) | [0, 2] (distance) |
| EUCLIDEAN | Spatial data, coordinate systems | No | [0, inf) |
| MANHATTAN | Feature counting, grid distances | No | [0, inf) |
| CHEBYSHEV | Worst-case dimension difference | No | [0, inf) |
| HAMMING | Binary features, hash comparison | N/A | [0, dim] |
| JACCARD | Set similarity | N/A | [0, 1] |
| PEARSON | Correlation-based similarity | No | [-1, 1] |
Most text embedding models produce normalized vectors, making COSINE the standard choice. If you are unsure, use COSINE.
```
### rules/deployment.md
```markdown
# SurrealDB Deployment and Operations Guide
This document covers installation, deployment patterns, operational tasks, and monitoring for SurrealDB 3.0.0 across local development, containerized, orchestrated, and cloud environments.
---
## Local Development
### Installation
```bash
# macOS (Homebrew) -- RECOMMENDED
brew install surrealdb/tap/surreal
# Linux (apt / package manager)
# See https://surrealdb.com/docs/surrealdb/installation for distro-specific instructions
# Docker (no host install required)
docker pull surrealdb/surrealdb:v3
# Verify installation
surreal version
```
> **Security note**: SurrealDB's website offers a `curl | sh` installer. For
> auditable installs, prefer your OS package manager (brew, apt, dnf) or Docker.
> If you must use a remote installer, download the script first, review it, then
> execute: `curl -sSf https://install.surrealdb.com -o install.sh && less install.sh && sh install.sh`
### Starting the Server
> **Credential warning**: Examples below use `root/root` for local development.
> For production, use strong credentials and DEFINE USER with least-privilege access.
```bash
# In-memory (data lost on restart, LOCAL DEVELOPMENT ONLY)
surreal start --log trace --user root --pass root memory
# RocksDB persistent storage (local dev)
surreal start --log info --user root --pass root rocksdb:./mydata.db
# SurrealKV persistent storage
surreal start --log info --user root --pass root surrealkv:./mydata.db
# SurrealKV with versioned storage (supports temporal queries)
surreal start --log info --user root --pass root surrealkv+versioned:./mydata.db
# TiKV distributed storage
surreal start --log info --user root --pass root tikv://pd0:2379
# Custom bind address and port (use 127.0.0.1 for local dev, 0.0.0.0 only in containers)
surreal start --bind 127.0.0.1:9000 --user root --pass root memory
# With TLS
surreal start --user root --pass root \
--web-crt ./cert.pem --web-key ./key.pem \
memory
```
### CLI Operations
```bash
# Interactive SQL shell
surreal sql \
--endpoint http://localhost:8000 \
--username root \
--password root \
--namespace test \
--database test
# Import data from a .surql file
surreal import \
--endpoint http://localhost:8000 \
--username root \
--password root \
--namespace test \
--database test \
data.surql
# Export database to a .surql file
surreal export \
--endpoint http://localhost:8000 \
--username root \
--password root \
--namespace test \
--database test \
> backup.surql
# Check version
surreal version
# Validate a SurrealQL file
surreal validate myfile.surql
# Upgrade storage from v1 to v2 format
surreal upgrade --path ./mydata.db
```
### Configuration Flags Reference
| Flag | Description | Default |
|------|-------------|---------|
| `--bind` | Listen address and port | `0.0.0.0:8000` (use `127.0.0.1:8000` for local dev) |
| `--log` | Log level (none, full, error, warn, info, debug, trace) | `info` |
| `--user` | Root username | Required |
| `--pass` | Root password | Required |
| `--strict` | Strict mode (require explicit namespace/database) | `false` |
| `--web-crt` | Path to TLS certificate | None |
| `--web-key` | Path to TLS private key | None |
| `--client-crt` | Path to client CA certificate (mTLS) | None |
| `--client-key` | Path to client key (mTLS) | None |
| `--no-banner` | Hide startup banner | `false` |
| `--query-timeout` | Maximum query execution time | None |
| `--transaction-timeout` | Maximum transaction duration | None |
---
## Docker Deployment
### Basic Docker Run
```bash
# In-memory
docker run --rm -p 8000:8000 \
surrealdb/surrealdb:v3 \
start --log info --user root --pass root memory
# With persistent volume
docker run --rm -p 8000:8000 \
-v $(pwd)/data:/data \
surrealdb/surrealdb:v3 \
start --log info --user root --pass root rocksdb:/data/mydb
```
### Dockerfile
```dockerfile
FROM surrealdb/surrealdb:v3
# Default environment variables
ENV SURREAL_LOG=info
ENV SURREAL_USER=root
ENV SURREAL_PASS=root
ENV SURREAL_BIND=0.0.0.0:8000
# Health check
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
CMD surreal version || exit 1
EXPOSE 8000
CMD ["start", "--log", "info", "--user", "root", "--pass", "root", "surrealkv:/data/db"]
```
### Docker Compose -- Single Node
```yaml
version: "3.8"
services:
surrealdb:
image: surrealdb/surrealdb:v3
command: start --log info --user root --pass root surrealkv:/data/db
ports:
- "8000:8000"
volumes:
- surrealdb-data:/data
restart: unless-stopped
healthcheck:
test: ["CMD", "surreal", "version"]
interval: 30s
timeout: 5s
retries: 3
deploy:
resources:
limits:
cpus: "2.0"
memory: 4G
reservations:
cpus: "0.5"
memory: 1G
volumes:
surrealdb-data:
driver: local
```
### Docker Compose -- TiKV Cluster
```yaml
version: "3.8"
services:
pd0:
image: pingcap/pd:latest
command:
- --name=pd0
- --client-urls=http://0.0.0.0:2379
- --peer-urls=http://0.0.0.0:2380
- --advertise-client-urls=http://pd0:2379
- --advertise-peer-urls=http://pd0:2380
- --initial-cluster=pd0=http://pd0:2380
- --data-dir=/data/pd0
volumes:
- pd0-data:/data
ports:
- "2379:2379"
tikv0:
image: pingcap/tikv:latest
command:
- --addr=0.0.0.0:20160
- --advertise-addr=tikv0:20160
- --pd=pd0:2379
- --data-dir=/data/tikv0
volumes:
- tikv0-data:/data
depends_on:
- pd0
tikv1:
image: pingcap/tikv:latest
command:
- --addr=0.0.0.0:20160
- --advertise-addr=tikv1:20160
- --pd=pd0:2379
- --data-dir=/data/tikv1
volumes:
- tikv1-data:/data
depends_on:
- pd0
tikv2:
image: pingcap/tikv:latest
command:
- --addr=0.0.0.0:20160
- --advertise-addr=tikv2:20160
- --pd=pd0:2379
- --data-dir=/data/tikv2
volumes:
- tikv2-data:/data
depends_on:
- pd0
surrealdb:
image: surrealdb/surrealdb:v3
command: start --log info --user root --pass root tikv://pd0:2379
ports:
- "8000:8000"
depends_on:
- tikv0
- tikv1
- tikv2
restart: unless-stopped
volumes:
pd0-data:
tikv0-data:
tikv1-data:
tikv2-data:
```
---
## Kubernetes Deployment
### Helm Chart (Single Node with SurrealKV)
```yaml
# values.yaml
replicaCount: 1
image:
repository: surrealdb/surrealdb
tag: v3.0.0
pullPolicy: IfNotPresent
service:
type: ClusterIP
port: 8000
storage:
engine: surrealkv
size: 50Gi
storageClass: standard
auth:
rootUsername: root
rootPassword: "" # Set via secret
existingSecret: surrealdb-auth
resources:
limits:
cpu: "4"
memory: 8Gi
requests:
cpu: "1"
memory: 2Gi
ingress:
enabled: true
className: nginx
hosts:
- host: surrealdb.example.com
paths:
- path: /
pathType: Prefix
tls:
- secretName: surrealdb-tls
hosts:
- surrealdb.example.com
```
### StatefulSet for Persistent Storage
```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: surrealdb
labels:
app: surrealdb
spec:
serviceName: surrealdb
replicas: 1
selector:
matchLabels:
app: surrealdb
template:
metadata:
labels:
app: surrealdb
spec:
containers:
- name: surrealdb
image: surrealdb/surrealdb:v3
args:
- start
- --log
- info
- --user
- $(SURREAL_USER)
- --pass
- $(SURREAL_PASS)
- surrealkv:/data/db
ports:
- containerPort: 8000
name: http
envFrom:
- secretRef:
name: surrealdb-auth
volumeMounts:
- name: data
mountPath: /data
resources:
limits:
cpu: "4"
memory: 8Gi
requests:
cpu: "1"
memory: 2Gi
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 5
periodSeconds: 10
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: standard
resources:
requests:
storage: 50Gi
```
### Deployment for Stateless Compute (TiKV Backend)
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: surrealdb
spec:
replicas: 3
selector:
matchLabels:
app: surrealdb
template:
metadata:
labels:
app: surrealdb
spec:
containers:
- name: surrealdb
image: surrealdb/surrealdb:v3
args:
- start
- --log
- info
- --user
- $(SURREAL_USER)
- --pass
- $(SURREAL_PASS)
- tikv://tikv-pd:2379
ports:
- containerPort: 8000
envFrom:
- secretRef:
name: surrealdb-auth
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: "500m"
memory: 1Gi
```
### Service and Ingress
```yaml
apiVersion: v1
kind: Service
metadata:
name: surrealdb
spec:
type: ClusterIP
ports:
- port: 8000
targetPort: 8000
protocol: TCP
name: http
selector:
app: surrealdb
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: surrealdb
annotations:
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/websocket-services: surrealdb
spec:
ingressClassName: nginx
tls:
- hosts:
- surrealdb.example.com
secretName: surrealdb-tls
rules:
- host: surrealdb.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: surrealdb
port:
number: 8000
```
### Horizontal Pod Autoscaling
```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: surrealdb
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: surrealdb
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
```
---
## Distributed Deployment (TiKV)
TiKV provides distributed, transactional key-value storage for SurrealDB. In this architecture, SurrealDB compute nodes are stateless and connect to a shared TiKV cluster.
### Architecture
```
Load Balancer
/ | \
SurrealDB SurrealDB SurrealDB
(compute) (compute) (compute)
\ | /
TiKV Cluster
/ | \
TiKV-0 TiKV-1 TiKV-2
\ | /
PD (Placement Driver)
/ | \
PD-0 PD-1 PD-2
```
### Component Configuration
**Placement Driver (PD)** -- manages cluster metadata and scheduling:
```bash
pd-server \
--name=pd0 \
--data-dir=/var/lib/pd \
--client-urls=http://0.0.0.0:2379 \
--peer-urls=http://0.0.0.0:2380 \
--advertise-client-urls=http://pd0:2379 \
--advertise-peer-urls=http://pd0:2380 \
--initial-cluster="pd0=http://pd0:2380,pd1=http://pd1:2380,pd2=http://pd2:2380"
```
**TiKV Nodes** -- store data:
```bash
tikv-server \
--addr=0.0.0.0:20160 \
--advertise-addr=tikv0:20160 \
--data-dir=/var/lib/tikv \
--pd=pd0:2379,pd1:2379,pd2:2379
```
**SurrealDB Compute Nodes** -- stateless query processing:
```bash
surreal start \
--log info \
--user root \
--pass root \
--bind 0.0.0.0:8000 \
tikv://pd0:2379,pd1:2379,pd2:2379
```
### Fault Tolerance
TiKV uses Raft consensus and maintains 3 replicas by default. The cluster can tolerate `replicas - 1` failures (1 failure with 3 replicas). Since SurrealDB compute nodes are stateless, they can be freely added or removed without data implications.
### Network Topology Considerations
- PD nodes communicate on port 2379 (client) and 2380 (peer)
- TiKV nodes communicate on port 20160
- SurrealDB nodes need access to PD nodes only (not directly to TiKV)
- Place TiKV nodes in the same datacenter for low latency
- For cross-region, configure TiKV placement rules to control replica locations
---
## Cloud Deployment
### SurrealDB Cloud (Managed Service)
SurrealDB Cloud is the managed database service available at app.surrealdb.com. It provides:
- Provisioning via web console or API
- Automatic backups
- Scaling controls
- Monitoring dashboard
- No infrastructure management
Connect using standard SDK connection strings with the provided cloud endpoint.
### AWS EKS
```bash
# Create EKS cluster
eksctl create cluster \
--name surrealdb-cluster \
--region us-east-1 \
--nodegroup-name standard \
--node-type m5.xlarge \
--nodes 3
# Install SurrealDB using Kubernetes manifests or Helm
kubectl apply -f surrealdb-statefulset.yaml
```
Use `gp3` storage class for EBS-backed persistent volumes. For TiKV deployments, use the TiKV Operator for Kubernetes.
### Google GKE
```bash
# Create GKE cluster
gcloud container clusters create surrealdb-cluster \
--zone us-central1-a \
--num-nodes 3 \
--machine-type n2-standard-4
# Deploy SurrealDB
kubectl apply -f surrealdb-statefulset.yaml
```
Use `standard-rwo` storage class for SSD-backed persistent volumes on GKE.
### Azure AKS
```bash
# Create AKS cluster
az aks create \
--resource-group surrealdb-rg \
--name surrealdb-cluster \
--node-count 3 \
--node-vm-size Standard_D4s_v3 \
--generate-ssh-keys
# Deploy SurrealDB
kubectl apply -f surrealdb-statefulset.yaml
```
Use `managed-premium` storage class for premium SSD volumes on AKS.
---
## Operational Tasks
### Backup and Restore
```bash
# Export a database (backup)
surreal export \
--endpoint http://localhost:8000 \
--username root \
--password root \
--namespace production \
--database app \
> backup_$(date +%Y%m%d_%H%M%S).surql
# Import a database (restore)
surreal import \
--endpoint http://localhost:8000 \
--username root \
--password root \
--namespace production \
--database app \
backup_20260219_120000.surql
```
For automated backups, create a cron job or Kubernetes CronJob:
```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: surrealdb-backup
spec:
schedule: "0 2 * * *" # Daily at 2 AM
jobTemplate:
spec:
template:
spec:
containers:
- name: backup
image: surrealdb/surrealdb:v3
command:
- /bin/sh
- -c
- |
surreal export \
--endpoint http://surrealdb:8000 \
--username $SURREAL_USER \
--password $SURREAL_PASS \
--namespace production \
--database app \
> /backups/backup_$(date +%Y%m%d_%H%M%S).surql
envFrom:
- secretRef:
name: surrealdb-auth
volumeMounts:
- name: backup-storage
mountPath: /backups
volumes:
- name: backup-storage
persistentVolumeClaim:
claimName: backup-pvc
restartPolicy: OnFailure
```
### Version Upgrades
When upgrading from SurrealDB v2 to v3:
1. Export data from v2 instance using `surreal export`.
2. Stop the v2 server.
3. Start the v3 server.
4. Import data using `surreal import`.
5. Test thoroughly in a staging environment first.
For storage format upgrades:
```bash
surreal upgrade --path ./mydata.db
```
### TLS/SSL Configuration
```bash
# Server-side TLS
surreal start \
--web-crt /etc/ssl/surrealdb/cert.pem \
--web-key /etc/ssl/surrealdb/key.pem \
--user root --pass root \
surrealkv:/data/db
# Mutual TLS (mTLS)
surreal start \
--web-crt /etc/ssl/surrealdb/cert.pem \
--web-key /etc/ssl/surrealdb/key.pem \
--client-crt /etc/ssl/surrealdb/ca.pem \
--client-key /etc/ssl/surrealdb/client-key.pem \
--user root --pass root \
surrealkv:/data/db
```
### Log Management
```bash
# Log levels (from least to most verbose)
# none -> full -> error -> warn -> info -> debug -> trace
# Production (standard)
surreal start --log info ...
# Debugging
surreal start --log debug ...
# Full trace (development only, very verbose)
surreal start --log trace ...
```
---
## High Availability
### Stateless Compute with TiKV
For high availability, deploy SurrealDB as stateless compute nodes backed by a TiKV cluster:
1. Run 3+ PD nodes for metadata HA.
2. Run 3+ TiKV nodes for data HA (Raft replication).
3. Run 2+ SurrealDB compute nodes behind a load balancer.
4. SurrealDB nodes can be added/removed without affecting data.
### Load Balancer Configuration
Place a load balancer (NGINX, HAProxy, cloud LB) in front of SurrealDB compute nodes:
```nginx
# NGINX configuration for SurrealDB
upstream surrealdb {
server surrealdb-0:8000;
server surrealdb-1:8000;
server surrealdb-2:8000;
}
server {
listen 443 ssl;
server_name surrealdb.example.com;
ssl_certificate /etc/ssl/cert.pem;
ssl_certificate_key /etc/ssl/key.pem;
location / {
proxy_pass http://surrealdb;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
}
```
The `Upgrade` and `Connection` headers are required for WebSocket support.
### Health Check Endpoints
SurrealDB exposes a `/health` endpoint for load balancers and orchestration health checks. Use this endpoint in your liveness and readiness probes.
### Failover Procedures
With TiKV:
- If a TiKV node fails, Raft automatically promotes a replica. No manual intervention needed for single-node failures.
- If a SurrealDB compute node fails, the load balancer routes traffic to healthy nodes. Replace the failed node.
- If a PD node fails, remaining PD nodes continue operation (quorum-based).
---
## Monitoring
### Prometheus Metrics
SurrealDB exposes metrics in Prometheus format. Configure Prometheus to scrape the metrics endpoint:
```yaml
# prometheus.yml
scrape_configs:
- job_name: "surrealdb"
static_configs:
- targets:
- "surrealdb-0:8000"
- "surrealdb-1:8000"
- "surrealdb-2:8000"
metrics_path: /metrics
scrape_interval: 15s
```
### Grafana Dashboard
Create dashboards tracking:
- Query throughput (queries per second)
- Query latency (p50, p95, p99)
- Active connections (WebSocket and HTTP)
- Memory usage
- CPU utilization
- Storage size and growth rate
- Error rates by type
- Live query subscription count
### Alert Configuration
Set alerts for:
- Query latency exceeding thresholds (e.g., p99 > 500ms)
- Error rate spikes (> 1% of requests)
- Memory usage above 80% of limit
- Disk usage above 85% of capacity
- No healthy instances (all health checks failing)
- TiKV replication lag (if using distributed storage)
### Performance Baselines
Establish baselines during normal operation and alert on deviations:
- Average query latency under normal load
- Typical memory and CPU utilization
- Normal connection count ranges
- Expected query throughput
```
### rules/security.md
```markdown
# SurrealDB Security
SurrealDB has a layered security model built into the database itself, eliminating the need for separate middleware or application-level permission checks in many cases. Authentication, authorization, and row-level security are all defined in SurrealQL and enforced by the database engine.
---
## Authentication Hierarchy
SurrealDB uses a four-level authentication hierarchy. Each level has access to its scope and everything beneath it.
```
Root User
|-- Full system access (all namespaces, databases, tables)
|
+-- Namespace User
|-- Access to all databases within a namespace
|
+-- Database User
|-- Access to all tables within a database
|
+-- Record User
|-- Access governed by table PERMISSIONS
|-- Authenticated via DEFINE ACCESS ... TYPE RECORD
```
### Root Users
Root users have unrestricted access to the entire SurrealDB instance. Use them only for administrative operations.
```surrealql
-- Define a root user (requires existing root access)
DEFINE USER admin ON ROOT PASSWORD 'strong-password-here' ROLES OWNER;
DEFINE USER operator ON ROOT PASSWORD 'another-password' ROLES EDITOR;
DEFINE USER readonly ON ROOT PASSWORD 'readonly-pass' ROLES VIEWER;
-- Roles:
-- OWNER: full control (create/drop namespaces, manage users)
-- EDITOR: read/write data, manage schemas
-- VIEWER: read-only access
```
### Namespace Users
Namespace users can access all databases within their namespace but cannot access other namespaces.
```surrealql
USE NS production;
DEFINE USER ns_admin ON NAMESPACE PASSWORD 'ns-password' ROLES OWNER;
DEFINE USER ns_dev ON NAMESPACE PASSWORD 'dev-password' ROLES EDITOR;
DEFINE USER ns_reader ON NAMESPACE PASSWORD 'reader-pass' ROLES VIEWER;
```
### Database Users
Database users can access tables and data within a single database.
```surrealql
USE NS production DB app_main;
DEFINE USER db_admin ON DATABASE PASSWORD 'db-password' ROLES OWNER;
DEFINE USER db_writer ON DATABASE PASSWORD 'writer-pass' ROLES EDITOR;
DEFINE USER db_reader ON DATABASE PASSWORD 'reader-pass' ROLES VIEWER;
```
### Record Users
Record users are end-user accounts authenticated via `DEFINE ACCESS ... TYPE RECORD`. They are the most granular level and are subject to table-level PERMISSIONS clauses.
```surrealql
-- See DEFINE ACCESS section below for full details
DEFINE ACCESS account ON DATABASE TYPE RECORD
SIGNUP (CREATE user SET email = $email, pass = crypto::argon2::generate($pass))
SIGNIN (SELECT * FROM user WHERE email = $email AND crypto::argon2::compare(pass, $pass))
DURATION FOR TOKEN 15m, FOR SESSION 12h;
```
---
## DEFINE ACCESS
The `DEFINE ACCESS` statement configures how users authenticate with SurrealDB. There are three access types: RECORD, JWT, and API KEY.
### Record-Based Authentication (End Users)
Record access allows end users to sign up and sign in. The `SIGNUP` and `SIGNIN` clauses define SurrealQL expressions that execute during authentication.
```surrealql
-- Basic email/password authentication
DEFINE ACCESS account ON DATABASE TYPE RECORD
SIGNUP (
CREATE user SET
email = $email,
pass = crypto::argon2::generate($pass),
created_at = time::now(),
role = 'member'
)
SIGNIN (
SELECT * FROM user
WHERE email = $email
AND crypto::argon2::compare(pass, $pass)
)
DURATION FOR TOKEN 15m, FOR SESSION 12h;
```
After successful authentication, `$auth` contains the authenticated user record and `$access` contains the access method name. These are available in PERMISSIONS clauses.
```surrealql
-- Advanced record access with validation
DEFINE ACCESS secure_account ON DATABASE TYPE RECORD
SIGNUP (
-- Validate email format
IF string::is::email($email) THEN
CREATE user SET
email = string::lowercase($email),
pass = crypto::argon2::generate($pass),
name = $name,
created_at = time::now(),
verified = false,
role = 'member'
ELSE
THROW "Invalid email format"
END
)
SIGNIN (
SELECT * FROM user
WHERE email = string::lowercase($email)
AND crypto::argon2::compare(pass, $pass)
)
DURATION FOR TOKEN 15m, FOR SESSION 12h;
```
```surrealql
-- Multi-tenant record access
DEFINE ACCESS tenant_account ON DATABASE TYPE RECORD
SIGNUP (
-- Verify tenant exists before creating user
LET $tenant = SELECT * FROM tenant WHERE id = $tenant_id;
IF count($tenant) > 0 THEN
CREATE user SET
email = $email,
pass = crypto::argon2::generate($pass),
tenant = $tenant_id,
role = 'member'
ELSE
THROW "Invalid tenant"
END
)
SIGNIN (
SELECT * FROM user
WHERE email = $email
AND crypto::argon2::compare(pass, $pass)
)
DURATION FOR TOKEN 30m, FOR SESSION 24h;
```
### JWT-Based Authentication
JWT access allows external identity providers to authenticate users with SurrealDB.
```surrealql
-- HMAC-based JWT (symmetric key)
DEFINE ACCESS jwt_auth ON DATABASE TYPE JWT
ALGORITHM HS256
KEY 'your-256-bit-secret-key-here'
DURATION FOR TOKEN 1h;
-- RSA-based JWT (asymmetric key, more secure)
DEFINE ACCESS jwt_rsa ON DATABASE TYPE JWT
ALGORITHM RS256
KEY '-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A...
-----END PUBLIC KEY-----'
DURATION FOR TOKEN 1h;
-- ECDSA-based JWT
DEFINE ACCESS jwt_ecdsa ON DATABASE TYPE JWT
ALGORITHM ES256
KEY '-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcD...
-----END PUBLIC KEY-----'
DURATION FOR TOKEN 1h;
-- JWT with record binding (map JWT claims to a user record)
DEFINE ACCESS external_auth ON DATABASE TYPE RECORD
WITH JWT ALGORITHM RS256 KEY '-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A...
-----END PUBLIC KEY-----'
DURATION FOR TOKEN 1h;
```
### Supported JWT Algorithms
| Algorithm | Type | Key |
|---|---|---|
| HS256, HS384, HS512 | HMAC (symmetric) | Shared secret key |
| RS256, RS384, RS512 | RSA (asymmetric) | Public key for verification |
| ES256, ES384, ES512 | ECDSA (asymmetric) | Public key for verification |
| PS256, PS384, PS512 | RSA-PSS (asymmetric) | Public key for verification |
| EdDSA | EdDSA (asymmetric) | Public key for verification |
### API Key Authentication
```surrealql
-- Define API key access
DEFINE ACCESS api_access ON DATABASE TYPE API KEY;
-- API keys are generated and managed through the SurrealDB API
-- They provide simple bearer token authentication
```
### Token Duration Configuration
```surrealql
-- Duration controls how long tokens and sessions last
DEFINE ACCESS account ON DATABASE TYPE RECORD
SIGNUP (...)
SIGNIN (...)
-- Token: short-lived, used for individual requests
-- Session: longer-lived, maintains authenticated state
DURATION FOR TOKEN 15m, FOR SESSION 12h;
-- Very short tokens for high-security environments
DEFINE ACCESS high_sec ON DATABASE TYPE RECORD
SIGNUP (...)
SIGNIN (...)
DURATION FOR TOKEN 5m, FOR SESSION 1h;
-- Longer durations for less sensitive applications
DEFINE ACCESS relaxed ON DATABASE TYPE RECORD
SIGNUP (...)
SIGNIN (...)
DURATION FOR TOKEN 1h, FOR SESSION 7d;
```
---
## Row-Level Security (Table Permissions)
Permissions are defined on tables and control what record users can do. They are enforced automatically by the database engine -- there is no way for a record user to bypass them.
### Basic Permissions
```surrealql
DEFINE TABLE post SCHEMALESS
PERMISSIONS
FOR select WHERE published = true OR user = $auth.id
FOR create WHERE $auth.id IS NOT NONE
FOR update WHERE user = $auth.id
FOR delete WHERE user = $auth.id OR $auth.role = 'admin';
```
### Granular Permission Examples
```surrealql
-- Public read, authenticated write
DEFINE TABLE article SCHEMALESS
PERMISSIONS
FOR select FULL -- anyone can read (including unauthenticated)
FOR create, update, delete WHERE $auth.id IS NOT NONE;
-- Owner-only access
DEFINE TABLE private_note SCHEMALESS
PERMISSIONS
FOR select, update, delete WHERE owner = $auth.id
FOR create WHERE $auth.id IS NOT NONE;
-- No access for record users (admin-only table)
DEFINE TABLE system_config SCHEMALESS
PERMISSIONS
FOR select, create, update, delete NONE;
-- Role-based permissions
DEFINE TABLE order SCHEMAFULL
PERMISSIONS
FOR select WHERE
customer = $auth.id
OR $auth.role IN ['admin', 'support']
FOR create WHERE $auth.id IS NOT NONE
FOR update WHERE
customer = $auth.id AND status = 'draft'
OR $auth.role = 'admin'
FOR delete WHERE $auth.role = 'admin';
```
### Multi-Tenant Permissions
```surrealql
-- Tenant isolation: users can only see data in their tenant
DEFINE TABLE customer SCHEMAFULL
PERMISSIONS
FOR select WHERE tenant = $auth.tenant
FOR create WHERE tenant = $auth.tenant
FOR update WHERE tenant = $auth.tenant AND $auth.role IN ['admin', 'editor']
FOR delete WHERE tenant = $auth.tenant AND $auth.role = 'admin';
DEFINE TABLE invoice SCHEMAFULL
PERMISSIONS
FOR select WHERE
tenant = $auth.tenant
AND (
created_by = $auth.id
OR $auth.role IN ['admin', 'finance']
)
FOR create WHERE tenant = $auth.tenant
FOR update WHERE
tenant = $auth.tenant
AND (created_by = $auth.id OR $auth.role = 'admin')
AND status != 'finalized'
FOR delete NONE; -- invoices cannot be deleted
```
### Complex Permission Conditions
```surrealql
-- Time-based permissions
DEFINE TABLE submission SCHEMAFULL
PERMISSIONS
FOR select FULL
FOR create WHERE
$auth.id IS NOT NONE
AND time::now() < d'2026-03-01T00:00:00Z' -- submission deadline
FOR update WHERE
author = $auth.id
AND status = 'draft'
AND time::now() < d'2026-03-01T00:00:00Z'
FOR delete WHERE author = $auth.id AND status = 'draft';
-- Permissions based on related records
DEFINE TABLE comment SCHEMAFULL
PERMISSIONS
FOR select WHERE
-- Can see comments on posts you can see
(SELECT VALUE published FROM ONLY post WHERE id = $parent.post) = true
OR $auth.id IS NOT NONE
FOR create WHERE $auth.id IS NOT NONE
FOR update WHERE author = $auth.id
FOR delete WHERE
author = $auth.id
OR $auth.role = 'moderator';
-- Access-method-specific permissions
DEFINE TABLE user SCHEMAFULL
PERMISSIONS
FOR select WHERE
$access = 'account' AND id = $auth.id -- record users see only themselves
OR $access = 'admin_access' -- admin access sees all
FOR update WHERE id = $auth.id
FOR delete NONE;
```
---
## Field-Level Permissions
### PERMISSIONS on DEFINE FIELD
```surrealql
-- Restrict field visibility and mutability
DEFINE TABLE user SCHEMAFULL
PERMISSIONS FOR select, create, update WHERE $auth.id = id OR $auth.role = 'admin';
DEFINE FIELD email ON TABLE user TYPE string
PERMISSIONS
FOR select WHERE id = $auth.id OR $auth.role = 'admin'
FOR update WHERE id = $auth.id;
DEFINE FIELD pass ON TABLE user TYPE string
PERMISSIONS
FOR select NONE -- password hash is never returned
FOR update WHERE id = $auth.id;
DEFINE FIELD role ON TABLE user TYPE string
PERMISSIONS
FOR select FULL
FOR update WHERE $auth.role = 'admin'; -- only admins can change roles
DEFINE FIELD salary ON TABLE user TYPE option<decimal>
PERMISSIONS
FOR select WHERE id = $auth.id OR $auth.role IN ['admin', 'hr']
FOR update WHERE $auth.role IN ['admin', 'hr'];
DEFINE FIELD internal_notes ON TABLE user TYPE option<string>
PERMISSIONS
FOR select WHERE $auth.role IN ['admin', 'hr']
FOR update WHERE $auth.role = 'admin';
```
### Computed Field Security
```surrealql
-- Computed fields can enforce derived security values
DEFINE FIELD created_by ON TABLE document TYPE record<user>
DEFAULT $auth.id
READONLY; -- cannot be overwritten after creation
DEFINE FIELD tenant ON TABLE document TYPE record<tenant>
DEFAULT $auth.tenant
READONLY;
-- Computed field that masks sensitive data based on viewer
DEFINE FIELD display_email ON TABLE user VALUE
IF $auth.role = 'admin' OR $auth.id = id THEN
email
ELSE
string::concat(
string::slice(email, 0, 2),
'***@',
string::split(email, '@')[1]
)
END;
```
### Sensitive Data Masking
```surrealql
-- Credit card masking
DEFINE TABLE payment_method SCHEMAFULL;
DEFINE FIELD card_number ON TABLE payment_method TYPE string
PERMISSIONS FOR select NONE; -- raw number never returned
DEFINE FIELD masked_card ON TABLE payment_method VALUE
string::concat('****-****-****-', string::slice(card_number, -4));
DEFINE FIELD cardholder ON TABLE payment_method TYPE string;
DEFINE FIELD expiry ON TABLE payment_method TYPE string;
-- SSN/sensitive ID masking
DEFINE FIELD ssn ON TABLE employee TYPE string
PERMISSIONS
FOR select WHERE $auth.role = 'hr_admin'
FOR update WHERE $auth.role = 'hr_admin';
DEFINE FIELD masked_ssn ON TABLE employee VALUE
string::concat('***-**-', string::slice(ssn, -4));
```
---
## Security Best Practices
### Principle of Least Privilege
```surrealql
-- Start with NONE and explicitly grant access
DEFINE TABLE sensitive_data SCHEMAFULL
PERMISSIONS
FOR select, create, update, delete NONE;
-- Then open up only what is needed
-- Better: grant specific access to specific roles
DEFINE TABLE project SCHEMAFULL
PERMISSIONS
FOR select WHERE
team_member = $auth.id
OR $auth.role IN ['admin', 'project_manager']
FOR create WHERE $auth.role IN ['admin', 'project_manager']
FOR update WHERE
lead = $auth.id
OR $auth.role = 'admin'
FOR delete WHERE $auth.role = 'admin';
```
### Password Hashing
SurrealDB provides built-in cryptographic hashing functions. Always use argon2 for password storage.
```surrealql
-- Argon2 (recommended -- memory-hard, resistant to GPU attacks)
crypto::argon2::generate($password)
crypto::argon2::compare($hash, $password)
-- Bcrypt (acceptable alternative)
crypto::bcrypt::generate($password)
crypto::bcrypt::compare($hash, $password)
-- Scrypt (another memory-hard option)
crypto::scrypt::generate($password)
crypto::scrypt::compare($hash, $password)
-- SHA-256/512 (NOT suitable for passwords, use for data integrity only)
crypto::sha256($data)
crypto::sha512($data)
-- PBKDF2 (acceptable but argon2 is preferred)
crypto::pbkdf2::generate($password)
crypto::pbkdf2::compare($hash, $password)
-- NEVER store plaintext passwords
-- BAD:
CREATE user SET pass = $pass;
-- GOOD:
CREATE user SET pass = crypto::argon2::generate($pass);
```
### Token Duration Guidelines
| Use Case | Token Duration | Session Duration |
|---|---|---|
| Banking/financial | 5m | 30m |
| Enterprise SaaS | 15m | 8h |
| Consumer web app | 30m | 24h |
| Mobile app | 1h | 30d |
| Internal tool | 1h | 12h |
| IoT device | 24h | 90d |
```surrealql
-- High-security pattern: short tokens, moderate sessions
DEFINE ACCESS banking ON DATABASE TYPE RECORD
SIGNUP (...)
SIGNIN (...)
DURATION FOR TOKEN 5m, FOR SESSION 30m;
```
### CORS Configuration
CORS is configured at the server level when starting SurrealDB, not in SurrealQL.
```bash
# Allow specific origins
surreal start --allow-origins "https://app.example.com,https://admin.example.com"
# Allow all origins (development only)
surreal start --allow-origins "*"
# Full server startup with security flags
surreal start \
--bind 0.0.0.0:8000 \
--user root \
--pass "strong-root-password" \
--allow-origins "https://app.example.com" \
--strict \
file:///var/data/surreal.db
```
### TLS/SSL Configuration
```bash
# Start with TLS
surreal start \
--web-crt /etc/ssl/certs/server.crt \
--web-key /etc/ssl/private/server.key \
--bind 0.0.0.0:8000
# Client connections should use wss:// or https://
# wss://db.example.com:8000/rpc (WebSocket with TLS)
# https://db.example.com:8000 (HTTP with TLS)
```
### Network Security for Distributed Deployments
```bash
# Bind only to private network interfaces
surreal start --bind 10.0.1.5:8000
# For TiKV distributed deployments, ensure:
# 1. TiKV nodes communicate over private network
# 2. PD (Placement Driver) is not exposed publicly
# 3. SurrealDB compute nodes connect to TiKV over private network
surreal start --kvs-ca /etc/tikv/ca.pem --kvs-crt /etc/tikv/client.crt --kvs-key /etc/tikv/client.key tikv://10.0.1.10:2379
# Use --strict mode in production to require authentication
surreal start --strict
```
### Audit Logging Patterns
```surrealql
-- Create an append-only audit log table
DEFINE TABLE audit_log SCHEMAFULL
PERMISSIONS
FOR select WHERE $auth.role = 'admin'
FOR create FULL
FOR update, delete NONE; -- immutable records
DEFINE FIELD action ON TABLE audit_log TYPE string;
DEFINE FIELD table_name ON TABLE audit_log TYPE string;
DEFINE FIELD record_id ON TABLE audit_log TYPE option<record>;
DEFINE FIELD actor ON TABLE audit_log TYPE option<record<user>>;
DEFINE FIELD timestamp ON TABLE audit_log TYPE datetime DEFAULT time::now();
DEFINE FIELD details ON TABLE audit_log TYPE option<object>;
-- Attach audit events to sensitive tables
DEFINE EVENT audit_user_changes ON TABLE user WHEN $event IN ["CREATE", "UPDATE", "DELETE"] THEN {
CREATE audit_log SET
action = $event,
table_name = 'user',
record_id = $after.id ?? $before.id,
actor = $auth.id,
details = {
before: IF $event != "CREATE" THEN $before ELSE NONE END,
after: IF $event != "DELETE" THEN $after ELSE NONE END
};
};
DEFINE EVENT audit_permission_changes ON TABLE system_config
WHEN $event IN ["CREATE", "UPDATE", "DELETE"] THEN {
CREATE audit_log SET
action = $event,
table_name = 'system_config',
record_id = $after.id ?? $before.id,
actor = $auth.id,
details = { event: $event };
};
```
### Common Security Pitfalls
1. Using `PERMISSIONS FOR select FULL` on tables with sensitive data. This allows unauthenticated access.
2. Forgetting that `$auth` is NONE for unauthenticated requests. Always check `$auth.id IS NOT NONE` where authentication is required.
3. Not setting PERMISSIONS at all. By default, tables without PERMISSIONS allow full access to all authenticated system users (root, namespace, database) but deny access to record users.
4. Storing plaintext passwords. Always use `crypto::argon2::generate()`.
5. Using overly long token durations. Keep tokens short (5-30 minutes) and sessions reasonable.
6. Not using `--strict` mode in production. Without it, unauthenticated connections may have elevated access.
7. Exposing SurrealDB directly to the internet without TLS. Always use TLS in production.
8. Using HS256 JWT with a weak secret key. Use at least 256 bits of entropy or prefer asymmetric algorithms (RS256, ES256).
---
## Authentication Flows
### Signup/Signin Flow for Web Applications
```surrealql
-- 1. Define the access method
DEFINE ACCESS account ON DATABASE TYPE RECORD
SIGNUP (
CREATE user SET
email = string::lowercase($email),
pass = crypto::argon2::generate($pass),
name = $name,
created_at = time::now(),
role = 'member'
)
SIGNIN (
SELECT * FROM user
WHERE email = string::lowercase($email)
AND crypto::argon2::compare(pass, $pass)
)
DURATION FOR TOKEN 15m, FOR SESSION 12h;
-- 2. User table with permissions
DEFINE TABLE user SCHEMAFULL
PERMISSIONS
FOR select WHERE id = $auth.id OR $auth.role = 'admin'
FOR update WHERE id = $auth.id
FOR delete WHERE $auth.role = 'admin'
FOR create NONE; -- users are created only through SIGNUP
DEFINE FIELD email ON TABLE user TYPE string ASSERT string::is::email($value);
DEFINE FIELD pass ON TABLE user TYPE string PERMISSIONS FOR select NONE;
DEFINE FIELD name ON TABLE user TYPE string;
DEFINE FIELD role ON TABLE user TYPE string DEFAULT 'member';
DEFINE FIELD created_at ON TABLE user TYPE datetime;
```
Client-side flow (using the JavaScript SDK):
```javascript
import Surreal from 'surrealdb';
const db = new Surreal();
await db.connect('wss://db.example.com/rpc');
await db.use({ namespace: 'production', database: 'app' });
// Sign up
const signupToken = await db.signup({
access: 'account',
variables: {
email: '[email protected]',
pass: 'secure-password',
name: 'Alice'
}
});
// Sign in
const signinToken = await db.signin({
access: 'account',
variables: {
email: '[email protected]',
pass: 'secure-password'
}
});
// Authenticate with existing token
await db.authenticate(signinToken);
// All subsequent queries are scoped to this user
const myProfile = await db.select('user');
// Returns only the authenticated user's record (due to PERMISSIONS)
```
### JWT Token Integration with External Identity Providers
```surrealql
-- Define JWT access that maps external tokens to SurrealDB permissions
DEFINE ACCESS external_idp ON DATABASE TYPE RECORD
WITH JWT ALGORITHM RS256 KEY '-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A...
-----END PUBLIC KEY-----'
DURATION FOR TOKEN 1h;
-- The JWT payload should include claims that map to user data
-- Example JWT payload:
-- {
-- "sub": "auth0|12345",
-- "email": "[email protected]",
-- "roles": ["editor"],
-- "tenant_id": "tenant:acme",
-- "exp": 1700000000
-- }
-- Access JWT claims in permissions via $token
DEFINE TABLE document SCHEMAFULL
PERMISSIONS
FOR select WHERE tenant = $token.tenant_id
FOR create WHERE 'editor' IN $token.roles
FOR update WHERE 'editor' IN $token.roles AND tenant = $token.tenant_id
FOR delete WHERE 'admin' IN $token.roles;
```
### Session Management
```surrealql
-- Define explicit session tracking
DEFINE TABLE session SCHEMAFULL
PERMISSIONS
FOR select WHERE user = $auth.id
FOR create WHERE $auth.id IS NOT NONE
FOR update, delete WHERE user = $auth.id;
DEFINE FIELD user ON TABLE session TYPE record<user>;
DEFINE FIELD device ON TABLE session TYPE string;
DEFINE FIELD ip_address ON TABLE session TYPE string;
DEFINE FIELD created_at ON TABLE session TYPE datetime DEFAULT time::now();
DEFINE FIELD last_active ON TABLE session TYPE datetime DEFAULT time::now();
DEFINE FIELD expires_at ON TABLE session TYPE datetime;
-- Create session on signin
DEFINE EVENT track_signin ON TABLE user WHEN $event = "UPDATE"
AND $after.last_signin != $before.last_signin THEN {
CREATE session SET
user = $after.id,
device = $after.current_device,
ip_address = $after.current_ip,
expires_at = time::now() + 12h;
};
-- Clean expired sessions (run periodically via application or cron)
DELETE session WHERE expires_at < time::now();
```
### Multi-Tenant Security with Namespaces
```surrealql
-- Strategy 1: One namespace per tenant (strongest isolation)
-- Each tenant gets their own namespace with identical schema
-- Tenant A
USE NS tenant_a DB main;
DEFINE ACCESS account ON DATABASE TYPE RECORD
SIGNUP (CREATE user SET email = $email, pass = crypto::argon2::generate($pass))
SIGNIN (SELECT * FROM user WHERE email = $email AND crypto::argon2::compare(pass, $pass))
DURATION FOR TOKEN 15m, FOR SESSION 12h;
-- Tenant B (completely isolated)
USE NS tenant_b DB main;
DEFINE ACCESS account ON DATABASE TYPE RECORD
SIGNUP (CREATE user SET email = $email, pass = crypto::argon2::generate($pass))
SIGNIN (SELECT * FROM user WHERE email = $email AND crypto::argon2::compare(pass, $pass))
DURATION FOR TOKEN 15m, FOR SESSION 12h;
```
```surrealql
-- Strategy 2: Shared database with tenant column (simpler, less isolation)
USE NS production DB shared;
DEFINE TABLE tenant SCHEMAFULL
PERMISSIONS FOR select, create, update, delete NONE; -- admin only
DEFINE TABLE user SCHEMAFULL
PERMISSIONS
FOR select WHERE tenant = $auth.tenant
FOR update WHERE id = $auth.id
FOR create, delete NONE;
DEFINE FIELD tenant ON TABLE user TYPE record<tenant> READONLY;
DEFINE TABLE data SCHEMAFULL
PERMISSIONS
FOR select WHERE tenant = $auth.tenant
FOR create WHERE tenant = $auth.tenant
FOR update WHERE tenant = $auth.tenant
FOR delete WHERE tenant = $auth.tenant AND $auth.role = 'admin';
DEFINE FIELD tenant ON TABLE data TYPE record<tenant> DEFAULT $auth.tenant READONLY;
-- The READONLY + DEFAULT ensures the tenant field is automatically set
-- and cannot be tampered with by the user
```
---
## Security Checklist for Production
Before deploying SurrealDB to production:
- Use `--strict` mode to require authentication for all connections
- Enable TLS with valid certificates (`--web-crt`, `--web-key`)
- Set strong root passwords (minimum 20 characters, random)
- Define PERMISSIONS on every table that record users access
- Use `PERMISSIONS FOR select NONE` on password/secret fields
- Use `crypto::argon2::generate()` for all password storage
- Set appropriate token and session durations
- Restrict CORS to specific origins (not `*`)
- Bind to private network interfaces where possible
- Enable audit logging on sensitive tables
- Review all `FULL` permissions for unintended public access
- Ensure `$auth.id IS NOT NONE` checks where authentication is required
- Use SCHEMAFULL tables to prevent schema injection
- Set `READONLY` on fields that should not be user-modifiable (tenant, created_by)
- Use asymmetric JWT algorithms (RS256, ES256) over symmetric (HS256) when possible
- Regularly rotate JWT signing keys
- Test permissions thoroughly with different user roles
```
### rules/performance.md
```markdown
# SurrealDB Performance
This guide covers storage engine selection, indexing strategies, query optimization, write and read performance tuning, distributed deployment considerations, and monitoring for SurrealDB.
---
## Storage Engine Selection
SurrealDB supports multiple storage backends, each suited to different deployment scenarios.
| Engine | Best For | Persistence | Distributed | Versioning |
|---|---|---|---|---|
| In-Memory | Development, testing, ephemeral caches | No | No | No |
| RocksDB | Single-node production, proven reliability | Yes | No | No |
| SurrealKV | Single-node production, time-travel queries | Yes | No | Yes |
| TiKV | Distributed HA, horizontal scaling | Yes | Yes | No |
| IndexedDB | Browser-based apps (WASM) | Yes (browser) | No | No |
### Starting with Each Engine
```bash
# In-Memory (data lost on restart)
surreal start memory
# RocksDB (single file path)
surreal start rocksdb:///var/data/surreal.db
# SurrealKV (default in SurrealDB 3.x for file-based storage)
surreal start surrealkv:///var/data/surreal.db
# Or simply:
surreal start file:///var/data/surreal.db
# TiKV (distributed, requires running TiKV cluster)
surreal start tikv://pd-host:2379
# IndexedDB (browser WASM only, configured in SDK)
# Not started from CLI; used within browser SDK initialization
```
### Engine Selection Criteria
Use **In-Memory** when:
- Running tests or CI/CD pipelines
- Prototyping and development
- Temporary data processing
Use **RocksDB** when:
- You need battle-tested persistence (RocksDB powers many production databases)
- Single-node deployment is sufficient
- You want well-understood tuning options
Use **SurrealKV** when:
- You want the SurrealDB-native storage engine
- You need time-travel queries (version history of records)
- Single-node deployment with SurrealDB-optimized performance
Use **TiKV** when:
- You need horizontal scaling across multiple nodes
- High availability with automatic failover is required
- Your dataset exceeds single-node capacity
- You need distributed transactions
Use **IndexedDB** when:
- Building browser-based applications with SurrealDB WASM
- Offline-first applications that sync later
---
## Indexing Strategies
Indexes are the single most impactful performance optimization. SurrealDB supports standard indexes, unique indexes, composite indexes, full-text search indexes, and vector (HNSW) indexes.
### Standard Indexes
```surrealql
-- Single-field index
DEFINE INDEX idx_email ON TABLE user COLUMNS email;
-- The index accelerates WHERE clauses on the indexed field:
-- FAST (uses index):
SELECT * FROM user WHERE email = '[email protected]';
-- SLOW (full table scan):
SELECT * FROM user WHERE name = 'Alice';
```
### Unique Indexes
```surrealql
-- Unique index prevents duplicate values
DEFINE INDEX idx_unique_email ON TABLE user COLUMNS email UNIQUE;
-- This also speeds up lookups and enforces data integrity
-- Attempting to insert a duplicate will fail:
CREATE user SET email = '[email protected]'; -- succeeds
CREATE user SET email = '[email protected]'; -- fails with unique constraint error
```
### Composite Indexes
```surrealql
-- Multi-column index for queries that filter on multiple fields
DEFINE INDEX idx_tenant_status ON TABLE order COLUMNS tenant, status;
-- FAST (uses composite index, left-to-right prefix match):
SELECT * FROM order WHERE tenant = tenant:acme AND status = 'active';
SELECT * FROM order WHERE tenant = tenant:acme; -- prefix match
-- SLOW (cannot use this index, wrong column order):
SELECT * FROM order WHERE status = 'active'; -- no leading 'tenant' filter
-- The order of columns matters: put the most selective or most frequently
-- filtered column first
DEFINE INDEX idx_status_created ON TABLE order COLUMNS status, created_at;
```
### Full-Text Search Indexes
```surrealql
-- Define an analyzer with tokenizers and filters
DEFINE ANALYZER english_analyzer
TOKENIZERS blank, class
FILTERS lowercase, snowball(english);
-- Define a full-text search index using BM25 scoring
DEFINE INDEX idx_ft_content ON TABLE article
FIELDS content
SEARCH ANALYZER english_analyzer BM25;
-- Full-text search query using the @@ operator
-- The number after @ is the scoring reference (used with search::score)
SELECT
id,
title,
search::score(1) AS relevance
FROM article
WHERE content @1@ 'distributed database performance'
ORDER BY relevance DESC
LIMIT 20;
-- Highlight matching terms
SELECT
id,
title,
search::highlight('<b>', '</b>', 1) AS highlighted
FROM article
WHERE content @1@ 'SurrealDB graph queries';
-- Combined analyzer for multiple languages
DEFINE ANALYZER multi_analyzer
TOKENIZERS blank, class, camel
FILTERS lowercase, ascii;
```
### Vector Indexes (HNSW)
See the vector-search rules file for detailed HNSW configuration. Summary:
```surrealql
-- Standard vector index
DEFINE INDEX idx_embedding ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE;
-- High-recall configuration
DEFINE INDEX idx_embedding_hr ON TABLE document
FIELDS embedding
HNSW DIMENSION 1536 DIST COSINE EFC 300 M 32
EXTEND_CANDIDATES KEEP_PRUNED_CONNECTIONS;
```
### When NOT to Index
- Columns with very low cardinality (e.g., boolean `active` with only true/false). The index overhead may exceed the scan cost.
- Tables with very few rows (under 1000). Full scans are fast on small tables.
- Columns that are rarely queried in WHERE clauses.
- Columns that are updated extremely frequently. Each update must also update the index.
- Temporary or staging tables used for bulk data processing.
### Index Rebuild Strategies
```surrealql
-- Remove and recreate an index to rebuild it
REMOVE INDEX idx_email ON TABLE user;
DEFINE INDEX idx_email ON TABLE user COLUMNS email;
-- Check current indexes on a table
INFO FOR TABLE user;
-- Returns index definitions and metadata
```
---
## Query Optimization
### EXPLAIN Statement
Use `EXPLAIN` to understand how SurrealDB executes a query and whether it uses indexes.
```surrealql
-- See the query execution plan
SELECT * FROM user WHERE email = '[email protected]' EXPLAIN;
-- EXPLAIN FULL provides more detail
SELECT * FROM user WHERE email = '[email protected]' EXPLAIN FULL;
-- Look for:
-- - "Index" operations (good: using an index)
-- - "Table" operations (bad: full table scan)
-- - "Iterate" step details show which index is used
-- Example output interpretation:
-- Operation: Iterate Index -> using idx_email (fast)
-- Operation: Iterate Table -> full scan (slow, needs index)
```
### Avoiding Full Table Scans
```surrealql
-- BAD: No index on 'status', causes full scan
SELECT * FROM order WHERE status = 'pending';
-- GOOD: Add an index first
DEFINE INDEX idx_order_status ON TABLE order COLUMNS status;
SELECT * FROM order WHERE status = 'pending';
-- BAD: Function call on indexed column prevents index use
SELECT * FROM user WHERE string::lowercase(email) = '[email protected]';
-- GOOD: Store normalized data, index it, query directly
-- (normalize at write time, not query time)
DEFINE FIELD email ON TABLE user VALUE string::lowercase($value);
SELECT * FROM user WHERE email = '[email protected]';
-- BAD: OR conditions across different columns may not use indexes efficiently
SELECT * FROM user WHERE email = '[email protected]' OR name = 'Alice';
-- GOOD: Use separate indexed queries if needed
-- or create a composite approach
```
### Efficient Graph Traversal
```surrealql
-- Index edge tables for fast traversal
DEFINE INDEX idx_knows_in ON TABLE knows COLUMNS in;
DEFINE INDEX idx_knows_out ON TABLE knows COLUMNS out;
-- GOOD: Bounded depth with limits
SELECT ->knows->(person LIMIT 20).name AS connections
FROM person:alice;
-- BAD: Unbounded multi-hop without limits
-- This can explode exponentially
SELECT ->knows->person->knows->person->knows->person
FROM person:alice;
-- GOOD: Use LIMIT at each hop to cap explosion
SELECT
->(knows LIMIT 10)->person
->(knows LIMIT 10)->person.name AS fof
FROM person:alice
LIMIT 50;
```
### Pagination Patterns
```surrealql
-- Basic offset pagination (simple but slower for deep pages)
SELECT * FROM article
ORDER BY created_at DESC
LIMIT 20
START 0; -- page 1
SELECT * FROM article
ORDER BY created_at DESC
LIMIT 20
START 20; -- page 2
SELECT * FROM article
ORDER BY created_at DESC
LIMIT 20
START 40; -- page 3
-- Cursor-based pagination (faster for deep pages)
-- First page
SELECT * FROM article
ORDER BY created_at DESC
LIMIT 20;
-- Subsequent pages: use the last record's timestamp as cursor
SELECT * FROM article
WHERE created_at < $last_seen_timestamp
ORDER BY created_at DESC
LIMIT 20;
-- For guaranteed uniqueness, combine with record ID
SELECT * FROM article
WHERE created_at < $cursor_time
OR (created_at = $cursor_time AND id < $cursor_id)
ORDER BY created_at DESC, id DESC
LIMIT 20;
```
### Projection Optimization
```surrealql
-- BAD: SELECT * fetches all fields, including large ones
SELECT * FROM document;
-- GOOD: Select only the fields you need
SELECT id, title, created_at FROM document;
-- Especially important for tables with embeddings or large text
-- BAD:
SELECT * FROM document; -- fetches embedding (6KB per row)
-- GOOD:
SELECT id, title, snippet FROM document;
-- GOOD: Use destructuring for nested/computed results
SELECT
id,
title,
author.name AS author_name
FROM document;
```
### Subquery Optimization
```surrealql
-- BAD: Correlated subquery that runs for every row
SELECT *,
(SELECT count() FROM comment WHERE post = $parent.id GROUP ALL) AS comment_count
FROM post;
-- GOOD: Use precomputed counts or batch the query
-- Option 1: Store count on the parent record
DEFINE EVENT update_comment_count ON TABLE comment WHEN $event = "CREATE" THEN {
UPDATE $after.post SET comment_count += 1;
};
-- Option 2: Use a single aggregation query
SELECT post, count() AS comment_count FROM comment GROUP BY post;
-- GOOD: Use LET to precompute and reuse
LET $active_users = SELECT VALUE id FROM user WHERE active = true;
SELECT * FROM order WHERE customer IN $active_users;
```
### Parallel Query Execution
```surrealql
-- SurrealDB can execute independent statements in parallel within a request
-- Sending multiple queries in a single request is more efficient than
-- sending them one at a time
-- These three queries can be sent together in one request:
SELECT count() FROM user GROUP ALL;
SELECT count() FROM order GROUP ALL;
SELECT count() FROM product GROUP ALL;
-- The SDK will pipeline these and SurrealDB processes them efficiently
```
---
## Write Performance
### Batch Operations
```surrealql
-- BAD: Individual inserts (one round trip each)
CREATE user:1 SET name = 'Alice', email = '[email protected]';
CREATE user:2 SET name = 'Bob', email = '[email protected]';
CREATE user:3 SET name = 'Charlie', email = '[email protected]';
-- GOOD: Batch insert (single operation)
INSERT INTO user [
{ id: user:1, name: 'Alice', email: '[email protected]' },
{ id: user:2, name: 'Bob', email: '[email protected]' },
{ id: user:3, name: 'Charlie', email: '[email protected]' }
];
-- For large batches, use groups of 100-1000 records
-- Too small = too many round trips
-- Too large = high memory usage per transaction
```
### Transaction Sizing
```surrealql
-- SurrealDB wraps each request in an implicit transaction
-- For large data modifications, consider chunking
-- BAD: Updating millions of rows in one transaction
UPDATE user SET verified = true WHERE created_at < d'2025-01-01';
-- BETTER: Process in chunks using LIMIT
-- Chunk 1
UPDATE user SET verified = true WHERE created_at < d'2025-01-01' AND verified = false LIMIT 1000;
-- Chunk 2 (repeat until no more matches)
UPDATE user SET verified = true WHERE created_at < d'2025-01-01' AND verified = false LIMIT 1000;
-- Use application-level loop:
-- while (affected > 0) { run UPDATE ... LIMIT 1000 }
```
### Bulk Import
```bash
# Import from SurrealQL file
surreal import --conn http://localhost:8000 --user root --pass root --ns test --db test data.surql
# Import from JSON/JSONL
# Prepare a .surql file with INSERT statements for best performance
# For very large imports:
# 1. Disable/remove indexes before import
# 2. Perform bulk insert
# 3. Recreate indexes after import
# This avoids index maintenance overhead during bulk loading
```
```surrealql
-- Pre-import: remove expensive indexes
REMOVE INDEX idx_embedding ON TABLE document;
REMOVE INDEX idx_ft_content ON TABLE document;
-- ... perform bulk import ...
-- Post-import: recreate indexes (they build from existing data)
DEFINE INDEX idx_embedding ON TABLE document
FIELDS embedding HNSW DIMENSION 1536 DIST COSINE;
DEFINE INDEX idx_ft_content ON TABLE document
FIELDS content SEARCH ANALYZER english_analyzer BM25;
```
### Concurrent Write Patterns
```surrealql
-- Use record-level operations to minimize contention
-- SurrealDB uses MVCC, so concurrent writes to different records do not block
-- Atomic counter increment (safe under concurrency)
UPDATE product:123 SET stock -= 1 WHERE stock > 0;
-- Conditional update (optimistic concurrency)
UPDATE order:456 SET status = 'shipped'
WHERE status = 'approved';
-- Returns empty if the order was already changed by another transaction
-- For high-contention counters, consider sharding
-- Instead of one counter record, use multiple shards:
UPDATE counter_shard:1 SET count += 1;
-- Total = SUM across all shards
SELECT math::sum(count) FROM counter_shard GROUP ALL;
```
---
## Read Performance
### Computed Views for Pre-Aggregation
```surrealql
-- Pre-compute expensive aggregations using events
DEFINE TABLE product_stats SCHEMAFULL;
DEFINE FIELD product ON TABLE product_stats TYPE record<product>;
DEFINE FIELD total_orders ON TABLE product_stats TYPE int DEFAULT 0;
DEFINE FIELD total_revenue ON TABLE product_stats TYPE decimal DEFAULT 0;
DEFINE FIELD avg_rating ON TABLE product_stats TYPE float DEFAULT 0;
DEFINE FIELD review_count ON TABLE product_stats TYPE int DEFAULT 0;
DEFINE FIELD last_ordered ON TABLE product_stats TYPE option<datetime>;
-- Update stats when orders are created
DEFINE EVENT update_product_stats ON TABLE order_item WHEN $event = "CREATE" THEN {
UPSERT product_stats:{$after.product} SET
product = $after.product,
total_orders += 1,
total_revenue += $after.price * $after.quantity,
last_ordered = time::now();
};
-- Update stats when reviews are created
DEFINE EVENT update_review_stats ON TABLE review WHEN $event = "CREATE" THEN {
LET $stats = SELECT * FROM product_stats:{$after.product};
LET $new_count = $stats.review_count + 1;
LET $new_avg = (($stats.avg_rating * $stats.review_count) + $after.rating) / $new_count;
UPSERT product_stats:{$after.product} SET
product = $after.product,
review_count = $new_count,
avg_rating = $new_avg;
};
-- Queries against the stats table are instant (no aggregation at query time)
SELECT * FROM product_stats ORDER BY total_revenue DESC LIMIT 10;
```
### Caching Strategies
```surrealql
-- Application-level cache table for expensive computations
DEFINE TABLE cache_entry SCHEMAFULL;
DEFINE FIELD key ON TABLE cache_entry TYPE string;
DEFINE FIELD value ON TABLE cache_entry TYPE object;
DEFINE FIELD expires_at ON TABLE cache_entry TYPE datetime;
DEFINE FIELD created_at ON TABLE cache_entry TYPE datetime DEFAULT time::now();
DEFINE INDEX idx_cache_key ON TABLE cache_entry COLUMNS key UNIQUE;
DEFINE INDEX idx_cache_expiry ON TABLE cache_entry COLUMNS expires_at;
-- Read from cache or compute
DEFINE FUNCTION fn::cached_query($key: string, $ttl: duration) {
LET $cached = SELECT VALUE value FROM cache_entry
WHERE key = $key AND expires_at > time::now()
LIMIT 1;
IF count($cached) > 0 {
RETURN $cached[0];
};
RETURN NONE; -- cache miss, application should compute and store
};
-- Store in cache
DEFINE FUNCTION fn::cache_set($key: string, $value: object, $ttl: duration) {
UPSERT cache_entry SET
key = $key,
value = $value,
expires_at = time::now() + $ttl;
};
-- Clean expired cache entries periodically
DELETE cache_entry WHERE expires_at < time::now();
```
### Live Query Efficiency
```surrealql
-- Live queries push changes to connected clients in real-time
-- Use them for dashboards, notifications, and collaborative features
-- Subscribe to changes on a table
LIVE SELECT * FROM message WHERE channel = 'general';
-- Subscribe to changes on a specific record
LIVE SELECT * FROM user:alice;
-- Performance tips for live queries:
-- 1. Use WHERE clauses to narrow the subscription scope
-- 2. Project only needed fields to reduce payload size
LIVE SELECT id, title, status FROM task WHERE project = project:123;
-- 3. Avoid subscribing to high-churn tables without filters
-- BAD:
LIVE SELECT * FROM log; -- fires on every log entry
-- GOOD:
LIVE SELECT * FROM log WHERE severity = 'error';
-- 4. Kill live queries when no longer needed
-- KILL $live_query_id;
```
### Connection Pooling (SDK-Level)
```javascript
// JavaScript SDK: Connection pooling is managed by the SDK
import Surreal from 'surrealdb';
// Create a single client instance and reuse it
const db = new Surreal();
await db.connect('wss://db.example.com/rpc');
await db.use({ namespace: 'production', database: 'app' });
// Reuse this 'db' instance across your application
// Do NOT create a new connection per request
// For server-side applications, consider a connection pool wrapper:
// - Maintain a pool of authenticated connections
// - Checkout/checkin connections per request
// - Set reasonable pool size (start with 10-20 per server)
```
```rust
// Rust SDK: Create one client, clone for concurrent tasks
use surrealdb::engine::remote::ws::Ws;
use surrealdb::Surreal;
let db = Surreal::new::<Ws>("ws://localhost:8000").await?;
db.use_ns("production").use_db("app").await?;
// Clone the client for use in spawned tasks
// The underlying connection is shared
let db_clone = db.clone();
tokio::spawn(async move {
let result = db_clone.select("user").await;
});
```
---
## Distributed Performance (TiKV)
### TiKV Architecture Overview
When using TiKV as the storage backend, SurrealDB operates in a distributed mode:
- **PD (Placement Driver)**: Manages cluster metadata and region scheduling
- **TiKV nodes**: Store data in regions, handle read/write requests
- **SurrealDB compute nodes**: Stateless query processors that connect to TiKV
```bash
# Start SurrealDB with TiKV backend
surreal start tikv://pd1:2379,pd2:2379,pd3:2379
# Multiple SurrealDB compute nodes can connect to the same TiKV cluster
# Each compute node is stateless and can handle any request
```
### Region Management
```
# TiKV automatically splits and merges regions based on data size
# Default region size: 96MB
# Regions are automatically distributed across TiKV nodes
# Monitor region distribution:
# - Use TiKV's built-in metrics (Prometheus/Grafana)
# - Watch for hot regions (regions receiving disproportionate traffic)
```
### Replication Factor Tuning
```
# TiKV uses Raft consensus for replication
# Default replication factor: 3 (data exists on 3 nodes)
# Configure via PD:
# Higher replication factor:
# + Better read availability
# + Survives more node failures
# - More storage used
# - Slightly higher write latency
# For most production deployments, 3 replicas is appropriate
# 5 replicas for critical data requiring higher durability
```
### Compute Node Scaling
```bash
# SurrealDB compute nodes are stateless -- scale horizontally
# Put a load balancer in front of multiple compute nodes
# Node 1
surreal start tikv://pd:2379 --bind 0.0.0.0:8000
# Node 2
surreal start tikv://pd:2379 --bind 0.0.0.0:8000
# Node 3
surreal start tikv://pd:2379 --bind 0.0.0.0:8000
# Load balancer distributes requests across nodes
# All nodes see the same data (shared TiKV cluster)
```
### Network Latency Considerations
```
# For distributed deployments:
# - Place SurrealDB compute nodes close to TiKV nodes (same datacenter)
# - Use dedicated network links between compute and storage tiers
# - Cross-region replication adds latency proportional to network distance
# - Read-heavy workloads benefit from local read replicas
# Latency budget (typical):
# - Same rack: <1ms
# - Same datacenter: 1-5ms
# - Cross-datacenter (same region): 5-20ms
# - Cross-region: 50-200ms
```
### Cross-Region Deployment Patterns
```
# Pattern 1: Active-passive (simpler)
# - Primary region handles all writes
# - Secondary region has read replicas
# - Failover to secondary if primary fails
# Pattern 2: Active-active (complex)
# - Both regions handle reads and writes
# - TiKV Raft handles consensus across regions
# - Higher write latency due to cross-region consensus
# - Use region-local reads with follower read feature
# Pattern 3: Region-local with async sync
# - Each region has its own SurrealDB + TiKV cluster
# - Application-level async replication between regions
# - Eventual consistency between regions
# - Lowest latency for local operations
```
---
## Monitoring and Benchmarking
### INFO FOR Statements
```surrealql
-- Introspect database structure and metadata
INFO FOR ROOT;
-- Shows all namespaces
INFO FOR NAMESPACE;
-- Shows all databases in the current namespace
INFO FOR DATABASE;
-- Shows all tables, access methods, and functions in the current database
INFO FOR TABLE user;
-- Shows fields, indexes, events, and permissions for a table
-- Use this to verify indexes exist and are correctly defined
```
### Query Timing
```surrealql
-- Use EXPLAIN to understand query performance
SELECT * FROM user WHERE email = '[email protected]' EXPLAIN FULL;
-- Time queries at the application level
-- Most SDKs support timing query execution
-- Compare query plans before and after adding indexes:
-- Before:
SELECT * FROM order WHERE status = 'pending' EXPLAIN;
-- Result: Iterate Table (full scan)
DEFINE INDEX idx_status ON TABLE order COLUMNS status;
-- After:
SELECT * FROM order WHERE status = 'pending' EXPLAIN;
-- Result: Iterate Index idx_status (index scan)
```
### Resource Monitoring
```bash
# Monitor SurrealDB process
# CPU and memory usage
ps aux | grep surreal
# Disk usage for RocksDB/SurrealKV
du -sh /var/data/surreal.db
# Network connections
ss -tnp | grep surreal
# For TiKV deployments, monitor:
# - PD dashboard (cluster overview)
# - TiKV metrics (region count, leader count, store size)
# - Raft metrics (proposal latency, log append latency)
# Use Prometheus + Grafana for comprehensive monitoring
```
### Benchmark Methodologies
```surrealql
-- Benchmark write throughput
-- Create a test table
DEFINE TABLE bench_write SCHEMALESS;
-- Time how long it takes to insert N records
-- Use batch inserts of 100-1000 records per request
-- Measure requests/second and records/second
-- Benchmark read throughput
-- Create test data, then measure SELECT performance
-- Test with and without indexes
-- Test with varying result set sizes (LIMIT 1, 10, 100, 1000)
-- Benchmark graph traversal
-- Create a known graph topology
-- Time traversals at different depths
-- Measure with and without edge table indexes
-- Benchmark vector search
-- Insert N vectors of known dimension
-- Time KNN queries with varying K values
-- Measure recall against brute-force results
```
### Common Bottlenecks and Solutions
| Bottleneck | Symptom | Solution |
|---|---|---|
| Missing index | Slow WHERE queries, EXPLAIN shows "Iterate Table" | Add appropriate index |
| Over-indexing | Slow writes, high disk usage | Remove unused indexes |
| Large result sets | High memory, slow response | Use LIMIT, pagination, projections |
| Deep graph traversal | Exponential query time | Limit depth, add LIMIT per hop, precompute |
| Vector search on large dataset | Slow KNN queries | Tune HNSW parameters (EFC, M), use metadata pre-filtering |
| Correlated subqueries | Slow queries on large tables | Precompute with events, use LET variables |
| Full-text search on large corpus | Slow text queries | Tune analyzer, use more specific search terms |
| Lock contention | Slow concurrent writes | Reduce transaction size, shard hot records |
| Network latency (TiKV) | High query latency | Co-locate compute and storage, use follower reads |
| Large transactions | Timeout or OOM | Chunk into smaller batches |
---
## Memory Management
### Cache Configuration
```bash
# RocksDB block cache size (controls memory usage for caching data blocks)
# Default is typically 50% of available RAM
# Adjust via environment variables or config
surreal start --rocksdb-cache-size 4GB rocksdb:///var/data/surreal.db
```
### Connection Limits
```bash
# Limit maximum concurrent connections
# Prevents resource exhaustion under load
surreal start --max-connections 1000
# Each connection consumes memory for:
# - Connection state
# - Query buffers
# - Live query subscriptions
# - Transaction context
```
### WASM Memory Considerations
```javascript
// When running SurrealDB in the browser via WASM:
// - Default WASM memory limit is typically 256MB-2GB depending on browser
// - Large datasets or vector indexes may exceed browser memory
// - Use pagination and lazy loading for large result sets
// - Consider server-side SurrealDB for heavy workloads
// For WASM deployments:
// - Keep dataset sizes small (thousands, not millions of records)
// - Avoid HNSW indexes on high-dimensional vectors in WASM
// - Use the server SDK for heavy analytics/search
```
### General Memory Guidelines
| Workload | Recommended RAM | Notes |
|---|---|---|
| Development | 1-2 GB | In-memory or small datasets |
| Small production (< 1M records) | 4-8 GB | Single-node, moderate indexes |
| Medium production (1-10M records) | 16-32 GB | Multiple indexes, vector search |
| Large production (10M+ records) | 64+ GB | Heavy indexing, HNSW on large datasets |
| TiKV node | 16-32 GB per node | Based on region count and data volume |
---
## Performance Checklist
Before deploying to production:
- Run EXPLAIN on all frequent queries to verify index usage
- Add indexes for every field used in WHERE clauses of frequent queries
- Index `in` and `out` columns on all edge tables used in graph traversals
- Use SCHEMAFULL tables to avoid schema inference overhead
- Select only needed fields (avoid `SELECT *` on wide tables)
- Use cursor-based pagination instead of OFFSET for deep pages
- Batch writes in groups of 100-1000 records
- Precompute aggregations using events and stats tables
- Set appropriate HNSW parameters for your dataset size and accuracy needs
- Monitor query latency and index usage regularly
- Use TiKV for workloads requiring horizontal scaling
- Co-locate compute and storage nodes for distributed deployments
- Test with realistic data volumes before production deployment
```
### rules/sdks.md
```markdown
# SurrealDB SDK Reference
This document covers SDK usage patterns for all officially supported languages. SurrealDB provides native SDKs in 9+ languages, each offering HTTP and WebSocket connectivity, with select SDKs also supporting embedded (in-process) database engines.
---
## JavaScript / TypeScript SDK
**Package**: `surrealdb` on npm
**Repository**: github.com/surrealdb/surrealdb.js
### Installation
```bash
npm install surrealdb
# For embedded Node.js engine
npm install @surrealdb/node
# For WASM browser engine
npm install @surrealdb/wasm
```
### Engine Options
The JS/TS SDK supports three distinct engines:
| Engine | Package | Use Case |
|--------|---------|----------|
| Remote (HTTP/WebSocket) | `surrealdb` | Client-server applications |
| Node.js embedded | `@surrealdb/node` | Server-side apps without separate DB process |
| WASM (browser) | `@surrealdb/wasm` | Browser-based applications, offline-first |
### Connection Patterns
```typescript
import { Surreal, RecordId, Table } from "surrealdb";
const db = new Surreal();
// --- Remote connections ---
// WebSocket (recommended for live queries and subscriptions)
await db.connect("wss://host:8000");
// HTTP (stateless, suitable for serverless)
await db.connect("https://host:8000");
// --- Embedded Node.js connections (requires @surrealdb/node) ---
// In-memory (data lost on process exit)
await db.connect("mem://");
// RocksDB persistent storage
await db.connect("rocksdb://path.db");
// SurrealKV persistent storage
await db.connect("surrealkv://path.db");
// SurrealKV with versioned storage (supports historical queries)
await db.connect("surrealkv+versioned://path.db");
// --- WASM browser connections (requires @surrealdb/wasm) ---
// In-memory
await db.connect("mem://");
// IndexedDB persistent storage (browser only)
await db.connect("indxdb://mydb");
```
### Authentication
```typescript
// Root-level authentication
await db.signin({
username: "root",
password: "root",
});
// Namespace-level authentication
await db.signin({
namespace: "my_ns",
username: "ns_user",
password: "ns_pass",
});
// Database-level authentication
await db.signin({
namespace: "my_ns",
database: "my_db",
username: "db_user",
password: "db_pass",
});
// Record-level (scope) authentication
const token = await db.signin({
namespace: "my_ns",
database: "my_db",
access: "user_access",
variables: {
email: "[email protected]",
password: "user_pass",
},
});
// Sign up a new record user
const token = await db.signup({
namespace: "my_ns",
database: "my_db",
access: "user_access",
variables: {
email: "[email protected]",
password: "new_pass",
name: "New User",
},
});
// Use an existing token
await db.authenticate(token);
// Invalidate the current session
await db.invalidate();
```
### Namespace and Database Selection
```typescript
await db.use({
namespace: "my_ns",
database: "my_db",
});
// Or set individually
await db.use({ namespace: "my_ns" });
await db.use({ database: "my_db" });
```
### CRUD Operations
```typescript
// --- Create ---
// Create with auto-generated ID
const person = await db.create("person", {
name: "Alice",
age: 30,
});
// Create with specific ID
const specific = await db.create(new RecordId("person", "alice"), {
name: "Alice",
age: 30,
});
// --- Select ---
// Select all records from a table
const allPeople = await db.select("person");
// Select a specific record
const alice = await db.select(new RecordId("person", "alice"));
// --- Update (full replacement) ---
// Replace all fields on a record
const updated = await db.update(new RecordId("person", "alice"), {
name: "Alice Smith",
age: 31,
email: "[email protected]",
});
// --- Merge (partial update) ---
// Update only specified fields, keep the rest
const merged = await db.merge(new RecordId("person", "alice"), {
age: 32,
});
// --- Patch (JSON Patch operations) ---
const patched = await db.patch(new RecordId("person", "alice"), [
{ op: "replace", path: "/age", value: 33 },
{ op: "add", path: "/verified", value: true },
]);
// --- Delete ---
// Delete a specific record
await db.delete(new RecordId("person", "alice"));
// Delete all records in a table
await db.delete("person");
```
### Queries
```typescript
// Simple query
const results = await db.query("SELECT * FROM person WHERE age > 25");
// Parameterized query (prevents injection)
const results = await db.query(
"SELECT * FROM person WHERE age > $min_age AND name = $name",
{
min_age: 25,
name: "Alice",
}
);
// Typed query results
interface Person {
id: RecordId;
name: string;
age: number;
}
const [people] = await db.query<[Person[]]>(
"SELECT * FROM person WHERE age > $min_age",
{ min_age: 25 }
);
// Multiple statements in one query
const [people, orders] = await db.query<[Person[], Order[]]>(`
SELECT * FROM person WHERE active = true;
SELECT * FROM order WHERE status = 'pending';
`);
```
### Live Queries
```typescript
// Subscribe to changes on a table
const stream = await db.live("person");
// Process events with async iteration
for await (const event of stream) {
switch (event.action) {
case "CREATE":
console.log("New person:", event.result);
break;
case "UPDATE":
console.log("Updated person:", event.result);
break;
case "DELETE":
console.log("Deleted person:", event.result);
break;
}
}
// Live query with a filter
const stream = await db.live("person", {
filter: "age > 25",
});
// Kill a live query
await db.kill(stream.id);
```
### RecordId Usage
```typescript
import { RecordId, Table } from "surrealdb";
// Create a RecordId
const id = new RecordId("person", "alice");
console.log(id.table); // "person"
console.log(id.id); // "alice"
// Numeric IDs
const numericId = new RecordId("person", 123);
// Complex IDs (arrays, objects)
const complexId = new RecordId("temperature", ["London", new Date()]);
// Table reference (for operations on all records)
const table = new Table("person");
```
### Error Handling
```typescript
import { Surreal, SurrealError, ConnectionError } from "surrealdb";
try {
await db.connect("wss://host:8000");
await db.signin({ username: "root", password: "root" });
} catch (error) {
if (error instanceof ConnectionError) {
console.error("Failed to connect:", error.message);
} else if (error instanceof SurrealError) {
console.error("SurrealDB error:", error.message);
}
}
```
### Connection Lifecycle and Reconnection
```typescript
const db = new Surreal();
// Connect with event handlers
db.on("connected", () => console.log("Connected"));
db.on("disconnected", () => console.log("Disconnected"));
db.on("error", (err) => console.error("Error:", err));
await db.connect("wss://host:8000");
// Graceful shutdown
await db.close();
```
---
## JavaScript / TypeScript SDK v2 (GA -- recommended for new projects)
**Package**: `surrealdb` on npm (v2.0.2)
**Status**: General availability. Full SurrealDB 3.0.x support. Recommended for
new projects. The v1 API above is maintained but v2 is the future.
**v2.0.1-v2.0.2 changes**:
- Streamed imports and exports (#563) -- large dataset import/export via streaming
- Blob import support with duplex streaming (#568)
- Return single value for `StringRecordId` (#569) -- no longer wraps in array
- RPC query stat duration parsing fix (#560)
The v2 SDK is a ground-up rewrite with an engine-based architecture, multi-session
support, client-side transactions, query builder patterns, streaming responses,
automatic token refresh, and full SurrealDB 3.0 compatibility.
**v2.0.0 GA highlights** (2026-02-25):
- Full SurrealDB 3.0.1 support (embedded WASM and Node engines updated)
- Engine-based architecture (createRemoteEngines, createNodeEngines, createWasmEngines)
- Multi-session support (newSession, forkSession, await using)
- Client-side transactions
- Automatic token refreshing with refresh token exchange
- Redesigned live query API with subscribe/async iteration
- Query builder pattern with chainable methods
- Expressions API (eq, ne, or, and, between, inside, raw, surql template tag)
- Diagnostics API for protocol-level inspection
- Codec visitor API for custom encode/decode
- User-defined API invocation (.api())
- Web Worker support via createWasmWorkerEngines with createWorker factory
### Installation
```bash
npm install surrealdb
# Embedded engines (published in sync with the SDK)
npm install @surrealdb/node
npm install @surrealdb/wasm
```
### Engine Architecture (v2)
The v2 SDK separates engines from the Surreal class. You compose engines
explicitly in the constructor.
```typescript
import { Surreal, createRemoteEngines } from "surrealdb";
import { createNodeEngines } from "@surrealdb/node";
import { createWasmEngines, createWasmWorkerEngines } from "@surrealdb/wasm";
// Remote only (HTTP + WebSocket)
const db = new Surreal({
engines: createRemoteEngines(),
});
// Remote + embedded Node.js
const db = new Surreal({
engines: {
...createRemoteEngines(),
...createNodeEngines(),
},
});
// Remote + WASM (browser)
const db = new Surreal({
engines: {
...createRemoteEngines(),
...createWasmEngines(),
},
});
// WASM in a Web Worker (offloads DB ops from main thread)
// NOTE: beta.2+ requires createWorker factory for Vite compatibility
import WorkerAgent from "@surrealdb/wasm/worker?worker";
const db = new Surreal({
engines: {
...createRemoteEngines(),
...createWasmWorkerEngines({
createWorker: () => new WorkerAgent(),
}),
},
});
```
### Connection with Auto Token Refresh (v2)
```typescript
const db = new Surreal();
await db.connect("wss://host:8000", {
namespace: "test",
database: "test",
renewAccess: true, // auto-refresh expired tokens (default: true)
authentication: {
username: "root",
password: "root",
},
});
// Or use a callable for async/deferred auth
await db.connect("wss://host:8000", {
namespace: "test",
database: "test",
authentication: () => ({
username: "root",
password: "root",
}),
});
```
### Event Listeners (v2)
```typescript
// Type-safe event subscriptions (replaces v1 .on() pattern)
const unsub = db.subscribe("connected", () => {
console.log("Connected");
});
// Cleanup
unsub();
// Access internal state
console.log(db.namespace); // current namespace
console.log(db.database); // current database
console.log(db.accessToken); // current access token
console.log(db.refreshToken); // current refresh token
console.log(db.params); // defined connection params
```
### Multi-Session Support (v2)
```typescript
// Create an isolated session (own namespace, database, auth state)
const session = await db.newSession();
await session.signin({ username: "other_user", password: "pass" });
await session.use({ namespace: "other_ns", database: "other_db" });
// Fork a session (clone its state)
const forked = await session.forkSession();
// Close a session
await session.closeSession();
// Automatic cleanup with await using (TC39 Explicit Resource Management)
{
await using session = await db.newSession();
// session is automatically closed at end of scope
}
```
### Query Builder Pattern (v2)
v2 introduces chainable builder methods on all query functions. `update` and
`upsert` no longer take contents as a second argument; use `.content()`,
`.merge()`, `.replace()`, or `.patch()` instead.
```typescript
import { Table, RecordId } from "surrealdb";
const usersTable = new Table("users");
// Select with field selection and fetch
const record = await db.select(new RecordId("person", "alice"))
.fields("age", "firstname", "lastname")
.fetch("orders");
// Select with where filter
const active = await db.select(usersTable)
.where(eq("active", true));
// Update with merge (v2 pattern)
await db.update(new RecordId("person", "alice")).merge({
age: 32,
verified: true,
});
// Update with content (full replace)
await db.update(new RecordId("person", "alice")).content({
name: "Alice Smith",
age: 32,
});
// Upsert with merge
await db.upsert(new RecordId("person", "bob")).merge({
name: "Bob",
active: true,
});
```
**IMPORTANT (v2 breaking change)**: Query functions no longer accept plain strings
as table names. You must use the `Table` class:
```typescript
// v1 (still works in v1 SDK)
await db.select("person");
// v2 (required)
await db.select(new Table("person"));
```
### Query Method Overhaul (v2)
```typescript
// Basic typed query
const [user] = await db.query<[User]>("SELECT * FROM user:foo");
// Collect specific result indexes from multi-statement queries
const [users, orders] = await db.query(
"LET $u = SELECT * FROM user; LET $o = SELECT * FROM order; RETURN $u; RETURN $o"
).collect<[User[], Order[]]>(2, 3);
// Auto-jsonify results
const [products] = await db.query<[Product[]]>(
"SELECT * FROM product"
).json();
// Get full response objects (including status, time, etc.)
const responses = await db.query<[Product[]]>(
"SELECT * FROM product"
).responses();
// Stream responses (prepare for future per-record streaming)
const stream = db.query("SELECT * FROM large_table").stream();
for await (const frame of stream) {
if (frame.isValue<Product>()) {
console.log(frame.value);
} else if (frame.isDone()) {
console.log("Stats:", frame.stats);
} else if (frame.isError()) {
console.error(frame.error);
}
}
```
### Expressions API (v2)
Compose dynamic, param-safe WHERE expressions:
```typescript
import { eq, ne, or, and, between, inside, raw, surql } from "surrealdb";
// Use with query builder .where()
await db.select(usersTable).where(eq("active", true));
// Compose complex expressions
await db.select(usersTable).where(
or(
eq("role", "admin"),
and(
eq("active", true),
between("age", 18, 65)
)
)
);
// Use with surql template tag
const isActive = true;
await db.query(surql`SELECT * FROM users WHERE ${eq("active", isActive)}`);
// Raw expression insertion (use with caution)
await db.query(surql`SELECT * FROM users ${raw("WHERE active = true")}`);
```
### Redesigned Live Queries (v2)
```typescript
const live = await db.live(new Table("users"));
// Callback-based (action, result, recordId)
live.subscribe((action, result, record) => {
console.log(action, result, record);
});
// Async iteration
for await (const { action, value } of live) {
console.log(action, value);
}
// Kill the live query
live.kill();
// Attach to an existing live query ID
const [id] = await db.query("LIVE SELECT * FROM users");
const existing = await db.liveOf(id);
```
### User-Defined API Invocation (v2)
```typescript
// Call user-defined APIs registered in SurrealDB
const result = await db.api("my_custom_endpoint", {
param1: "value",
});
```
### Diagnostics API (v2)
Intercept protocol-level communication for debugging:
```typescript
import { applyDiagnostics, createRemoteEngines } from "surrealdb";
const db = new Surreal({
engines: applyDiagnostics(createRemoteEngines(), (event) => {
// event: { type, key, phase, duration?, success?, result? }
console.log(`[${event.type}] ${event.phase}`, event.duration);
}),
});
```
Event types: `open`, `version`, `use`, `signin`, `query`, `reset`.
Each event has `before`, `progress` (queries only), and `after` phases.
### Codec Visitor API (v2)
Custom encode/decode processing for SurrealDB values:
```typescript
const db = new Surreal({
codecOptions: {
valueDecodeVisitor(value) {
// Transform RecordIds, Dates, or custom types on decode
return value;
},
valueEncodeVisitor(value) {
// Transform values before sending to SurrealDB
return value;
},
},
});
```
### Migration Guide: v1 to v2
| v1 Pattern | v2 Equivalent |
|------------|---------------|
| `new Surreal()` | `new Surreal({ engines: createRemoteEngines() })` |
| `db.connect("wss://...")` | Same, but with `authentication` option for auto-refresh |
| `db.on("connected", fn)` | `db.subscribe("connected", fn)` -- returns unsub function |
| `db.select("person")` | `db.select(new Table("person"))` |
| `db.update(id, data)` | `db.update(id).content(data)` or `.merge(data)` |
| `db.merge(id, data)` | `db.update(id).merge(data)` |
| `db.query(q).then(([r]) => ...)` | `db.query(q).collect<[T]>(0)` |
| `db.live("person")` then iterate | `db.live(new Table("person"))` then `.subscribe()` or `for await` |
| N/A | `db.newSession()` -- isolated sessions |
| N/A | `db.query(q).stream()` -- streaming responses |
| N/A | `db.api("endpoint")` -- user-defined APIs |
| N/A | `applyDiagnostics()` -- protocol inspection |
---
## Python SDK
**Package**: `surrealdb` on PyPI (v2.0.0-alpha.1, released 2026-02-26)
**Repository**: github.com/surrealdb/surrealdb.py
**Status**: Pre-release alpha. SurrealDB 3.x support, Python 3.10+ required (3.9 dropped).
**v2.0.0-alpha.1 changes**:
- SurrealDB 3.x feature support added (#230)
- Python 3.9 dropped; minimum is now Python 3.10
- Fixed WebSocket session transaction ID bug (#236)
- Added musl Linux support for Alpine/container deployments (#241)
- Improved error handling with structured error types (#233)
- Pydantic Logfire instrumentation with code examples (#229)
- README and dev docs moved to CONTRIBUTING.md (#243)
### Installation
```bash
pip install surrealdb
```
### Synchronous API
```python
from surrealdb import Surreal
# Context manager ensures clean connection lifecycle
with Surreal("ws://localhost:8000/rpc") as db:
db.signin({"username": "root", "password": "root"})
db.use("test", "test")
# Create
person = db.create("person", {"name": "Alice", "age": 30})
# Create with specific ID
db.create("person:alice", {"name": "Alice", "age": 30})
# Select all
people = db.select("person")
# Select one
alice = db.select("person:alice")
# Update (full replace)
db.update("person:alice", {"name": "Alice Smith", "age": 31})
# Merge (partial update)
db.merge("person:alice", {"age": 32})
# Delete
db.delete("person:alice")
# Query with parameters
result = db.query(
"SELECT * FROM person WHERE age > $min_age",
{"min_age": 25}
)
# Raw query
result = db.query("SELECT * FROM person")
```
### Asynchronous API
```python
import asyncio
from surrealdb import AsyncSurreal
async def main():
async with AsyncSurreal("ws://localhost:8000/rpc") as db:
await db.signin({"username": "root", "password": "root"})
await db.use("test", "test")
# All CRUD operations are async
person = await db.create("person", {"name": "Bob", "age": 25})
people = await db.select("person")
result = await db.query("SELECT * FROM person WHERE age > $min", {"min": 20})
asyncio.run(main())
```
### Embedded Connections
```python
# In-memory (data lost when process exits)
async with AsyncSurreal("memory") as db:
await db.use("test", "test")
await db.create("person", {"name": "Alice"})
# SurrealKV persistent storage
async with AsyncSurreal("surrealkv://mydb") as db:
await db.use("test", "test")
await db.create("person", {"name": "Alice"})
# RocksDB persistent storage
async with AsyncSurreal("rocksdb://mydb") as db:
await db.use("test", "test")
await db.create("person", {"name": "Alice"})
```
### Authentication Patterns
```python
# Root authentication
db.signin({"username": "root", "password": "root"})
# Namespace authentication
db.signin({
"namespace": "my_ns",
"username": "ns_user",
"password": "ns_pass",
})
# Database authentication
db.signin({
"namespace": "my_ns",
"database": "my_db",
"username": "db_user",
"password": "db_pass",
})
# Record user authentication
token = db.signin({
"namespace": "my_ns",
"database": "my_db",
"access": "user_access",
"variables": {
"email": "[email protected]",
"password": "secret",
},
})
# Sign up
token = db.signup({
"namespace": "my_ns",
"database": "my_db",
"access": "user_access",
"variables": {
"email": "[email protected]",
"password": "new_pass",
},
})
# Token-based authentication
db.authenticate(token)
```
### Sessions and Transactions (WebSocket only)
```python
# Transactions are only available over WebSocket connections
with Surreal("ws://localhost:8000/rpc") as db:
db.signin({"username": "root", "password": "root"})
db.use("test", "test")
# Run multiple statements atomically
result = db.query("""
BEGIN TRANSACTION;
CREATE account:alice SET balance = 100;
CREATE account:bob SET balance = 50;
UPDATE account:alice SET balance -= 25;
UPDATE account:bob SET balance += 25;
COMMIT TRANSACTION;
""")
```
### Pydantic and Dataclass Mapping
```python
from dataclasses import dataclass
from typing import Optional
@dataclass
class Person:
name: str
age: int
email: Optional[str] = None
# Query results can be mapped to dataclasses
result = db.query("SELECT * FROM person")
people = [Person(**record) for record in result]
```
### Pydantic Logfire Observability
The Python SDK integrates with Pydantic Logfire for tracing and observability of database operations. Refer to the Logfire documentation for setup details.
---
## Go SDK
**Package**: `github.com/surrealdb/surrealdb.go` (v1.4.0, released 2026-03-03)
**Repository**: github.com/surrealdb/surrealdb.go
**v1.4.0 changes**:
- SurrealDB v3 structured error handling: new `surrealdb.ServerError` type for extracting v3 error fields. Existing `RPCError` and `QueryError` continue to work for v2 compatibility.
- Identifier sanitization in restore to prevent SQL injection (#375)
- Added `models.Table` example for select operations (#379)
### Installation
```bash
go get github.com/surrealdb/surrealdb.go
```
### Connection
```go
package main
import (
"context"
"fmt"
surrealdb "github.com/surrealdb/surrealdb.go"
)
func main() {
ctx := context.Background()
// WebSocket connection
db, err := surrealdb.New("ws://localhost:8000")
if err != nil {
panic(err)
}
defer db.Close()
// HTTP connection
db, err = surrealdb.New("http://localhost:8000")
if err != nil {
panic(err)
}
// Embedded in-memory
db, err = surrealdb.New("mem://")
// Embedded on-disk
db, err = surrealdb.New("surrealkv://path.db")
}
```
### Authentication and Namespace Selection
```go
// Sign in
_, err = db.Signin(ctx, &surrealdb.Auth{
Username: "root",
Password: "root",
})
if err != nil {
panic(err)
}
// Select namespace and database
err = db.Use(ctx, "my_ns", "my_db")
if err != nil {
panic(err)
}
```
### CRUD with Struct Mapping
```go
type Person struct {
ID string `json:"id,omitempty"`
Name string `json:"name"`
Age int `json:"age"`
Email string `json:"email,omitempty"`
}
// Create
person, err := surrealdb.Create[Person](db, ctx, "person", Person{
Name: "Alice",
Age: 30,
})
// Select all
people, err := surrealdb.Select[[]Person](db, ctx, "person")
// Select one
alice, err := surrealdb.Select[Person](db, ctx, "person:alice")
// Update (full replace)
updated, err := surrealdb.Update[Person](db, ctx, "person:alice", Person{
Name: "Alice Smith",
Age: 31,
Email: "[email protected]",
})
// Merge (partial update)
merged, err := surrealdb.Merge[Person](db, ctx, "person:alice", map[string]interface{}{
"age": 32,
})
// Delete
err = db.Delete(ctx, "person:alice")
// Query
results, err := surrealdb.Query[[]Person](db, ctx,
"SELECT * FROM person WHERE age > $min_age",
map[string]interface{}{"min_age": 25},
)
```
### Context-Based Operations
All Go SDK operations accept a `context.Context` parameter, enabling timeout control, cancellation propagation, and deadline management.
```go
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
people, err := surrealdb.Select[[]Person](db, ctx, "person")
if err != nil {
// Handle timeout or cancellation
}
```
---
## Rust SDK
**Crate**: `surrealdb`
**Repository**: github.com/surrealdb/surrealdb
### Cargo.toml
```toml
[dependencies]
surrealdb = "3"
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
```
### Connection
```rust
use surrealdb::Surreal;
use surrealdb::engine::remote::ws::Ws;
use surrealdb::engine::remote::http::Http;
use surrealdb::engine::local::{Mem, RocksDb, SurrealKv};
// WebSocket
let db = Surreal::new::<Ws>("localhost:8000").await?;
// HTTP
let db = Surreal::new::<Http>("localhost:8000").await?;
// Embedded in-memory
let db = Surreal::new::<Mem>(()).await?;
// Embedded RocksDB
let db = Surreal::new::<RocksDb>("path.db").await?;
// Embedded SurrealKV
let db = Surreal::new::<SurrealKv>("path.db").await?;
```
### Authentication
```rust
use surrealdb::opt::auth::Root;
db.signin(Root {
username: "root",
password: "root",
}).await?;
db.use_ns("my_ns").use_db("my_db").await?;
```
### CRUD with Serde
```rust
use serde::{Deserialize, Serialize};
use surrealdb::RecordId;
#[derive(Debug, Serialize, Deserialize)]
struct Person {
name: String,
age: u32,
email: Option<String>,
}
#[derive(Debug, Deserialize)]
struct Record {
id: RecordId,
}
// Create
let created: Vec<Record> = db.create("person")
.content(Person {
name: "Alice".to_string(),
age: 30,
email: None,
})
.await?;
// Create with specific ID
let alice: Option<Record> = db.create(("person", "alice"))
.content(Person {
name: "Alice".to_string(),
age: 30,
email: None,
})
.await?;
// Select all
let people: Vec<Person> = db.select("person").await?;
// Select one
let alice: Option<Person> = db.select(("person", "alice")).await?;
// Update
let updated: Option<Person> = db.update(("person", "alice"))
.content(Person {
name: "Alice Smith".to_string(),
age: 31,
email: Some("[email protected]".to_string()),
})
.await?;
// Merge
let merged: Option<Person> = db.update(("person", "alice"))
.merge(serde_json::json!({ "age": 32 }))
.await?;
// Delete
let _: Option<Person> = db.delete(("person", "alice")).await?;
```
### Queries
```rust
// Parameterized query
let mut result = db.query("SELECT * FROM person WHERE age > $min_age")
.bind(("min_age", 25))
.await?;
let people: Vec<Person> = result.take(0)?;
// Multiple statements
let mut result = db.query("SELECT * FROM person; SELECT * FROM order;").await?;
let people: Vec<Person> = result.take(0)?;
let orders: Vec<Order> = result.take(1)?;
```
### Live Queries
```rust
use surrealdb::Notification;
use futures::StreamExt;
let mut stream = db.select("person").live().await?;
while let Some(notification) = stream.next().await {
let notification: Notification<Person> = notification?;
match notification.action {
Action::Create => println!("Created: {:?}", notification.data),
Action::Update => println!("Updated: {:?}", notification.data),
Action::Delete => println!("Deleted: {:?}", notification.data),
_ => {}
}
}
```
---
## Java SDK
**Package**: Available on Maven Central
**Repository**: github.com/surrealdb/surrealdb.java
### Maven Dependency
```xml
<dependency>
<groupId>com.surrealdb</groupId>
<artifactId>surrealdb</artifactId>
<version>3.0.0</version>
</dependency>
```
### Connection and Authentication
```java
import com.surrealdb.Surreal;
// WebSocket connection
Surreal db = new Surreal();
db.connect("ws://localhost:8000");
db.signin("root", "root");
db.use("my_ns", "my_db");
// HTTP connection
db.connect("http://localhost:8000");
```
### CRUD Operations
```java
// Create
db.create("person", Map.of(
"name", "Alice",
"age", 30
));
// Select
List<Map<String, Object>> people = db.select("person");
// Query with parameters
List<Map<String, Object>> results = db.query(
"SELECT * FROM person WHERE age > $min_age",
Map.of("min_age", 25)
);
// Update
db.update("person:alice", Map.of(
"name", "Alice Smith",
"age", 31
));
// Merge
db.merge("person:alice", Map.of("age", 32));
// Delete
db.delete("person:alice");
```
### Async Operations with CompletableFuture
```java
CompletableFuture<List<Map<String, Object>>> future = db.queryAsync(
"SELECT * FROM person WHERE age > $min_age",
Map.of("min_age", 25)
);
future.thenAccept(results -> {
results.forEach(System.out::println);
}).exceptionally(error -> {
System.err.println("Query failed: " + error.getMessage());
return null;
});
```
---
## .NET SDK
**Package**: `SurrealDb.Net` on NuGet
**Repository**: github.com/surrealdb/surrealdb.net
### Installation
```bash
dotnet add package SurrealDb.Net
```
### Basic Usage
```csharp
using SurrealDb.Net;
using SurrealDb.Net.Models;
// Create client
var db = new SurrealDbClient("ws://localhost:8000/rpc");
await db.SignIn(new RootAuth { Username = "root", Password = "root" });
await db.Use("my_ns", "my_db");
// Create
var person = await db.Create("person", new Person
{
Name = "Alice",
Age = 30
});
// Select all
var people = await db.Select<Person>("person");
// Query
var results = await db.Query(
"SELECT * FROM person WHERE age > $min_age",
new { min_age = 25 }
);
// Dispose
await db.DisposeAsync();
```
### Dependency Injection
```csharp
// In Startup.cs or Program.cs
builder.Services.AddSurreal(options =>
{
options.Endpoint = "ws://localhost:8000/rpc";
options.Namespace = "my_ns";
options.Database = "my_db";
});
// In a service or controller
public class PersonService
{
private readonly ISurrealDbClient _db;
public PersonService(ISurrealDbClient db)
{
_db = db;
}
public async Task<IEnumerable<Person>> GetPeople()
{
return await _db.Select<Person>("person");
}
}
```
### LINQ Integration
```csharp
// Query using LINQ-style expressions where supported
var adults = await db.Select<Person>("person")
.Where(p => p.Age >= 18)
.OrderBy(p => p.Name)
.ToListAsync();
```
---
## PHP SDK
**Package**: `surrealdb/surrealdb.php` on Packagist
**Repository**: github.com/surrealdb/surrealdb.php
### Installation
```bash
composer require surrealdb/surrealdb.php
```
### Basic Usage
```php
use Surreal\Surreal;
$db = new Surreal();
// Connect via WebSocket
$db->connect("ws://localhost:8000/rpc");
// Or via HTTP
$db->connect("http://localhost:8000");
// Authenticate
$db->signin([
"username" => "root",
"password" => "root",
]);
$db->use(["namespace" => "my_ns", "database" => "my_db"]);
// Create
$person = $db->create("person", [
"name" => "Alice",
"age" => 30,
]);
// Select
$people = $db->select("person");
$alice = $db->select("person:alice");
// Query
$results = $db->query(
"SELECT * FROM person WHERE age > $min_age",
["min_age" => 25]
);
// Update
$db->update("person:alice", [
"name" => "Alice Smith",
"age" => 31,
]);
// Merge
$db->merge("person:alice", ["age" => 32]);
// Delete
$db->delete("person:alice");
// Close
$db->close();
```
---
## SDK Selection Guide
### Decision Matrix
| Factor | JS/TS | Python | Go | Rust | Java | .NET | PHP |
|--------|-------|--------|----|------|------|------|-----|
| Embedded engine | Yes | Yes | Yes | Yes | No | No | No |
| WebSocket | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| HTTP | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Live queries | Yes | Yes | Yes | Yes | Limited | Limited | No |
| WASM (browser) | Yes | No | No | No | No | No | No |
| Async API | Yes | Yes | Yes | Yes | Yes | Yes | No |
| Type safety | TS generics | Type hints | Generics | Strong | Generics | Generics | Weak |
### When to Use Each SDK
- **JavaScript/TypeScript**: Web applications, full-stack JS projects, browser-based apps (WASM), serverless functions, real-time apps with live queries.
- **Python**: Data science, machine learning pipelines, scripting, backend APIs (FastAPI/Django), prototyping.
- **Go**: Microservices, high-concurrency servers, CLI tools, cloud-native applications.
- **Rust**: Performance-critical applications, systems programming, embedded databases in Rust apps, when you need direct library integration without network overhead.
- **Java**: Enterprise applications, Spring Boot services, Android development.
- **.NET**: ASP.NET applications, Windows services, enterprise C# codebases, Blazor apps.
- **PHP**: Laravel/Symfony applications, WordPress plugins, traditional web applications.
### Embedded vs Remote Trade-offs
**Embedded** (in-process database):
- No network latency
- Single-process deployment
- No separate database server to manage
- Limited to single-node (no distributed queries)
- Uses process memory for database operations
- Available in: JS/TS (Node.js, WASM), Python, Go, Rust
**Remote** (HTTP/WebSocket to server):
- Shared database across multiple application instances
- Supports TiKV for distributed storage
- Independent scaling of compute and storage
- Network latency overhead
- Requires running SurrealDB server
- Available in: All SDKs
### Performance Characteristics
- **Rust SDK**: Lowest overhead; direct library calls when embedded, minimal serialization.
- **Go SDK**: Low overhead; efficient goroutine-based concurrency.
- **Node.js embedded**: V8 + native bindings; good for I/O-heavy workloads.
- **Python SDK**: Higher overhead due to GIL; use async API for I/O-bound workloads.
- **WASM (browser)**: Runs entirely client-side; performance depends on browser WASM runtime.
```
### rules/surrealism.md
```markdown
# Surrealism -- SurrealDB WASM Extension System
Surrealism is SurrealDB's plugin/extension system, introduced in v3.0.0. It enables developers to write custom functions in Rust, compile them to WebAssembly (WASM), and register them as database modules callable from SurrealQL queries.
---
## Overview
Surrealism bridges the Rust ecosystem with SurrealDB's query engine. You can write arbitrary Rust functions, use any Rust crate, compile to WASM, and expose the resulting functions as first-class SurrealQL functions.
Key properties:
- Functions run inside the SurrealDB process as WASM modules
- Sandboxed execution with controlled memory and CPU
- Access to the full Rust crate ecosystem
- Called from SurrealQL using the `module_name::function_name()` syntax
- Managed via the `surreal module` CLI command
---
## Development Workflow
### Step 1: Create a Rust Project
Create a new Rust library project with Surrealism support:
```bash
cargo new --lib my_module
cd my_module
```
### Step 2: Configure Cargo.toml
```toml
[package]
name = "my_module"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["cdylib"]
[dependencies]
surrealism = "3"
```
### Step 3: Configure surrealism.toml
Create a `surrealism.toml` in the project root:
```toml
[module]
name = "my_module"
version = "0.1.0"
description = "A custom SurrealDB module"
```
### Step 4: Write Functions
Annotate functions with the `#[surrealism]` attribute:
```rust
// src/lib.rs
use surrealism::prelude::*;
#[surrealism]
fn hello(name: String) -> String {
format!("Hello, {}!", name)
}
#[surrealism]
fn add(a: f64, b: f64) -> f64 {
a + b
}
#[surrealism]
fn reverse_string(input: String) -> String {
input.chars().rev().collect()
}
```
### Step 5: Build the WASM Module
```bash
surreal module build --path ./my_module
```
This compiles the Rust project to a WASM binary optimized for SurrealDB.
### Step 6: Register in the Database
```surrealql
-- Define a bucket for storing WASM modules
DEFINE BUCKET wasm_modules;
-- Register the compiled module
DEFINE MODULE my_module FROM "wasm_modules/my_module.wasm";
```
### Step 7: Call from SurrealQL
```surrealql
-- Call module functions using module_name::function_name syntax
SELECT my_module::hello("World");
-- Returns: "Hello, World!"
SELECT my_module::add(10, 25.5);
-- Returns: 35.5
SELECT my_module::reverse_string("SurrealDB");
-- Returns: "BDlaerruS"
-- Use in WHERE clauses
SELECT * FROM product WHERE my_module::validate_sku(sku) = true;
-- Use in field projections
SELECT name, my_module::format_price(price) AS formatted_price FROM product;
```
---
## Type Mapping
Surrealism maps between Rust types and SurrealQL types:
| Rust Type | SurrealQL Type | Notes |
|-----------|---------------|-------|
| `String` | `string` | UTF-8 text |
| `bool` | `bool` | Boolean |
| `i64` | `int` | 64-bit integer |
| `f64` | `float` | 64-bit float |
| `Vec<T>` | `array` | Array of mapped type |
| `Option<T>` | `T \| NONE` | Nullable values |
| `HashMap<String, T>` | `object` | Key-value object |
| `()` | `NONE` | No return value |
| `Result<T, E>` | `T` or error | Errors propagate to SurrealQL |
---
## Error Handling
Use Rust's `Result` type to handle errors gracefully. Errors propagate as SurrealQL query errors.
```rust
use surrealism::prelude::*;
#[surrealism]
fn divide(a: f64, b: f64) -> Result<f64, String> {
if b == 0.0 {
Err("Division by zero".to_string())
} else {
Ok(a / b)
}
}
#[surrealism]
fn parse_json(input: String) -> Result<String, String> {
let parsed: serde_json::Value = serde_json::from_str(&input)
.map_err(|e| format!("Invalid JSON: {}", e))?;
Ok(parsed.to_string())
}
```
When called from SurrealQL:
```surrealql
SELECT my_module::divide(10, 0);
-- Error: "Division by zero"
SELECT my_module::divide(10, 3);
-- Returns: 3.3333333333333335
```
---
## Advanced Examples
### Fake Data Generation
Use the `fake` crate to generate test data:
```toml
[dependencies]
surrealism = "3"
fake = { version = "2", features = ["derive"] }
rand = "0.8"
```
```rust
use surrealism::prelude::*;
use fake::{Fake, faker::name::en::*, faker::internet::en::*, faker::address::en::*};
use rand::thread_rng;
#[surrealism]
fn fake_name() -> String {
let mut rng = thread_rng();
Name().fake_with_rng(&mut rng)
}
#[surrealism]
fn fake_email() -> String {
let mut rng = thread_rng();
FreeEmail().fake_with_rng(&mut rng)
}
#[surrealism]
fn fake_city() -> String {
let mut rng = thread_rng();
CityName().fake_with_rng(&mut rng)
}
```
```surrealql
-- Generate fake test data
CREATE person SET
name = faker::fake_name(),
email = faker::fake_email(),
city = faker::fake_city();
```
### Custom Validation Logic
```rust
use surrealism::prelude::*;
#[surrealism]
fn validate_email(email: String) -> bool {
let parts: Vec<&str> = email.split('@').collect();
if parts.len() != 2 {
return false;
}
let domain_parts: Vec<&str> = parts[1].split('.').collect();
!parts[0].is_empty() && domain_parts.len() >= 2 && domain_parts.iter().all(|p| !p.is_empty())
}
#[surrealism]
fn validate_phone(phone: String) -> bool {
let digits: String = phone.chars().filter(|c| c.is_ascii_digit()).collect();
digits.len() >= 10 && digits.len() <= 15
}
#[surrealism]
fn sanitize_html(input: String) -> String {
input
.replace('<', "<")
.replace('>', ">")
.replace('"', """)
.replace('\'', "'")
}
```
```surrealql
-- Use in field definitions for validation
DEFINE FIELD email ON person VALUE $value
ASSERT validation::validate_email($value) = true;
-- Use to sanitize user input
UPDATE post SET content = validation::sanitize_html(content);
```
### Business Rule Engine
```rust
use surrealism::prelude::*;
#[surrealism]
fn calculate_discount(total: f64, customer_tier: String) -> f64 {
let discount_rate = match customer_tier.as_str() {
"platinum" => 0.20,
"gold" => 0.15,
"silver" => 0.10,
"bronze" => 0.05,
_ => 0.0,
};
let volume_discount = if total > 1000.0 {
0.05
} else if total > 500.0 {
0.02
} else {
0.0
};
total * (discount_rate + volume_discount)
}
#[surrealism]
fn calculate_shipping(weight_kg: f64, zone: String) -> Result<f64, String> {
let base_rate = match zone.as_str() {
"domestic" => 5.0,
"continental" => 15.0,
"international" => 30.0,
_ => return Err(format!("Unknown shipping zone: {}", zone)),
};
Ok(base_rate + (weight_kg * 2.5))
}
```
```surrealql
SELECT
*,
pricing::calculate_discount(total, customer.tier) AS discount,
pricing::calculate_shipping(weight, shipping_zone) AS shipping_cost
FROM order
WHERE status = 'pending';
```
### Quantitative Finance
```rust
use surrealism::prelude::*;
#[surrealism]
fn compound_interest(principal: f64, rate: f64, periods: f64) -> f64 {
principal * (1.0 + rate).powf(periods)
}
#[surrealism]
fn present_value(future_value: f64, rate: f64, periods: f64) -> f64 {
future_value / (1.0 + rate).powf(periods)
}
#[surrealism]
fn moving_average(values: Vec<f64>, window: i64) -> Vec<f64> {
let window = window as usize;
if values.len() < window {
return vec![];
}
values
.windows(window)
.map(|w| w.iter().sum::<f64>() / window as f64)
.collect()
}
```
### Text Processing
```rust
use surrealism::prelude::*;
#[surrealism]
fn slug(input: String) -> String {
input
.to_lowercase()
.chars()
.map(|c| if c.is_alphanumeric() { c } else { '-' })
.collect::<String>()
.split('-')
.filter(|s| !s.is_empty())
.collect::<Vec<&str>>()
.join("-")
}
#[surrealism]
fn word_count(text: String) -> i64 {
text.split_whitespace().count() as i64
}
#[surrealism]
fn truncate(text: String, max_len: i64) -> String {
let max_len = max_len as usize;
if text.len() <= max_len {
text
} else if max_len > 3 {
format!("{}...", &text[..max_len - 3])
} else {
text[..max_len].to_string()
}
}
```
---
## Use Cases
- **Custom data generation**: Generate realistic test data using Rust crate ecosystem (fake, rand)
- **Domain-specific functions**: Financial calculations, scientific formulas, statistical functions
- **Validation logic**: Complex input validation beyond what SurrealQL assertions support natively
- **Business rule engines**: Pricing rules, discount calculations, eligibility checks
- **Text processing**: Slugification, templating, sanitization, NLP preprocessing
- **External library access**: Leverage any Rust crate compiled to WASM
- **Encoding and hashing**: Custom encoding schemes, checksums, hash functions
- **Data transformation**: Complex ETL transformations within queries
---
## Best Practices
**Keep modules small and focused.** Each module should serve a single domain. Prefer multiple small modules over one monolithic module. This improves build times, reduces WASM binary size, and makes version management simpler.
**Handle all errors gracefully.** Always use `Result` return types for functions that can fail. Provide descriptive error messages that help diagnose issues when viewed in SurrealQL query results.
**Version modules alongside database schemas.** When your database schema depends on module functions (e.g., in DEFINE FIELD assertions), keep module versions synchronized with schema migrations.
**Test modules independently before deployment.** Write standard Rust unit tests for your module functions. Test with edge cases, empty inputs, and boundary values before compiling to WASM and registering in the database.
**Be mindful of memory and execution time.** WASM modules run inside the database process. Functions that allocate large amounts of memory or perform long-running computations can affect query performance. For heavy computation, consider running it outside the database and storing results.
**Avoid side effects.** Surrealism functions should be pure -- given the same inputs, they produce the same outputs. Do not rely on global mutable state, file system access, or network calls within module functions.
**Use descriptive module and function names.** Since functions are called as `module_name::function_name()` in SurrealQL, choose names that clearly indicate the function's purpose without requiring documentation lookup.
---
## Limitations and Status
Surrealism was introduced in SurrealDB v3.0.0 and is under active development. Current considerations:
- The API may evolve in future SurrealDB releases
- WASM modules cannot make network calls or access the filesystem
- Module execution is sandboxed with memory and CPU limits
- Complex dependencies may increase WASM binary size significantly
- Debugging WASM modules requires Rust-level tooling (the database provides limited introspection)
- Community feedback is encouraged to shape the direction of the extension system
```
### rules/surrealist.md
```markdown
# Surrealist -- SurrealDB IDE and Management Interface
Surrealist is the official graphical management interface for SurrealDB. It is available as a web application at app.surrealdb.com and as a standalone desktop application built with Tauri.
Current version: **v3.7.3** (2026-03-13). New: PrivateLink support, streamed import/export, org ID in settings, improved node rendering performance, dataset rename, better ticket display/filtering.
---
## Overview
Surrealist provides a comprehensive interface for managing SurrealDB instances, including query authoring, data exploration, schema design, and cloud instance management. It supports connections to local, remote, and cloud-hosted SurrealDB instances.
---
## Views and Features
### Query View
The primary interface for writing and executing SurrealQL queries.
**Capabilities:**
- Full SurrealQL syntax highlighting with autocomplete
- Multiple query tabs for working on several queries simultaneously
- Saved queries library for frequently used queries
- Query history with search
- Multiple result display modes:
- Table view for tabular results
- JSON view for raw response data
- Graph visualization for relationship queries (displays nodes and edges interactively)
- Query execution timing and statistics
- Variable binding panel for parameterized queries
- Query formatting and validation
**Usage Patterns:**
```surrealql
-- Write queries with syntax highlighting and autocomplete
SELECT * FROM person WHERE age > 25 ORDER BY name;
-- Use the variable panel to bind parameters
-- Set $min_age = 25 in the variables panel, then:
SELECT * FROM person WHERE age > $min_age;
-- Graph queries are visualized as interactive node-edge diagrams
SELECT *, ->knows->person AS friends FROM person;
```
### Explorer View
Browse and inspect database records interactively.
**Capabilities:**
- Table listing with record counts
- Record browsing with pagination
- Individual record inspection and editing
- Follow record links and graph edges to navigate between related records
- Inline record creation and deletion
- Field-level filtering and sorting
- Export selected records
### Designer View
Visual schema design and entity relationship diagrams.
**Capabilities:**
- Visual entity-relationship diagrams showing tables and their fields
- Table relationship visualization (record links, graph edges)
- Schema definition viewing and editing
- Field type information display
- Index visualization
- Access rule overview
- Event and changefeed display
### GraphQL View
Execute GraphQL queries against SurrealDB's built-in GraphQL endpoint.
**Capabilities:**
- GraphQL syntax highlighting
- Query variable panel
- Response viewer
- Schema introspection
### Cloud Panel
Manage SurrealDB Cloud instances directly from Surrealist.
**Capabilities:**
- Provision new cloud database instances
- View and manage existing instances
- Monitor instance health and metrics
- Scale instance resources
- Manage billing and usage
---
## Key Features
### Surreal Sidekick
AI-powered assistant integrated into Surrealist that helps with query writing and schema design.
**What it does:**
- Suggests SurrealQL queries based on natural language descriptions
- Explains existing queries
- Recommends schema improvements
- Helps debug query errors
- Suggests index optimizations
### One-Click Local Database (Desktop Only)
The desktop application can start a local SurrealDB instance with a single click, without requiring a separate server installation. This is useful for development and testing.
### Sandbox Environment
A built-in sandbox environment for learning SurrealQL without connecting to any database. It provides an in-memory database instance where you can experiment freely.
### Command Palette
Quick navigation and action system accessible via keyboard shortcut:
- Switch between views
- Open saved queries
- Connect to different databases
- Execute common actions
- Search across the interface
### SurrealML Model Upload
Upload and manage SurrealML machine learning models through the interface. This integrates with SurrealDB's built-in ML capabilities.
### Auto-Generated API Documentation
View auto-generated REST and WebSocket API documentation for your database schema, including available tables, fields, and access rules.
### Access Rule Management
Visual interface for managing SurrealDB access rules:
- Define and edit access methods
- Configure record-level access
- Set up namespace and database authentication
- Manage user permissions
### Connection Management
Manage multiple database connections:
- Save connection profiles with authentication credentials
- Switch between connections quickly
- Support for local, remote, and cloud connections
- Connection health indicators
- WebSocket and HTTP connection options
---
## Configuration
### Connection Strings
Surrealist accepts standard SurrealDB connection strings:
```
# WebSocket (recommended)
ws://localhost:8000
wss://your-server.example.com:8000
# HTTP
http://localhost:8000
https://your-server.example.com:8000
# Cloud
wss://your-instance.surrealdb.cloud
```
### Authentication Setup
When connecting to a SurrealDB instance, configure authentication in the connection dialog:
- **Root credentials**: Username and password for root-level access
- **Namespace credentials**: For namespace-scoped access
- **Database credentials**: For database-scoped access
- **Token-based**: Paste an existing authentication token
- **Access method**: Use a defined access method for record-level authentication
### Namespace and Database Selection
After connecting, select the target namespace and database from the dropdowns in the toolbar, or specify them in the connection configuration.
---
## Desktop Application
The desktop version of Surrealist is built with Tauri and provides additional capabilities over the web version:
- **Local database serving**: Start an embedded SurrealDB instance without installing the server separately
- **File system access**: Import and export files directly
- **Native performance**: Runs as a native application
- **Offline mode**: Work with local databases without internet access
- **System tray integration**: Keep Surrealist running in the background
### Installation
Download the desktop application from the SurrealDB website or the Surrealist GitHub releases page. Available for macOS, Windows, and Linux.
---
## Web Application
Access the web version at app.surrealdb.com. The web application provides the same core functionality as the desktop version except for local database serving and file system access.
No installation required -- works in any modern browser.
---
## Keyboard Shortcuts
Common keyboard shortcuts for efficient navigation:
| Action | Shortcut |
|--------|----------|
| Execute query | Ctrl/Cmd + Enter |
| New query tab | Ctrl/Cmd + T |
| Close query tab | Ctrl/Cmd + W |
| Format query | Ctrl/Cmd + Shift + F |
| Command palette | Ctrl/Cmd + K |
| Toggle sidebar | Ctrl/Cmd + B |
| Save query | Ctrl/Cmd + S |
| Switch to Query view | Ctrl/Cmd + 1 |
| Switch to Explorer view | Ctrl/Cmd + 2 |
| Switch to Designer view | Ctrl/Cmd + 3 |
---
## Tips
- Use the graph visualization in Query View to understand relationships between records. Execute queries that include graph traversals and the results will render as an interactive node-edge diagram.
- The Explorer View is the fastest way to inspect individual records and navigate relationships by following record links.
- Save frequently used queries in the saved queries library. Organize them with descriptive names for quick retrieval.
- Use the sandbox to test destructive operations (DELETE, REMOVE TABLE) safely before running them on production data.
- The command palette provides the fastest way to navigate between views and execute common actions without leaving the keyboard.
```
### rules/surreal-sync.md
```markdown
# Surreal-Sync -- Data Migration and Synchronization
Surreal-Sync is SurrealDB's data migration and synchronization tool. It enables full and incremental data transfer from external databases and streaming sources into SurrealDB, supporting both one-time migrations and continuous synchronization.
**Recent updates** (main branch, 2026-03-13):
- SurrealDB v3 compatibility: version detection, v2/v3 query handling, value sanitization
- PostgreSQL: foreign key support and automatic relation handling
- TOML config file support for PostgreSQL sources
- Neo4j: fixed relationship in/out ID mismatch
- Improved test infrastructure with shared containers and dynamic port binding
---
## Supported Sources
| Source | Sync Type | Method | Incremental Tracking |
|--------|-----------|--------|---------------------|
| MongoDB | Full + incremental | Change streams | Resume token |
| MySQL | Full + incremental | Trigger-based CDC + sequence checkpoints | Trigger tables |
| PostgreSQL (triggers) | Full + incremental | Trigger-based CDC | Trigger tables |
| PostgreSQL (wal2json) | Full + incremental | Logical replication | LSN position |
| Neo4j | Full + incremental | Timestamp-based tracking | Timestamp watermark |
| JSONL | Bulk import | JSON Lines file processing | N/A (one-time) |
| Kafka | Streaming | Consumer subscriptions with deduplication | Consumer offset |
---
## CLI Usage
### General Syntax
```bash
surreal-sync from <SOURCE> <COMMAND> \
--connection-string <CONN_STRING> \
--surreal-endpoint <ENDPOINT> \
--surreal-username <USER> \
--surreal-password <PASS> \
--to-namespace <NS> \
--to-database <DB>
```
### Common Flags
| Flag | Description | Required |
|------|-------------|----------|
| `--connection-string` | Source database connection string | Yes |
| `--surreal-endpoint` | SurrealDB endpoint URL | Yes |
| `--surreal-username` | SurrealDB username | Yes |
| `--surreal-password` | SurrealDB password | Yes |
| `--to-namespace` | Target SurrealDB namespace | Yes |
| `--to-database` | Target SurrealDB database | Yes |
| `--batch-size` | Records per batch (default varies by source) | No |
| `--tables` | Comma-separated list of tables to sync | No |
| `--exclude-tables` | Comma-separated list of tables to exclude | No |
---
## Source-Specific Guides
### MongoDB
MongoDB synchronization uses change streams for incremental updates, which requires a replica set or sharded cluster.
**Prerequisites:**
- MongoDB 3.6+ with replica set enabled
- Read access to the source database
- Change stream access for incremental sync
**Full Sync:**
```bash
surreal-sync from mongodb full \
--connection-string "mongodb://user:pass@host:27017/mydb?replicaSet=rs0" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
**Incremental Sync (Change Streams):**
```bash
surreal-sync from mongodb watch \
--connection-string "mongodb://user:pass@host:27017/mydb?replicaSet=rs0" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
**Data Mapping:**
- MongoDB collections map to SurrealDB tables
- BSON `ObjectId` maps to SurrealDB string IDs
- Nested documents are preserved as SurrealDB objects
- Arrays are preserved as SurrealDB arrays
- BSON types (Date, Decimal128, etc.) are converted to appropriate SurrealQL types
**Example Schema Mapping:**
```
MongoDB SurrealDB
------ ---------
db.users (collection) -> users (table)
_id: ObjectId("abc123") -> users:abc123 (record ID)
{ name: "Alice", -> { name: "Alice",
address: { -> address: {
city: "NYC" -> city: "NYC"
}, -> },
tags: ["admin"] -> tags: ["admin"]
} -> }
```
### PostgreSQL (Trigger-Based CDC)
Trigger-based CDC installs database triggers on source tables to capture INSERT, UPDATE, and DELETE operations.
**Prerequisites:**
- PostgreSQL 10+
- Permission to create triggers and tables on the source database
- Surreal-sync will create a CDC tracking table
**Setup:**
```bash
# Install triggers on specified tables
surreal-sync from postgres-triggers setup \
--connection-string "postgresql://user:pass@host:5432/mydb" \
--tables "users,orders,products"
```
**Full Sync:**
```bash
surreal-sync from postgres-triggers full \
--connection-string "postgresql://user:pass@host:5432/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
**Incremental Sync:**
```bash
surreal-sync from postgres-triggers watch \
--connection-string "postgresql://user:pass@host:5432/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
### PostgreSQL (wal2json)
Logical replication with wal2json provides CDC without modifying the source database schema (no triggers needed).
**Prerequisites:**
- PostgreSQL 9.4+ with `wal2json` extension installed
- `wal_level = logical` in postgresql.conf
- Replication slot permissions
**Configure PostgreSQL:**
```sql
-- postgresql.conf
-- wal_level = logical
-- max_replication_slots = 4
-- max_wal_senders = 4
-- Create a replication slot
SELECT pg_create_logical_replication_slot('surreal_sync', 'wal2json');
```
**Sync:**
```bash
surreal-sync from postgres-wal full \
--connection-string "postgresql://user:pass@host:5432/mydb" \
--replication-slot "surreal_sync" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
surreal-sync from postgres-wal watch \
--connection-string "postgresql://user:pass@host:5432/mydb" \
--replication-slot "surreal_sync" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
### MySQL
MySQL synchronization uses trigger-based CDC with sequence checkpoints for incremental tracking.
**Prerequisites:**
- MySQL 5.7+
- Permission to create triggers and tables
- Surreal-sync will create CDC tracking tables
**Setup:**
```bash
surreal-sync from mysql setup \
--connection-string "mysql://user:pass@host:3306/mydb" \
--tables "users,orders,products"
```
**Full Sync:**
```bash
surreal-sync from mysql full \
--connection-string "mysql://user:pass@host:3306/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
**Incremental Sync:**
```bash
surreal-sync from mysql watch \
--connection-string "mysql://user:pass@host:3306/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
### Neo4j
Neo4j synchronization maps nodes and relationships to SurrealDB records and edges.
**Prerequisites:**
- Neo4j 4.0+
- Read access to the source database
- Nodes should have a timestamp property for incremental sync
**Full Sync:**
```bash
surreal-sync from neo4j full \
--connection-string "bolt://user:pass@host:7687" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
**Incremental Sync:**
```bash
surreal-sync from neo4j watch \
--connection-string "bolt://user:pass@host:7687" \
--timestamp-field "updatedAt" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
**Data Mapping:**
```
Neo4j SurrealDB
----- ---------
(:Person {name: "Alice"}) -> person:xxx { name: "Alice" }
(:Company {name: "Acme"}) -> company:xxx { name: "Acme" }
-[:WORKS_AT {since: 2020}]-> works_at:xxx { since: 2020, in: person:xxx, out: company:xxx }
```
- Neo4j node labels map to SurrealDB table names (lowercased)
- Neo4j relationship types map to SurrealDB edge table names (lowercased)
- Properties are preserved
- Graph structure is maintained through SurrealDB's graph edge model
### JSONL (JSON Lines)
Bulk import from JSON Lines files.
```bash
surreal-sync from jsonl import \
--file /path/to/data.jsonl \
--table "my_table" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app \
--batch-size 1000
```
**JSONL Format:**
Each line is a complete JSON object:
```json
{"name": "Alice", "age": 30, "email": "[email protected]"}
{"name": "Bob", "age": 25, "email": "[email protected]"}
{"name": "Charlie", "age": 35, "email": "[email protected]"}
```
### Kafka
Streaming sync from Kafka topics with deduplication support.
**Prerequisites:**
- Kafka 2.0+
- Topic must contain JSON messages
- Consumer group permissions
```bash
surreal-sync from kafka subscribe \
--brokers "broker1:9092,broker2:9092" \
--topic "events" \
--group-id "surreal-sync-group" \
--table "events" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
**Deduplication:**
Surreal-sync tracks Kafka message offsets and can use a configurable deduplication key to prevent duplicate records:
```bash
surreal-sync from kafka subscribe \
--brokers "broker1:9092" \
--topic "events" \
--group-id "surreal-sync-group" \
--table "events" \
--dedup-key "event_id" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace production \
--to-database app
```
---
## Schema Mapping Strategies
### Automatic Table Creation
By default, Surreal-sync creates tables automatically based on the source schema. Tables are created in the target namespace and database as needed during sync.
### Field Type Mapping
| Source Type | SurrealDB Type |
|-------------|---------------|
| String/VARCHAR/TEXT | `string` |
| Integer/INT/BIGINT | `int` |
| Float/DOUBLE/DECIMAL | `float` or `decimal` |
| Boolean | `bool` |
| Date/DateTime/Timestamp | `datetime` |
| JSON/JSONB | `object` |
| Array | `array` |
| Binary/BLOB | `bytes` |
| NULL | `NONE` |
| UUID | `string` |
### Relationship Preservation
Surreal-sync attempts to preserve relationships from the source:
- Foreign keys in SQL databases become record links in SurrealDB
- Neo4j relationships become graph edges
- MongoDB document references (DBRef) are converted to record links where detectable
### Index Recreation
Primary keys and unique indexes from the source database are recreated as SurrealDB indexes. Non-unique indexes are created based on a best-effort mapping.
---
## Best Practices
### Test Migrations with a Subset
Always test with a limited dataset before running a full migration:
```bash
surreal-sync from postgres-triggers full \
--connection-string "postgresql://user:pass@host:5432/mydb" \
--tables "users" \
--batch-size 100 \
--limit 1000 \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace staging \
--to-database migration_test
```
### Monitor CDC Lag
For incremental sync, monitor the lag between the source database and SurrealDB:
- Check the sync checkpoint position against the source's current position
- Set up alerts if lag exceeds acceptable thresholds
- For Kafka, monitor consumer group lag using standard Kafka tools
### Handle Schema Drift
When the source schema changes during continuous sync:
- New columns/fields are automatically added to SurrealDB records
- Removed columns result in the field being absent from new records (old records retain it)
- Type changes may require manual intervention; monitor sync logs for type conversion errors
### Plan for Cutover
For production migrations with minimal downtime:
1. Run full sync to establish baseline data in SurrealDB.
2. Start incremental sync to capture changes during migration.
3. Monitor lag until incremental sync is caught up.
4. Pause writes to the source database.
5. Wait for final incremental sync to complete.
6. Switch application connections to SurrealDB.
7. Resume normal operations.
### Rollback Strategies
- Keep the source database running during migration (do not decommission immediately)
- Export the SurrealDB data after migration as a checkpoint
- If issues are found, switch application connections back to the source
- For incremental sync, the source database remains the authoritative copy until cutover
```
### rules/surrealfs.md
```markdown
# SurrealFS -- AI Agent Virtual Filesystem
SurrealFS is a virtual filesystem backed by SurrealDB, designed primarily for AI agent workflows. It provides a familiar file and directory interface with persistent storage, enabling AI agents to manage workspace files, notes, and project artifacts through standard filesystem operations.
---
## Overview
SurrealFS consists of two components:
1. **Rust Core** (`surrealfs` crate): The filesystem engine providing async API and CLI REPL
2. **Python Agent** (`surrealfs-ai`): A Pydantic AI agent that exposes SurrealFS as a conversational interface
All filesystem data is stored in SurrealDB, making it queryable, persistent, and shareable across processes and machines.
---
## Rust Core (`surrealfs`)
### Installation
```bash
cargo install surrealfs
```
### Storage Backends
SurrealFS supports two storage backends:
**Embedded RocksDB** (local, single-process):
```bash
surrealfs --storage rocksdb://./my_fs_data
```
**Remote SurrealDB** (shared, multi-process):
```bash
surrealfs --storage ws://localhost:8000 \
--namespace my_ns \
--database my_db \
--username root \
--password root
```
### CLI REPL
SurrealFS provides an interactive REPL with standard filesystem commands:
```
$ surrealfs --storage rocksdb://./data
surrealfs> pwd
/
surrealfs> mkdir /projects
Created directory: /projects
surrealfs> mkdir /projects/my-app
Created directory: /projects/my-app
surrealfs> cd /projects/my-app
/projects/my-app
surrealfs> write_file README.md "# My Application\n\nA sample project."
Written: /projects/my-app/README.md (36 bytes)
surrealfs> cat README.md
# My Application
A sample project.
surrealfs> ls
README.md
surrealfs> touch notes.txt
Created: /projects/my-app/notes.txt
surrealfs> edit notes.txt "Meeting notes from today's standup."
Written: /projects/my-app/notes.txt (35 bytes)
surrealfs> ls /
projects/
```
### Available Commands
| Command | Description | Example |
|---------|-------------|---------|
| `ls [path]` | List directory contents | `ls /projects` |
| `cat <path>` | Display file contents | `cat /readme.md` |
| `tail <path> [n]` | Display last N lines of a file | `tail /logs/app.log 20` |
| `nl <path>` | Display file with line numbers | `nl /src/main.rs` |
| `grep <pattern> <path>` | Search for pattern in file | `grep "TODO" /src/main.rs` |
| `touch <path>` | Create an empty file | `touch /notes.txt` |
| `mkdir <path>` | Create a directory | `mkdir /projects` |
| `write_file <path> <content>` | Write content to a file | `write_file /hello.txt "Hello"` |
| `edit <path> <content>` | Replace file contents | `edit /hello.txt "Updated"` |
| `cp <src> <dst>` | Copy a file or directory | `cp /a.txt /b.txt` |
| `cd <path>` | Change working directory | `cd /projects` |
| `pwd` | Print working directory | `pwd` |
### Path Normalization and Safety
SurrealFS enforces path safety:
- All paths are normalized to prevent directory traversal
- Paths cannot escape the root `/` directory
- Relative paths like `../../../etc/passwd` are resolved safely within the virtual filesystem
- Symbolic links are not supported (no escape vectors)
### Async Rust API
For programmatic use in Rust applications:
```rust
use surrealfs::SurrealFs;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
// Connect to a remote SurrealDB instance
let fs = SurrealFs::connect("ws://localhost:8000", "my_ns", "my_db").await?;
// Or use embedded storage
let fs = SurrealFs::open_rocksdb("./my_fs_data").await?;
// Create directories
fs.mkdir("/projects/my-app").await?;
// Write files
fs.write_file("/projects/my-app/README.md", "# My App").await?;
// Read files
let content = fs.read_file("/projects/my-app/README.md").await?;
println!("{}", content);
// List directories
let entries = fs.ls("/projects").await?;
for entry in entries {
println!("{}", entry.name);
}
// Copy files
fs.cp("/projects/my-app/README.md", "/backups/readme_backup.md").await?;
// Search within files
let matches = fs.grep("TODO", "/projects/my-app/src/main.rs").await?;
Ok(())
}
```
### Curl Piping Support
SurrealFS supports piping content from external sources:
```bash
# In the REPL, pipe curl output to a file
surrealfs> write_file /data/response.json $(curl -s https://api.example.com/data)
```
---
## Python Agent (`surrealfs-ai`)
### Installation
```bash
pip install surrealfs-ai
```
### Quick Start
```python
from surrealfs_ai import build_chat_agent
import uvicorn
# Build the agent with default configuration
agent = build_chat_agent()
# Convert to a web application
app = agent.to_web()
# Serve on port 7932
uvicorn.run(app, host="127.0.0.1", port=7932)
```
### Architecture
The Python agent wraps SurrealFS operations in a Pydantic AI agent:
- **Agent Framework**: Built on Pydantic AI agents
- **Default LLM**: Claude Haiku (fast, cost-effective for filesystem operations)
- **Observability**: Pydantic Logfire integration for tracing and monitoring
- **Interface**: HTTP API served via Uvicorn (default port 7932)
### Configuration
```python
from surrealfs_ai import build_chat_agent, AgentConfig
config = AgentConfig(
# SurrealDB connection
surreal_endpoint="ws://localhost:8000",
surreal_namespace="my_ns",
surreal_database="my_db",
surreal_username="root",
surreal_password="root",
# LLM configuration
model="claude-haiku", # Default LLM
# Server configuration
host="127.0.0.1",
port=7932,
)
agent = build_chat_agent(config)
```
### Conversational Interface
Once running, interact with the agent via HTTP:
```bash
# Create a directory
curl -X POST http://localhost:7932/chat \
-H "Content-Type: application/json" \
-d '{"message": "Create a directory called /notes/meetings"}'
# Write a file
curl -X POST http://localhost:7932/chat \
-H "Content-Type: application/json" \
-d '{"message": "Write a file at /notes/meetings/standup.md with the agenda for today"}'
# Read a file
curl -X POST http://localhost:7932/chat \
-H "Content-Type: application/json" \
-d '{"message": "Show me the contents of /notes/meetings/standup.md"}'
# Search files
curl -X POST http://localhost:7932/chat \
-H "Content-Type: application/json" \
-d '{"message": "Find all files that mention deployment in /notes"}'
```
### Logfire Telemetry
The Python agent integrates with Pydantic Logfire for observability:
```python
import logfire
logfire.configure()
agent = build_chat_agent()
# All filesystem operations are automatically traced
```
---
## Use Cases
### AI Agent Workspace Management
SurrealFS provides a persistent workspace for AI agents that need to create, modify, and organize files as part of their workflows.
```python
# An AI coding agent can use SurrealFS to manage its workspace
agent_workspace = SurrealFs.connect("ws://localhost:8000", "agents", "workspace")
# Store generated code
await agent_workspace.write_file("/output/generated_module.py", generated_code)
# Keep scratch notes
await agent_workspace.write_file("/scratch/analysis.md", analysis_notes)
# Maintain state between sessions
await agent_workspace.write_file("/state/last_run.json", state_json)
```
### Persistent Note-Taking Systems
Build note-taking systems where all data is stored in SurrealDB and queryable:
```python
# Notes are stored as files but backed by SurrealDB
# This means they can be queried using SurrealQL
# Write notes
await fs.write_file("/notes/2026-02-19.md", daily_notes)
# The underlying SurrealDB data can be queried directly
# for advanced search, aggregation, and analysis
```
### Multi-Agent Collaboration
Multiple AI agents can share a filesystem backed by the same remote SurrealDB instance:
```python
# Agent 1: Research agent writes findings
research_fs = SurrealFs.connect("ws://surrealdb:8000", "project", "shared")
await research_fs.write_file("/research/findings.md", research_results)
# Agent 2: Writing agent reads findings and produces output
writer_fs = SurrealFs.connect("ws://surrealdb:8000", "project", "shared")
findings = await writer_fs.read_file("/research/findings.md")
await writer_fs.write_file("/output/report.md", generate_report(findings))
# Agent 3: Review agent reads output
reviewer_fs = SurrealFs.connect("ws://surrealdb:8000", "project", "shared")
report = await reviewer_fs.read_file("/output/report.md")
```
### Content Organization
Organize and manage content with directory hierarchies:
```
/
├── projects/
│ ├── project-alpha/
│ │ ├── README.md
│ │ ├── spec.md
│ │ └── notes/
│ └── project-beta/
│ ├── README.md
│ └── design.md
├── templates/
│ ├── project-template.md
│ └── meeting-template.md
└── archive/
└── 2025/
```
---
## Integration Patterns
### As an MCP Tool
SurrealFS can be exposed as an MCP (Model Context Protocol) tool for AI coding agents:
```json
{
"name": "surrealfs",
"description": "Virtual filesystem for persistent file management",
"tools": [
{
"name": "read_file",
"description": "Read the contents of a file",
"parameters": {
"path": { "type": "string", "description": "File path" }
}
},
{
"name": "write_file",
"description": "Write content to a file",
"parameters": {
"path": { "type": "string", "description": "File path" },
"content": { "type": "string", "description": "File content" }
}
},
{
"name": "list_directory",
"description": "List files and directories at a path",
"parameters": {
"path": { "type": "string", "description": "Directory path" }
}
}
]
}
```
### Embedded in Agentic Workflows
```python
from surrealfs_ai import build_chat_agent
# Use SurrealFS as a tool within a larger agent pipeline
fs_agent = build_chat_agent()
async def research_pipeline(topic: str):
# Step 1: Research agent gathers information
research = await research_agent.run(topic)
# Step 2: Store research in SurrealFS
await fs_agent.run(f"Write the following research to /research/{topic}.md: {research}")
# Step 3: Analysis agent reads and analyzes
content = await fs_agent.run(f"Read /research/{topic}.md")
analysis = await analysis_agent.run(content)
# Step 4: Store analysis
await fs_agent.run(f"Write analysis to /analysis/{topic}.md: {analysis}")
```
### REST API Access
The Python agent's HTTP interface provides a REST-like API:
```
POST /chat
Content-Type: application/json
{
"message": "Your natural language filesystem command"
}
```
The agent interprets the natural language request and executes the appropriate filesystem operations.
### CLI Integration
Use SurrealFS from shell scripts and automation:
```bash
#!/bin/bash
# Backup script that stores results in SurrealFS
BACKUP_DATE=$(date +%Y-%m-%d)
# Run backup
pg_dump mydb > /tmp/backup.sql
# Store in SurrealFS via the REPL
echo "write_file /backups/${BACKUP_DATE}/database.sql $(cat /tmp/backup.sql)" | surrealfs --storage ws://localhost:8000
# Clean up
rm /tmp/backup.sql
```
---
## Storage Model
All filesystem data is stored in SurrealDB tables:
- **Files**: Stored as records with path, content, metadata (size, created, modified timestamps)
- **Directories**: Stored as records with path and metadata
- **Queryable**: Since data lives in SurrealDB, you can query it directly using SurrealQL for operations beyond standard filesystem commands (e.g., find all files modified in the last 24 hours, search across all file contents)
```surrealql
-- Find recently modified files
SELECT path, modified_at FROM fs_file
WHERE modified_at > time::now() - 24h
ORDER BY modified_at DESC;
-- Search across all file contents
SELECT path, content FROM fs_file
WHERE content CONTAINS "deployment";
-- Get total storage usage
SELECT math::sum(size) AS total_bytes FROM fs_file;
```
```
---
## Skill Companion Files
> Additional files collected from the skill directory layout.
### README.md
```markdown
# surreal-skills
[](https://github.com/24601/surreal-skills/actions/workflows/ci.yml)
[](LICENSE)
[](https://github.com/24601/surreal-skills/releases)
[](https://skills.sh)
Expert SurrealDB 3 skill for AI coding agents. Complete coverage of SurrealQL, multi-model data modeling, graph traversal, vector search, security, deployment, performance tuning, SDK integration, WASM extensions, and the full SurrealDB ecosystem.
## Features
- **SurrealQL mastery** -- Complete language reference with statements, functions, operators, and idioms
- **Multi-model data modeling** -- Document, graph, vector, relational, time-series, and geospatial patterns in a single schema
- **Graph queries** -- First-class edge creation and traversal without JOINs
- **Vector search** -- HNSW indexes, similarity functions, and RAG pipeline patterns
- **Security** -- Row-level permissions, JWT auth, namespace/database/record-level access control
- **Deployment** -- Storage engine selection, Docker, Kubernetes, production hardening
- **Performance** -- Index strategies, EXPLAIN analysis, batch operations, connection pooling
- **9+ SDK integrations** -- JavaScript/TypeScript, Python, Go, Rust, Java, .NET, C, PHP, Dart
- **Surrealism WASM extensions** -- Custom functions and analyzers compiled from Rust (new in v3)
- **Full ecosystem** -- Surrealist IDE, Surreal-Sync CDC, SurrealFS agent filesystem, SurrealML
- **Health checks and introspection** -- Doctor script and schema introspection for any SurrealDB instance
- **Universal agent support** -- Works with 30+ AI coding agents via skills.sh
## Installation
### Claude Code (recommended)
**Option 1 -- Install as a Claude Code skill (global)**
```bash
npx skills add 24601/surreal-skills -a claude-code -g -y
```
This installs the skill globally so it is available in every Claude Code session. The `-g` flag installs globally, `-y` auto-confirms prompts.
**Option 2 -- Install per-project via CLAUDE.md**
Clone the repo and reference it from your project's `CLAUDE.md`:
```bash
git clone https://github.com/24601/surreal-skills.git ~/.claude/skills/surrealdb
```
Then add to your project's `CLAUDE.md` (or `~/.claude/CLAUDE.md` for global):
```markdown
# SurrealDB Skill
@import ~/.claude/skills/surrealdb/AGENTS.md
```
Or inline the reference:
```markdown
# SurrealDB Skill
For SurrealDB work, read the rules at ~/.claude/skills/surrealdb/rules/ and
use the scripts at ~/.claude/skills/surrealdb/scripts/ for health checks
and schema introspection.
```
**Option 3 -- Add as a Claude Code custom slash command**
Create `~/.claude/commands/surrealdb.md`:
```markdown
Load the SurrealDB 3 skill from ~/.claude/skills/surrealdb/AGENTS.md
and use its rules for all SurrealDB architecture, development, and operations tasks.
Available rules: surrealql, data-modeling, graph-queries, vector-search, security,
deployment, performance, sdks, surrealism, surrealist, surreal-sync, surrealfs.
```
Then invoke with `/surrealdb` in any Claude Code session.
**Option 4 -- Project-scoped slash commands**
Add SurrealDB-specific commands to your project:
```bash
mkdir -p .claude/commands
```
Create `.claude/commands/surreal-doctor.md`:
```markdown
Run the SurrealDB health check: uv run ~/.claude/skills/surrealdb/scripts/doctor.py
Report any issues found and suggest fixes based on the deployment rules.
```
Create `.claude/commands/surreal-schema.md`:
```markdown
Introspect the current SurrealDB schema: uv run ~/.claude/skills/surrealdb/scripts/schema.py introspect
Analyze the output using the data-modeling rules and suggest improvements.
```
### Other AI Agents
```bash
# skills.sh (universal -- works with all supported agents)
npx skills add 24601/surreal-skills
# Amp
npx skills add 24601/surreal-skills -a amp -g -y
# Codex
npx skills add 24601/surreal-skills -a codex -g -y
# Gemini CLI
npx skills add 24601/surreal-skills -a gemini-cli -g -y
# OpenCode
npx skills add 24601/surreal-skills -a opencode -g -y
# Pi (badlogic/pi-mono)
npx skills add 24601/surreal-skills -a pi -g -y
# OpenClaw / Clawdbot
npx skills add 24601/surreal-skills -a openclaw -g -y
```
### GitHub Copilot (native agent skills)
Copilot supports the [Agent Skills standard](https://agentskills.io/) natively in VS Code,
Copilot CLI, and the Copilot coding agent. This skill ships a Copilot-native
`.github/skills/surrealdb/SKILL.md` that Copilot auto-loads when your prompt
is SurrealDB-related.
**Option 1 -- Project-level (recommended for teams)**
Copy the entire skill into your project's `.github/skills/` directory:
```bash
# From the surreal-skills repo
cp -r .github/skills/surrealdb <your-project>/.github/skills/surrealdb
cp -r rules/ <your-project>/.github/skills/surrealdb/rules/
```
Copilot discovers this automatically -- no config needed. Type `/surrealdb` in
chat or let Copilot auto-load it when it detects SurrealQL context.
**Option 2 -- Personal (all projects)**
Clone into `~/.copilot/skills/`:
```bash
git clone https://github.com/24601/surreal-skills.git ~/.copilot/skills/surrealdb
```
Or add a custom search location in VS Code settings:
```json
{
"chat.agentSkillsLocations": [
"~/.copilot/skills"
]
}
```
**Option 3 -- Use `/skills` menu**
Type `/skills` in Copilot chat to open the Configure Skills menu, then browse
to the cloned `surrealdb` directory.
### Other IDE Integrations
```bash
# Cursor -- add skill to .cursor/skills/ (same Agent Skills standard)
cp -r .github/skills/surrealdb <your-project>/.cursor/skills/surrealdb
# Windsurf -- append AGENTS.md to .windsurfrules
cat AGENTS.md >> .windsurfrules
# Cline / Continue -- reference in your config
# Add the AGENTS.md path to your system prompt configuration
```
### Manual installation
```bash
# Clone to any location
git clone https://github.com/24601/surreal-skills.git ~/.claude/skills/surrealdb
# Verify installation
uv run ~/.claude/skills/surrealdb/scripts/doctor.py --check
```
## Quick Start
> **Credential warning**: Examples below use `root/root` for **local development
> only**. Never use default credentials against production or shared instances.
```bash
# Start SurrealDB in-memory for LOCAL DEVELOPMENT ONLY
surreal start memory --user root --pass root --bind 127.0.0.1:8000
# Connect via CLI REPL (local dev)
surreal sql --endpoint http://localhost:8000 --user root --pass root --ns test --db test
# Create records with SurrealQL
CREATE person:alice SET name = 'Alice', email = '[email protected]';
CREATE person:bob SET name = 'Bob', email = '[email protected]';
# Create graph edges
RELATE person:alice->follows->person:bob SET since = time::now();
# Traverse the graph
SELECT ->follows->person.name AS following FROM person:alice;
# Run the health check
uv run scripts/doctor.py
```
## Architecture
```
surreal-skills/
SKILL.md # Skill manifest (frontmatter + body)
AGENTS.md # Structured agent briefing
README.md # This file
LICENSE # MIT license
scripts/
onboard.py # Setup wizard / capabilities manifest
doctor.py # Health check (CLI, server, auth, storage)
schema.py # Schema introspection and export
rules/
surrealql.md # SurrealQL language reference
data-modeling.md # Multi-model schema design patterns
graph-queries.md # Graph traversal and RELATE patterns
vector-search.md # Vector indexes, similarity search, RAG
security.md # Permissions, auth, access control
deployment.md # Storage engines, Docker, K8s, production
performance.md # Indexes, EXPLAIN, optimization
sdks.md # Official SDK integration (9+ languages)
surrealism.md # WASM extension system (new in v3)
surrealist.md # Surrealist IDE/GUI
surreal-sync.md # CDC migration tool
surrealfs.md # AI agent filesystem
```
## Rules
| Rule | Description |
|------|-------------|
| `surrealql.md` | Complete SurrealQL language reference: CREATE, SELECT, UPDATE, DELETE, RELATE, INSERT, UPSERT, LIVE SELECT, DEFINE, REMOVE, INFO, subqueries, transactions, futures, all built-in functions, v2-to-v3 migration notes |
| `data-modeling.md` | Schema design patterns: record IDs (typed, generated, composite), field types, schemafull vs schemaless, normalization strategies, multi-model design (document + graph + vector in one schema), time-series and geospatial data |
| `graph-queries.md` | Graph edge creation with RELATE, traversal operators (-> outgoing, <- incoming, <-> bidirectional), path expressions, recursive queries, filtering and aggregation on edges, graph-specific DEFINE TABLE TYPE RELATION |
| `vector-search.md` | Vector field definitions, HNSW and brute-force index creation, distance metrics (cosine, euclidean, manhattan, minkowski), vector::similarity functions, RAG pipeline patterns, hybrid search combining vector + metadata filtering |
| `security.md` | Row-level permissions with WHERE predicates, DEFINE ACCESS for JWT and record-based auth, DEFINE USER for system users, namespace/database/table permission scoping, $auth and $session runtime variables, authentication flow patterns |
| `deployment.md` | Installation methods (curl, brew, Docker, binary), storage engine selection (memory, RocksDB, SurrealKV with time-travel, TiKV for distributed), Docker Compose and Kubernetes Helm charts, production hardening, backup/restore, log levels, monitoring |
| `performance.md` | Index strategies (unique, full-text search analyzers, HNSW vector, MTree), EXPLAIN statement for query analysis, batch operations, connection pooling, storage engine trade-offs by workload, parallel queries, resource limits, compute-to-storage ratios |
| `sdks.md` | Official SDK usage for JavaScript/TypeScript (Node, Deno, Bun, browser), Python, Go, Rust, Java, .NET, C, PHP, Dart: connection setup (HTTP vs WebSocket), authentication flows, CRUD operations, live query subscriptions, typed record handling, error patterns |
| `surrealism.md` | Surrealism WASM extension system introduced in SurrealDB 3: Rust SDK for authoring, custom function registration, custom analyzer creation, module compilation to wasm32-unknown-unknown, deployment to running instances, versioning, testing |
| `surrealist.md` | Surrealist IDE and GUI: schema designer with visual table editing, query editor with syntax highlighting and auto-complete, graph visualizer for relationships, table explorer, connection profiles, import/export, embedding in applications |
| `surreal-sync.md` | Surreal-Sync CDC migration tool: source connectors (PostgreSQL, MySQL, MongoDB, etc.), SurrealDB as target, incremental change data capture, schema translation rules, migration workflow orchestration, conflict resolution, monitoring |
| `surrealfs.md` | SurrealFS AI agent filesystem: file storage backed by SurrealDB, metadata management with SurrealQL queries, directory structures, file versioning, agent-friendly API patterns, integration with AI agent frameworks |
## Scripts
| Script | Usage | Description |
|--------|-------|-------------|
| `onboard.py` | `uv run scripts/onboard.py --check` | Verify prerequisites (surreal CLI, Python, uv, server connectivity) |
| `onboard.py` | `uv run scripts/onboard.py --agent` | Output JSON capabilities manifest for agent integration |
| `doctor.py` | `uv run scripts/doctor.py` | Full health check: CLI version, server reachability, auth, namespace, database, storage engine |
| `doctor.py` | `uv run scripts/doctor.py --check` | Quick pass/fail (exit code 0 = healthy, 1 = issues) |
| `doctor.py` | `uv run scripts/doctor.py --endpoint URL` | Check a specific SurrealDB endpoint |
| `schema.py` | `uv run scripts/schema.py introspect` | Full schema dump of all tables, fields, indexes, events, accesses |
| `schema.py` | `uv run scripts/schema.py tables` | List all tables with field/index counts |
| `schema.py` | `uv run scripts/schema.py table <name>` | Inspect a single table in detail |
| `schema.py` | `uv run scripts/schema.py export --format surql` | Export schema as reproducible DEFINE statements |
| `schema.py` | `uv run scripts/schema.py export --format json` | Export schema as structured JSON |
| `check_upstream.py` | `uv run scripts/check_upstream.py` | Compare upstream repos against skill snapshot; shows what changed |
| `check_upstream.py` | `uv run scripts/check_upstream.py --stale` | Only show repos with new commits since snapshot |
| `check_upstream.py` | `uv run scripts/check_upstream.py --json` | JSON-only output (no Rich table) |
All scripts follow the dual-output convention: stderr for Rich-formatted human output, stdout for machine-readable JSON.
## Sub-Skills
### Surrealism (WASM Extensions)
New in SurrealDB 3. Extend the database with custom functions and analyzers written in Rust, compiled to WebAssembly, and deployed to running instances. The `rules/surrealism.md` rule covers the full Surrealism SDK, module authoring, compilation, deployment, and testing workflow.
### Surreal-Sync (CDC Migration)
Change Data Capture tool for migrating data from external databases (PostgreSQL, MySQL, MongoDB, and others) into SurrealDB. Supports incremental sync, schema translation, and conflict resolution. See `rules/surreal-sync.md`.
### SurrealFS (AI Agent Filesystem)
A filesystem abstraction built on SurrealDB, designed for AI agent workflows. Store files with rich metadata queryable via SurrealQL, version files automatically, and integrate with agent frameworks. See `rules/surrealfs.md`.
## Use Cases
### API Backend
Use SurrealDB as the primary datastore for REST or GraphQL APIs. Define tables with schemafull validation, set up row-level permissions for multi-tenant security, connect via the JavaScript or Python SDK over WebSocket for real-time live queries.
### Real-Time Application
Leverage LIVE SELECT for push-based data subscriptions. Clients receive changes as they happen without polling. Combine with WebSocket SDK connections for chat applications, collaborative editors, dashboards, and notification systems.
### Graph Analytics
Model complex relationships (social networks, organizational hierarchies, dependency trees, knowledge graphs) using RELATE and typed edge tables. Traverse paths of arbitrary depth with `->` operators. Filter and aggregate at each hop without writing JOINs.
### Vector Search and RAG
Store document embeddings alongside content. Create HNSW vector indexes with configurable distance metrics. Query with `vector::similarity::cosine` for semantic search. Build retrieval-augmented generation pipelines that combine vector similarity with metadata filtering in a single SurrealQL query.
### IoT and Time-Series
Ingest high-volume sensor data with schemaless tables. Use datetime fields and range queries for time-series analysis. Aggregate with built-in math and time functions. SurrealKV storage engine enables time-travel queries to access historical state at any point in time.
### Geospatial Applications
Store geometry types (points, polygons, multipoints) as native SurrealDB values. Use built-in geo functions (`geo::distance`, `geo::bearing`, `geo::area`, `geo::contains`) for spatial queries. Combine with other data models -- a single query can traverse a graph, filter by location, and rank by vector similarity.
### Data Migration
Migrate from PostgreSQL, MySQL, MongoDB, or other databases using Surreal-Sync CDC. Translate schemas automatically, sync incrementally, and validate with schema introspection. For SurrealDB v2-to-v3 upgrades, use surreal export/import with the migration notes in `rules/surrealql.md`.
### WASM Extensions
Extend SurrealDB with custom business logic using Surrealism. Write functions and analyzers in Rust, compile to WASM, and deploy without restarting the server. Use cases include custom validation, domain-specific scoring, proprietary tokenizers, and specialized aggregation functions.
### AI Agent Filesystem
Use SurrealFS as a persistent, queryable filesystem for AI agent workflows. Agents can store and retrieve files with rich metadata, query across files with SurrealQL, and leverage SurrealDB's permissions system for multi-agent access control.
### Full-Text Search
Define custom analyzers (tokenizers, filters, stemmers) and create search indexes on text fields. Query with full-text search predicates that integrate with the rest of SurrealQL -- combine text search with graph traversal, vector similarity, and relational filters in a single statement.
## Configuration
Set these environment variables to configure the skill scripts. All are optional with sensible defaults.
| Variable | Description | Default |
|----------|-------------|---------|
| `SURREAL_ENDPOINT` | SurrealDB server URL | `http://localhost:8000` |
| `SURREAL_USER` | Root or namespace username | `root` |
| `SURREAL_PASS` | Root or namespace password | `root` |
| `SURREAL_NS` | Default namespace | `test` |
| `SURREAL_DB` | Default database | `test` |
These variables are also recognized by the surreal CLI and official SurrealDB SDKs.
## Source Provenance
This skill was built on **2026-02-22** from these upstream sources. Use `check_upstream.py`
to detect what changed since this snapshot for incremental updates.
| Repository | Release | SHA | Snapshot Date | Rules Affected |
|------------|---------|-----|---------------|----------------|
| [surrealdb/surrealdb](https://github.com/surrealdb/surrealdb) | v3.0.0 | `2e0a61fd4daf` | 2026-02-19 | surrealql, data-modeling, security, performance, deployment, surrealism |
| [surrealdb/surrealist](https://github.com/surrealdb/surrealist) | v3.7.2 | `a87e89e23796` | 2026-02-21 | surrealist |
| [surrealdb/surrealdb.js](https://github.com/surrealdb/surrealdb.js) | v1.3.2 | `48894dfe70bd` | 2026-02-20 | sdks |
| [surrealdb/surrealdb.js](https://github.com/surrealdb/surrealdb.js) (v2 beta) | v2.0.0-beta.1 | `48894dfe70bd` | 2026-02-20 | sdks |
| [surrealdb/surrealdb.py](https://github.com/surrealdb/surrealdb.py) | v1.0.8 | `1ff4470e6ec0` | 2026-02-03 | sdks |
| [surrealdb/surrealdb.go](https://github.com/surrealdb/surrealdb.go) | v1.3.0 | `89d0f8d1b4c6` | 2026-02-12 | sdks |
| [surrealdb/surreal-sync](https://github.com/surrealdb/surreal-sync) | v0.3.4 | `8166b2b041b1` | 2026-02-12 | surreal-sync |
| [surrealdb/surrealfs](https://github.com/surrealdb/surrealfs) | -- | `0008a3a94dbe` | 2026-01-29 | surrealfs |
Documentation: [surrealdb.com/docs](https://surrealdb.com/docs) snapshot 2026-02-22.
Machine-readable provenance: [`SOURCES.json`](SOURCES.json).
## Registries
This skill is published to multiple agent skill registries:
| Registry | Install Command |
|----------|----------------|
| [skills.sh](https://skills.sh) | `npx skills add 24601/surreal-skills` |
| [ClawHub](https://clawhub.ai) | `npx clawhub install surrealdb` |
| [OpenClaw / Clawdbot](https://github.com/openclaw) | `clawhub install surrealdb` |
| GitHub | `git clone https://github.com/24601/surreal-skills.git` |
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, code style, and PR process.
## Security
To report a vulnerability, use [GitHub Security Advisories](https://github.com/24601/surreal-skills/security/advisories/new). See [SECURITY.md](SECURITY.md) for details.
This skill declares the following security properties in `SKILL.md` frontmatter:
| Property | Value | Meaning |
|----------|-------|---------|
| `no_network` | **false** | `doctor.py`/`schema.py` connect to user-specified SurrealDB endpoint (WebSocket). `check_upstream.py` calls GitHub API via `gh` CLI. No other third-party calls. |
| `no_credentials` | **false** | Scripts accept `SURREAL_USER`/`SURREAL_PASS` for DB auth. No credentials are stored in the skill itself. |
| `no_env_write` | true | Scripts do not modify environment variables |
| `no_file_write` | true | Rules are read-only; scripts write only to stdout/stderr |
| `no_shell_exec` | false | Scripts invoke `surreal` CLI and `gh` CLI |
| `scripts_auditable` | true | All scripts are readable Python with no obfuscation |
| `scripts_use_pep723` | true | Dependencies declared inline via PEP 723, no requirements.txt |
| `no_obfuscated_code` | true | No obfuscated, encoded, or encrypted code |
| `no_binary_blobs` | true | No compiled binaries or WASM files |
| `no_minified_scripts` | true | No minified JavaScript or compressed code |
| `no_curl_pipe_sh` | **false** | Documentation mentions `curl\|sh` as one install option; safer alternatives (brew, Docker) are listed first. The skill itself never executes `curl\|sh`. |
### Required Environment Variables
Declared in `SKILL.md` `requires.env_vars`:
| Variable | Sensitive | Default | Purpose |
|----------|-----------|---------|---------|
| `SURREAL_ENDPOINT` | No | `http://localhost:8000` | SurrealDB server URL |
| `SURREAL_USER` | **Yes** | `root` | Authentication username |
| `SURREAL_PASS` | **Yes** | `root` | Authentication password |
| `SURREAL_NS` | No | `test` | Default namespace |
| `SURREAL_DB` | No | `test` | Default database |
### Required Binaries
Declared in `SKILL.md` `requires.binaries`:
| Binary | Required | Install |
|--------|----------|---------|
| `surreal` | Yes | `brew install surrealdb/tap/surreal` |
| `python3` (>=3.10) | Yes | System package manager |
| `uv` | Yes | `brew install uv` or `pip install uv` |
| `docker` | No | Optional for containerized instances |
| `gh` | No | Optional -- only used by `check_upstream.py` to compare upstream repo SHAs via GitHub API |
### Script Safety
- All user-provided table names are validated against `[a-zA-Z_][a-zA-Z0-9_]*` before interpolation into SurrealQL queries (prevents SurrealQL injection)
- `doctor.py` and `schema.py` connect only to the SurrealDB endpoint specified by the user (via env var or CLI flag)
- `check_upstream.py` calls GitHub API via `gh` CLI to compare upstream repo SHAs (optional maintenance script, not needed for normal usage)
- No data is sent to third-party services
- Credential warning labels are present on all `root/root` examples
## License
[MIT](LICENSE)
## Credits
Built for the [SurrealDB](https://surrealdb.com) community. SurrealDB is created and maintained by [SurrealDB Ltd](https://github.com/surrealdb/surrealdb).
Published on [skills.sh](https://skills.sh), [ClawHub](https://clawhub.ai), and [GitHub](https://github.com/24601/surreal-skills).
```
### _meta.json
```json
{
"owner": "24601",
"slug": "surrealdb",
"displayName": "SurrealDB 3",
"latest": {
"version": "1.2.1",
"publishedAt": 1773432169951,
"commit": "https://github.com/openclaw/skills/commit/a435feb1537caf3750160cfd98a2af0326929bb6"
},
"history": [
{
"version": "1.2.0",
"publishedAt": 1772592340840,
"commit": "https://github.com/openclaw/skills/commit/d798e40897e4d277475721658088a82e77a67a9e"
},
{
"version": "1.1.1",
"publishedAt": 1772134475646,
"commit": "https://github.com/openclaw/skills/commit/2c45c867fceb317d3b34dc3bbe2d0b151cc6b7ff"
},
{
"version": "1.1.0",
"publishedAt": 1772082040233,
"commit": "https://github.com/openclaw/skills/commit/5f2c906a91e2067b9302fd8b9211eed0adba6a2a"
},
{
"version": "1.0.6",
"publishedAt": 1771969871750,
"commit": "https://github.com/openclaw/skills/commit/0301665956dc846803b9b9227fc610ab5161423f"
},
{
"version": "1.0.3",
"publishedAt": 1771783747470,
"commit": "https://github.com/openclaw/skills/commit/62a186e49823ab0fd251087b75f4d706d1e05a2f"
}
]
}
```
### references/online_docs.md
```markdown
# SurrealDB Online Documentation and Resources
A comprehensive directory of official SurrealDB documentation, repositories, and community resources.
## Official Documentation
| Resource | URL |
|----------|-----|
| Main Documentation | https://surrealdb.com/docs |
| SurrealQL Reference | https://surrealdb.com/docs/surrealdb/surrealql |
| Data Model | https://surrealdb.com/docs/surrealdb/datamodel |
| Extensions | https://surrealdb.com/docs/surrealdb/extensions |
| Security | https://surrealdb.com/docs/surrealdb/security |
| Deployment | https://surrealdb.com/docs/surrealdb/deployment |
| CLI Reference | https://surrealdb.com/docs/surrealdb/cli |
## SDKs
| SDK | Documentation | Repository |
|-----|--------------|------------|
| JavaScript/TypeScript | https://surrealdb.com/docs/sdk/javascript | https://github.com/surrealdb/surrealdb.js |
| Python | https://surrealdb.com/docs/sdk/python | https://github.com/surrealdb/surrealdb.py |
| Rust | https://surrealdb.com/docs/sdk/rust | https://github.com/surrealdb/surrealdb (core) |
| Go | https://surrealdb.com/docs/sdk/go | https://github.com/surrealdb/surrealdb.go |
| Java | https://surrealdb.com/docs/sdk/java | https://github.com/surrealdb/surrealdb.java |
| .NET (C#) | https://surrealdb.com/docs/sdk/dotnet | https://github.com/surrealdb/surrealdb.net |
## GitHub Repositories
| Repository | Description |
|-----------|-------------|
| https://github.com/surrealdb/surrealdb | Core database engine (Rust) |
| https://github.com/surrealdb/surrealdb.js | JavaScript/TypeScript SDK |
| https://github.com/surrealdb/surrealdb.py | Python SDK |
| https://github.com/surrealdb/surrealdb.go | Go SDK |
| https://github.com/surrealdb/surrealdb.java | Java SDK |
| https://github.com/surrealdb/surrealdb.net | .NET SDK |
| https://github.com/surrealdb/surrealist | Surrealist IDE/Query Explorer |
| https://github.com/surrealdb/docs.surrealdb.com | Documentation source |
## Tools and Applications
| Tool | URL | Description |
|------|-----|-------------|
| Surrealist (Cloud) | https://app.surrealdb.com | Browser-based query explorer and IDE |
| Surrealist (Desktop) | https://surrealdb.com/surrealist | Downloadable desktop application |
| SurrealDB Cloud | https://surrealdb.com/cloud | Managed SurrealDB hosting |
## Community
| Resource | URL |
|----------|-----|
| Discord | https://discord.gg/surrealdb |
| GitHub Discussions | https://github.com/surrealdb/surrealdb/discussions |
| Blog | https://surrealdb.com/blog |
| YouTube | https://youtube.com/@surrealdb |
| Twitter / X | https://x.com/surrealdb |
## Learning Resources
| Resource | URL |
|----------|-----|
| Getting Started Guide | https://surrealdb.com/docs/surrealdb/introduction/start |
| SurrealQL Tutorial | https://surrealdb.com/docs/surrealdb/surrealql/overview |
| University (Video Courses) | https://surrealdb.com/learn |
```
### references/surrealql_cheatsheet.md
```markdown
# SurrealQL Quick Reference
A concise cheatsheet for the most commonly used SurrealQL statements, types, functions, and patterns.
## Namespace and Database
```surql
USE NS my_namespace DB my_database;
```
## CRUD Operations
```surql
-- Create with auto-generated ID
CREATE person SET name = "Alice", age = 30;
-- Create with specific ID
CREATE person:alice SET name = "Alice", age = 30;
-- Insert (upsert semantics)
INSERT INTO person { id: person:alice, name: "Alice", age: 30 };
-- Select all
SELECT * FROM person;
-- Select with conditions
SELECT name, age FROM person WHERE age > 25 ORDER BY name LIMIT 10;
-- Update (merge)
UPDATE person:alice SET age = 31;
-- Update with MERGE (partial update)
UPDATE person:alice MERGE { email: "[email protected]" };
-- Replace (full overwrite)
UPDATE person:alice CONTENT { name: "Alice", age: 31, email: "[email protected]" };
-- Delete
DELETE person:alice;
DELETE FROM person WHERE age < 18;
```
## Record Links and Graph Relations
```surql
-- Create a graph edge
RELATE person:alice -> knows -> person:bob SET since = d"2024-01-15";
-- Traverse outbound
SELECT ->knows->person.name FROM person:alice;
-- Traverse inbound
SELECT <-knows<-person.name FROM person:bob;
-- Multi-hop traversal
SELECT ->knows->person->works_at->company.name FROM person:alice;
-- Bidirectional
SELECT <->knows<->person.name FROM person:alice;
```
## Data Types
| Type | Example |
|------|---------|
| String | `"hello"` |
| Int | `42` |
| Float | `3.14` |
| Bool | `true`, `false` |
| None/Null | `NONE`, `NULL` |
| Datetime | `d"2026-02-19T12:00:00Z"` |
| Duration | `1h30m`, `7d`, `100ms` |
| Record ID | `person:alice`, `post:ulid()` |
| Array | `[1, 2, 3]` |
| Object | `{ key: "value" }` |
| Geometry | `{ type: "Point", coordinates: [-73.97, 40.77] }` |
| Bytes | `<bytes>"base64encoded"` |
| UUID | `u"550e8400-e29b-41d4-a716-446655440000"` |
## Schema Definition
```surql
-- Define table with schema enforcement
DEFINE TABLE person SCHEMAFULL;
-- Define fields
DEFINE FIELD name ON person TYPE string;
DEFINE FIELD age ON person TYPE int DEFAULT 0;
DEFINE FIELD email ON person TYPE option<string> ASSERT string::is::email($value);
DEFINE FIELD tags ON person TYPE array<string> DEFAULT [];
DEFINE FIELD created ON person TYPE datetime DEFAULT time::now() READONLY;
-- Define indexes
DEFINE INDEX idx_email ON person FIELDS email UNIQUE;
DEFINE INDEX idx_name ON person FIELDS name SEARCH ANALYZER ascii BM25;
-- Define vector index
DEFINE INDEX idx_embedding ON document FIELDS embedding MTREE DIMENSION 1536 DIST COSINE;
```
## Common Functions
### String Functions
```surql
string::concat("a", "b") -- "ab"
string::lowercase("ABC") -- "abc"
string::split("a,b,c", ",") -- ["a", "b", "c"]
string::trim(" hello ") -- "hello"
string::len("hello") -- 5
string::contains("hello", "ell") -- true
```
### Array Functions
```surql
array::len([1, 2, 3]) -- 3
array::distinct([1, 1, 2]) -- [1, 2]
array::flatten([[1, 2], [3]]) -- [1, 2, 3]
array::sort([3, 1, 2]) -- [1, 2, 3]
array::push([1, 2], 3) -- [1, 2, 3]
array::filter([1, 2, 3], |$v| $v > 1) -- [2, 3]
```
### Math Functions
```surql
math::sum([1, 2, 3]) -- 6
math::mean([1, 2, 3]) -- 2
math::max([1, 2, 3]) -- 3
math::min([1, 2, 3]) -- 1
math::round(3.7) -- 4
math::abs(-5) -- 5
```
### Time Functions
```surql
time::now() -- Current datetime
time::day(d"2026-02-19") -- 19
time::month(d"2026-02-19") -- 2
time::year(d"2026-02-19") -- 2026
time::format(time::now(), "%Y-%m-%d")
```
### Crypto and Rand
```surql
rand::uuid::v7() -- Generate UUIDv7
rand::int(1, 100) -- Random integer
rand::float(0.0, 1.0) -- Random float
crypto::argon2::generate("password")
crypto::argon2::compare(hash, "password")
```
### Type Functions
```surql
type::is::string("hello") -- true
type::is::int(42) -- true
type::thing("person", "alice") -- person:alice
```
## Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `=` | Equals | `WHERE age = 30` |
| `!=` | Not equals | `WHERE age != 30` |
| `>`, `>=`, `<`, `<=` | Comparison | `WHERE age > 25` |
| `AND`, `OR`, `NOT` | Logical | `WHERE age > 25 AND name != "Bob"` |
| `IN` | Containment | `WHERE status IN ["active", "pending"]` |
| `CONTAINS` | Array contains | `WHERE tags CONTAINS "admin"` |
| `CONTAINSALL` | Contains all | `WHERE tags CONTAINSALL ["a", "b"]` |
| `CONTAINSANY` | Contains any | `WHERE tags CONTAINSANY ["a", "b"]` |
| `~` | Fuzzy match | `WHERE name ~ "alice"` |
| `@@` | Full-text match | `WHERE content @@ "search term"` |
| `??` | Null coalesce | `email ?? "no-email"` |
| `?:` | Ternary | `age >= 18 ?: "minor"` |
## Subqueries and Expressions
```surql
-- Subquery in SELECT
SELECT *, (SELECT count() FROM ->wrote->post GROUP ALL) AS post_count FROM person;
-- LET bindings
LET $adults = SELECT * FROM person WHERE age >= 18;
SELECT * FROM $adults WHERE name STARTS WITH "A";
-- IF expression
SELECT *, IF age >= 18 THEN "adult" ELSE "minor" END AS category FROM person;
-- FOR loop (scripting)
FOR $p IN (SELECT * FROM person) {
UPDATE $p.id SET verified = true;
};
```
## Transactions
```surql
BEGIN TRANSACTION;
CREATE account:a SET balance = 100;
CREATE account:b SET balance = 200;
UPDATE account:a SET balance -= 50;
UPDATE account:b SET balance += 50;
COMMIT TRANSACTION;
```
## Live Queries
```surql
LIVE SELECT * FROM person;
LIVE SELECT * FROM person WHERE age > 25;
KILL $live_query_id;
```
## Access Control
```surql
-- Define access method
DEFINE ACCESS user_auth ON DATABASE TYPE RECORD
SIGNUP (CREATE user SET email = $email, pass = crypto::argon2::generate($pass))
SIGNIN (SELECT * FROM user WHERE email = $email AND crypto::argon2::compare(pass, $pass))
DURATION FOR SESSION 24h;
-- Define scope-level permissions
DEFINE TABLE post SCHEMAFULL
PERMISSIONS
FOR select FULL
FOR create WHERE $auth.id != NONE
FOR update WHERE author = $auth.id
FOR delete WHERE author = $auth.id;
```
## Vector Search
```surql
-- Define vector index
DEFINE INDEX idx_vec ON document FIELDS embedding MTREE DIMENSION 1536 DIST COSINE;
-- Query nearest neighbors
SELECT *, vector::similarity::cosine(embedding, $query_vec) AS score
FROM document
WHERE embedding <|10|> $query_vec
ORDER BY score DESC;
```
## Common One-Liners
```surql
-- Count records
SELECT count() FROM person GROUP ALL;
-- Group and aggregate
SELECT department, count(), math::mean(salary) FROM employee GROUP BY department;
-- Deduplicate
SELECT array::distinct(->tagged->tag.name) FROM post;
-- Paginate
SELECT * FROM post ORDER BY created DESC LIMIT 20 START 40;
-- Check if record exists
IF (SELECT * FROM person:alice) THEN "exists" ELSE "not found" END;
-- Bulk upsert from array
INSERT INTO person $data ON DUPLICATE KEY UPDATE name = $input.name, age = $input.age;
```
```
### scripts/check_upstream.py
```python
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "rich>=13.0.0",
# ]
# ///
"""Check upstream SurrealDB repos for changes since the last skill snapshot.
Compares current HEAD SHAs and release tags against the baselines recorded
in SOURCES.json. Prints a Rich table on stderr and structured JSON on stdout
so both humans and agents can consume the output.
Usage:
uv run scripts/check_upstream.py # full diff report
uv run scripts/check_upstream.py --json # stdout JSON only (no Rich)
uv run scripts/check_upstream.py --stale # only repos that changed
"""
from __future__ import annotations
import argparse
import json
import subprocess
import sys
from pathlib import Path
from typing import Any
from rich.console import Console
from rich.table import Table
stderr = Console(stderr=True)
SOURCES_PATH = Path(__file__).resolve().parent.parent / "SOURCES.json"
def _gh_api(endpoint: str) -> Any:
"""Call gh api and return parsed JSON."""
result = subprocess.run(
["gh", "api", endpoint],
capture_output=True,
text=True,
timeout=15,
)
if result.returncode != 0:
return None
return json.loads(result.stdout)
def load_sources() -> dict[str, Any]:
if not SOURCES_PATH.exists():
stderr.print(f"[red]SOURCES.json not found at {SOURCES_PATH}[/red]")
sys.exit(1)
return json.loads(SOURCES_PATH.read_text())
def check_repo(name: str, baseline: dict[str, Any]) -> dict[str, Any]:
"""Compare a single repo against its baseline."""
commits = _gh_api(f"repos/{name}/commits?per_page=1")
if not commits or not isinstance(commits, list):
return {"repo": name, "status": "error", "message": "Failed to fetch commits"}
current_sha = commits[0]["sha"]
current_date = commits[0]["commit"]["committer"]["date"]
release_data = _gh_api(f"repos/{name}/releases/latest")
current_release = None
if isinstance(release_data, dict) and "tag_name" in release_data:
current_release = release_data["tag_name"]
baseline_sha = baseline["sha"]
baseline_release = baseline.get("release")
sha_changed = current_sha != baseline_sha
release_changed = current_release != baseline_release and current_release is not None
# Count commits since baseline
commits_behind = 0
if sha_changed:
compare = _gh_api(f"repos/{name}/compare/{baseline_sha[:12]}...{current_sha[:12]}")
if isinstance(compare, dict):
commits_behind = compare.get("ahead_by", 0)
return {
"repo": name,
"status": "changed" if (sha_changed or release_changed) else "current",
"baseline_sha": baseline_sha[:12],
"current_sha": current_sha[:12],
"sha_changed": sha_changed,
"commits_behind": commits_behind,
"baseline_release": baseline_release,
"current_release": current_release,
"release_changed": release_changed,
"current_date": current_date,
"rules_affected": baseline.get("rules_affected", []),
}
def render_table(results: list[dict[str, Any]]) -> None:
"""Print a Rich table to stderr."""
table = Table(title="Upstream Source Status", show_lines=True)
table.add_column("Repository", style="bold")
table.add_column("Status")
table.add_column("Baseline SHA")
table.add_column("Current SHA")
table.add_column("Commits")
table.add_column("Release")
table.add_column("Rules Affected")
for r in results:
if r["status"] == "error":
table.add_row(r["repo"], "[red]ERROR[/red]", "-", "-", "-", "-", r.get("message", ""))
continue
status = "[green]CURRENT[/green]" if r["status"] == "current" else "[yellow]CHANGED[/yellow]"
sha_display = r["current_sha"]
if r["sha_changed"]:
sha_display = f"[yellow]{r['current_sha']}[/yellow]"
commits = str(r["commits_behind"]) if r["commits_behind"] else "-"
release_display = r.get("current_release") or "-"
if r["release_changed"]:
release_display = f"[yellow]{r['baseline_release']} -> {r['current_release']}[/yellow]"
rules = ", ".join(Path(p).name for p in r["rules_affected"])
table.add_row(r["repo"], status, r["baseline_sha"], sha_display, commits, release_display, rules)
stderr.print(table)
changed = [r for r in results if r["status"] == "changed"]
if changed:
all_rules = sorted({rule for r in changed for rule in r["rules_affected"]})
stderr.print(f"\n[yellow]{len(changed)} repo(s) changed.[/yellow] Rules to review:")
for rule in all_rules:
stderr.print(f" - {rule}")
else:
stderr.print("\n[green]All sources are current. No updates needed.[/green]")
def main() -> None:
parser = argparse.ArgumentParser(description="Check upstream repos for changes since last snapshot.")
parser.add_argument("--json", action="store_true", help="JSON-only output (no Rich table)")
parser.add_argument("--stale", action="store_true", help="Only show repos that changed")
args = parser.parse_args()
sources = load_sources()
results: list[dict[str, Any]] = []
for name, baseline in sources.get("repos", {}).items():
if not args.json:
stderr.print(f"Checking {name}...", end=" ")
result = check_repo(name, baseline)
results.append(result)
if not args.json:
mark = "[green]OK[/green]" if result["status"] == "current" else "[yellow]CHANGED[/yellow]"
stderr.print(mark)
if args.stale:
results = [r for r in results if r["status"] == "changed"]
if not args.json:
stderr.print()
render_table(results)
report = {
"skill_version": sources.get("skill_version"),
"snapshot_date": sources.get("snapshot_date"),
"check_date": __import__("datetime").date.today().isoformat(),
"repos": results,
"stale_count": sum(1 for r in results if r["status"] == "changed"),
"rules_to_update": sorted({rule for r in results if r["status"] == "changed" for rule in r["rules_affected"]}),
}
print(json.dumps(report, indent=2))
if __name__ == "__main__":
main()
```
### scripts/doctor.py
```python
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "rich>=13.0.0",
# "websockets>=12.0",
# ]
# ///
"""SurrealDB environment health check tool.
Runs quick (offline) or full (online) diagnostics against a SurrealDB
installation and reports results as a Rich table on stderr and structured
JSON on stdout.
"""
from __future__ import annotations
import argparse
import asyncio
import json
import os
import shutil
import socket
import subprocess
import sys
from typing import Any
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
stderr_console = Console(stderr=True)
DEFAULT_ENDPOINT = "ws://localhost:8000"
# ---------------------------------------------------------------------------
# Types
# ---------------------------------------------------------------------------
STATUS_PASS = "pass"
STATUS_WARN = "warn"
STATUS_FAIL = "fail"
def _env(name: str) -> str | None:
val = os.environ.get(name, "").strip()
return val if val else None
# ---------------------------------------------------------------------------
# Quick checks (no server connection required)
# ---------------------------------------------------------------------------
def check_cli() -> dict[str, Any]:
"""Check if the surreal CLI is installed and get its version."""
path = shutil.which("surreal")
if path is None:
return {"status": STATUS_FAIL, "message": "surreal CLI not found in PATH"}
try:
result = subprocess.run(
[path, "version"], capture_output=True, text=True, timeout=10,
)
output = (result.stdout or result.stderr).strip()
version = None
for token in output.split():
if token[0].isdigit():
version = token
break
return {"status": STATUS_PASS, "version": version or output, "path": path}
except Exception as exc:
return {"status": STATUS_FAIL, "message": str(exc)}
def check_env_vars() -> dict[str, Any]:
"""Check that essential environment variables are set."""
required = ["SURREAL_ENDPOINT", "SURREAL_USER", "SURREAL_PASS", "SURREAL_NS", "SURREAL_DB"]
present = {k: _env(k) is not None for k in required}
missing = [k for k, v in present.items() if not v]
status = STATUS_PASS if not missing else STATUS_WARN
return {"status": status, "variables": present, "missing": missing}
def check_uv() -> dict[str, Any]:
"""Check if uv is available."""
path = shutil.which("uv")
if path:
return {"status": STATUS_PASS, "path": path}
return {"status": STATUS_WARN, "message": "uv not found -- install from https://docs.astral.sh/uv/"}
def check_port(endpoint: str | None = None) -> dict[str, Any]:
"""Check if the SurrealDB port is open (TCP-level only)."""
ep = endpoint or _env("SURREAL_ENDPOINT") or DEFAULT_ENDPOINT
host, port = _parse_endpoint(ep)
try:
with socket.create_connection((host, port), timeout=3):
return {"status": STATUS_PASS, "endpoint": ep, "host": host, "port": port}
except OSError as exc:
return {"status": STATUS_FAIL, "endpoint": ep, "host": host, "port": port, "message": str(exc)}
def _parse_endpoint(endpoint: str) -> tuple[str, int]:
url = endpoint
for prefix in ("ws://", "wss://", "http://", "https://"):
if url.startswith(prefix):
url = url[len(prefix):]
break
url = url.rstrip("/")
if ":" in url:
host, port_str = url.rsplit(":", 1)
try:
return host, int(port_str)
except ValueError:
pass
return url, 8000
# ---------------------------------------------------------------------------
# Full checks (require server connection)
# ---------------------------------------------------------------------------
async def _ws_query(endpoint: str, user: str, password: str, ns: str | None, db: str | None, query: str) -> Any:
"""Open a WebSocket to SurrealDB, authenticate, optionally USE ns/db, and execute a query."""
import websockets # type: ignore[import-untyped]
ws_ep = endpoint.rstrip("/") + "/rpc"
async with websockets.connect(ws_ep, open_timeout=5, close_timeout=5) as ws:
# Sign in
signin_msg = json.dumps({
"id": "signin",
"method": "signin",
"params": [{"user": user, "pass": password}],
})
await ws.send(signin_msg)
signin_resp = json.loads(await ws.recv())
if signin_resp.get("error"):
raise RuntimeError(f"Auth failed: {signin_resp['error']}")
# USE namespace/database if provided
if ns and db:
use_msg = json.dumps({
"id": "use",
"method": "use",
"params": [ns, db],
})
await ws.send(use_msg)
use_resp = json.loads(await ws.recv())
if use_resp.get("error"):
raise RuntimeError(f"USE failed: {use_resp['error']}")
# Query
query_msg = json.dumps({
"id": "query",
"method": "query",
"params": [query],
})
await ws.send(query_msg)
query_resp = json.loads(await ws.recv())
if query_resp.get("error"):
raise RuntimeError(f"Query failed: {query_resp['error']}")
return query_resp.get("result")
return None # unreachable but keeps type-checker happy
def _run_ws_query(endpoint: str, user: str, password: str, ns: str | None, db: str | None, query: str) -> Any:
"""Synchronous wrapper around _ws_query."""
return asyncio.get_event_loop().run_until_complete(
_ws_query(endpoint, user, password, ns, db, query),
)
def check_auth(endpoint: str, user: str, password: str) -> dict[str, Any]:
"""Authenticate against the SurrealDB server."""
try:
import websockets # noqa: F401
asyncio.get_event_loop().run_until_complete(
_ws_auth_only(endpoint, user),
)
# If we get here, at least the connection works. We test auth below.
except Exception:
pass
try:
_run_ws_query(endpoint, user, password, None, None, "INFO FOR ROOT")
return {"status": STATUS_PASS, "user": user}
except RuntimeError as exc:
return {"status": STATUS_FAIL, "user": user, "message": str(exc)}
except Exception as exc:
return {"status": STATUS_FAIL, "user": user, "message": str(exc)}
async def _ws_auth_only(endpoint: str, user: str) -> None:
"""Minimal connection test (no auth)."""
import websockets # type: ignore[import-untyped]
ws_ep = endpoint.rstrip("/") + "/rpc"
async with websockets.connect(ws_ep, open_timeout=5, close_timeout=5):
pass
def check_namespace(endpoint: str, user: str, password: str, ns: str) -> dict[str, Any]:
"""Check that the namespace is accessible."""
try:
_run_ws_query(endpoint, user, password, None, None, "INFO FOR ROOT")
return {"status": STATUS_PASS, "namespace": ns}
except Exception as exc:
return {"status": STATUS_WARN, "namespace": ns, "message": str(exc)}
def check_database(endpoint: str, user: str, password: str, ns: str, db: str) -> dict[str, Any]:
"""Check the database and count tables."""
try:
result = _run_ws_query(endpoint, user, password, ns, db, "INFO FOR DB")
table_count = 0
if isinstance(result, list) and result:
first = result[0]
db_info = first.get("result") if isinstance(first, dict) else first
if isinstance(db_info, dict):
tables = db_info.get("tables") or db_info.get("tb") or {}
table_count = len(tables)
return {"status": STATUS_PASS, "database": db, "tables": table_count}
except Exception as exc:
return {"status": STATUS_FAIL, "database": db, "message": str(exc)}
def check_server_version(endpoint: str, user: str, password: str) -> dict[str, Any]:
"""Retrieve the server version via a version query or CLI."""
# Try the CLI first as it is more reliable
cli_check = check_cli()
if cli_check["status"] == STATUS_PASS:
return {"status": STATUS_PASS, "version": cli_check.get("version", "unknown")}
return {"status": STATUS_WARN, "message": "Could not determine server version"}
def check_storage_engine(endpoint: str, user: str, password: str) -> dict[str, Any]:
"""Attempt to determine the storage engine in use."""
# SurrealDB does not expose storage engine via query in all versions;
# we infer from the CLI start flags or just report as available.
path = shutil.which("surreal")
if path:
try:
result = subprocess.run([path, "version"], capture_output=True, text=True, timeout=10)
output = (result.stdout or result.stderr).strip()
# The engine is typically in the startup config, not the version string.
# We report what we can.
return {"status": STATUS_PASS, "engine": "unknown (check server startup flags)"}
except Exception:
pass
return {"status": STATUS_WARN, "message": "Could not determine storage engine"}
# ---------------------------------------------------------------------------
# Aggregation
# ---------------------------------------------------------------------------
def run_quick(endpoint_override: str | None = None) -> dict[str, Any]:
"""Run quick (offline) checks."""
checks: dict[str, Any] = {}
checks["cli_installed"] = check_cli()
checks["env_vars"] = check_env_vars()
checks["uv_available"] = check_uv()
checks["port_open"] = check_port(endpoint_override)
warnings = [k for k, v in checks.items() if v.get("status") == STATUS_WARN]
errors = [k for k, v in checks.items() if v.get("status") == STATUS_FAIL]
overall = STATUS_FAIL if errors else STATUS_WARN if warnings else STATUS_PASS
return {
"status": "healthy" if overall == STATUS_PASS else "degraded" if overall == STATUS_WARN else "unhealthy",
"checks": checks,
"warnings": warnings,
"errors": errors,
}
def run_full(
endpoint: str | None = None,
user: str | None = None,
password: str | None = None,
ns: str | None = None,
db: str | None = None,
) -> dict[str, Any]:
"""Run full (online) checks."""
ep = endpoint or _env("SURREAL_ENDPOINT") or DEFAULT_ENDPOINT
u = user or _env("SURREAL_USER") or "root"
p = password or _env("SURREAL_PASS") or "root"
namespace = ns or _env("SURREAL_NS")
database = db or _env("SURREAL_DB")
checks: dict[str, Any] = {}
# Quick checks first
checks["cli_installed"] = check_cli()
checks["env_vars"] = check_env_vars()
checks["uv_available"] = check_uv()
checks["port_open"] = check_port(ep)
# Online checks
if checks["port_open"]["status"] == STATUS_PASS:
checks["auth_valid"] = check_auth(ep, u, p)
checks["server_version"] = check_server_version(ep, u, p)
checks["storage_engine"] = check_storage_engine(ep, u, p)
if namespace:
checks["namespace_accessible"] = check_namespace(ep, u, p, namespace)
if namespace and database:
checks["database_accessible"] = check_database(ep, u, p, namespace, database)
else:
checks["server_reachable"] = {"status": STATUS_FAIL, "message": "Skipped: port not reachable"}
warnings = [k for k, v in checks.items() if v.get("status") == STATUS_WARN]
errors = [k for k, v in checks.items() if v.get("status") == STATUS_FAIL]
overall = STATUS_FAIL if errors else STATUS_WARN if warnings else STATUS_PASS
return {
"status": "healthy" if overall == STATUS_PASS else "degraded" if overall == STATUS_WARN else "unhealthy",
"checks": checks,
"warnings": warnings,
"errors": errors,
}
# ---------------------------------------------------------------------------
# Output
# ---------------------------------------------------------------------------
def print_results_table(report: dict[str, Any]) -> None:
"""Print a Rich table of results to stderr."""
overall = report["status"]
style_map = {"healthy": "green", "degraded": "yellow", "unhealthy": "red"}
color = style_map.get(overall, "white")
table = Table(title=f"SurrealDB Health Check -- [{color}]{overall.upper()}[/{color}]", show_lines=True)
table.add_column("Check", style="bold")
table.add_column("Status")
table.add_column("Details")
status_render = {
STATUS_PASS: "[green]PASS[/green]",
STATUS_WARN: "[yellow]WARN[/yellow]",
STATUS_FAIL: "[red]FAIL[/red]",
}
for name, info in report["checks"].items():
rendered_status = status_render.get(info.get("status", ""), info.get("status", ""))
detail_parts = []
for k, v in info.items():
if k == "status":
continue
detail_parts.append(f"{k}: {v}")
table.add_row(name.replace("_", " ").title(), rendered_status, "\n".join(detail_parts) if detail_parts else "-")
stderr_console.print(table)
if report["warnings"]:
stderr_console.print(f"\n[yellow]Warnings:[/yellow] {', '.join(report['warnings'])}")
if report["errors"]:
stderr_console.print(f"[red]Errors:[/red] {', '.join(report['errors'])}")
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="doctor",
description="SurrealDB environment health check.",
)
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--quick", "--check", action="store_true", help="Fast checks only (no server connection); exit code 0 = healthy, 1 = issues")
mode.add_argument("--full", action="store_true", help="Complete health check (default)")
parser.add_argument("--endpoint", type=str, default=None, help="SurrealDB endpoint (overrides SURREAL_ENDPOINT)")
parser.add_argument("--user", type=str, default=None, help="Username (overrides SURREAL_USER)")
parser.add_argument("--pass", dest="password", type=str, default=None, help="Password (overrides SURREAL_PASS)")
parser.add_argument("--ns", type=str, default=None, help="Namespace (overrides SURREAL_NS)")
parser.add_argument("--db", type=str, default=None, help="Database (overrides SURREAL_DB)")
return parser
def main() -> None:
parser = build_parser()
args = parser.parse_args()
if args.quick:
report = run_quick(endpoint_override=args.endpoint)
else:
report = run_full(
endpoint=args.endpoint,
user=args.user,
password=args.password,
ns=args.ns,
db=args.db,
)
print_results_table(report)
print(json.dumps(report, indent=2))
# Exit with non-zero if unhealthy
if report["status"] == "unhealthy":
sys.exit(1)
if __name__ == "__main__":
main()
```
### scripts/onboard.py
```python
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "rich>=13.0.0",
# ]
# ///
"""SurrealDB skill onboarding: setup wizard, configuration check, and capabilities manifest."""
from __future__ import annotations
import argparse
import json
import os
import shutil
import subprocess
import sys
from pathlib import Path
from rich.console import Console
from rich.panel import Panel
from rich.prompt import Confirm, Prompt
from rich.table import Table
stderr_console = Console(stderr=True)
stdout_console = Console(file=sys.stdout)
SKILL_VERSION = "1.0.0"
DEFAULT_ENDPOINT = "ws://localhost:8000"
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _run(cmd: list[str], timeout: int = 10) -> subprocess.CompletedProcess[str]:
"""Run a subprocess, capturing output."""
return subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
def _surreal_version() -> str | None:
"""Return the surreal CLI version string, or None if not found."""
path = shutil.which("surreal")
if path is None:
return None
try:
result = _run([path, "version"])
output = (result.stdout or result.stderr).strip()
# surreal CLI may print "surreal 3.x.x" or just the version
for token in output.split():
if token[0].isdigit():
return token
return output or None
except Exception:
return None
def _env(name: str) -> str | None:
"""Return an environment variable or None."""
val = os.environ.get(name, "").strip()
return val if val else None
def _check_uv() -> bool:
"""Return True if uv is available."""
return shutil.which("uv") is not None
def _check_port(host: str, port: int, timeout: float = 2.0) -> bool:
"""Return True if a TCP connection can be made to host:port."""
import socket
try:
with socket.create_connection((host, port), timeout=timeout):
return True
except OSError:
return False
def _parse_endpoint(endpoint: str) -> tuple[str, int]:
"""Extract host and port from a SurrealDB endpoint URL."""
url = endpoint
for prefix in ("ws://", "wss://", "http://", "https://"):
if url.startswith(prefix):
url = url[len(prefix):]
break
url = url.rstrip("/")
if ":" in url:
host, port_str = url.rsplit(":", 1)
try:
return host, int(port_str)
except ValueError:
pass
return url, 8000
# ---------------------------------------------------------------------------
# --check mode
# ---------------------------------------------------------------------------
def run_check() -> dict:
"""Run configuration checks and return a results dict."""
results: dict = {}
# surreal CLI
version = _surreal_version()
results["cli_installed"] = {
"status": "pass" if version else "fail",
"version": version,
"path": shutil.which("surreal"),
}
# Endpoint
endpoint = _env("SURREAL_ENDPOINT") or DEFAULT_ENDPOINT
host, port = _parse_endpoint(endpoint)
reachable = _check_port(host, port)
results["server_reachable"] = {
"status": "pass" if reachable else "fail",
"endpoint": endpoint,
}
# Credentials
user = _env("SURREAL_USER")
password = _env("SURREAL_PASS")
results["credentials_configured"] = {
"status": "pass" if user and password else "warn" if user or password else "fail",
"user_set": user is not None,
"pass_set": password is not None,
}
# Namespace / Database
ns = _env("SURREAL_NS")
db = _env("SURREAL_DB")
results["namespace_database"] = {
"status": "pass" if ns and db else "warn" if ns or db else "fail",
"namespace": ns,
"database": db,
}
# uv
results["uv_available"] = {
"status": "pass" if _check_uv() else "warn",
"path": shutil.which("uv"),
}
return results
def print_check_table(results: dict) -> None:
"""Render a Rich table of check results to stderr."""
table = Table(title="SurrealDB Configuration Check", show_lines=True)
table.add_column("Check", style="bold")
table.add_column("Status")
table.add_column("Details")
status_style = {"pass": "[green]PASS[/green]", "warn": "[yellow]WARN[/yellow]", "fail": "[red]FAIL[/red]"}
for name, info in results.items():
status = status_style.get(info["status"], info["status"])
details_parts = []
for k, v in info.items():
if k == "status":
continue
details_parts.append(f"{k}: {v}")
table.add_row(name.replace("_", " ").title(), status, "\n".join(details_parts))
stderr_console.print(table)
# ---------------------------------------------------------------------------
# --agent mode
# ---------------------------------------------------------------------------
CAPABILITIES_MANIFEST: dict = {
"skill": "surrealdb",
"version": SKILL_VERSION,
"description": "Expert SurrealDB 3 architect and developer",
"commands": [
{"name": "onboard", "subcommands": ["--check", "--agent", "--interactive"], "description": "Setup and capabilities"},
{"name": "doctor", "subcommands": ["--full", "--quick"], "description": "Environment health check"},
{"name": "schema", "subcommands": ["export", "inspect", "diff"], "description": "Schema introspection"},
],
"rules": [
{"file": "surrealql.md", "topic": "SurrealQL syntax, statements, functions, operators"},
{"file": "data-modeling.md", "topic": "Multi-model data modeling patterns"},
{"file": "graph-queries.md", "topic": "Graph traversal and RELATE patterns"},
{"file": "vector-search.md", "topic": "Vector indexes, HNSW, RAG patterns"},
{"file": "security.md", "topic": "Authentication, permissions, access control"},
{"file": "performance.md", "topic": "Indexing, query optimization, storage engines"},
{"file": "sdks.md", "topic": "SDK patterns for JS, Python, Go, Rust, Java, .NET"},
{"file": "deployment.md", "topic": "Docker, Kubernetes, cloud, distributed deployment"},
{"file": "surrealism.md", "topic": "WASM extension development"},
{"file": "surreal-sync.md", "topic": "Data migration from other databases"},
{"file": "surrealist.md", "topic": "Surrealist IDE/GUI"},
{"file": "surrealfs.md", "topic": "AI agent virtual filesystem"},
],
"decision_trees": {
"new_project": "Run doctor -> Design schema (data-modeling.md) -> Choose deployment (deployment.md) -> Configure security (security.md)",
"data_modeling": "Identify models needed -> Read data-modeling.md -> Use graph-queries.md for relationships -> Use vector-search.md for AI features",
"performance_issue": "Run doctor -> Check indexes (performance.md) -> Review queries (surrealql.md) -> Consider storage engine change",
"migration": "Identify source DB -> Read surreal-sync.md -> Plan schema mapping (data-modeling.md) -> Execute migration",
"extension_development": "Read surrealism.md -> Write Rust module -> Compile WASM -> Register with DEFINE MODULE",
},
"environment_variables": {
"SURREAL_ENDPOINT": "SurrealDB server endpoint (default: ws://localhost:8000)",
"SURREAL_USER": "Root/admin username",
"SURREAL_PASS": "Root/admin password",
"SURREAL_NS": "Default namespace",
"SURREAL_DB": "Default database",
},
}
def run_agent() -> dict:
"""Return the capabilities manifest dict."""
return CAPABILITIES_MANIFEST
# ---------------------------------------------------------------------------
# --interactive mode
# ---------------------------------------------------------------------------
def run_interactive() -> dict:
"""Guided setup interview. Returns a summary dict."""
stderr_console.print(Panel("SurrealDB Skill -- Interactive Setup", style="bold cyan"))
result: dict = {"steps": []}
# 1. Check surreal CLI
version = _surreal_version()
if version:
stderr_console.print(f" surreal CLI found: v{version}")
result["steps"].append({"step": "cli_check", "status": "pass", "version": version})
else:
stderr_console.print("[red] surreal CLI not found.[/red] Install from https://surrealdb.com/install")
result["steps"].append({"step": "cli_check", "status": "fail"})
# 2. Endpoint
default_ep = _env("SURREAL_ENDPOINT") or DEFAULT_ENDPOINT
endpoint = Prompt.ask("SurrealDB endpoint", default=default_ep, console=stderr_console)
result["endpoint"] = endpoint
# 3. Connectivity
host, port = _parse_endpoint(endpoint)
reachable = _check_port(host, port)
if reachable:
stderr_console.print(f" Server reachable at {endpoint}")
result["steps"].append({"step": "connectivity", "status": "pass"})
else:
stderr_console.print(f"[yellow] Cannot reach {endpoint}. Continue anyway.[/yellow]")
result["steps"].append({"step": "connectivity", "status": "fail"})
# 4. Credentials
default_user = _env("SURREAL_USER") or "root"
user = Prompt.ask("Username", default=default_user, console=stderr_console)
password = Prompt.ask("Password", password=True, default=_env("SURREAL_PASS") or "", console=stderr_console)
result["user"] = user
result["steps"].append({"step": "credentials", "status": "pass"})
# 5. Namespace / Database
ns = Prompt.ask("Namespace", default=_env("SURREAL_NS") or "test", console=stderr_console)
db = Prompt.ask("Database", default=_env("SURREAL_DB") or "test", console=stderr_console)
result["namespace"] = ns
result["database"] = db
result["steps"].append({"step": "namespace_database", "status": "pass"})
# 6. Generate .env
if Confirm.ask("Generate a .env file?", default=True, console=stderr_console):
env_path = Path.cwd() / ".env"
lines = [
f"SURREAL_ENDPOINT={endpoint}",
f"SURREAL_USER={user}",
f"SURREAL_PASS={password}",
f"SURREAL_NS={ns}",
f"SURREAL_DB={db}",
]
env_path.write_text("\n".join(lines) + "\n")
stderr_console.print(f" Wrote {env_path}")
result["env_file"] = str(env_path)
result["steps"].append({"step": "env_file", "status": "pass"})
else:
result["steps"].append({"step": "env_file", "status": "skipped"})
return result
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="onboard",
description="SurrealDB skill onboarding: setup wizard, configuration check, and capabilities manifest.",
)
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--check", action="store_true", help="Check configuration status (non-interactive)")
mode.add_argument("--agent", action="store_true", help="Output JSON capabilities manifest")
mode.add_argument("--interactive", action="store_true", help="Guided setup interview (default for TTY)")
return parser
def main() -> None:
parser = build_parser()
args = parser.parse_args()
if args.agent:
manifest = run_agent()
stderr_console.print("[bold]Capabilities manifest:[/bold]")
print(json.dumps(manifest, indent=2))
return
if args.check:
results = run_check()
print_check_table(results)
print(json.dumps(results, indent=2))
return
if args.interactive:
result = run_interactive()
print(json.dumps(result, indent=2))
return
# Default: interactive if TTY, otherwise check
if sys.stdin.isatty():
result = run_interactive()
print(json.dumps(result, indent=2))
else:
results = run_check()
print_check_table(results)
print(json.dumps(results, indent=2))
if __name__ == "__main__":
main()
```
### scripts/schema.py
```python
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "rich>=13.0.0",
# "websockets>=12.0",
# ]
# ///
"""SurrealDB schema introspection and export tool.
Subcommands:
export -- Export the complete schema as SurrealQL DEFINE statements.
inspect -- Structured view of tables, fields, indexes, events, and access.
diff -- Compare two SurrealQL schema files and show differences.
"""
from __future__ import annotations
import argparse
import asyncio
import difflib
import json
import os
import sys
from pathlib import Path
from typing import Any
from rich.console import Console
from rich.panel import Panel
from rich.syntax import Syntax
from rich.table import Table
from rich.tree import Tree
stderr_console = Console(stderr=True)
DEFAULT_ENDPOINT = "ws://localhost:8000"
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _env(name: str) -> str | None:
val = os.environ.get(name, "").strip()
return val if val else None
import re as _re
_SAFE_IDENT = _re.compile(r"^[a-zA-Z_][a-zA-Z0-9_]*$")
def _sanitize_identifier(name: str) -> str:
"""Validate that a name is a safe SurrealQL identifier.
Prevents SurrealQL injection by rejecting names that contain
anything other than alphanumeric characters and underscores.
"""
name = name.strip()
if not name or not _SAFE_IDENT.match(name):
raise ValueError(
f"Invalid identifier: {name!r}. "
"Identifiers must match [a-zA-Z_][a-zA-Z0-9_]*."
)
return name
async def _ws_query(endpoint: str, user: str, password: str, ns: str, db: str, query: str) -> Any:
"""Connect via WebSocket RPC, authenticate, USE ns/db, and execute a query."""
import websockets # type: ignore[import-untyped]
ws_ep = endpoint.rstrip("/") + "/rpc"
async with websockets.connect(ws_ep, open_timeout=10, close_timeout=5) as ws:
# Sign in
signin_msg = json.dumps({
"id": "signin",
"method": "signin",
"params": [{"user": user, "pass": password}],
})
await ws.send(signin_msg)
signin_resp = json.loads(await ws.recv())
if signin_resp.get("error"):
raise RuntimeError(f"Auth failed: {signin_resp['error']}")
# USE
use_msg = json.dumps({
"id": "use",
"method": "use",
"params": [ns, db],
})
await ws.send(use_msg)
use_resp = json.loads(await ws.recv())
if use_resp.get("error"):
raise RuntimeError(f"USE failed: {use_resp['error']}")
# Query
query_msg = json.dumps({
"id": "query",
"method": "query",
"params": [query],
})
await ws.send(query_msg)
query_resp = json.loads(await ws.recv())
if query_resp.get("error"):
raise RuntimeError(f"Query failed: {query_resp['error']}")
return query_resp.get("result")
def run_query(endpoint: str, user: str, password: str, ns: str, db: str, query: str) -> Any:
"""Synchronous wrapper for WebSocket query."""
return asyncio.get_event_loop().run_until_complete(
_ws_query(endpoint, user, password, ns, db, query),
)
def _extract_result(raw: Any) -> Any:
"""Extract the inner result from a SurrealDB query response list."""
if isinstance(raw, list) and raw:
first = raw[0]
if isinstance(first, dict) and "result" in first:
return first["result"]
return first
return raw
# ---------------------------------------------------------------------------
# export subcommand
# ---------------------------------------------------------------------------
def cmd_export(args: argparse.Namespace) -> None:
"""Export the complete database schema as SurrealQL DEFINE statements."""
ep = args.endpoint or _env("SURREAL_ENDPOINT") or DEFAULT_ENDPOINT
user = args.user or _env("SURREAL_USER") or "root"
password = args.password or _env("SURREAL_PASS") or "root"
ns = args.ns or _env("SURREAL_NS") or "test"
db = args.db or _env("SURREAL_DB") or "test"
stderr_console.print(f"Exporting schema from {ep} / {ns} / {db} ...")
try:
db_info_raw = run_query(ep, user, password, ns, db, "INFO FOR DB")
except Exception as exc:
stderr_console.print(f"[red]Failed to query database:[/red] {exc}")
print(json.dumps({"error": str(exc)}))
sys.exit(1)
db_info = _extract_result(db_info_raw)
statements: list[str] = []
# Header
statements.append(f"-- Schema export for namespace: {ns}, database: {db}")
statements.append(f"-- Endpoint: {ep}")
statements.append("")
if not isinstance(db_info, dict):
stderr_console.print("[yellow]Unexpected INFO FOR DB format. Dumping raw result.[/yellow]")
raw_str = json.dumps(db_info_raw, indent=2, default=str)
statements.append(f"-- Raw INFO FOR DB:\n-- {raw_str}")
_output_schema(statements, args)
return
# Iterate over known schema categories
category_order = ["accesses", "analyzers", "functions", "models", "params", "tables", "users"]
# Some versions use shorthand keys
key_aliases = {"ac": "accesses", "az": "analyzers", "fn": "functions", "ml": "models",
"pa": "params", "tb": "tables", "us": "users"}
normalized: dict[str, dict] = {}
for k, v in db_info.items():
canon = key_aliases.get(k, k)
if isinstance(v, dict):
normalized[canon] = v
for category in category_order:
items = normalized.get(category, {})
if not items:
continue
statements.append(f"-- {category.upper()}")
for name, definition in sorted(items.items()):
if isinstance(definition, str):
stmt = definition.rstrip(";") + ";"
statements.append(stmt)
else:
statements.append(f"-- {name}: {json.dumps(definition, default=str)}")
statements.append("")
# Per-table detail
tables = normalized.get("tables", {})
for tbl_name in sorted(tables.keys()):
statements.append(f"-- TABLE: {tbl_name}")
try:
tbl_info_raw = run_query(ep, user, password, ns, db, f"INFO FOR TABLE {_sanitize_identifier(tbl_name)}")
tbl_info = _extract_result(tbl_info_raw)
if isinstance(tbl_info, dict):
for section_key in ("fields", "fd", "indexes", "ix", "events", "ev", "lives", "lv"):
section = tbl_info.get(section_key)
if isinstance(section, dict) and section:
for item_name, item_def in sorted(section.items()):
if isinstance(item_def, str):
statements.append(item_def.rstrip(";") + ";")
else:
statements.append(f"-- {item_name}: {json.dumps(item_def, default=str)}")
else:
statements.append(f"-- (raw) {json.dumps(tbl_info, default=str)}")
except Exception as exc:
statements.append(f"-- Error fetching table info: {exc}")
statements.append("")
_output_schema(statements, args)
def _output_schema(statements: list[str], args: argparse.Namespace) -> None:
"""Write schema to stderr (pretty), stdout (JSON), and optionally to a file."""
schema_text = "\n".join(statements)
# Rich syntax highlighting on stderr
stderr_console.print(Syntax(schema_text, "sql", theme="monokai", line_numbers=True))
# Machine-readable JSON on stdout
output = {"schema": schema_text, "line_count": len(statements)}
if hasattr(args, "output_dir") and args.output_dir:
out_dir = Path(args.output_dir)
out_dir.mkdir(parents=True, exist_ok=True)
out_file = out_dir / "schema.surql"
out_file.write_text(schema_text)
stderr_console.print(f"Schema written to {out_file}")
output["file"] = str(out_file)
print(json.dumps(output, indent=2))
# ---------------------------------------------------------------------------
# inspect subcommand
# ---------------------------------------------------------------------------
def cmd_inspect(args: argparse.Namespace) -> None:
"""Structured inspection of the database schema."""
ep = args.endpoint or _env("SURREAL_ENDPOINT") or DEFAULT_ENDPOINT
user = args.user or _env("SURREAL_USER") or "root"
password = args.password or _env("SURREAL_PASS") or "root"
ns = args.ns or _env("SURREAL_NS") or "test"
db = args.db or _env("SURREAL_DB") or "test"
stderr_console.print(f"Inspecting schema on {ep} / {ns} / {db} ...")
try:
db_info_raw = run_query(ep, user, password, ns, db, "INFO FOR DB")
except Exception as exc:
stderr_console.print(f"[red]Failed:[/red] {exc}")
print(json.dumps({"error": str(exc)}))
sys.exit(1)
db_info = _extract_result(db_info_raw)
key_aliases = {"ac": "accesses", "az": "analyzers", "fn": "functions", "ml": "models",
"pa": "params", "tb": "tables", "us": "users"}
normalized: dict[str, dict] = {}
if isinstance(db_info, dict):
for k, v in db_info.items():
canon = key_aliases.get(k, k)
if isinstance(v, dict):
normalized[canon] = v
# Build a Rich tree for stderr
tree = Tree(f"[bold]Database: {ns}/{db}[/bold]")
inspection: dict[str, Any] = {"namespace": ns, "database": db, "tables": {}}
tables = normalized.get("tables", {})
for tbl_name in sorted(tables.keys()):
tbl_branch = tree.add(f"[cyan]{tbl_name}[/cyan]")
tbl_detail: dict[str, Any] = {"fields": {}, "indexes": {}, "events": {}, "accesses": {}}
try:
tbl_info_raw = run_query(ep, user, password, ns, db, f"INFO FOR TABLE {_sanitize_identifier(tbl_name)}")
tbl_info = _extract_result(tbl_info_raw)
if isinstance(tbl_info, dict):
section_map = {
"fields": ["fields", "fd"],
"indexes": ["indexes", "ix"],
"events": ["events", "ev"],
"accesses": ["accesses", "ac"],
}
for label, keys in section_map.items():
section_data: dict = {}
for key in keys:
if key in tbl_info and isinstance(tbl_info[key], dict):
section_data.update(tbl_info[key])
if section_data:
sec_branch = tbl_branch.add(f"[yellow]{label}[/yellow]")
for item_name, item_def in sorted(section_data.items()):
display = item_def if isinstance(item_def, str) else json.dumps(item_def, default=str)
sec_branch.add(f"{item_name}: {display}")
tbl_detail[label] = section_data
except Exception as exc:
tbl_branch.add(f"[red]Error: {exc}[/red]")
tbl_detail["error"] = str(exc)
inspection["tables"][tbl_name] = tbl_detail
# Summary table
summary_table = Table(title="Schema Summary", show_lines=True)
summary_table.add_column("Metric", style="bold")
summary_table.add_column("Value")
summary_table.add_row("Namespace", ns)
summary_table.add_row("Database", db)
summary_table.add_row("Tables", str(len(tables)))
total_fields = sum(len(t.get("fields", {})) for t in inspection["tables"].values())
total_indexes = sum(len(t.get("indexes", {})) for t in inspection["tables"].values())
summary_table.add_row("Total Fields", str(total_fields))
summary_table.add_row("Total Indexes", str(total_indexes))
stderr_console.print(summary_table)
stderr_console.print(tree)
# JSON on stdout
print(json.dumps(inspection, indent=2, default=str))
# ---------------------------------------------------------------------------
# diff subcommand
# ---------------------------------------------------------------------------
def cmd_diff(args: argparse.Namespace) -> None:
"""Compare two SurrealQL schema files."""
file1 = Path(args.file1)
file2 = Path(args.file2)
if not file1.exists():
stderr_console.print(f"[red]File not found:[/red] {file1}")
print(json.dumps({"error": f"File not found: {file1}"}))
sys.exit(1)
if not file2.exists():
stderr_console.print(f"[red]File not found:[/red] {file2}")
print(json.dumps({"error": f"File not found: {file2}"}))
sys.exit(1)
lines1 = file1.read_text().splitlines(keepends=True)
lines2 = file2.read_text().splitlines(keepends=True)
diff_lines = list(difflib.unified_diff(
lines1, lines2,
fromfile=str(file1),
tofile=str(file2),
lineterm="",
))
# Categorize changes
additions = [l for l in diff_lines if l.startswith("+") and not l.startswith("+++")]
removals = [l for l in diff_lines if l.startswith("-") and not l.startswith("---")]
diff_text = "\n".join(diff_lines) if diff_lines else "(no differences)"
# Rich output on stderr
if diff_lines:
stderr_console.print(Panel(f"Differences: {file1.name} vs {file2.name}", style="bold"))
stderr_console.print(Syntax(diff_text, "diff", theme="monokai"))
stderr_console.print(f"\n[green]+{len(additions)} additions[/green], [red]-{len(removals)} removals[/red]")
else:
stderr_console.print("[green]Schemas are identical.[/green]")
# JSON on stdout
output = {
"file1": str(file1),
"file2": str(file2),
"identical": len(diff_lines) == 0,
"additions": len(additions),
"removals": len(removals),
"diff": diff_text,
}
print(json.dumps(output, indent=2))
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="schema",
description="SurrealDB schema introspection and export tool.",
)
subparsers = parser.add_subparsers(dest="command", help="Available subcommands")
# Shared connection args
def add_connection_args(p: argparse.ArgumentParser) -> None:
p.add_argument("--endpoint", type=str, default=None, help="SurrealDB endpoint (default: SURREAL_ENDPOINT or ws://localhost:8000)")
p.add_argument("--user", type=str, default=None, help="Username (default: SURREAL_USER or root)")
p.add_argument("--pass", dest="password", type=str, default=None, help="Password (default: SURREAL_PASS or root)")
p.add_argument("--ns", type=str, default=None, help="Namespace (default: SURREAL_NS or test)")
p.add_argument("--db", type=str, default=None, help="Database (default: SURREAL_DB or test)")
# export
export_parser = subparsers.add_parser("export", help="Export the complete schema as SurrealQL")
add_connection_args(export_parser)
export_parser.add_argument("--output-dir", type=str, default=None, help="Directory to write schema.surql file")
# inspect
inspect_parser = subparsers.add_parser("inspect", help="Structured schema inspection")
add_connection_args(inspect_parser)
# diff
diff_parser = subparsers.add_parser("diff", help="Compare two SurrealQL schema files")
diff_parser.add_argument("--file1", type=str, required=True, help="First schema file")
diff_parser.add_argument("--file2", type=str, required=True, help="Second schema file")
return parser
def main() -> None:
parser = build_parser()
args = parser.parse_args()
if args.command is None:
parser.print_help(sys.stderr)
sys.exit(1)
dispatch = {
"export": cmd_export,
"inspect": cmd_inspect,
"diff": cmd_diff,
}
dispatch[args.command](args)
if __name__ == "__main__":
main()
```