Hello friends!

In this series of articles I talk about Convex - a new open and free BaaS (Backend as a Service) solution that looks very promising and is quickly gaining popularity among developers.

Today, Convex provides a mixed-mode reactive database, authentication/authorization mechanism, file storage, task scheduler, and smart search tools.

This is the first part of a series in which we will talk about the Convex features and database.

❯ Functions

❯ Queries

Requests are the heart of the backend. They request data from the database, check authentication or perform other business logic, and return the data to the client.

An example of a query that takes named arguments, reads data from the database and returns the result:

import { query } from "./_generated/server";
import { v } from "convex/values";

// Возвращает последние 100 задач из определенного списка
export const getTaskList = query({
  args: { taskListId: v.id("taskLists") },
  handler: async (ctx, args) => {
    const tasks = await ctx.db
      .query("tasks")
      .filter((q) => q.eq(q.field("taskListId"), args.taskListId))
      .order("desc")
      .take(100);
    return tasks;
  },
});

Request name

Queries are defined in TypeScript/JavaScript files in the directory convex.

The path, file name, and how the function is exported from it determine how the client will call it:

// convex/myFunctions.ts
// Эта функция будет вызываться как `api.myFunctions.myQuery`
export const myQuery = …;

// Эта функция будет вызываться как `api.myFunctions.sum`
export const sum = …;

Directories can be nested:

// convex/foo/myQueries.ts
// Эта функция будет вызываться как `api.foo.myQueries.listMessages`
export const listMessages = …;

Exports are named by default default:

// convex/myFunctions.ts
// Эта функция будет вызываться как `api.myFunctions.default`.
export default …;

Similar rules apply to mutations and operations. HTTP operations use a different approach to routing.

Constructor query

The constructor function is used to define the request query. Function handler should return the result of the query call:

import { query } from "./_generated/server";

export const myConstantString = query({
  handler: () => {
    return "Константная строка";
  },
});

Request Arguments

Queries accept named parameters. They are available as a second parameter handler():

import { query } from "./_generated/server";

export const sum = query({
  handler: (_, args: { a: number; b: number }) => {
    return args.a + args.b;
  },
});

Arguments and responses are automatically serialized and deserialized, so that almost any data can be passed freely to and from the request.

An object is used to define argument types and validate them args with validators v:

import { query } from "./_generated/server";
import { v } from "convex/values";

export const sum = query({
  args: { a: v.number(), b: v.number() },
  handler: (_, args) => {
    return args.a + args.b;
  },
});

First parameter handler() contains the request context.

Answer

Queries can return almost any data, which is automatically serialized and deserialized.

Queries can return undefined, which is not a valid Convex value. On the client undefined from the request is converted to null.

Request context

Constructor query allows you to query data and perform other operations using an object QueryCtx, transmitted handler() as the first argument:

import { query } from "./_generated/server";
import { v } from "convex/values";

export const myQuery = query({
  args: { a: v.number(), b: v.number() },
  handler: (ctx, args) => {
    // Работаем с `ctx`
  },
});

Which part of the context is used depends on the purpose of the request:

• The field is intended for retrieving data from the database db. Please note that the function handler can be asynchronous:

import { query } from "./_generated/server";
import { v } from "convex/values";

export const getTask = query({
  args: { id: v.id("tasks") },
  handler: async (ctx, args) => {
    return await ctx.db.get(args.id);
  },
});

• to obtain the URLs of files stored on the server, the field is intended storage
• The field is intended to verify user authentication auth

Separating query code using utilities

To separate query code and reuse logic across multiple Convex functions, you can use the following utilities:

import { Id } from "./_generated/dataModel";
import { query, QueryCtx } from "./_generated/server";
import { v } from "convex/values";

export const getTaskAndAuthor = query({
  args: { id: v.id("tasks") },
  handler: async (ctx, args) => {
    const task = await ctx.db.get(args.id);
    if (task === null) {
      return null;
    }
    return { task, author: await getUserName(ctx, task.authorId ?? null) };
  },
});

// Утилита, возвращающая имя пользователя (при наличии)
async function getUserName(ctx: QueryCtx, userId: Id<"users"> | null) {
  if (userId === null) {
    return null;
  }
  return (await ctx.db.get(userId))?.name;
}

Using NPM Packages

Requests can import NPM packages from node_modules. Please note that not all packages are supported.

npm i @faker-js/faker

import { query } from "./_generated/server";
import { faker } from "@faker-js/faker";

export const randomName = query({
  args: {},
  handler: () => {
    faker.seed();
    return faker.person.fullName();
  },
});

Calling a request on the client

A hook is used to call a request from React useQuery along with the generated object api:

import { useQuery } from "convex/react";
import { api } from "../convex/_generated/api";

export function MyApp() {
  const data = useQuery(api.myFunctions.sum, { a: 1, b: 2 });
  // Работаем с `data`
}

Caching and reactivity

Queries have two great features:

1. Caching: Convex automatically caches query results. Repeated requests with similar arguments retrieve data from the cache.
2. Responsiveness: Clients can subscribe to queries to obtain new results when underlying data changes.

For these features to work, the function handler must be deterministic: it must return the same results for the same arguments (including the request context).

For this reason, requests cannot request data from third-party APIs (operations are used for this).

Limits

Queries have limits on the amount of data they can read at a time to ensure good performance.

❯ Mutations

Mutations add, update, and delete data from the database, check authentication, or perform other business logic, and optionally return a response to the client.

An example of a mutation that takes named arguments, writes data to the database and returns the result:

import { mutation } from "./_generated/server";
import { v } from "convex/values";

// Создает новую задачу с определенным текстом
export const createTask = mutation({
  args: { text: v.string() },
  handler: async (ctx, args) => {
    const newTaskId = await ctx.db.insert("tasks", { text: args.text });
    return newTaskId;
  },
});

Mutation name

Mutations follow the same naming rules as queries.

Queries and mutations can be defined in the same file when using named exports.

Constructor mutation

To define a mutation, a constructor function is used mutation. The mutation itself is performed by the function handler:

import { mutation } from "./_generated/server";

export const mutateSomething = mutation({
  handler: () => {
    // Логика мутации
  },
});

Unlike a query, a mutation can, but does not have to, return a response.

Mutation arguments

Like queries, mutations take named arguments, which are accessible through a second parameter handler():

import { mutation } from "./_generated/server";

export const mutateSomething = mutation({
  handler: (_, args: { a: number; b: number }) => {
    // Работаем с `args.a` и `args.b`

    // Опционально возвращаем ответ
    return "Успешный успех";
  },
});

Arguments and responses are automatically serialized and deserialized, so that almost any data can be passed freely to/from the mutation.

An object is used to define argument types and validate them args with validators v:

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const mutateSomething = mutation({
  args: { a: v.number(), b: v.number() },
  handler: (_, args) => {
    // Работаем с `args.a` и `args.b`
  },
});

The first parameter handler() is the mutation context.

Answer

Mutations can return almost any data, which is automatically serialized and deserialized.

Mutations can return undefined, which is not a valid Convex value. On the client undefined from mutation transforms into null.

Context of mutation

Constructor mutation allows you to write data to the database and perform other operations using the object MutationCtxpassed as the first argument handler():

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const mutateSomething = mutation({
  args: { a: v.number(), b: v.number() },
  handler: (ctx, args) => {
    // Работаем с `ctx`
  },
});

Which part of the mutation context will be used depends on the mutation task:

• a field is used to read and write to the database db. Please note that the function handler can be asynchronous:

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const addItem = mutation({
  args: { text: v.string() },
  handler: async (ctx, args) => {
    await ctx.db.insert("tasks", { text: args.text });
  },
});

• to generate URLs for files stored on the server, the field is used storage
• the field is used to check user authentication auth
• the field is used to plan the launch of functions in the future scheduler

Splitting Mutation Code Using Utilities

Helper utilities can be used to separate mutation code and reuse logic across multiple Convex functions:

import { v } from "convex/values";
import { mutation, MutationCtx } from "./_generated/server";

export const addItem = mutation({
  args: { text: v.string() },
  handler: async (ctx, args) => {
    await ctx.db.insert("tasks", { text: args.text });
    await trackChange(ctx, "addItem");
  },
});

// Утилита для фиксации изменения
async function trackChange(ctx: MutationCtx, type: "addItem" | "removeItem") {
  await ctx.db.insert("changes", { type });
}

Mutations can be caused by utilities that accept QueryCtx (query context) as a parameter, since mutations can do the same thing as queries.

Using NPM Packages

Mutations can import NPM packages from node_modules. Please note that not all packages are supported.

npm i @faker-js/faker

import { faker } from "@faker-js/faker";
import { mutation } from "./_generated/server";

export const randomName = mutation({
  args: {},
  handler: async (ctx) => {
    faker.seed();
    await ctx.db.insert("tasks", { text: "Привет, " + faker.person.fullName() });
  },
});

Calling mutation on the client

To call a mutation from React, use a hook useMutation along with the generated object api:

import { useMutation } from "convex/react";
import { api } from "../convex/_generated/api";

export function MyApp() {
  const mutateSomething = useMutation(api.myFunctions.mutateSomething);
  const handleClick = () => {
    mutateSomething({ a: 1, b: 2 });
  };
  // Вешаем `handleClick` на кнопку
  // ...
}

When mutations are called on the client, they are executed one at a time using a single ordered queue. Therefore, the data in the database is edited in the order in which mutations are called.

Transactions

Mutations are triggered transactionally. This means the following:

1. All database reads within a transaction receive a consistent display of data in the database. Therefore, you don't have to worry about concurrent data updates in the middle of a mutation.
2. All records in the database are recorded together. If a mutation writes data to the DB and then throws an exception, the data will not be written to the DB.

In order for this to work, mutations must be deterministic and cannot be called by third party APIs (operations should be used for this).

Limits

Mutations have limits on the amount of data they can read and write at a time to ensure good performance.

❯ Operations / Actions

Operations can call third-party services to do things like process a payment using Stripe. They can run in the JS Convex framework or in Node.js. They can interact with the database through queries and mutations.

Operation name

Operations follow the same naming conventions as queries.

Constructor action

A constructor function is used to define an operation action. The operation itself is performed by the function handler:

import { action } from "./_generated/server";

export const doSomething = action({
  handler: () => {
    // Логика операции

    // Опционально возвращаем ответ
    return "Успешный успех";
  },
});

Unlike a request, an operation can, but does not have to, return a response.

Arguments and answers

The operation's arguments and responses follow the same rules as the mutation's arguments and responses:

import { action } from "./_generated/server";
import { v } from "convex/values";

export const doSomething = action({
  args: { a: v.number(), b: v.number() },
  handler: (_, args) => {
    // Работем с `args.a` и `args.b`

    // Опционально возвращаем ответ
    return "Успешный успех";
  },
});

The first argument handler() is the context of the operation.

Operation context

Constructor action allows you to interact with the database and perform other operations using the object ActionCtx, transmitted handler() as the first argument:

import { action } from "./_generated/server";
import { v } from "convex/values";

export const doSomething = action({
  args: { a: v.number(), b: v.number() },
  handler: (ctx, args) => {
    // Работаем с `ctx`
  },
});

Which part of the operation context will be used depends on the purpose of the operation:

• a field is used to read data from the database runQueryexecuting the request:

import { action, internalQuery } from "./_generated/server";
import { internal } from "./_generated/api";
import { v } from "convex/values";

export const doSomething = action({
  args: { a: v.number() },
  handler: async (ctx, args) => {
    const data = await ctx.runQuery(internal.myFunctions.readData, {
      a: args.a,
    });
    // Работаем с `data`
  },
});

export const readData = internalQuery({
  args: { a: v.number() },
  handler: async (ctx, args) => {
    // Читаем данные из `ctx.db`
  },
});

readData is an internal request that is not directly accessible to the client.

Operations, mutations and queries can be defined in a single file.

• A field is used to record data in the database runMutation, performing the mutation:

import { v } from "convex/values";
import { action } from "./_generated/server";
import { internal } from "./_generated/api";

export const doSomething = action({
  args: { a: v.number() },
  handler: async (ctx, args) => {
    const data = await ctx.runMutation(internal.myMutations.writeData, {
      a: args.a,
    });
    // ...
  },
});

writeData — an internal mutation that is not directly accessible to the client.

• to generate URLs for files stored on the server, the field is used storage
• the field is used to check user authentication auth
• the field is used to plan the launch of functions in the future scheduler
• for vector search by index, the field is used vectorSearch

Calling third party APIs and using NPM packages

Operations can be performed in a custom JS Convex runtime or in Node.js.

By default, operations are performed in the Convex environment. This environment supports the function fetch:

import { action } from "./_generated/server";

export const doSomething = action({
  args: {},
  handler: async () => {
    const data = await fetch("https://api.thirdpartyservice.com");
    // ...
  },
});

In the Convex environment, operations are faster than in Node.js because they do not require time to start the environment before execution (cold start).

Operations can import NPM packages, but not all packages are supported.

To perform an operation in Node.js, you need to add a directive to the beginning of the file "use node". Please note that other Convex functions cannot be performed in Node.js.

"use node";

import { action } from "./_generated/server";
import SomeNpmPackage from "some-npm-package";

export const doSomething = action({
  args: {},
  handler: () => {
    // Работаем с `SomeNpmPackage`
  },
});

Splitting opcode using utilities

Helper utilities can be used to separate opcodes and reuse logic across multiple Convex functions.

Please note that between objects ActionCtx, QueryCtx And MutationCtx only the field is common auth.

Calling an operation on the client

A hook is used to call operations from React useAction along with the generated object api:

import { useAction } from "convex/react";
import { api } from "../convex/_generated/api";

export function MyApp() {
  const performMyAction = useAction(api.myFunctions.doSomething);
  const handleClick = () => {
    performMyAction({ a: 1 });
  };
  // Вешаем `handleClick` на кнопку
  // ...
}

Unlike mutations, operations called on the same client are executed in parallel. Each operation is performed upon reaching the server. Consistent execution of operations is the task of the developer.

Note that in most cases calling an operation directly on the client is an anti-pattern. Instead, operations should be scheduled by mutations:

import { v } from "convex/values";
import { internal } from "./_generated/api";
import { internalAction, mutation } from "./_generated/server";

export const mutationThatSchedulesAction = mutation({
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    const taskId = await ctx.db.insert("tasks", { text });
    // Планируем выполнение операции
    await ctx.scheduler.runAfter(0, internal.myFunctions.actionThatCallsAPI, {
      taskId,
      text,
    });
  },
});

export const actionThatCallsAPI = internalAction({
  args: { taskId: v.id("tasks"), text: v.string() },
  handler: (_, args): void => {
    // Работаем с `taskId` и `text`, например, обращаемся к апи
    // и запускаем другую мутацию для сохранения результата
  },
});

Limits

The operation timeout is 10 minutes. Memory limits are 512 and 64 MB for Node.js and the Convex runtime, respectively.

Operations can perform up to 1000 simultaneous operations such as query execution, mutation or fetch().

Error Handling

Unlike queries and mutations, operations can have side effects and are therefore not re-executed by Convex when errors occur. It is the developer's responsibility to handle such errors and retry the operation.

Dangling Promises

Make sure all promises in operations are awaited. Dangling promises can lead to subtle bugs.

❯ HTTP operations / HTTP actions

HTTP operations allow you to create HTTP APIs directly in Convex.

HTTP operations accept Request and return Response from Fetch API. HTTP operations can manipulate the request and response directly and interact with data in Convex through queries, mutations, and operations. HTTP operations can be used to receive webhooks from third-party applications or to define public HTTP APIs.

HTTP operations are provided via https://<ваш-урл>.convex.site (For example, https://happy-animal-123.convex.site).

HTTP Operation Definition

HTTP operation handlers are defined using a constructor httpAction, similar to a constructor action for normal operations:

import { httpAction } from "./_generated/server";

export const doSomething = httpAction(async () => {
  // Логика операции
  return new Response();
});

First parameter handler() - object ActionCtx, providing auth, storage And scheduler, and also runQuery(), runMutation() And runAction().

The second parameter contains the request details. HTTP operations do not support argument validation; parsing the arguments from the incoming request is the developer's job.

Example:

import { httpAction } from "./_generated/server";
import { internal } from "./_generated/api";

export const postMessage = httpAction(async (ctx, request) => {
  const { author, body } = await request.json();

  await ctx.runMutation(internal.messages.sendOne, {
    body: `Отправлено с помощью операции HTTP: ${body}`,
    author,
  });

  return new Response(null, {
    status: 200,
  });
});

To create an HTTP operation from a file convex/http.ts|js instance should be exported by default HttpRouter. To create it, use the function httpRouter. Routes are determined using the method route:

// convex/http.ts
import { httpRouter } from "convex/server";
import { postMessage, getByAuthor, getByAuthorPathSuffix } from "./messages";

const http = httpRouter();

http.route({
  path: "/postMessage",
  method: "POST",
  handler: postMessage,
});

// Дополнительные роуты
http.route({
  path: "/getMessagesByAuthor",
  method: "GET",
  handler: getByAuthor,
});

// Определение роута с помощью префикса пути
http.route({
  // Будет совпадать с /getAuthorMessages/User+123, /getAuthorMessages/User+234 и т.п.
  pathPrefix: "/getAuthorMessages/",
  method: "GET",
  handler: getByAuthorPathSuffix,
});

// Роутер должен экспортироваться по умолчанию
export default http;

The operation can then be called via HTTP and interact with data stored in the Convex database:

export DEPLOYMENT_NAME="happy-animal-123"
curl -d '{ "author": "User 123", "body": "Hello world" }' \
  -H 'content-type: application/json' "https://$DEPLOYMENT_NAME.convex.site/postMessage"

Limits

HTTP operations run in the same environment as requests and mutations, so they do not have access to Node.js APIs. However, they can call operations that can be performed in Node.js.

HTTP operations can have side effects and are not re-executed by Convex when errors occur. Handling such errors and re-executing the HTTP operation is the developer's responsibility.

Request and response sizes are limited to 20 MB.

Types of supported request and response bodies: .text(), .json(), .blob() And .arrayBuffer().

Popular patterns

File storage

HTTP operations can be used to download and retrieve files.

CORS

HTTP operations must contain headers CORS to receive requests from the client:

import { httpRouter } from "convex/server";
import { httpAction } from "./_generated/server";
import { api } from "./_generated/api";
import { Id } from "./_generated/dataModel";

const http = httpRouter();

http.route({
  path: "/sendImage",
  method: "POST",
  handler: httpAction(async (ctx, request) => {
    // Шаг 1: сохраняем файл
    const blob = await request.blob();
    const storageId = await ctx.storage.store(blob);

    // Шаг 2: сохраняем идентификатор файла в БД с помощью мутации
    const author = new URL(request.url).searchParams.get("author");
    await ctx.runMutation(api.messages.sendImage, { storageId, author });

    // Шаг 3: возвращаем ответ с правильными заголовками CORS
    return new Response(null, {
      status: 200,
      // Заголовки CORS
      headers: new Headers({
        // Например, https://mywebsite.com (настраивается с помощью панели управления Convex)
        "Access-Control-Allow-Origin": process.env.CLIENT_ORIGIN!,
        Vary: "origin",
      }),
    });
  }),
});

export default http;

Example of processing a preliminary request OPTIONS:

// Предварительный запрос для /sendImage
http.route({
  path: "/sendImage",
  method: "OPTIONS",
  handler: httpAction(async (_, request) => {
    // Проверяем наличие необходимых заголовков
    const headers = request.headers;
    if (
      headers.get("Origin") !== null &&
      headers.get("Access-Control-Request-Method") !== null &&
      headers.get("Access-Control-Request-Headers") !== null
    ) {
      return new Response(null, {
        headers: new Headers({
          "Access-Control-Allow-Origin": process.env.CLIENT_ORIGIN!,
          "Access-Control-Allow-Methods": "POST",
          "Access-Control-Allow-Headers": "Content-Type, Digest",
          "Access-Control-Max-Age": "86400",
        }),
      });
    } else {
      return new Response();
    }
  }),
});

Authentication

Authenticated user details can be obtained using ctx.auth.getUserIdentity(). Then tokenIdentifier can be added to the title Authorization:

const jwtToken = "...";

fetch("https://happy-animal-123.convex.site/myAction", {
  headers: {
    Authorization: `Bearer ${jwtToken}`,
  },
});

❯ Internal functions

Internal functions can only be called by other functions, i.e. cannot be called directly by the Convex client.

By default, all Convex functions are public and available to clients. Public functions can be called by attackers in ways that lead to "interesting" results. Internal features help reduce this risk. It is recommended to always use such functions when writing logic that should not be accessible to the client.

Internal functions can use argument validation and/or authentication.

Use Cases

Internal functions are for:

• call from operations using runQuery() And runMutation()
• call from HTTP operations using runQuery(), runMutation() And runAction()
• planning their launch in the future in other functions
• scheduling them to run periodically in cron tasks
• launch using the control panel
• launch using CLI

Defining an internal function

An internal function is defined using constructors internalQuery, internalMutation or internalAction. For example:

import { internalMutation } from "./_generated/server";
import { v } from "convex/values";

export const markPlanAsProfessional = internalMutation({
  args: { planId: v.id("plans") },
  handler: async (ctx, args) => {
    await ctx.db.patch(args.planId, { planType: "professional" });
  },
});

()
If you are passing a complex object to an internal function, you may not need to use argument validation. However, please note that when using internalQuery() or internalMutation()It's a good idea to pass document IDs instead of documents to ensure that the query or mutation will work with the actual state of the database.

Calling an internal function

Internal functions can be called from and scheduled within operations and mutations using an object internal.

Example of a public operation upgrade, causing internal mutation plans.markPlanAsProfessional, defined above:

import { action } from "./_generated/server";
import { internal } from "./_generated/api";
import { v } from "convex/values";

export const upgrade = action({
  args: {
    planId: v.id("plans"),
  },
  handler: async (ctx, args) => {
    // Обращаемся к платежной системе
    const response = await fetch("https://...");
    if (response.ok) {
      // Обновляем план на "professional" в БД Convex
      await ctx.runMutation(internal.plans.markPlanAsProfessional, {
        planId: args.planId,
      });
    }
  },
});

In the example given, the user should not be able to directly call internal.plans.markPlanAsProfessional().

Public and internal functions can be defined in the same file.

❯ Validation of arguments and return values

Argument and return value validators help ensure that queries, mutations, and operations are called with arguments and return values ​​of the correct types.

This is important for safety. Without argument validation, an attacker can call your public functions with any arguments, which can lead to adverse consequences. TS won't help because it's missing at runtime. It is recommended that argument validation be added to all production applications. For non-public functions that are not called by the client, it is recommended to use internal functions with optional validation.

Adding a validator

To add argument validation you need to pass an object with properties args And handler to constructors query, mutation or action. To validate the returned values, the property is used returns this object:

import { mutation, query } from "./_generated/server";
import { v } from "convex/values";

export const send = mutation({
  // Валидация аргументов
  args: {
    body: v.string(),
    author: v.string(),
  },
  // Валидация возвращаемого значения
  returns: v.null(),
  handler: async (ctx, args) => {
    const { body, author } = args;
    await ctx.db.insert("messages", { body, author });
  },
});

When using validators, value types are inferred automatically.

Unlike TS, object validation throws an exception if the object contains properties that are not specified in the validator.

Validator args: {} can also be useful as TS will show an error on the client when trying to pass arguments to a parameterless function.

Supported Types

All functions, both public and internal, support the following data types. Each type has a corresponding validator, which is accessible through the object v, imported from "convex/values".

The database can store the same types of data.

In addition, you can define unions, literals, type any and optional fields.

Convex values

Convex supports the following value types:

Associations

Unions are defined using v.union():

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export default mutation({
  args: {
    stringOrNumber: v.union(v.string(), v.number()),
  },
  handler: async ({ db }, { stringOrNumber }) => {
    //...
  },
});

Literals

Literals are defined using v.literal(). They are usually used in combination with joins:

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export default mutation({
  args: {
    oneTwoOrThree: v.union(
      v.literal("one"),
      v.literal("two"),
      v.literal("three"),
    ),
  },
  handler: async ({ db }, { oneTwoOrThree }) => {
    //...
  },
});

Any

Fields that can contain any value are defined using v.any():

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export default mutation({
  args: {
    anyValue: v.any(),
  },
  handler: async ({ db }, { anyValue }) => {
    //...
  },
});

This matches the type any in TS.

Optional fields

Optional fields are defined using v.optional():

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export default mutation({
  args: {
    optionalString: v.optional(v.string()),
    optionalNumber: v.optional(v.number()),
  },
  handler: async ({ db }, { optionalString, optionalNumber }) => {
    //...
  },
});

This matches the modifier ? in TS.

Extracting TS Types

Type Infer allows you to convert a Convex validator to a TS type. This avoids duplication:

import { mutation } from "./_generated/server";
import { Infer, v } from "convex/values";

const nestedObject = v.object({
  property: v.string(),
});

// Разрешается в `{ property: string }`.
export type NestedObject = Infer<typeof nestedObject>;

export default mutation({
  args: {
    nested: nestedObject,
  },
  handler: async ({ db }, { nested }) => {
    //...
  },
});

❯ Error handling

There are 4 reasons why errors can occur in queries and mutations:

1. Application errors: the function code reached a logical condition to stop further execution and was thrown away ConvexError.
2. Developer errors: a bug in a function (for example, calling db.get(null) instead of db.get(id)).
3. Read/Write Limit Errors: The function is trying to retrieve or write too much data.
4. Convex internal errors: There is a problem within Convex.

Convex automatically handles internal errors. In such cases, queries and mutations are executed repeatedly until they complete successfully.

Handling other types of errors is the developer's job. Best practices:

1. Show the user the corresponding UI.
2. Send the error to the appropriate service.
3. Print the error to the console and configure report streaming.

Additionally, client errors can be sent to services like Sentry, for more information for debugging and monitoring.

Errors in requests

If an error occurs while executing a request, it is sent to the client and thrown at the point where the hook is called useQuery. The best way to handle such errors is with error boundaries.

The fuse allows you to intercept errors thrown in child components, render a fallback UI, and send information to a special service. Sentry even provides a special component Sentry.ErrorBoundary.

The more fuses used in an application, the more granular the fallback UI will be. The simplest thing is to wrap the entire application in one fuse:

<StrictMode>
  <ErrorBoundary>
    <ConvexProvider client={convex}>
      <App />
    </ConvexProvider>
  </ErrorBoundary>
</StrictMode>

However, with this approach, an error in any child component will cause the entire application to crash. Therefore, it is better to wrap individual parts of the application in fuses. Then, if an error occurs in one component, the others will function normally.

Unlike other frameworks, queries in Convex are not re-executed when an error occurs: queries are deterministic, so re-executing them with the same arguments will always result in the same errors.

Errors in mutations

An error in mutation results in the following:

1. The promise returned from the mutation is rejected.
2. The optimistic update is being rolled back.

Sentry should automatically report "unhandled promise rejection". In this case, no additional mutation error handling is required.

Note that errors in mutations are not caught by fuses, since such errors are not part of component rendering.

To render a fallback UI if the mutation fails, you can use .catch() after calling mutation:

sendMessage(newMessageText).catch((error) => {
  // Работаем с `error`
});

In an asynchronous handler you can use try...catch:

try {
  await sendMessage(newMessageText);
} catch (error) {
  // Работаем с `error`
}

Errors in operations

Unlike queries and mutations, operations can have side effects, so they are not re-executed by Convex if errors occur. Handling such errors is the developer's task.

Differences between bug reporting in development and production modes

In development mode, any server error thrown on the client will include the original error message and the server stack trace to aid debugging.

In production mode, the server error will only include the function name and a general message "Server Error" no stack trace. Application server errors will contain custom data (in the data).

Full error reports for both modes can be found on the "Logs" page of a specific deployment.

Application errors, expected failures

If there are expected failures, the function may return other values ​​or throw ConvexError. We'll talk about this in the next section.

Read/Write Limit Errors

To ensure high performance, Convex rejects queries and mutations that attempt to read or write too much data.

Requests and mutations are rejected in the following cases:

• more than 16_384 documents are scanned
• more than 8 MB of data is scanned
• called more than 4_096 db.get() or db.query()
• Function JS code takes longer than 1 second to execute

In addition, mutations are rejected in the following cases:

• more than 8_192 documents are recorded
• more than 8 MB of data is recorded

Documents are "scanned" by the database to determine which documents are returned from db.query(). For example, db.query("table").take(5).collect() scans only 5 documents, and db.query("table").filter(...).first() scans all documents contained in the table table to determine the first document that matches the filter.

Number of calls db.get() And db.query() limited to prevent the query from subscribing to too many index ranges.

If these limits are frequently reached, it is recommended to index queries to reduce the number of documents scanned, thereby avoiding unnecessary database reads.

Application errors

If there are expected failures, the function may return other values ​​or throw ConvexError.

Returning other values

When using TS, a different return type may indicate an error handling scenario. For example, mutation createUser() may return:

Id<"users"> | { error: "EMAIL_ADDRESS_IN_USE" };

This allows you to remember about the need to handle errors in the UI.

Throwing Application Errors

Throwing an exception may be preferable for the following reasons:

• you can use the built-in mechanism for bubbling exceptions from deeply nested function calls instead of manually pushing the error up the call stack. This also works for calls runQuery(), runMutation() And runAction() in operations
• throwing an exception in a mutation prevents its transaction from committing
• It might be easier to handle all types of errors equally on the client

Convex provides an error subclass ConvexError to transfer information from the server to the client:

import { ConvexError } from "convex/values";
import { mutation } from "./_generated/server";

export const assignRole = mutation({
  args: {
    // ...
  },
  handler: (ctx, args) => {
    const isTaken = isRoleTaken(/* ... */);
    if (isTaken) {
      throw new ConvexError("Роль уже назначена");
    }
    // ...
  },
});

Payload data

Constructor ConvexError accepts all data types supported by Convex. Data is written to the error property data:

// error.data === "Сообщение об ошибке"
throw new ConvexError("Сообщение об ошибке");

// error.data === {message: "Сообщение об ошибке", code: 123, severity: "high"}
throw new ConvexError({
  message: "Сообщение об ошибке",
  code: 123,
  severity: "high",
});

// error.data === {code: 123, severity: "high"}
throw new ConvexError({
  code: 123,
  severity: "high",
});

Error payload more complex than string, useful for more structured error reporting, as well as for handling different types of errors on the client.

Handling application errors on the client

On the client, application errors are also used ConvexError. The error payload is contained in the property data:

import { ConvexError } from "convex/values";
import { useMutation } from "convex/react";
import { api } from "../convex/_generated/api";

export function MyApp() {
  const doSomething = useMutation(api.myFunctions.mutateSomething);
  const handleSomething = async () => {
    try {
      await doSomething({ a: 1, b: 2 });
    } catch (error) {
      const errorMessage =
        // Проверяем, что имеем дело с ошибкой приложения
        error instanceof ConvexError
          ? // Получаем доступ к данным и приводим их к ожидаемому типу
            (error.data as { message: string }).message
          : // Вероятно, имеет место ошибка разработчика,
            // производственная среда не предоставляет
            // дополнительной информации клиенту
            "Возникла неожиданная ошибка";
      // Работаем с `errorMessage`
    }
  };
  // ...
}

❯ Runtime environments

Convex functions can run in two environments:

• default Convex environment
• optional Node.js environment

Convex Default Environment

All Convex server functions are written in JS or TS. By default they run in a custom JS environment, much like Cloudflare Workers environment, with access to most global variables defined by web standards.

A default environment has many benefits, including the following:

• no cold start. The environment is always running and ready to immediately perform functions
• latest JS features. The environment is based on the V8 engine from Google Chrome. This provides an interface very similar to the client code, helping to keep the code as simple as possible.
• low costs for data access. The framework is designed for low data access costs through queries and mutations, allowing you to access the database using a simple JS interface

Query and Mutation Limits

Queries and mutations are constrained by the environment to ensure they are deterministic. This allows Convex to re-run them automatically if necessary.

Determinism means that a function that is called with the same arguments will always return the same values.

Convex provides helpful error messages when writing "illegal" functions.

Using arbitrary values ​​and times in queries and mutations

Convex provides a seeded pseudo-random number generator Math.random(), guaranteeing the determinism of functions. Generator padding is a hidden parameter of the function. Multiple calls Math.random() one function will return different arbitrary values. Note that Convex does not re-evaluate JS modules every time the function is run, so the result of the call Math.random(), stored in a global variable, will not change between function calls.

To ensure reproducibility of function logic, system time used globally (outside of any function) is “frozen” at deployment time. The system time in the function is "frozen" at the beginning of its execution. Date.now() will return the same result throughout the entire execution of the function.

const globalRand = Math.random(); // `globalRand` не меняется между запусками
const globalNow = Date.now(); // `globalNow` - это время деплоя функций

export const updateSomething = mutation({
  handler: () => {
    const now1 = Date.now(); // `now1` - время начала выполнения функции
    const rand1 = Math.random(); // `rand1` имеет новое значения при каждом запуске функции
    // implementation
    const now2 = Date.now(); // `now2` === `now1`
    const rand2 = Math.random(); // `rand1` !== `rand2`
  },
});

Operations

Operations are not limited by rules that ensure determinism of functions. They can access third party HTTP endpoints using the standard function fetch.

By default, operations are also performed in a custom JS environment. They can be defined in the same file with queries and mutations.

Node.js environment

Some JS/TS libraries are not supported by the default Convex environment. Therefore Convex allows you to switch to Node.js 18 using the directive "use node" at the beginning of the corresponding file.

Only operations can be performed in Node.js. To interact the library for Node.js and the Convex database, you can use utilities runQuery or runMutation to call a query or mutation, respectively.

❯ Database

The Convex database provides a relational data model that stores JSON-like documents that can be used with or without a schema. It "just works", delivering predictable performance through an easy-to-use interface.

Queries and mutations read and write data through a lightweight JS interface. You don't need to configure anything, you don't need to write SQL.

❯ Tables and documents

Tables

The Convex deployment contains tables that store data. Initially, the deployment does not contain any tables or data.

The table is created when the first document is added to it:

// Таблица `friends` не существует
await ctx.db.insert("friends", { name: "Алекс" });
// Теперь она существует и содержит один документ

There is no need to explicitly define a schema or create tables.

Documents

Tables contain documents. Documents are very similar to JS objects. They contain fields and values, and can contain nested arrays and objects.

Examples of valid Convex documents:

{}
{"name": "Алекс"}
{"name": {"first": "Иван", "second": "Петров"}, "age": 34}

Documents can also contain links to documents in other tables. We'll talk about this in one of the following sections.

❯ Reading data

Queries and mutations can read data from database tables using document ids and document queries.

Reading one document

Method db.get allows you to read a document by its ID:

import { query } from "./_generated/server";
import { v } from "convex/values";

export const getTask = query({
  args: { taskId: v.id("tasks") },
  handler: async (ctx, args) => {
    const task = await ctx.db.get(args.taskId);
    // Работаем с `task`
  },
});

A validator should be used to limit the data retrieved from a table v.id.

Request for documents

Document queries always begin by selecting a table using the method db.query:

import { query } from "./_generated/server";

export const listTasks = query({
  handler: async (ctx) => {
    const tasks = await ctx.db.query("tasks").collect();
    // Работаем с `tasks`
  },
});

Then we can:

• filter
• sort
• and wait (await) results

Filtration

Method filter allows you to filter documents returned by a query. This method accepts a filter created using FilterBuilder, and selects only matching documents.

To filter documents that contain specific keywords, you should use a search query, which we'll talk about later.

Equality Test

The following query searches for documents in a table users, Where doc.name === "Алекс":

// Возвращает всех пользователей с именем "Алекс"
const usersNamedAlex = await ctx.db
  .query("users")
  .filter((q) => q.eq(q.field("name"), "Алекс"))
  .collect();

q is an auxiliary object FilterBuilder. It contains methods for all supported filtering operators.

This filter runs on all documents in the table. For each document q.field("name") is assessed as a property name. Then q.eq() checks whether this property is equal to "Алекс".

If the request references a field that is not in the document, it returns undefined.

Comparisons

Filters can also be used to compare values. The following query searches for documents where doc.age >= 18:

// Возвращает всех пользователей, старше 18 лет
const adults = await ctx.db
  .query("users")
  .filter((q) => q.gte(q.field("age"), 18))
  .collect();

Operator q.gte checks that the first argument (doc.age) is greater than or equal to the second (18).

Full list of comparison operators:

Arithmetic

Queries can contain simple arithmetic. The following query searches for documents in a table carpets, Where doc.height * doc.width > 100:

// Возвращает все ковры, площадью свыше 100
const largeCarpets = await ctx.db
  .query("carpets")
  .filter((q) => q.gt(q.mul(q.field("height"), q.field("width")), 100))
  .collect();

Complete list of arithmetic operators:

Combining Operators

Complex filters can be created using q.and(), q.or() And q.not(). The following query searches for documents where doc.name === "Алекс" && doc.age >= 18:

// Возвращает всех пользователей по имени "Алекс", старше 18 лет
const adultAlexes = await ctx.db
  .query("users")
  .filter((q) =>
    q.and(q.eq(q.field("name"), "Алекс"), q.gte(q.field("age"), 18)),
  )
  .collect();

The following query searches for documents where doc.name === "Алекс" || doc.name === "Вера":

// Возвращает всех пользователей по имени "Алекс" или "Вера"
const usersNamedAlexOrEmma = await ctx.db
  .query("users")
  .filter((q) =>
    q.or(q.eq(q.field("name"), "Алекс"), q.eq(q.field("name"), "Вера")),
  )
  .collect();

Order

By default, Convex returns documents sorted by field _creationTime (time of creation).

To select the sort order, use .ord("asc" | "desc"). By default, documents are sorted in ascending order (asc).

// Возвращает все сообщения, от старых к новым
const messages = await ctx.db.query("messages").order("asc").collect();

// Возвращает все сообщения, от новым к старым
const messages = await ctx.db.query("messages").order("desc").collect();

If you need to sort documents by another field and a small number of documents, sorting can be done in JS:

// Возвращает 10 лучших (по количеству лайков) сообщений
// (предполагается, что таблица "messages" маленькая)
const messages = await ctx.db.query("messages").collect();
const topTenMostLikedMessages = messages
  .sort((a, b) => b.likes - a.likes)
  .slice(0, 10);

For queries that return a large number of documents, indexes should be used to improve performance. Documents returned by queries using indexes are sorted by the columns specified in the indexes, avoiding slow table scans:

// Возвращает 20 лучших (по количеству лайков) сообщений с помощью индекса "by_likes"
const messages = await ctx.db
  .query("messages")
  .withIndex("by_likes")
  .order("desc")
  .take(20);

Sorting values ​​of different types

One field can contain values ​​of different types. If an indexed field contains values ​​of different types, the sort order will be as follows: undefined < null < bigint < number < boolean < string < ArrayBuffer < Array < Object.

The same sort order is used by comparison operators q.lt(), q.lte(), q.gt() And q.gte().

Retrieving Results

In most of the examples above, the requests ended with a method call collect, which returns all documents that match the filter. There are other options.

Return n results

.take(n) returns n results matching the query:

// Возвращает 5 первых пользователей
const users = await ctx.db.query("users").take(5);

Return first result

.first() returns the first document matching the request, or null:

// Возвращает первого пользователя с указанным email
const userOrNull = await ctx.db
  .query("users")
  .filter((q) => q.eq(q.field("email"), "test@example.com"))
  .first();

Returning a unique result

.unique() returns the first document matching the request, or null. If there are multiple documents matching the request, an exception is thrown:

// Предполагается, что таблица "counter" содержит только один документ
const counterOrNull = await ctx.db.query("counter").unique();

Loading results page by page

.paginate(opts) loads the results page and returns Cursor to download additional results. We'll talk about this later.

Advanced queries

Convex doesn't have a dedicated query language for complex logic like joins, aggregations, and groupings. This logic is implemented using JS. Convex guarantees consistent results.

Association

// join
import { query } from "./_generated/server";
import { v } from "convex/values";

export const eventAttendees = query({
  args: { eventId: v.id("events") },
  handler: async (ctx, args) => {
    const event = await ctx.db.get(args.eventId);
    return Promise.all(
      (event?.attendeeIds ?? []).map((userId) => ctx.db.get(userId)),
    );
  },
});

Aggregation

// aggregation
import { query } from "./_generated/server";
import { v } from "convex/values";

export const averagePurchasePrice = query({
  args: { email: v.string() },
  handler: async (ctx, args) => {
    const userPurchases = await ctx.db
      .query("purchases")
      .filter((q) => q.eq(q.field("buyer"), args.email))
      .collect();
    const sum = userPurchases.reduce((a, { value: b }) => a + b, 0);
    return sum / userPurchases.length;
  },
});

Grouping

// group by
import { query } from "./_generated/server";
import { v } from "convex/values";

export const numPurchasesPerBuyer = query({
  args: { email: v.string() },
  handler: async (ctx, args) => {
    const userPurchases = await ctx.db.query("purchases").collect();

    return userPurchases.reduce(
      (counts, { buyer }) => ({
        ...counts,
        [buyer]: counts[buyer] ?? 0 + 1,
      }),
      {} as Record<string, number>,
    );
  },
});

❯ Data recording

Mutations can add, update and delete documents from database tables.

Adding new documents

To create new documents, use the method db.insert:

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const createTask = mutation({
  args: { text: v.string() },
  handler: async (ctx, args) => {
    const taskId = await ctx.db.insert("tasks", { text: args.text });
    // Работаем с `taskId`
  },
});

Second parameter db.insert() — a JS object with the data of the new document.

Method insert returns the globally unique ID of the created document.

Updating existing documents

To update an existing document, use its ID and one of the following methods:

1. Method db.patch partially updates the document, superficially combining it with new data. New fields are added. Existing fields are overwritten. Fields with value undefined are deleted.

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const updateTask = mutation({
  args: { id: v.id("tasks") },
  handler: async (ctx, args) => {
    const { id } = args;
    console.log(await ctx.db.get(id));
    // { text: "foo", status: { done: true }, _id: ... }

    // Добавляем `tag` и перезаписываем `status`
    await ctx.db.patch(id, { tag: "bar", status: { archived: true } });
    console.log(await ctx.db.get(id));
    // { text: "foo", tag: "bar", status: { archived: true }, _id: ... }

    // Удаляем `tag` путем установки его значения в `undefined`
    await ctx.db.patch(id, { tag: undefined });
    console.log(await ctx.db.get(id));
    // { text: "foo", status: { archived: true }, _id: ... }
  },
});

1. Method db.replace replaces the document completely, potentially removing existing fields:

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const replaceTask = mutation({
  args: { id: v.id("tasks") },
  handler: async (ctx, args) => {
    const { id } = args;
    console.log(await ctx.db.get(id));
    // { text: "foo", _id: ... }

    // Полностью заменяем документ
    await ctx.db.replace(id, { invalid: true });
    console.log(await ctx.db.get(id));
    // { invalid: true, _id: ... }
  },
});

Deleting documents

To delete a document, use its ID and method db.delete:

import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const deleteTask = mutation({
  args: { id: v.id("tasks") },
  handler: async (ctx, args) => {
    await ctx.db.delete(args.id);
  },
});

❯ Document ID

Each document in Convex has a globally unique string ID, which is automatically generated by the system:

const userId = await ctx.db.insert("users", { name: "Михаил Лермонтов" });

This ID can be used to read the document:

const retrievedUser = await ctx.db.get(userId);

The ID is stored in the field _id:

const userId = retrievedUser._id;

The ID can also be used to update a document in place:

await ctx.db.patch(userId, { name: "Федор Достоевский" });

Convex generates type Id for TS based on a schema that is parameterized by table names:

import { Id } from "./_generated/dataModel";

const userId: Id<"users"> = user._id;

The IDs are a string at runtime, but the type Id can be used to separate IDs from other strings at compile time.

Links and relationships

In Convex, you can link to a document by simply adding it Id to another document:

await ctx.db.insert("books", {
  title,
  ownerId: user._id,
});

Links allow you to receive documents:

const user = await ctx.db.get(book.ownerId);

And request documents:

const myBooks = await ctx.db
  .query("books")
  .filter((q) => q.eq(q.field("ownerId"), user._id))
  .collect();

Id how links enable the creation of complex data models.

Although Convex supports nested objects and arrays, documents must be relatively small in size. To structure data, it is better to create separate tables, documents and links.

ID Serialization

IDs are strings that can be easily inserted into URLs and stored outside of Convex.

You can pass an ID string from an external source (such as a URL) to the Convex function and get the corresponding object. When using TS on the client, you can cast a string to type Id.

import { useQuery } from "convex/react";
import { Id } from "../convex/_generated/dataModel";
import { api } from "../convex/_generated/api";

export function App() {
  const id = localStorage.getItem("myIDStorage");
  const task = useQuery(api.tasks.getTask, { taskId: id as Id<"tasks"> });
  // ...
}

Because this ID is retrieved from an external source, you must use an argument validator or method ctx.db.normalizeId to confirm that the ID belongs to the specified table before returning the document:

import { query } from "./_generated/server";
import { v } from "convex/values";

export const getTask = query({
  args: {
    taskId: v.id("tasks"),
  },
  handler: async (ctx, args) => {
    const task = await ctx.db.get(args.taskId);
    // ...
  },
});

❯ Data types

All Convex documents are defined as JS objects. The field values ​​of these objects can be of any supported type (see the section on validating arguments and return values).

The shape of documents can be determined using a schema, which we will talk about in the next section.

System fields

Each Convex document has two automatically generated system fields:

• _id: Document ID (see previous section)
• _creationTime: document creation time (number of ms since epoch)

Limits

The maximum value size is 1 MB. The maximum nesting of fields is 16 levels.

Table names must consist of Latin characters, numbers and underscores (_), but cannot begin with the last one (except for system fields).

Working with dates and times

Convex does not provide a special data type for working with date and time. They are stored and retrieved from the database as strings.

❯ Schemes

A diagram is a description

1. Project tables.
2. Type of documents in tables.

Although a schema is not required, defining one ensures that documents in tables are of the correct type. Adding a schema also makes the code more type safe.

Create a schema

The schema is defined in the file convex/schema.ts and looks like this:

import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  messages: defineTable({
    body: v.string(),
    user: v.id("users"),
  }),
  users: defineTable({
    name: v.string(),
    tokenIdentifier: v.string(),
  }).index("by_token", ["tokenIdentifier"]),
});

This schema contains 2 tables: messages And users. Tables are defined using the function defineTable. The document type is determined using a validator v. In addition to the specified fields, Convex automatically adds fields _id And _creationTime.

Validators

The document type is determined using a validator v:

import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  documents: defineTable({
    id: v.id("documents"),
    string: v.string(),
    number: v.number(),
    boolean: v.boolean(),
    nestedObject: v.object({
      property: v.string(),
    }),
  }),
});

v also allows you to define unions, optional properties, string literals, etc.

Optional fields

Optional fields are created using v.optional():

defineTable({
  optionalString: v.optional(v.string()),
  optionalNumber: v.optional(v.number()),
});

This matches the modifier ? in TS.

Associations

Unions are created using v.union():

defineTable({
  stringOrNumber: v.union(v.string(), v.number()),
});

v.union() can be used at the top level (if the table can contain different types of documents):

defineTable(
  v.union(
    v.object({
      kind: v.literal("StringDocument"),
      value: v.string(),
    }),
    v.object({
      kind: v.literal("NumberDocument"),
      value: v.number(),
    }),
  ),
);

Literals

Literals are created using v.literal(). This method is usually used in combination with v.union():

defineTable({
  oneTwoOrThree: v.union(
    v.literal("one"),
    v.literal("two"),
    v.literal("three"),
  ),
});

Any

Fields that can contain a value of any type are defined using v.any():

defineTable({
  anyValue: v.any(),
});

This matches the type any in TS.

Settings

As the second parameter defineTable() accepts an object with settings.

schemaValidation: boolean

This setting determines whether Convex should check that new and existing documents conform to the schema. By default, this check is performed.

You can disable this check like this:

defineSchema(
  {
    // Таблицы
  },
  {
    schemaValidation: false,
  },
);

TS types are generated anyway.

strictTableNameTypes: boolean

This setting determines whether the TS allows you to work with tables not specified in the schema. By default, TS disables this.

You can disable this setting like this:

defineSchema(
  {
    // Таблицы
  },
  {
    strictTableNameTypes: false,
  },
);

The type of documents from tables not specified in the schema is any.

Schema Validation

Circuits are automatically “puffed” when commands are executed npx convex dev And npx convex deploy.

The first push after adding or modifying a schema checks all documents for compliance with the schema. If validation fails, the scheme is not pushed.

Please note that only documents from tables specified in the schema are validated.

Circular links

Consider this example:

import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  users: defineTable({
    preferencesId: v.id("preferences"),
  }),
  preferences: defineTable({
    userId: v.id("users"),
  }),
});

In this schema the documents are in the table users contain links to documents in the table preferences, and vice versa.

It is not possible to create such links in Convex.

The simplest way to solve this problem is to make one of the links "null":

import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  users: defineTable({
    preferencesId: v.id("preferences"),
  }),
  preferences: defineTable({
    userId: v.union(v.id("users"), v.null()),
  }),
});

After this, the document is first created preferences, then the document is created users, then a link to the document is set preferences:

import { mutation } from "./_generated/server";

export default mutation(async (ctx) => {
  const preferencesId = await ctx.db.insert("preferences", {});
  const userId = await ctx.db.insert("users", { preferencesId });
  await ctx.db.patch(preferencesId, { userId });
});

TS types

After defining the schema, the command npx convex dev generates new files dataModel.d.ts And server.d.ts with schema-based types.

Doc<TableName>

Type TS Doc from dataModel.d.ts provides document types for all tables. It can be used in both Convex functions and React components:

import { Doc } from "../convex/_generated/dataModel";

function MessageView(props: { message: Doc<"messages"> }) {
  // ...
}

To extract part of a document type, you can use a utility like Infer.

query And mutation

Functions query And mutation have the same api, but provide db with more precise types. Functions like db.insert understand the diagram. The queries return the correct document types (not any).

❯ Pagination

Paginated queries are queries that return a page-by-page list of results.

This can be useful when developing components with a "Load More" button or infinite scrolling.

Pagination in Convex involves:

1. Writing a paginated query that calls paginate(paginationOpts).
2. Using a Hook usePaginatedQuery.

Please note: paginated queries are currently an experimental feature.

Creating a paginated request

Convex uses cursor-based pagination. This means that paginated queries return the string Cursor, which represents the end of the current page's results. To load additional results, the query is called again, but with a cursor.

To do this, Convex provides a query that:

1. Takes a single argument - an object with a property paginationOpts type PaginationOptions.
   
   • PaginationOptions is an object with fields numItems And cursor
   • to validate this argument is used paginationOptsValidator from convex/server
   • object args may also contain other fields
2. Calls paginate(paginationOpts) on the request and returns the result.
   
   • Returnable page (page) in PaginationResult is an array of documents.

import { v } from "convex/values";
import { query, mutation } from "./_generated/server";
import { paginationOptsValidator } from "convex/server";

export const list = query({
  args: { paginationOpts: paginationOptsValidator },
  handler: async (ctx, args) => {
    const foo = await ctx.db
      .query("messages")
      .order("desc")
      .paginate(args.paginationOpts);
    return foo;
  },
});

Additional Arguments

Except paginationOpts, object args may contain other fields:

export const listWithExtraArg = query({
  args: { paginationOpts: paginationOptsValidator, author: v.string() },
  handler: async (ctx, args) => {
    return await ctx.db
      .query("messages")
      .filter((q) => q.eq(q.field("author"), args.author))
      .order("desc")
      .paginate(args.paginationOpts);
  },
});

Converting Results

Property page the object returned paginate(), may be subject to additional transformations:

export const listWithTransformation = query({
  args: { paginationOpts: paginationOptsValidator },
  handler: async (ctx, args) => {
    const results = await ctx.db
      .query("messages")
      .order("desc")
      .paginate(args.paginationOpts);

    return {
      ...results,
      page: results.page.map((message) => ({
        author: message.author.slice(0, 1),
        body: message.body.toUpperCase(),
      })),
    };
  },
});

Client-side pagination

To get paginated results on the client, use a hook usePaginatedQuery. This hook provides a simple interface for rendering current documents and requesting additional ones. It controls the cursor automatically.

The hook accepts the following parameters:

• name of the paginated request
• object of arguments passed to the request, except paginationOpts (which are added by the hook itself)
• settings object with field initialNumItems to load the first page

The hook returns an object with the following fields:

• results - array of documents
• isLoading — results loading indicator
• status — pagination status:
  
  • "LoadingFirstPage" - hook loads the first page
  • "CanLoadMore" — additional documents are available. You can call the function loadMore to load next page
  • "LoadingMore" — loading another page
  • "Exhausted" — there are no documents left to download (the end of the list has been reached)
• loadMore(n) — function for loading additional results. Runs only when status === "CanLoadMore"

import { usePaginatedQuery } from "convex/react";
import { api } from "../convex/_generated/api";

export function App() {
  const { results, status, loadMore } = usePaginatedQuery(
    api.messages.list,
    {},
    { initialNumItems: 5 },
  );

  return (
    <div>
      {results?.map(({ _id, text }) => <div key={_id}>{text}</div>)}
      <button onClick={() => loadMore(5)} disabled={status !== "CanLoadMore"}>
        Загрузить еще
      </button>
    </div>
  );
}

Additional arguments can be passed to the request:

import { usePaginatedQuery } from "convex/react";
import { api } from "../convex/_generated/api";

export function App() {
  const { results, status, loadMore } = usePaginatedQuery(
    api.messages.listWithExtraArg,
    { author: "Алекс" },
    { initialNumItems: 5 },
  );

  return (
    <div>
      {results?.map(({ _id, text }) => <div key={_id}>{text}</div>)}
      <button onClick={() => loadMore(5)} disabled={status !== "CanLoadMore"}>
        Загрузить еще
      </button>
    </div>
  );
}

Reactivity

Like other Convex features, paginated queries are completely reactive. React components are automatically re-rendered when a paginated list is added, removed, or modified.

One consequence of this is that page sizes in Convex can change dynamically.

Manual pagination

Paginated queries can be used outside of React:

import { ConvexHttpClient } from "convex/browser";
import { api } from "../convex/_generated/api";

require("dotenv").config();

const client = new ConvexHttpClient(process.env.VITE_CONVEX_URL!);

/**
 * Выводит в консоль массив,
 * содержащий все сообщения из пагинированного запроса "list",
 * путем объединения страниц результатов в один массив
 */
async function getAllMessages() {
  let continueCursor = null;
  let isDone = false;
  let page;

  const results = [];

  while (!isDone) {
    ({ continueCursor, isDone, page } = await client.query(api.messages.list, {
      paginationOpts: { numItems: 5, cursor: continueCursor },
    }));
    console.log("получено", page.length);
    results.push(...page);
  }

  console.log(results);
}

getAllMessages();

❯ Indexes

Indexes are a data structure that can speed up queries by telling Convex how to organize documents. Indexes also allow you to change the sort order of documents in query results.

Index Definition

Indexes are defined as part of the schema. Each index consists of:

1. Titles
   
   • must be unique within the table
2. Ordered list of indexed fields
   
   • to specify a nested field, a path with a dot is used, for example, properties.name

To add an index to a table, use the method index:

import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

// Определяем таблицу "messages" с двумя индексами
export default defineSchema({
  messages: defineTable({
    channel: v.id("channels"),
    body: v.string(),
    user: v.id("users"),
  })
    .index("by_channel", ["channel"])
    .index("by_channel_user", ["channel", "user"]),
});

Index by_channel sorted by field channel in the diagram. Messages from one channel are sorted by field _creationTime, generated automatically by the system.

Index by_channel_user sorts messages from one channel first by userwho sent them, then by _creationTime.

Indexes are created when commands are executed npx convex dev And npx convex deploy.

Querying Documents Using Indexes

Request messages from channel, created 1-2 minutes ago, will look like this:

const messages = await ctx.db
  .query("messages")
  .withIndex("by_channel", (q) =>
    q
      .eq("channel", channel)
      .gt("_creationTime", Date.now() - 2 * 60000)
      .lt("_creationTime", Date.now() - 60000),
  )
  .collect();

Method withIndex determines which index to query and how to use it to retrieve documents. The first argument is the name of the index, the second is the index range expression. An index range expression is a description of which documents should be considered when executing a query.

The choice of index affects both the form of the query range expression and the order of the results returned. For example, when adding indexes by_channel And by_channel_user, you can get results from one channel, ordered as by _creationTime, and by user, respectively.

const messages = await ctx.db
  .query("messages")
  .withIndex("by_channel_user", (q) => q.eq("channel", channel))
  .collect();

The result of this query will be all messages from channel, sorted first by user, then by _creationTime.

const messages = await ctx.db
  .query("messages")
  .withIndex("by_channel_user", (q) =>
    q.eq("channel", channel).eq("user", user),
  )
  .collect();

The result of this request will be messages from channel, sent user, sorted by _creationTime.

An index range expression is always a chain of:

1. 0 or more equality expressions defined with .eq.
2. [optional] Expressions of the lower threshold defined using .gt or .gte.
3. [optional] Expressions of the upper threshold defined using .lt or .lte.

The fields must be iterated in index order.

Each equality expression must compare another indexed field, starting from the beginning and in order. The upper and lower bounds must follow the equality expressions and compare the next field.

// Не компилируется!
const messages = await ctx.db
  .query("messages")
  .withIndex("by_channel", (q) =>
    q
      .gt("_creationTime", Date.now() - 2 * 60000)
      .lt("_creationTime", Date.now() - 60000),
  )
  .collect();

This request is invalid because the index by_channel sorted by (channel, _creationTime) and this query range contains comparison by _creationTime without range limitation to one channel. Since the index is sorted first by channel, then by _creationTime, the index for searching messages from all channels created 1-2 minutes ago is useless.

Query performance is based on range specificity.

For example, in this request:

const messages = await ctx.db
  .query("messages")
  .withIndex("by_channel", (q) =>
    q
      .eq("channel", channel)
      .gt("_creationTime", Date.now() - 2 * 60000)
      .lt("_creationTime", Date.now() - 60000),
  )
  .collect();

performance will be based on the number of messages in channel, created 1-2 minutes ago.

If there is no index range, the query will consider all documents in the index.

withIndex() designed to define ranges that Convex can effectively use for index lookups. For other filtering you should use the method filter.

An example of requesting other people's messages in channel:

const messages = await ctx.db
  .query("messages")
  .withIndex("by_channel", q => q.eq("channel", channel))
  .filter(q => q.neq(q.field("user"), myUserId)
  .collect();

In this case, the performance of the request will depend on the number of messages in the channel.

Sorting using indexes

Queries using withIndex(), are sorted by the columns specified in the index.

The order of the columns in the index determines the sorting priority. Subsequent columns are compared only if the previous ones match. The last column to compare is _creationTime, which Convex automatically adds to the end of each index.

For example, by_channel_user includes channel, user And _creationTime. Therefore, the results of the query to messagesusing .withIndex("by_channel_user"), will be sorted first by channel, then by user, then by creation time.

Sorting by index makes it easy to retrieve data such as n the best (by number of points) players, n latest transactions, n best (by number of likes) messages, etc.

For example, to get the top 10 players, you can define the following index:

export default defineSchema({
  players: defineTable({
    username: v.string(),
    highestScore: v.number(),
  }).index("by_highest_score", ["highestScore"]),
});

And get them with .take(10):

const topScoringPlayers = await ctx.db
  .query("users")
  .withIndex("by_highest_score")
  .order("desc")
  .take(10);

There is no range expression in this example because we are interested in the top players of all time. This query will be effective even for a large amount of data due to the use take().

When using an index without a range expression, in conjunction with withIndex() One of the following methods should always be used:

• first()
• unique()
• take(n)
• paginate(opts)

These methods allow you to limit the query to a reasonable size, avoiding a full table scan.

You can use a range expression to refine your query. Example query for the top 10 players in Canada:

const topScoringPlayers = await ctx.db
  .query("users")
  .withIndex("by_country_highest_score", (q) => q.eq("country", "CA"))
  .order("desc")
  .take(10);

Restrictions

Convex supports indexes containing up to 16 fields. Each table can contain up to 32 indexes. Columns in indexes cannot be duplicated.

System fields (starting with _) cannot be used in indexes. Field _creationTime added to each index automatically to ensure stable sorting. In other words, the index by_creation_time is created automatically. Index by_id is reserved.

This completes the first part of the guide. See you in the next part.

---

News, product reviews and competitions from the Timeweb.Cloud team - in our Telegram channel ↩