Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,14 @@ To be released.

### @fedify/mysql

- Added `MysqlMessageQueue` class to the `@fedify/mysql` package, a
MySQL/MariaDB-backed `MessageQueue` implementation. It uses periodic
polling (`SELECT … FOR UPDATE SKIP LOCKED`) to deliver messages and
MySQL advisory locks (`GET_LOCK`/`RELEASE_LOCK`) for ordering-key
serialization. Supports delayed delivery, ordering keys,
`enqueueMany()`, and concurrent workers. Requires MySQL 8.0+ or
MariaDB 10.6+. [[#586], [#599]]

- Added `@fedify/mysql` package, a MySQL/MariaDB-backed `KvStore`
implementation. It provides `MysqlKvStore`, which stores key–value
pairs in a MySQL table using the [`mysql2`] driver. Supports TTL,
Expand All @@ -73,7 +81,9 @@ To be released.

[`mysql2`]: https://www.npmjs.com/package/mysql2
[#585]: https://github.com/fedify-dev/fedify/issues/585
[#586]: https://github.com/fedify-dev/fedify/issues/586
[#597]: https://github.com/fedify-dev/fedify/pull/597
[#599]: https://github.com/fedify-dev/fedify/pull/599


Version 2.0.3
Expand Down
4 changes: 3 additions & 1 deletion docs/manual/deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,9 @@ Development
Production
: Consider [`PostgresKvStore`](./kv.md#postgreskvstore) and
[`PostgresMessageQueue`](./mq.md#postgresmessagequeue) if you already use
PostgreSQL, or [`RedisKvStore`](./kv.md#rediskvstore) and
PostgreSQL, [`MysqlKvStore`](./kv.md#mysqlkvstore) and
[`MysqlMessageQueue`](./mq.md#mysqlmessagequeue) if you already use
MySQL or MariaDB, or [`RedisKvStore`](./kv.md#rediskvstore) and
[`RedisMessageQueue`](./mq.md#redismessagequeue) for dedicated caching
infrastructure. There is also [`AmqpMessageQueue`](./mq.md#amqpmessagequeue)
for RabbitMQ users.
Expand Down
7 changes: 5 additions & 2 deletions docs/manual/federation.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,10 @@ runtime).

As separate packages, [`@fedify/redis`] provides [`RedisMessageQueue`] class,
which is a Redis-backed implementation for production use,
and [`@fedify/postgres`] provides [`PostgresMessageQueue`] class, which is a
PostgreSQL-backed implementation for production use, and [`@fedify/amqp`]
[`@fedify/postgres`] provides [`PostgresMessageQueue`] class, which is a
PostgreSQL-backed implementation for production use,
[`@fedify/mysql`] provides [`MysqlMessageQueue`] class, which is a
MySQL/MariaDB-backed implementation for production use, and [`@fedify/amqp`]
provides [`AmqpMessageQueue`] class, which is an AMQP broker-backed
implementation for production use.

Expand Down Expand Up @@ -187,6 +189,7 @@ Further details are explained in the [*Message queue* section](./mq.md).

[`RedisMessageQueue`]: https://jsr.io/@fedify/redis/doc/mq/~/RedisMessageQueue
[`PostgresMessageQueue`]: https://jsr.io/@fedify/postgres/doc/mq/~/PostgresMessageQueue
[`MysqlMessageQueue`]: https://jsr.io/@fedify/mysql/doc/mq/~/MysqlMessageQueue
[`@fedify/amqp`]: https://github.com/fedify-dev/fedify/tree/main/packages/amqp
[`AmqpMessageQueue`]: https://jsr.io/@fedify/amqp/doc/mq/~/AmqpMessageQueue

Expand Down
7 changes: 5 additions & 2 deletions docs/manual/inbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -349,8 +349,9 @@ const federation = createFederation({
> The `InProcessMessageQueue` is a simple in-memory message queue that is
> suitable for development and testing. For production use, you should
> consider using a more robust message queue, such as [`DenoKvMessageQueue`]
> from [`@fedify/denokv`] package or [`RedisMessageQueue`] from
> [`@fedify/redis`] package.
> from [`@fedify/denokv`] package, [`RedisMessageQueue`] from
> [`@fedify/redis`] package, or [`MysqlMessageQueue`] from
> [`@fedify/mysql`] package.
>
> For more information, see the [*Message queue* section](./mq.md).

Expand Down Expand Up @@ -389,6 +390,8 @@ duplicate retry mechanisms and leverages the backend's optimized retry features.
[`@fedify/denokv`]: https://github.com/fedify-dev/fedify/tree/main/packages/denokv
[`RedisMessageQueue`]: https://jsr.io/@fedify/redis/doc/mq/~/RedisMessageQueue
[`@fedify/redis`]: https://github.com/fedify-dev/fedify/tree/main/packages/redis
[`MysqlMessageQueue`]: https://jsr.io/@fedify/mysql/doc/mq/~/MysqlMessageQueue
[`@fedify/mysql`]: https://github.com/fedify-dev/fedify/tree/main/packages/mysql


Activity idempotency
Expand Down
86 changes: 86 additions & 0 deletions docs/manual/mq.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,88 @@ const federation = createFederation({
[`AmqpMessageQueue`]: https://jsr.io/@fedify/amqp/doc/mq/~/AmqpMessageQueue
[RabbitMQ]: https://www.rabbitmq.com/

### [`MysqlMessageQueue`]

*This API is available since Fedify 2.1.0.*

To use [`MysqlMessageQueue`], you need to install the *@fedify/mysql* package
first:

::: code-group

~~~~ bash [Deno]
deno add jsr:@fedify/mysql
~~~~

~~~~ bash [npm]
npm add @fedify/mysql mysql2
~~~~

~~~~ bash [pnpm]
pnpm add @fedify/mysql mysql2
~~~~

~~~~ bash [Yarn]
yarn add @fedify/mysql mysql2
~~~~

~~~~ bash [Bun]
bun add @fedify/mysql mysql2
~~~~

:::

[`MysqlMessageQueue`] is a message queue implementation that uses a MySQL or
MariaDB database as the backend. Since MySQL and MariaDB do not provide a
`LISTEN`/`NOTIFY` mechanism, it uses **polling** to discover new messages.
The polling interval is configurable and defaults to 1 second to minimize
latency; this is shorter than the default for PostgreSQL-backed queues.

Concurrent workers are safely supported via `SELECT … FOR UPDATE SKIP LOCKED`
and MySQL advisory locks (`GET_LOCK`/`RELEASE_LOCK`).

> [!NOTE]
> `MysqlMessageQueue` requires MySQL 8.0+ or MariaDB 10.6+ for
> `SELECT … FOR UPDATE SKIP LOCKED` support.

> [!NOTE]
> Because `MysqlMessageQueue` uses polling rather than a push-based
> notification system, there is an inherent latency between when a message
> is enqueued and when it is delivered. With the default 1-second poll
> interval, messages may take up to 1 second to be picked up. You can
> lower the `pollInterval` option to reduce this latency at the cost of
> additional database load.

Best for
: Production use in systems that already use MySQL or MariaDB.

Pros
: Persistent, supports multiple workers, minimal additional infrastructure
for MySQL/MariaDB users.

Cons
: Polling-based delivery (up to `pollInterval` latency); requires
MySQL 8.0+ or MariaDB 10.6+.

~~~~ typescript twoslash
import type { KvStore } from "@fedify/fedify";
// ---cut-before---
import { createFederation } from "@fedify/fedify";
import { MysqlMessageQueue } from "@fedify/mysql";
import mysql from "mysql2/promise";

const pool = mysql.createPool("mysql://user:pass@localhost/db");
const federation = createFederation<void>({
// ---cut-start---
kv: null as unknown as KvStore,
// ---cut-end---
queue: new MysqlMessageQueue(pool), // [!code highlight]
// ... other options
});
~~~~

[`MysqlMessageQueue`]: https://jsr.io/@fedify/mysql/doc/mq/~/MysqlMessageQueue

### `SqliteMessageQueue`

*This API is available since Fedify 2.0.0.*
Expand Down Expand Up @@ -767,6 +849,9 @@ The following implementations do not yet support native retry:
[`PostgresMessageQueue`]
: Native retry support planned for future release.

[`MysqlMessageQueue`]
: No native retry support (`~MessageQueue.nativeRetrial` is `false`).

[`AmqpMessageQueue`]
: Native retry support planned for future release.

Expand Down Expand Up @@ -835,6 +920,7 @@ The following implementations support ordering keys:
| [`DenoKvMessageQueue`] | Yes |
| [`RedisMessageQueue`] | Yes |
| [`PostgresMessageQueue`] | Yes |
| [`MysqlMessageQueue`] | Yes |
| [`AmqpMessageQueue`] | Yes[^1] |
| [`SqliteMessageQueue`] | Yes |
| `WorkersMessageQueue` | Yes[^2] |
Expand Down
4 changes: 3 additions & 1 deletion docs/manual/relay.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,8 @@ Configuration options
~~~~

> [!NOTE]
> For production, use [`RedisMessageQueue`] or [`PostgresMessageQueue`].
> For production, use [`RedisMessageQueue`], [`PostgresMessageQueue`],
> or [`MysqlMessageQueue`].

`subscriptionHandler` (required)
: Callback to approve or reject subscription requests. See
Expand All @@ -177,6 +178,7 @@ Configuration options

[`RedisMessageQueue`]: https://jsr.io/@fedify/redis/doc/mq/~/RedisMessageQueue
[`PostgresMessageQueue`]: https://jsr.io/@fedify/postgres/doc/mq/~/PostgresMessageQueue
[`MysqlMessageQueue`]: https://jsr.io/@fedify/mysql/doc/mq/~/MysqlMessageQueue


Relay types
Expand Down
7 changes: 5 additions & 2 deletions docs/manual/send.md
Original file line number Diff line number Diff line change
Expand Up @@ -390,8 +390,9 @@ const federation = createFederation({
> The `InProcessMessageQueue` is a simple in-memory message queue that is
> suitable for development and testing. For production use, you should
> consider using a more robust message queue, such as [`DenoKvMessageQueue`]
> from [`@fedify/denokv`] package or [`RedisMessageQueue`] from
> [`@fedify/redis`] package.
> from [`@fedify/denokv`] package, [`RedisMessageQueue`] from
> [`@fedify/redis`] package, or [`MysqlMessageQueue`] from
> [`@fedify/mysql`] package.
>
> For further information, see the [*Message queue* section](./mq.md).

Expand All @@ -414,6 +415,8 @@ an error and does not retry the delivery.
[`@fedify/denokv`]: https://github.com/fedify-dev/fedify/tree/main/packages/denokv
[`RedisMessageQueue`]: https://jsr.io/@fedify/redis/doc/mq/~/RedisMessageQueue
[`@fedify/redis`]: https://github.com/fedify-dev/fedify/tree/main/packages/redis
[`MysqlMessageQueue`]: https://jsr.io/@fedify/mysql/doc/mq/~/MysqlMessageQueue
[`@fedify/mysql`]: https://github.com/fedify-dev/fedify/tree/main/packages/mysql


Optimizing activity delivery for large audiences
Expand Down
10 changes: 7 additions & 3 deletions packages/mysql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,22 @@
[![JSR][JSR badge]][JSR]
[![npm][npm badge]][npm]

This package provides [Fedify]'s [`KvStore`] implementation for
MySQL/MariaDB:
This package provides [Fedify]'s [`KvStore`] and [`MessageQueue`]
implementations for MySQL/MariaDB:

- [`MysqlKvStore`]
- [`MysqlMessageQueue`]

~~~~ typescript
import { createFederation } from "@fedify/fedify";
import { MysqlKvStore } from "@fedify/mysql";
import { MysqlKvStore, MysqlMessageQueue } from "@fedify/mysql";
import mysql from "mysql2/promise";

const pool = mysql.createPool("mysql://user:password@localhost/dbname");

const federation = createFederation({
kv: new MysqlKvStore(pool),
queue: new MysqlMessageQueue(pool),
});
~~~~

Expand All @@ -29,7 +31,9 @@ const federation = createFederation({
[npm]: https://www.npmjs.com/package/@fedify/mysql
[Fedify]: https://fedify.dev/
[`KvStore`]: https://jsr.io/@fedify/fedify/doc/federation/~/KvStore
[`MessageQueue`]: https://jsr.io/@fedify/fedify/doc/federation/~/MessageQueue
[`MysqlKvStore`]: https://jsr.io/@fedify/mysql/doc/~/MysqlKvStore
[`MysqlMessageQueue`]: https://jsr.io/@fedify/mysql/doc/mq/~/MysqlMessageQueue


Installation
Expand Down
3 changes: 2 additions & 1 deletion packages/mysql/deno.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"license": "MIT",
"exports": {
".": "./src/mod.ts",
"./kv": "./src/kv.ts"
"./kv": "./src/kv.ts",
"./mq": "./src/mq.ts"
},
"exclude": [
"dist",
Expand Down
10 changes: 10 additions & 0 deletions packages/mysql/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@
"require": "./dist/kv.cjs",
"default": "./dist/kv.js"
},
"./mq": {
"types": {
"import": "./dist/mq.d.ts",
"require": "./dist/mq.d.cts",
"default": "./dist/mq.d.ts"
},
"import": "./dist/mq.js",
"require": "./dist/mq.cjs",
"default": "./dist/mq.js"
},
"./package.json": "./package.json"
},
"files": [
Expand Down
1 change: 1 addition & 0 deletions packages/mysql/src/mod.ts
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export { MysqlKvStore, type MysqlKvStoreOptions } from "./kv.ts";
export { MysqlMessageQueue, type MysqlMessageQueueOptions } from "./mq.ts";
Loading
Loading