FastEmbed/ONNX embedding model is loaded multiple times per process, leaking ~25 GB of native memory

[ThomasSteinbach]

Summary

A single long-running basic-memory mcp server process grows to 25+ GB of memory and never releases it, eventually putting the whole machine under memory pressure (heavy swap/compression, UI stalls). This is not explained by the embedding model's normal size (~2.3 GB). The model appears to be loaded several times inside one process, and the native memory of the superseded ONNX sessions is never reclaimed.

Environment

• basic-memory 0.21.5 (pipx/mise install), Python 3.14
• macOS (Apple Silicon, 48 GB RAM)
• Backend: SQLite + sqlite-vec, semantic search enabled
• Embedding: provider fastembed, model jinaai/jina-embeddings-v3, 1024 dims
• fastembed 0.8.0, onnxruntime 1.26.0
• Server: basic-memory mcp --transport streamable-http --port 7331 (one shared daemon, multiple clients)
• 3 projects configured, sync_changes: true (file watch active)

What I observed

After ~30 h uptime, the server process footprint (footprint -p <pid>) looked like this:

phys_footprint:  26 GB
  25 GB   MALLOC_LARGE   (native heap)
 2.1 GB   mapped file    (the ONNX model on disk)
 ~0.6 GB  everything else

So ~25 GB sits in native large allocations — far beyond the model itself, and it is not Python-object memory.

The server log shows the embedding model being loaded multiple times within one continuous process (process start 11:39:18; this is the only process that has the model open, so every line below belongs to it):

11:41:59  FastEmbed model loaded: model_name=jinaai/jina-embeddings-v3 ...
11:44:54  FastEmbed model loaded: ...
12:22:26  FastEmbed model loaded: ...
12:22:36  FastEmbed model loaded: ...

Each "model loaded" line means a fresh TextEmbedding / onnxruntime.InferenceSession was created (~2.3 GB of weights plus its own CPU memory arena). The loads line up with vector-sync batches and incoming project-resolve / MCP requests.

Resident memory grows step-wise during embedding/sync activity and then stays flat — it does not shrink back. Over a day of normal multi-client use it reaches 25+ GB.

Why this matters

On a machine doing other work, this one process can eat roughly half of RAM, forcing the system into constant swap/compression and making everything sluggish. Restarting the server frees the memory instantly, but it grows back with use.

Likely root cause (hypothesis)

embedding_provider_factory.create_embedding_provider() is designed to return a process-wide cached singleton (_EMBEDDING_PROVIDER_CACHE), so the model should load exactly once. In practice it loads many times per process, which suggests the cache is being bypassed — i.e. a new provider instance is created on a cache miss, probably because the provider / search repository is rebuilt per request or per project context instead of reusing the cached provider.

Compounding factor: onnxruntime does not return CPU arena memory to the OS, so every superseded session's memory is retained for the life of the process. The effect is cumulative growth.

Confirmed: repeated model loads in one process, ~25 GB of native MALLOC_LARGE, step-wise retained growth.
Not yet pinned down: the exact code path that bypasses the provider cache.

Suggested directions

• Guarantee exactly one embedding provider / ONNX session per process. Verify the factory cache is actually shared across the request and sync paths, and log a warning if a second session is ever created.
• Consider disabling or capping the ONNX CPU arena (SessionOptions.enable_cpu_mem_arena = False); note that fastembed does not currently expose session options, so this may need an upstream hook or a workaround.
• Add a one-time "model loaded" guard so accidental re-instantiation is immediately visible.

How to reproduce (approximate)

1. Enable semantic search with fastembed + jinaai/jina-embeddings-v3.
2. Run basic-memory mcp --transport streamable-http as a long-lived server with sync_changes: true and a few projects.
3. Drive normal usage (note writes + searches) for a few hours.
4. Watch the server process RSS/footprint and grep the log for FastEmbed model loaded — it shows up more than once per process, and memory climbs accordingly.

[phernandez]

Hi @ThomasSteinbach thanks for the bug report. this seems very strange. would you mind sending me your full server logs so I can see this? I would not expect the transport to load fastembed more than once. It may be loading the server per each request, and that would cause terrible performance.

can you also let me know what client you are using and how you have the mcp connection setup? If you want, you can email me the logs at paul@basicmemory.com or find me on our discord (I'm paul).

[phernandez]

Thanks @ThomasSteinbach — excellent report. We dug into the code on current main; here's where we landed.

What we confirmed in code

• A new SQLiteSearchRepository is constructed per request, per sync batch, and per project, and each construction calls create_embedding_provider(). The embedding_provider injection param exists but no caller passes it — so the only thing preventing N model loads is the process-wide provider cache (_EMBEDDING_PROVIDER_CACHE).
• That cache itself is correct (process-global, thread-safe, double-checked) and should return one provider — and load the model once — for a stable config.

What we could not confirm from code alone
We couldn't find a deterministic path that bypasses the cache. Our leading hypothesis is cache-key drift: the cache key includes CPU-count-derived thread/parallel knobs (os.process_cpu_count() / affinity). In a container or under changing CPU affinity those can shift between calls → new key → new provider → a fresh ~2.3 GB model load. Compounding it, onnxruntime's CPU arena doesn't return memory to the OS, so any extra load permanently inflates RSS — consistent with your "grows step-wise, never releases."

To confirm the trigger, could you share:

1. The server log around several FastEmbed model loaded lines — ideally the Vector batch runtime settings: ... threads=... parallel=... request_concurrency=... line logged near each load. If threads/parallel differ between loads, that confirms the key drift.
2. Your config.json (the semantic_* fields).
3. How you run it: Docker/container? Any CPU limits (cgroup --cpus)? Output of nproc in the runtime.
4. Any BASIC_MEMORY_SEMANTIC_EMBEDDING_THREADS / _PARALLEL env vars set?
5. Confirm versions (you listed Python 3.14 / fastembed 0.8.0 / onnxruntime 1.26.0).

What we'll fix regardless (defensive hardening, not blocked on the above):

• Inject the single cached provider into the per-project/per-request repositories so every path reuses one instance.
• Drop the volatile CPU/thread knobs from the provider cache key (they affect execution, not the loaded weights) — or add a process-wide "model loaded once per (model, cache_dir)" guard.
• Set enable_cpu_mem_arena=False so any transient extra load returns memory to the OS.
• Log a warning if a second distinct provider is ever created in one process, so this is visible going forward.

We'll track the hardening here; your logs will let us confirm and close out the exact root cause.