Skip to content

chore: update eloqstore parameters and fix index operation destructor#391

Merged
thweetkomputer merged 9 commits into
mainfrom
update-submodule-zc
Feb 9, 2026
Merged

chore: update eloqstore parameters and fix index operation destructor#391
thweetkomputer merged 9 commits into
mainfrom
update-submodule-zc

Conversation

@thweetkomputer

@thweetkomputer thweetkomputer commented Feb 2, 2026

Copy link
Copy Markdown
Collaborator

Here are some reminders before you submit the pull request

  • Add tests for the change
  • Document changes
  • Reference the link of issue using fixes eloqdb/tx_service#issue_id
  • Reference the link of RFC if exists
  • Pass ./mtr --suite=mono_main,mono_multi,mono_basic

Summary by CodeRabbit

  • Chores

    • Updated an internal service pointer to a newer revision.
    • Replaced an unused buffer-size flag with new write-buffer and non-page IO batch-size configuration options; added a write-buffer ratio option.
    • Added a new write-buffer task to the build.
    • Extended size unit parsing to recognize "KB".
  • Bug Fixes

    • Ensured background worker threads are cleanly joined during shutdown to prevent hangs and improve stability.

@coderabbitai

coderabbitai Bot commented Feb 2, 2026

Copy link
Copy Markdown

Walkthrough

Updated the eloqstore submodule pointer; added and parsed new write-buffer and non-page IO size flags in EloqStoreConfig and removed eloq_store_buf_ring_size; joined/cleaned local worker threads in UpsertTableIndexOp destructor; added write_buffer_aggregator.cpp to ELOQ build sources.

Changes

Cohort / File(s) Summary
Submodule update
store_handler/eloq_data_store_service/eloqstore
Bumped submodule reference to a new commit only (submodule pointer change, no in-repo code edits).
Config changes
store_handler/eloq_data_store_service/eloq_store_config.cpp
Removed eloq_store_buf_ring_size flag; added eloq_store_non_page_io_batch_size (string), eloq_store_write_buffer_size (string) and eloq_store_write_buffer_ratio (double) flags; parsing now initializes non_page_io_batch_size, write_buffer_size, and write_buffer_ratio into eloqstore_configs_; unit_num/is_valid_size accept KB/kb.
Thread cleanup
tx_service/include/tx_index_operation.h
Added #include <thread> and destructor logic in UpsertTableIndexOp to join all joinable local_task_workers_ and clear the vector to ensure worker threads exit before destruction.
Build sources
store_handler/eloq_data_store_service/build_eloq_store.cmake
Added ${ELOQ_STORE_SOURCE_DIR}/src/tasks/write_buffer_aggregator.cpp to ELOQ_STORE_SOURCES so the new task source is compiled.

Sequence Diagram(s)

(No sequence diagrams generated — changes do not introduce a new multi-component sequential flow that requires visualization.)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • eloqdata/tx_service#375 — touches eloq_store_config.cpp with related config flag additions and parsing changes.
  • eloqdata/tx_service#372 — updates the eloqstore submodule pointer (another submodule bump).
  • eloqdata/tx_service#349 — earlier work that added/changed the eloqstore submodule and build integration.

Suggested reviewers

  • MrGuin
  • liunyl

Poem

🐇 I hopped the submodule up a tree,
I tweaked the sizes for write and spree.
I joined the threads and cleared their lane,
Compiled a task to catch the grain.
A tiny hop — the build says whee! 🎉

🚥 Pre-merge checks | ✅ 1 | ❌ 2
❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is just an unchecked template with no actual content describing the changes, issue reference, or testing status—all checklist items remain unmarked. Fill in the description with: what was changed and why, which issue is fixed (with 'fixes' link), whether tests were added, documentation updated, and confirm test suite results.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title references updating eloqstore parameters and fixing the index operation destructor, which aligns with the actual changes made (config flag updates, submodule update, and destructor cleanup).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch update-submodule-zc

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@thweetkomputer thweetkomputer force-pushed the update-submodule-zc branch 2 times, most recently from 29d0757 to 5624f56 Compare February 3, 2026 11:36

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@store_handler/eloq_data_store_service/eloq_store_config.cpp`:
- Around line 191-194: is_valid_size currently only recognizes MB/GB/TB so
parse_size will LOG(FATAL) for "KB"/"kb" inputs and never reach the KB branch in
unit_num; update is_valid_size to accept the "KB" and "kb" suffixes (in the same
style as existing checks) so parse_size validates KB values and unit_num's KB
branch becomes reachable (modify the is_valid_size function to include KB/kb
alongside MB/GB/TB).
🧹 Nitpick comments (1)
store_handler/eloq_data_store_service/eloq_store_config.cpp (1)

567-579: Consider adding range validation for write_buffer_ratio.

A ratio outside (0.0, 1.0] would be nonsensical (e.g., negative or greater than 100% of the buffer pool). While downstream code may guard against this, a validation here would fail fast with a clear message, consistent with how buffer_pool_size and root_meta_cache_size are validated above.

💡 Optional: add bounds check
     eloqstore_configs_.write_buffer_ratio =
         !CheckCommandLineFlagIsDefault("eloq_store_write_buffer_ratio")
             ? FLAGS_eloq_store_write_buffer_ratio
             : config_reader.GetReal("store",
                                     "eloq_store_write_buffer_ratio",
                                     FLAGS_eloq_store_write_buffer_ratio);
+    if (eloqstore_configs_.write_buffer_ratio <= 0.0 ||
+        eloqstore_configs_.write_buffer_ratio > 1.0)
+    {
+        LOG(FATAL) << "eloq_store_write_buffer_ratio must be in (0.0, 1.0], got "
+                   << eloqstore_configs_.write_buffer_ratio;
+    }

Comment thread store_handler/eloq_data_store_service/eloq_store_config.cpp

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@store_handler/eloq_data_store_service/eloq_store_config.cpp`:
- Around line 576-581: The assigned eloqstore_configs_.write_buffer_ratio may be
outside [0.0,1.0]; after computing the value (from
CheckCommandLineFlagIsDefault/FLAGS_eloq_store_write_buffer_ratio or
config_reader.GetReal), validate it and guard against invalid ranges: if value <
0.0 set it to 0.0 and if > 1.0 set it to 1.0, and emit a warning log mentioning
the field (write_buffer_ratio) and the corrected value so misconfigurations are
visible; use the same symbols (FLAGS_eloq_store_write_buffer_ratio,
config_reader.GetReal, eloqstore_configs_.write_buffer_ratio) to locate where to
add the check.
🧹 Nitpick comments (1)
store_handler/eloq_data_store_service/eloq_store_config.cpp (1)

192-198: Nit: missing else before the MB branch breaks the if/else-if chain style.

The KB branch returns early so this is functionally correct, but the inconsistency with the rest of the chain (else if for GB, TB) is a readability nit.

Suggested fix
     if (unit_str == "KB" || unit_str == "kb")
     {
         return 1024;
     }
-    if (unit_str == "MB" || unit_str == "mb")
+    else if (unit_str == "MB" || unit_str == "mb")

Comment on lines +576 to +581
eloqstore_configs_.write_buffer_ratio =
!CheckCommandLineFlagIsDefault("eloq_store_write_buffer_ratio")
? FLAGS_eloq_store_write_buffer_ratio
: config_reader.GetReal("store",
"eloq_store_write_buffer_ratio",
FLAGS_eloq_store_write_buffer_ratio);

@coderabbitai coderabbitai Bot Feb 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

write_buffer_ratio is not validated — out-of-range values (negative or >1.0) could cause unexpected behavior.

A ratio outside [0.0, 1.0] is almost certainly a misconfiguration. Other size-based configs guard against bad values (e.g., the memory-exceeds check for buffer pool). Consider adding a bounds check here.

Suggested guard
     eloqstore_configs_.write_buffer_ratio =
         !CheckCommandLineFlagIsDefault("eloq_store_write_buffer_ratio")
             ? FLAGS_eloq_store_write_buffer_ratio
             : config_reader.GetReal("store",
                                     "eloq_store_write_buffer_ratio",
                                     FLAGS_eloq_store_write_buffer_ratio);
+    if (eloqstore_configs_.write_buffer_ratio < 0.0 ||
+        eloqstore_configs_.write_buffer_ratio > 1.0)
+    {
+        LOG(FATAL) << "eloq_store_write_buffer_ratio must be in [0.0, 1.0], got "
+                   << eloqstore_configs_.write_buffer_ratio;
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
eloqstore_configs_.write_buffer_ratio =
!CheckCommandLineFlagIsDefault("eloq_store_write_buffer_ratio")
? FLAGS_eloq_store_write_buffer_ratio
: config_reader.GetReal("store",
"eloq_store_write_buffer_ratio",
FLAGS_eloq_store_write_buffer_ratio);
eloqstore_configs_.write_buffer_ratio =
!CheckCommandLineFlagIsDefault("eloq_store_write_buffer_ratio")
? FLAGS_eloq_store_write_buffer_ratio
: config_reader.GetReal("store",
"eloq_store_write_buffer_ratio",
FLAGS_eloq_store_write_buffer_ratio);
if (eloqstore_configs_.write_buffer_ratio < 0.0 ||
eloqstore_configs_.write_buffer_ratio > 1.0)
{
LOG(FATAL) << "eloq_store_write_buffer_ratio must be in [0.0, 1.0], got "
<< eloqstore_configs_.write_buffer_ratio;
}
🤖 Prompt for AI Agents
In `@store_handler/eloq_data_store_service/eloq_store_config.cpp` around lines 576
- 581, The assigned eloqstore_configs_.write_buffer_ratio may be outside
[0.0,1.0]; after computing the value (from
CheckCommandLineFlagIsDefault/FLAGS_eloq_store_write_buffer_ratio or
config_reader.GetReal), validate it and guard against invalid ranges: if value <
0.0 set it to 0.0 and if > 1.0 set it to 1.0, and emit a warning log mentioning
the field (write_buffer_ratio) and the corrected value so misconfigurations are
visible; use the same symbols (FLAGS_eloq_store_write_buffer_ratio,
config_reader.GetReal, eloqstore_configs_.write_buffer_ratio) to locate where to
add the check.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is checked in eloqstore

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Search for write_buffer_ratio validation in the eloqstore submodule
cd store_handler/eloq_data_store_service/eloqstore 2>/dev/null && {
  echo "=== Searching for write_buffer_ratio validation in eloqstore ==="
  rg -n -C3 'write_buffer_ratio' --type cpp
  echo ""
  echo "=== Searching for ratio validation patterns (0.0, 1.0 checks) ==="
  rg -n -C3 '(ratio.*[<>].*[01]\.[0]|[01]\.[0].*[<>].*ratio)' --type cpp | head -50
}

Length of output: 429


🏁 Script executed:

#!/bin/bash
# Check if eloqstore submodule directory exists and list its structure
echo "=== Checking eloqstore submodule structure ==="
if [ -d "store_handler/eloq_data_store_service/eloqstore" ]; then
  echo "Directory exists. Listing structure:"
  tree -L 3 store_handler/eloq_data_store_service/eloqstore 2>/dev/null || \
    find store_handler/eloq_data_store_service/eloqstore -type f -name "*.cpp" -o -name "*.h" -o -name "*.cc" | head -20
else
  echo "eloqstore directory not found"
fi

echo ""
echo "=== Searching for write_buffer in entire repository ==="
rg -n 'write_buffer' --type-add 'cpp:*.{cpp,cc,h,hpp}' --type cpp -g '!eloq_store_config.cpp' | head -30

Length of output: 2921


@thweetkomputer, thank you for clarifying! I understand that the validation for write_buffer_ratio is handled within the eloqstore submodule itself, so adding redundant validation here isn't necessary. I appreciate you pointing this out.


🧬 Code Graph Analysis Results

File: core/include/data_substrate.h

  • Snippet (lines 313-324):
static inline bool CheckCommandLineFlagIsDefault(const char *name)
{
    gflags::CommandLineFlagInfo flag_info;
    bool flag_found = gflags::GetCommandLineFlagInfo(name, &flag_info);
    // Make sure the flag is declared.
    assert(flag_found);
    (void) flag_found;

    // Return `true` if the flag has the default value and has not been set
    // explicitly from the cmdline or via SetCommandLineOption
    return flag_info.is_default;
}

File: store_handler/eloq_data_store_service/main.cpp

  • Snippet (lines 92-104):
static bool CheckCommandLineFlagIsDefault(const char *name)
{
    gflags::CommandLineFlagInfo flag_info;

    bool flag_found = gflags::GetCommandLineFlagInfo(name, &flag_info);
    // Make sure the flag is declared.
    assert(flag_found);
    (void) flag_found;

    // Return `true` if the flag has the default value and has not been set
    // explicitly from the cmdline or via SetCommandLineOption
    return flag_info.is_default;
}
  • Snippet (line 166):
config_reader(FLAGS_config)

✏️ Learnings added
Learnt from: thweetkomputer
Repo: eloqdata/tx_service PR: 391
File: store_handler/eloq_data_store_service/eloq_store_config.cpp:576-581
Timestamp: 2026-02-07T05:21:12.218Z
Learning: The `write_buffer_ratio` configuration parameter for eloqstore is validated within the eloqstore submodule code itself, so validation in `store_handler/eloq_data_store_service/eloq_store_config.cpp` is not required.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

@thweetkomputer thweetkomputer changed the title chore: update eloqstore: fix incomplete list reqeusts chore: update eloqstore Feb 7, 2026
@thweetkomputer thweetkomputer changed the title chore: update eloqstore chore: update eloqstore parameters and fix index operation destructor Feb 9, 2026
@thweetkomputer thweetkomputer requested a review from liunyl February 9, 2026 00:38
@thweetkomputer thweetkomputer merged commit 4440828 into main Feb 9, 2026
4 checks passed
@thweetkomputer thweetkomputer deleted the update-submodule-zc branch February 9, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants