The issue was first observed as a flaky test in CI resulting in a segfault.
In multi-GPU replicated mode, the search query is divided in batches. These search batches are ran in parallel with OpenMP. In some cases, there may be more batches than available GPUs/indexes causing multiple searches to be performed on the same index at once. This may possibly cause a thread safety issue.
The issue was first observed as a flaky test in CI resulting in a segfault.
In multi-GPU replicated mode, the search query is divided in batches. These search batches are ran in parallel with OpenMP. In some cases, there may be more batches than available GPUs/indexes causing multiple searches to be performed on the same index at once. This may possibly cause a thread safety issue.