Skip to content

DATAREDIS-1151 - change clean method keys command to scan #532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
eyison wants to merge 4 commits into from
Closed

DATAREDIS-1151 - change clean method keys command to scan #532

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
95200fc
change clean method `keys` command to `scan`
eyison May 21, 2020
8060462
batch delete directly
eyison Jun 17, 2020
c2aafda
change clean method `keys` command to `scan`,and batch delete directly
eyison May 21, 2020
209e328
Merge remote-tracking branch 'origin/master'
eyison Jun 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 13 additions & 10 deletions src/main/java/org/springframework/data/redis/cache/DefaultRedisCacheWriter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@

import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.util.Collections;
import java.util.Optional;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.function.Consumer;
import java.util.function.Function;

import org.springframework.dao.PessimisticLockingFailureException;
import com.sun.istack.internal.Nullable;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.connection.RedisStringCommands.SetOption;
import org.springframework.data.redis.core.Cursor;
import org.springframework.data.redis.core.ScanOptions;
import org.springframework.data.redis.core.types.Expiration;
import org.springframework.lang.Nullable;
import org.springframework.util.Assert;

/**
* {@link RedisCacheWriter} implementation capable of reading/writing binary data from/to Redis in {@literal standalone}
Expand Down Expand Up @@ -179,11 +179,14 @@ public void clean(String name, byte[] pattern) {
wasLocked = true;
}

byte[][] keys = Optional.ofNullable(connection.keys(pattern)).orElse(Collections.emptySet())
.toArray(new byte[0][]);

if (keys.length > 0) {
connection.del(keys);
List<byte[]> keys = new ArrayList<byte[]>();
Copy link

@cnhongwei cnhongwei Jun 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use LinkedList replace ArrayList because the list may be huge.

Cursor<byte[]> cursor = connection.scan(new ScanOptions.ScanOptionsBuilder().match(new String(pattern)).build());
while (cursor.hasNext()) {
byte[] bytes = cursor.next();
keys.add(bytes);
}
if (!keys.isEmpty()) {
connection.del(keys.toArray(new byte[][]{}));
Copy link

@chengchen chengchen Jun 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the complexity of DEL is O(N), if the number of keys is huge, DEL command could time out as well, right?
Maybe we could do it batch by batch as well?

I was thinking of using UNLINK, but it's only supported in higher Redis version and it's a change of behavior (from sync to async)...

}
} finally {

Expand Down