Spring Data Redis indexes not cleaned up when setting TTLs [DATAREDIS-748]

[spring-projects-issues]

Rajesh A opened DATAREDIS-748 and commented

When using the Spring-data-redis library to manage repositories, Spring creates new keys and sets under the hood to facilitate various type of key lookups. I noticed that when we set TTLs on the same objects (as opposed to a straight delete), these additional keys (indexes) and set members don't get deleted.
This has caused us to run out of disk space on the redis server as the associated keys never get removed.

More details:
In my setup, lets say I have a PersonRepository storing a Person class. The Person class (hash is "person") has say a bunch of fields, of note - name (identifier), email (indexed) & ttl (default value 0). Now lets say the first time the person object gets saved, the TTL is not set as expected as its 0. A bunch of other objects get created and updated, like person:johnsmith:idx etc.

Now when I go ahead and update the object by reading it (say using repo.findOne()), updated the ttl value to something and saved - the TTL gets set as expected. At this point, once the key expires, the related indices and key references do not get removed. If instead of updating (the TTL) on the object, we went ahead and straight away deleted, I believe the references get removed

---

Issue Links:

• DATAREDIS-963 Spring Data Redis indexes not cleaned up when setting TTLs
  ("is duplicated by")

[spring-projects-issues]

Mark Paluch commented

Can you provide more details on your setup, how you tested? It's essential that your application is running so it receives expired events for secondary index cleanup

[spring-projects-issues]

Mark Paluch commented

Ping! Any update?

[spring-projects-issues]

Rajesh A commented

Apologies for the absence, I have added some detail but might be missing something. Is it expected that an application is responsible to perform cleanup by subscribing to keyspace notifications and remove indices and references event by event? If that is the case, this would not be a bug.

[spring-projects-issues]

Rajesh A commented

Updated bug with some additional information and a followup question

[spring-projects-issues]

Mark Paluch commented

It's expected that Spring Data Redis subscribes to keyspace notifications (which means that you need to have your application running) but you need to enable it:

1. Configure redis with CONFIG SET notify-keyspace-events Ex
2. Enable the listener via @EnableRedisRepositories(enableKeyspaceEvents = ON_STARTUP)

See also: https://docs.spring.io/spring-data/redis/docs/current/reference/html/#redis.repositories.expirations

[spring-projects-issues]

Rajesh A commented

From the link:
Additionally to persisting the original, a phantom copy is persisted in Redis and set to expire 5 minutes after the original one. This is done to enable the Repository support to publish RedisKeyExpiredEvent holding the expired value via Springs ApplicationEventPublisher whenever a key expires even though the original values have already been gone.

Are you saying that by enabling keyspace-notifications on the redis server and enabling keyspaceEvent listener within the spring application, Spring will self-handle the expiration events and delete the respective indexes? If this is true, I certainly do not see this happen on Spring Boot - 1.5.9.RELEASE version.

FWIW I did follow step 1 and enabled keyspace notification on Amazon Elasticache. Then in my application, I have a listener subscribed to a pattern which is capturing events as it receives, and then act upon the keys that we know have indexes and delete certain keys from sets as no longer necessary

[spring-projects-issues]

Mark Paluch commented

Yes, you need to enable keyspace events. Please pay attention to exceptions during startup. ElastiCache has some quirks (e.g. CONFIG SET causes an error). Spring Data Redis attempts to configure your Redis server properly on startup. That's why you find an option in @EnableRedisRepositories to configure your Redis server properly. Setting keyspaceNotificationsConfigParameter to an empty string will not change your Redis server config.

I need to investigate about Spring Boot and the cleanup issue you mentioned

[spring-projects-issues]

Mark Paluch commented

I wasn't able to reproduce the issue. See https://gist.github.com/mp911de/581151f6394607998371d9f0910a7ac8.

I think it makes only sense to follow up this issue if you can provide an example that reproduces the issue

[spring-projects-issues]

Rajesh A commented

In the gist, you perform a save() with a TTL set. I see events arrive when we provide a TTL value. After 10 seconds, did you check if the :idx key(s) were removed too?

[spring-projects-issues]

Mark Paluch commented

Indeed, everything works as it's expected to work