Skip to content

Fix split package LazySoftDeletesDirectoryReaderWrapper #81981

@ChrisHegarty

Description

@ChrisHegarty

The last split package issue that we have between server and lucene is LazySoftDeletesDirectoryReaderWrapper. This was added early in 2021, so not one of the legacy hangovers from years gone by.

By inspecting the code, the three package-private member dependencies in org.apache.lucene.index become clear:

  1. access to field SegmentReader.isNRT,
  2. CacheKey::new, and
  3. PendingSoftDeletes::applySoftDeletes.

First, LazySoftDeletesDirectoryReaderWrapper is only useful for read-only indices that are lazily loaded (a.k.a. frozen tier), so it is doubtful that Lucene would find it generally useful.

A note on each of the specific package-private API points:

  1. Access to field SegmentReader.isNRT is only for assertion code, and exposing an accessor from Lucene for this seems like it could be reasonable.
  2. The CacheKey constructor is deliberately non-accessible from outside the package. There were past issues where problems arose with short-lived reader wrappers that would not reuse caches across queries. Maybe there's a way to make DelegatingCacheHelper only work with a DirectoryReader, since directory readers would generally not be short-lived. Or otherwise exposing DelegatingCacheHelper as a more generally useful API in Lucene itself.
  3. For PendingSoftDeletes::applySoftDeletes one possibility is to just duplicate the code in ES. There are additional package-private issues when this is attempted, but maybe they are more straightforward to resolve.

[ the above notes are a summary of a discussion that included @romseygeek @jpountz, and @ywelsch ]

see #78166

Metadata

Metadata

Assignees

Labels

:Distributed Indexing/EngineAnything around managing Lucene and the Translog in an open shard.:Search/SearchSearch-related issues that do not fall into other categories>bugTeam:Distributed (Obsolete)Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.Team:SearchMeta label for search team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions