Add ingest processor existence helper method #45156
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds a helper method to the ingest service allowing it to inspect a pipeline by id and verify the existence of a processor in the pipeline. This work exposed a potential bug in that some processors contain inner processors that are passed in at instantiation. These processors needed a common way to expose their inner processors, so the WrappedProcessor was created in order to expose the inner processor.
hub-cap
added
>enhancement
:Data Management/Ingest Node
Collaborator
|
Pinging @elastic/es-core-features |
Contributor
Author
|
@elasticmachine run elasticsearch-ci/2 |
martijnvg
reviewed
Aug 5, 2019
Member
martijnvg
left a comment
martijnvg
left a comment
There was a problem hiding this comment.
Besides the comment about hasProcessor(...) method, this PR LGTM.
| } | ||
|
|
||
| for (Processor processor: pipeline.flattenAllProcessors()) { | ||
| if (processor instanceof WrappedProcessor) { |
Member
There was a problem hiding this comment.
I think this method does not work as expected in the following cases:
hasProcessor("_id", ForEachProcessor.class/ConditionalProcessor.class)invocations, because both classes implementedWrappedProcessor, we never end up checking whetherprocessoris assignable fromclazzand straight check the inner processor.- In the case that a wrapped processor is wrapped by another wrapped processor (for example: foreach processor wraps a
conditionalprocessor) then we never check the 2nd wrapped processor or any processor below that.
I think something like the following fixes the above cases:
if (clazz.isAssignableFrom(processor.getClass())) {
return true;
}
while (processor instanceof WrappedProcessor) {
WrappedProcessor wrappedProcessor = (WrappedProcessor) processor;
if (clazz.isAssignableFrom(wrappedProcessor.getInnerProcessor().getClass())) {
return true;
}
processor = wrappedProcessor.getInnerProcessor();
if (wrappedProcessor == processor) {
break;
}
}
(replaces the body of the for loop)
jbaiera
approved these changes
Aug 6, 2019
Member
jbaiera
left a comment
jbaiera
left a comment
There was a problem hiding this comment.
This LGTM, pending the name nitpicks
| * A wrapped processor is one that encapsulates an inner processor, or a processor that the wrapped processor enacts upon. All processors | ||
| * that contain an "inner" processor should implement this interface, such that the actual processor can be obtained. | ||
| */ | ||
| public interface WrappedProcessor extends Processor { |
Member
There was a problem hiding this comment.
I think originally we had wanted to call this WrappingProcessor
|
|
||
| import java.util.function.Consumer; | ||
|
|
||
| class WrappedProcessorImpl extends FakeProcessor implements WrappedProcessor { |
Member
There was a problem hiding this comment.
Same thing here re: WrappingProcessor vs WrappedProcessor
Contributor
Author
|
@elasticmachine update branch |
hub-cap
added a commit
that referenced
this pull request
Aug 7, 2019
This commit adds a helper method to the ingest service allowing it to inspect a pipeline by id and verify the existence of a processor in the pipeline. This work exposed a potential bug in that some processors contain inner processors that are passed in at instantiation. These processors needed a common way to expose their inner processors, so the WrappingProcessor was created in order to expose the inner processor.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds a helper method to the ingest service allowing it to
inspect a pipeline by id and verify the existence of a processor in the
pipeline. This work exposed a potential bug in that some processors
contain inner processors that are passed in at instantiation. These
processors needed a common way to expose their inner processors, so the
WrappedProcessor was created in order to expose the inner processor.