-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow nested parsers on MessageProducer level (v2) #2078
base: master
Are you sure you want to change the base?
Conversation
item.resolve(Some(Err(String::from( | ||
"Fail to extract printable message", | ||
)))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missed continue;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @DmitryAstafyev for the alternative approach to solve the nested parsers problem. Here are my thoughts about it regarding the plugins system and some general thoughts:
I think this approach wouldn't affect the current implementation of the plugins system because we are only allowing the parser to return strings, so they will fall into the category of the default implementation to the new methods.
But if we decide to let the plugins return types other than strings then they would be involved like the other methods on LogMessage
trait, but this wouldn't increase the complexity of the code by much at that point.
Another point to keep in mind that we want to change return type of the parse()
method to be an iterator of results which could add some complexity for the solution in the producer loop.
I still think the main point here is how often such a situation could happen, because changing the main logic because of one special case wouldn't make much sense in the case of one -or very few- special cases. On the other hand, having a lot of parsers that have another parser nested in them wouldn't be a good solution if many parsers need a nested parser within them to performe the parser correctly.
From the performance point of view, I don't think these changes will have an impact on the parsers that don't need nested one, because all the code for that case will be removed at the compile time thanks generics in the producer loop.
But for the parsers that need the nesting, I think the direct nesting would be better for the performance because it's saving as moving data around between parser and producer and having to save them in a HashMap
. We still need to measure here to know which tradeoff are we facing here
Something, that I didn't expect to be honest :/ Test inputs:
Test workflow:
|
The major approach is: that the parser doesn't own any nested parsers, but still can request additional (nested) parsing. At the same time, only the initial parser knows how a complete message should be represented. The Message Producer doesn't know it, the initial parser - knows. That's why the initial parser has a way to request an addition (nested) parsing. At the same time, this solution gives flexibility and can be easily scaled. To compare two ways:
Parsers doesn't own nested parser(s) and requires parsing
In the same time a weak point of suggested solution - an initial parser should keep in memory result of nested parsing |
This PR isn't for to be merged. It's a suggestion in the scope of work on #2073
Solution limitations:
The previous version is here #2077