Merge branch 'master' into '6.0/stage' #224
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
= Motivation
The current parsing logic of sdb based on split() and the shlex library
has resulted in multiple workarounds in the implementation of commands
and the overall user-experience of the tool. In addition, its lack of
proper error-handling and reporting frequently result in the user not
knowing what is wrong with his input. Trying to fix the aforementioned
shortcomings in the existing logic has proven difficult as fixing one
problem brings up a new one.
= Patch
This patch replaces this code a simple hand-written parser that provides
the bare-minimum that we need and improved error-reporting. Unit tests
are also provided for the parser to test its behavior and also highlight
its behavior in extreme cases of input. This patch also does a first
pass in undoing most of the workarounds that we have in existing commands
due to the old parsing logic.
= Things To Note: Quoted Strings
Proper support for single and double quote strings is added with this
patch. Double-quote strings are allowed to escape a double-quote by
inserting a backslash before it. Single-quote strings can escape a
single quote the same way. E.g. the following examples are valid:
```
... | filter "obj.spa_name == 'rpool'" | ...
... | filter "obj.spa_name == \"rpool\"" | ...
... | filter 'obj.spa_name == "rpool"' | ...
... | filter 'obj.spa_name == \'rpool\'' | ...
```
The purpose of strings is solely to allow the ability to pass multiple
words separated by space as a single argument to commands. The `filter`
examples show above get the whole predicate passed in string form as a
single argument. The actual quotes of the string are not part of the
arguments passed to the command. This behavior was modelled after bash.
= Examples of new errors
```
// Before
sdb> echo 1 2 3 |
sdb: cannot recognize command:
// After
sdb: syntax error: freestanding pipe with no command
echo 1 2 3 |
^
```
```
// Before
sdb> echo 1 2 3 | filter obj != 1
sdb: filter: invalid input: comparison operator is missing
// After
sdb> echo 1 2 3 | filter obj != 1
sdb: syntax error: predicates that use != as an operator should be quoted
echo 1 2 3 | filter obj != 1
^
```
```
// Before
sdb> echo 1 2 3 | filter "obj != 1
sdb encountered an internal error due to a bug. Here's the
information you need to file the bug:
----------------------------------------------------------
Target Info:
ProgramFlags.IS_LIVE|IS_LINUX_KERNEL
Platform(<Architecture.X86_64: 1>, <PlatformFlags.IS_LITTLE_ENDIAN|IS_64_BIT: 3>)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/sdb/internal/repl.py", line 107, in eval_cmd
for obj in invoke(self.target, [], input_):
File "/usr/lib/python3/dist-packages/sdb/pipeline.py", line 107, in invoke
all_tokens = list(lexer)
File "/usr/lib/python3.6/shlex.py", line 295, in __next__
token = self.get_token()
File "/usr/lib/python3.6/shlex.py", line 105, in get_token
raw = self.read_token()
File "/usr/lib/python3.6/shlex.py", line 187, in read_token
raise ValueError("No closing quotation")
ValueError: No closing quotation
----------------------------------------------------------
Link: https://github.com/delphix/sdb/issues/new
// After
sdb> echo 1 2 3 | filter "obj != 1
sdb: syntax error: unfinished string expression
echo 1 2 3 | filter "obj != 1
^
```
```
// Before
sdb> ! pwd
sdb: cannot recognize command:
sdb> ! echo hello!
Multiple ! not supported
// After
sdb> ! pwd
/export/home/delphix/sdb
sdb> ! echo hello!
hello!
```
Introduce Basic Single Pass Parser
prakashsurya
approved these changes
May 13, 2020
sdimitro
approved these changes
May 13, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.