From a46cdee07fe24a3004d3fd21d16b52815f249fd7 Mon Sep 17 00:00:00 2001 From: Scott Carda Date: Mon, 18 Oct 2021 16:04:12 -0700 Subject: [PATCH 1/4] Combined information from the SPECIFICATION into the README and updated information. --- src/QsFmt/README.md | 93 +++++++++++++++++++++++++++++++------- src/QsFmt/SPECIFICATION.md | 68 ---------------------------- 2 files changed, 76 insertions(+), 85 deletions(-) delete mode 100644 src/QsFmt/SPECIFICATION.md diff --git a/src/QsFmt/README.md b/src/QsFmt/README.md index d10bc8d5eb..91a9fc3af1 100644 --- a/src/QsFmt/README.md +++ b/src/QsFmt/README.md @@ -1,37 +1,96 @@ # QsFmt: Q# Formatter -QsFmt is a source code formatter for Q#. +QsFmt is a source code formatter and updater for Q#. It's in the very early stages of development and is currently experimental. It will very likely eat your code when it tries to format it! -You can use the command-line tool by running `dotnet run -p App` from this folder. -This will only print the formatted code to the console, and won't overwrite your files, so it's safe to use. +## Usage + +Updates the source code in input files: +    `qsfmt update --inputs Path\To\My\File1.qs Path\To\My\File2.qs` +Updates the source code in project: +    `qsfmt update --project Path\To\My\Project.csproj` + +Command Line Options: +    `-i`, `--inputs`: Required. Files or folders to update. +    `-p`, `--project`: Required. The project file for the project to update. +    `-b`, `--backup`: Option to create backup files of input files. +    `-r`, `--recurse`: Option to process input folders recursively. +    `--qsharp-version`: Option to provide a Q# version to the tool. +    `--help`: Display this help screen. +    `--version`: Display version information. + +Either the `--inputs` option or the `--project` must be used to specify the input files to the tool. +The `--recurse` and `--qsharp-version` options can only be used with the `--inputs` option. + +## Input and Output +Input to the formatter can be specified in one of two ways. +Individual files or folders can be specified with the `--input` command-line argument. +Multiple files and folders can be specified after the argument, but at least one is expected. +.qs extension directly under the folder will be processed. If the `--recurse` option is +specified, subfolders will be processed recursively for Q# files. +The other method of providing input to the formatter is by specifying a Q# project file +with the `--project` command-line argument. When this method is used, exactly one project file +is expected after the argument, and the tool will use MSBuild to determine all applicable Q# source +files under this project. Source files from referenced libraries and other projects will not be processed. +It is worth noting that the tool used to support input directly from the command-line. That is no +longer supported. If there is interest in supporting that method of input, a third input command-line +option specific to that input method may be created in the future. + +The output of the formatter is to overwrite the input files that it processes. The `--backup` +option can be specified to create backup files for all input files with the original content in them. + +## Rules that are run + +Updating rules remove outdated syntax that is deprecated and will be no longer supported in the future. + +Updating rules currently in use: + - Array Syntax - [proposal](https://github.com/microsoft/qsharp-language/blob/main/Approved/2-enhanced-array-literals.md) + - Using and Borrowing Syntax - [proposal](https://github.com/microsoft/qsharp-language/blob/main/Approved/1-implicitly-scoped-qubit-allocation.md) + - Parentheses in For Loop Syntax - Removes deprecated parentheses around for-loop range expressions. + - Unit Syntax - Replaces deprecated unit syntax `()` for `Unit`. + +## Update vs. Format + +Currently the formatter only supports rules that update deprecated syntax +through the use of the `update` command. A `format` command is being worked on +but is not yet ready for release. This command, when ready, will format Q# +code by running such rules like: + - Collapsed Spaces - removes duplicate spaces so that there is only a single space where spaces are used + - Operator Spacing - ensure that operators have the appropriate spaces in them + - New Lines - removes excessive new lines + - Indentation - ensures that there are appropriate indentation for the text ## Limitations -* The current formatting capabilities are very simple, with only basic indentation and whitespace normalization. -* Many types of syntax are not yet supported and will remain unchanged by the formatter. -* There are several bugs related to other types of syntax, especially callable declaration syntax like attributes, type parameters, and specializations. - These may be swallowed by the formatter, resulting in incorrect output. +- Currently the formatter only supports rules that update deprecated syntax through + the use of the `update` command. The `format` command is not yet ready for release. +- Many types of syntax are not yet supported and will remain unchanged by the formatter. +- There are several bugs related to other types of syntax, especially callable declaration syntax like + attributes, type parameters, and specializations. These may be swallowed by the formatter, + resulting in incorrect output. ## Design -QsFmt uses a [concrete syntax tree](https://en.wikipedia.org/wiki/Parse_tree), which is lossless: a Q# program parsed into a CST can be converted back into a string without any loss of information. -QsFmt's syntax tree is modeled on the Q# compiler's abstract syntax tree, but with additional information on every node for tokens like semicolons and curly braces that are unnecessary in an AST. +QsFmt uses a [concrete syntax tree](https://en.wikipedia.org/wiki/Parse_tree), which is lossless: a +Q# source file parsed into a CST can be converted back into a string without any loss of information. +QsFmt's syntax tree is modeled on the Q# compiler's abstract syntax tree, but with additional +information on every node for tokens like semicolons and curly braces that are unnecessary in an AST. Whitespace and comment tokens, known as *trivia tokens*, are attached as a prefix to a non-trivia token. -For example, in the expression `x + y`, `x` has prefix `""`, `+` has prefix `" "`, and `y` has prefix `" "`. +For example, in the expression `x + y`, `x` has prefix `""`, `+` has prefix `" "`, and `y` has +prefix `" "`. QsFmt uses [ANTLR](https://www.antlr.org/) to parse Q# programs. It uses a grammar based on the [grammar in the Q# language specification](https://github.com/microsoft/qsharp-language/tree/main/Specifications/Language/5_Grammar). ANTLR's parse tree is then converted into Q# Formatter's concrete syntax tree. -Formatting rules are a mapping from one CST to another CST. -Then the formatting pipeline is: +Transformation rules are a mapping from one CST to another CST. +Then the transformation pipeline is: -1. Parse a Q# program into a CST. -2. Apply formatting rules to the CST in order. -3. Unparse the CST into a Q# program. +1. Parse a Q# source file into a CST. +2. Apply transformation rules to the CST in order. +3. Unparse the CST into a Q# source file. -This allows formatting transformations (indentation, brace position, etc.) to be written separately, and in many cases may be independent from each other. +This allows transformations to be written separately, and in many cases may be independent from each other. (However, there may be dependencies where one transformation must run before another.) -This will hopefully make the formatting transformations more modular and simpler to write. +This will hopefully make the transformation rules more modular and simpler to write. diff --git a/src/QsFmt/SPECIFICATION.md b/src/QsFmt/SPECIFICATION.md deleted file mode 100644 index 6f0b342f73..0000000000 --- a/src/QsFmt/SPECIFICATION.md +++ /dev/null @@ -1,68 +0,0 @@ -# Q# Formatter Design Specification Document - -QsFmt is a source code formatter for Q#. - -## Where to get the formatter - -The `Microsoft.Quantum.QSharp.Formatter` NuGet package will be distributed with the rest of Q# on NuGet.org. -As a dotnet tool, it is installed with the command `dotnet tool install Microsoft.Quantum.QSharp.Formatter`. - -## How to use the command line formatter - -After installing the tool the command `qsfmt` is used to run the formatter. -There are two commands supported for the tool: 'format' and 'update'. -The 'format' command runs the tool as a formatter for Q# code, running only those transformation rules that update whitespace and affect indentation. The underlying code will not be changed. -The 'update' command allows the tool to be used as a means of updating old Q# code, replacing deprecated syntax with newer supported syntax. These transformation rules change the actual code given to them, and may occasionally fail. -These two commands work separately and can't be run together. If the user wishes to both format and update their code, they would need to run the tool twice, first with 'update' and then with 'format'. -Both of these commands should be idempotent, meaning that iterated uses of the commands will not continue to have effects on the inputs. - -The tool will initially only support the 'update' command, as the formatting functionality is still being worked on. The 'format' command will be unavailable until it is supported. - -## Input and Output -Input files to the formatter will be the last argument(s) to the tool. -At least one input argument is expected. If no inputs are given, usage information is printed. -Inputs can be given as one to many paths to directories, files, or any combination. In directories specified, all files with the '.qs' extension found will be taken as input. -Basic wild cards may be used in the paths. -The `--recurse` option can be specified to process the input directories recursively for input files. You can specify this for all input directories, not per-directory. -The `--backup` option can be specified to create backup files for all input files with the original content in them. - -The output of the formatter is to overwrite the input files that it processes. - -## Rules that are run - -The tool will parse the input into a concrete syntax tree using ANTLR. This tree will then have rules applied to it in the form of transformations on the tree. -The transformations are separated into two groups: transformations run by the 'format' command, and transformations run by the 'update' command. -Listed here are some of the transformations we intend to use. - -### Formatting Transformations - - - Collapsed Spaces - removes duplicate spaces so that there is only a single space where spaces are used - - Operator Spacing - ensure that operators have the appropriate spaces in them - - New Lines - removes excessive new lines - - Indentation - ensures that there are appropriate indentation for the text - -### Updating Transformations - -Updating transformations remove outdated syntax that is deprecated and will be no longer supported in the future. - -Some syntaxes that can be updated are: - - Array Syntax - [proposal](https://github.com/microsoft/qsharp-language/blob/main/Approved/2-enhanced-array-literals.md) - - Using and Borrowing Syntax - [proposal](https://github.com/microsoft/qsharp-language/blob/main/Approved/1-implicitly-scoped-qubit-allocation.md) - - Parentheses in For Loop Syntax - Removes deprecated parentheses around for-loop range expressions. - - Boolean Operator Syntax - Replaces deprecated boolean operators (`&&`, `||`, `!`) for keywords (`and`, `or`, `not`). - - Unit Syntax - Replaces deprecated unit syntax `()` for `Unit`. - - Body and Adjoint Functor Argument Syntax - When specifying `body` and `adjoint` functors on an operation, replaces the deprecated empty argument `()` with the required `(...)`. - -## How errors are handled - -If an internal unhandled exception is thrown by the tool, that will be surfaced to the user and is a bug with the tool that should be filed and addressed. -Exceptions like these will prevent any output from being written on the file that caused the exception. The tool will try to recover from the exception to process other input files. - -Parsing errors may occur if the formatter is given text that is not proper Q# code. In this case ANTLR gives the appropriate error message. -Parsing errors prevent the given input file from being processed. Other input files may still be processed. -Errors should not occur during a formatting transformation. All formatting transformations should never encounter the syntax that they are expected to change but are unable to change. -Warnings may occur during an updating transformation of the concrete syntax tree if an updating transformation encounters an issue, such as syntax that it would be expected to update, but can't for some reason. -Warnings like this will prevent the referenced code piece from being updated, but the rest of the file will still be processed. - -Errors and warnings of all kinds should be collected and reported to the user through stderr. -Errors and warnings should be handled in a way so as not to affect the tool's idempotency. From 8d496261941a38c00893abbab259472035afce93 Mon Sep 17 00:00:00 2001 From: Scott Carda Date: Tue, 19 Oct 2021 17:13:45 -0700 Subject: [PATCH 2/4] Added Building From Source section --- src/QsFmt/README.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/QsFmt/README.md b/src/QsFmt/README.md index 91a9fc3af1..af5273fcb6 100644 --- a/src/QsFmt/README.md +++ b/src/QsFmt/README.md @@ -94,3 +94,15 @@ Then the transformation pipeline is: This allows transformations to be written separately, and in many cases may be independent from each other. (However, there may be dependencies where one transformation must run before another.) This will hopefully make the transformation rules more modular and simpler to write. + +## Building From Source + +To build the tool from the source code, you will need to have Java installed. The latest version of +Java can be downloaded from Oracle's website found [here](https://www.oracle.com/java/technologies/downloads/). +Java is used by ANTLR to build the concrete syntax tree. + +Once Java in installed, you may build the App project by executing the command +``` +dotnet build ./App/App.fsproj +``` +from this directory. From a41ca458e519058f8c759c46b640b970a2c8b310 Mon Sep 17 00:00:00 2001 From: Scott Carda Date: Thu, 21 Oct 2021 14:12:47 -0700 Subject: [PATCH 3/4] renamed --inputs to --input, moved Building From Source to top. Removed comments about 'experimental' from opening section. --- src/QsFmt/README.md | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/src/QsFmt/README.md b/src/QsFmt/README.md index af5273fcb6..8ce59614f3 100644 --- a/src/QsFmt/README.md +++ b/src/QsFmt/README.md @@ -1,18 +1,27 @@ # QsFmt: Q# Formatter QsFmt is a source code formatter and updater for Q#. -It's in the very early stages of development and is currently experimental. -It will very likely eat your code when it tries to format it! + +## Building From Source + +To build the tool from the source code, you will need to have Java installed. The latest version of +Java can be downloaded from Oracle's website found [here](https://www.oracle.com/java/technologies/downloads/). +Java is used by ANTLR to build the concrete syntax tree. + +Once Java in installed, you may build the App project by executing the following command from this directory: +``` +dotnet build ./App/App.fsproj +``` ## Usage Updates the source code in input files: -    `qsfmt update --inputs Path\To\My\File1.qs Path\To\My\File2.qs` +    `qsfmt update --input Path\To\My\File1.qs Path\To\My\File2.qs` Updates the source code in project:     `qsfmt update --project Path\To\My\Project.csproj` Command Line Options: -    `-i`, `--inputs`: Required. Files or folders to update. +    `-i`, `--input`: Required. Files or folders to update.     `-p`, `--project`: Required. The project file for the project to update.     `-b`, `--backup`: Option to create backup files of input files.     `-r`, `--recurse`: Option to process input folders recursively. @@ -20,8 +29,8 @@ Command Line Options:     `--help`: Display this help screen.     `--version`: Display version information. -Either the `--inputs` option or the `--project` must be used to specify the input files to the tool. -The `--recurse` and `--qsharp-version` options can only be used with the `--inputs` option. +Either the `--input` option or the `--project` must be used to specify the input files to the tool. +The `--recurse` and `--qsharp-version` options can only be used with the `--input` option. ## Input and Output Input to the formatter can be specified in one of two ways. @@ -94,15 +103,3 @@ Then the transformation pipeline is: This allows transformations to be written separately, and in many cases may be independent from each other. (However, there may be dependencies where one transformation must run before another.) This will hopefully make the transformation rules more modular and simpler to write. - -## Building From Source - -To build the tool from the source code, you will need to have Java installed. The latest version of -Java can be downloaded from Oracle's website found [here](https://www.oracle.com/java/technologies/downloads/). -Java is used by ANTLR to build the concrete syntax tree. - -Once Java in installed, you may build the App project by executing the command -``` -dotnet build ./App/App.fsproj -``` -from this directory. From 3c205629e636c9f23894c13efbe5dffd27d8370e Mon Sep 17 00:00:00 2001 From: Scott Carda Date: Wed, 27 Oct 2021 14:45:52 -0700 Subject: [PATCH 4/4] Updated for new boolean operator rule. --- src/QsFmt/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/QsFmt/README.md b/src/QsFmt/README.md index 8ce59614f3..91a7c7ac20 100644 --- a/src/QsFmt/README.md +++ b/src/QsFmt/README.md @@ -58,6 +58,8 @@ Updating rules currently in use: - Using and Borrowing Syntax - [proposal](https://github.com/microsoft/qsharp-language/blob/main/Approved/1-implicitly-scoped-qubit-allocation.md) - Parentheses in For Loop Syntax - Removes deprecated parentheses around for-loop range expressions. - Unit Syntax - Replaces deprecated unit syntax `()` for `Unit`. + - Boolean Operator Syntax - Replaces deprecated use of boolean operators `&&`, `||`, and `!` with + their keyword equivalence `and`, `or`, and `not` respectively. ## Update vs. Format