[RFC FS-1001] String Interpolation (dotnet#8907)

* string interploation implementation

* string interploation tests

* escape {{ }}, test verbatim and triple quote, implement .NET specifiers

* fix tests

* string interpolation tests: internal representation corner cases

* string-interp tests should have --langversion:preview

* string interop tests: sprintf

* string interp tests: format specifier negative cases

* string interp tests: format specifier negative cases, .NET-style padding

* fix nested interp strings

* style cleanup

* lex: unify string interp stack and counter

* string-interp: add test cases

* fix mixed quote nested string interpolation

* string-interp: add test case for multiple interpolation points with different indentation

* lexfilter: push new CtxtParen at endPos for INTERP_STRING_PART and INTERP_STRING_BEGIN_PART

* lexfilter: do not check undentation limit for string interpolation tokens.

* FormattableString prototype

* add FormattableString support

* negative error checking

* remove diagnostics

* simpler FormattableString implementation

* fix test

* add testing for nested

* add IFormattable support

* tweak error message

* tests: StringInterpolation: fix case errors

* fix error message

* check number of values matches

* allow use of format strings with printf and friends

* update baselines

* fix baselines

* add Experimental attributes

* update string interp negative tests

* stringinterp test: add PrintFormat tests

* printf: fix empty interpolation string evaluates to null in printf env

* enable test corectly

* Revert "printf: fix empty interpolation string evaluates to null in printf env"

This reverts commit 7f39617.

* simplify codegen for interpolated strings

* fix build

* fix build

* Merge master to feature/string-interp (dotnet#9580)

* Update dependencies from https://github.com/dotnet/arcade build 20200626.2 (dotnet#9577)

Microsoft.DotNet.Arcade.Sdk
 From Version 1.0.0-beta.20302.3 -> To Version 1.0.0-beta.20326.2

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Improve perf for String.filter up to 3x (dotnet#9509)

* Improve perf for String.filter 2-2.5x

* Cleanup: remove "foo" etc in tests

* Add tests for new execution path for LOH in String.filter

* Change test string

* String map performance improvement (dotnet#9470)

* Simplify and improve perf of String.length

* Improve performance of String.map

* Revert "Simplify and improve perf of String.length"

* Resolves dotnet#9470 (comment)

* Lingering space

* Change `String` to use `new` to clarify use of ctor

* Add some better tests for String.map, add side-effect test

* Add tests to ensure the mapping function is called a deterministically amount of times

* Fix typo

* Remove "foo" from String.map tests

* Perf: String.replicate from O(n) to O(log(n)), up to 12x speed improvement (dotnet#9512)

* Turn String.replicate from O(n) into O(log(n))

* Cleanup String.replicate tests by removing usages of "foo"

* String.replicate: add tests for missing cases, and for the new O(log(n)) cut-off points

* Improve String.replicate algorithm further

* Add tests for String.replicate covering all lines/branches of algo

* Fix accidental comment

Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Abel Braaksma <[email protected]>

* Re enable tests for operators: OperatorsModule1.fs and OperatorsModule2.fs (dotnet#9516) (dotnet#9589)

* Re-enabling tests from OperatorsModule1/2.fs (compile errors)

* Fix compile errors in OperatorsModule1/2.fs, fix tests. Note tanh test comment.

* Fix `tanh` test, ensure stable result on x86 vs x64 runtimes

* Stop using exception AssertionException, so that test window shows useful info

* Whitespace cleanup and redundant code removal

* Cleanup spelling etc

* Re-enabling int, int16, int32, int64, nativeint, incr, nullArg etc tests

* Special-case floating-point assertion messages for higher precision output

* Fix/update/add tests (some still failing)

* Separate Checked tests, add & fix others, differentiate framework/bitness for some tests

* Add branch for .NET Native (ignore cos test)

* Resorting to comparing floats with a delta using Assert.AreNearEqual

* Add some more tests

Co-authored-by: Abel Braaksma <[email protected]>

* Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) (dotnet#9599)

Co-authored-by: Thorsten Reichert <[email protected]>

* Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) (dotnet#9604)

Co-authored-by: Thorsten Reichert <[email protected]>

* Merge master to feature/string-interp (dotnet#9615)

* Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576)

* Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (dotnet#9611)

* Migrated PreComputedTupleConstructor01.fs test case

* Migrated PreComputedTupleConstructor02.fs test case

* Migrated DU.fs and Record.fs test cases

* Allow notebook to discover location of shared framework (dotnet#9596)

Co-authored-by: Thorsten Reichert <[email protected]>
Co-authored-by: Kevin Ransom (msft) <[email protected]>
Co-authored-by: Phillip Carter <[email protected]>

* Merge master to feature/string-interp (dotnet#9619)

* Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576)

* Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (dotnet#9611)

* Migrated PreComputedTupleConstructor01.fs test case

* Migrated PreComputedTupleConstructor02.fs test case

* Migrated DU.fs and Record.fs test cases

* Allow notebook to discover location of shared framework (dotnet#9596)

Co-authored-by: Thorsten Reichert <[email protected]>
Co-authored-by: Kevin Ransom (msft) <[email protected]>

* Text tweeks

* don't auto-resolve types from System.Runtime.WindowsRuntime (dotnet#9644) (dotnet#9648)

Co-authored-by: Brett V. Forsgren <[email protected]>

* yeet (dotnet#9657) (dotnet#9661)

yeet

Co-authored-by: Phillip Carter <[email protected]>

* yeet (dotnet#9657) (dotnet#9670)

yeet

Co-authored-by: Phillip Carter <[email protected]>

* fix up tokenizer tests

* fix code review things

* fix code review things

* fix code review things

* fix code review things

* add various testing

* correct continuations for interpolated strings

* fix lexer continuations and colorization for multi-line interpolated strings

* revert xlf changes

* fix assert

* completion and brace matching (not all tests passing yet)

* Fix rebuild

* fix various niggles and get tests working

* fix printf when '%a' in final position

* fix test case

* interpolated string specifer highlighting

* fix triple quote interpolated string specifer highlighting

* fix triple quote interpolated string specifer highlighting

* fix build

* fix missing error message

* fix % specifiers for interpolated strings

* fix % specifiers for interpolated strings

* fix FCS tests

* minor nits from code review

* code review feedback and use struct tuples in more places

* revert struct tuples

* use struct tuples where possible, byrefs for index

* fix byref for index

* fix ksprintf block size

* make recent cache entry more explicit (cleanup)

* improve performance

* remove unused code

* Move existing Compiler.ComponentTests to a new Compiler.fs framework (dotnet#9839) (dotnet#9848)

* Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function

* Changed some wording in error messages

Co-authored-by: Vlad Zarytovskii <[email protected]>

* Move existing Compiler.ComponentTests to a new Compiler.fs framework (dotnet#9839)

* Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function

* Changed some wording in error messages

* fix dotnet#9893

* fix unmantched right brace in interp string

Co-authored-by: Yatao Li <[email protected]>
Co-authored-by: Kevin Ransom (msft) <[email protected]>
Co-authored-by: dotnet bot <[email protected]>
Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Abel Braaksma <[email protected]>
Co-authored-by: Thorsten Reichert <[email protected]>
Co-authored-by: Phillip Carter <[email protected]>
Co-authored-by: Brett V. Forsgren <[email protected]>
Co-authored-by: Vlad Zarytovskii <[email protected]>

Author: 10 people

AutomaticCompletion/BraceCompletionSessionProvider.fs

@@ -459,30 +459,50 @@ type AsteriskCompletionSession() =
 [<ExportLanguageService(typeof<IEditorBraceCompletionSessionFactory>, FSharpConstants.FSharpLanguageName)>]
 type EditorBraceCompletionSessionFactory() =
 
+    let spanIsNotCommentOrString (span: ClassifiedSpan) =
+        match span.ClassificationType with
+        | ClassificationTypeNames.Comment
+        | ClassificationTypeNames.StringLiteral -> false
+        | _ -> true 
+
     member __.IsSupportedOpeningBrace openingBrace =
         match openingBrace with
         | Parenthesis.OpenCharacter | CurlyBrackets.OpenCharacter | SquareBrackets.OpenCharacter
         | DoubleQuote.OpenCharacter | VerticalBar.OpenCharacter | AngleBrackets.OpenCharacter 
         | Asterisk.OpenCharacter -> true
         | _ -> false
 
-    member __.CheckCodeContext(document: Document, position: int, _openingBrace, cancellationToken) =
-        // We need to know if we are inside a F# comment. If we are, then don't do automatic completion.
+    member __.CheckCodeContext(document: Document, position: int, _openingBrace:char, cancellationToken) =
+        // We need to know if we are inside a F# string or comment. If we are, then don't do automatic completion.
         let sourceCodeTask = document.GetTextAsync(cancellationToken)
         sourceCodeTask.Wait(cancellationToken)
         let sourceCode = sourceCodeTask.Result
 
         position = 0 
-        || let colorizationData = Tokenizer.getClassifiedSpans(document.Id, sourceCode, TextSpan(position - 1, 1), Some (document.FilePath), [ ], cancellationToken)
-           in colorizationData.Count = 0 
-              || colorizationData.Exists(fun classifiedSpan -> 
-                    classifiedSpan.TextSpan.IntersectsWith position &&
-                    (
-                        match classifiedSpan.ClassificationType with
-                        | ClassificationTypeNames.Comment
-                        | ClassificationTypeNames.StringLiteral -> false
-                        | _ -> true // anything else is a valid classification type
-                    ))                
+        || (let colorizationData = Tokenizer.getClassifiedSpans(document.Id, sourceCode, TextSpan(position - 1, 1), Some (document.FilePath), [ ], cancellationToken)
+            colorizationData.Count = 0
+            || 
+            colorizationData.Exists(fun classifiedSpan -> 
+                classifiedSpan.TextSpan.IntersectsWith position &&
+                spanIsNotCommentOrString classifiedSpan)) 
+
+            // This would be the case where '{' has been pressed in a string and the next position
+            // is known not to be a string.  This corresponds to the end of an interpolated string part.
+            //
+            // However, Roslyn doesn't activate BraceCompletionSessionProvider for string text at all (and at the time '{
+            // is pressed the text is classified as a string). So this code doesn't get called at all and so
+            // no brace completion is available inside interpolated strings.
+            //
+            // || (openingBrace = '{' &&
+            // colorizationData.Exists(fun classifiedSpan -> 
+            //    classifiedSpan.TextSpan.IntersectsWith (position-1) &&
+            //    spanIsString classifiedSpan) && 
+            // let colorizationData2 = Tokenizer.getClassifiedSpans(document.Id, sourceCode, TextSpan(position, 1), Some (document.FilePath), [ ], cancellationToken)
+            // (colorizationData2.Count = 0 
+            //  || 
+            //  colorizationData2.Exists(fun classifiedSpan -> 
+            //    classifiedSpan.TextSpan.IntersectsWith position &&
+            //    not (spanIsString classifiedSpan)))))
 
     member __.CreateEditorSession(_document, _openingPosition, openingBrace, _cancellationToken) =
         match openingBrace with

Completion/CompletionUtils.fs

@@ -89,7 +89,8 @@ module internal CompletionUtils =
         let triggerLine = textLines.GetLineFromPosition triggerPosition
         let classifiedSpans = Tokenizer.getClassifiedSpans(documentId, sourceText, triggerLine.Span, Some filePath, defines, CancellationToken.None)
         classifiedSpans.Count = 0 || // we should provide completion at the start of empty line, where there are no tokens at all
-        classifiedSpans.Exists (fun classifiedSpan -> 
+        let result =
+          classifiedSpans.Exists (fun classifiedSpan -> 
             classifiedSpan.TextSpan.IntersectsWith triggerPosition &&
             (
                 match classifiedSpan.ClassificationType with
@@ -100,6 +101,7 @@ module internal CompletionUtils =
                 | ClassificationTypeNames.NumericLiteral -> false
                 | _ -> true // anything else is a valid classification type
             ))
+        result
 
     let inline getKindPriority kind =
         match kind with