From ef87ed646fb429c59bad767e4489c96560accf80 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Sun, 8 Sep 2024 06:20:42 +0300 Subject: [PATCH 1/6] Raw lifetimes --- src/tokens.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/tokens.md b/src/tokens.md index fe2b5534f..711b39a65 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -759,10 +759,15 @@ r[lex.token.life.syntax] > _(not immediately followed by `'`)_\ >    | `'_` > _(not immediately followed by `'`)_ +>    | RAW_LIFETIME > > LIFETIME_OR_LABEL :\ >       `'` [NON_KEYWORD_IDENTIFIER][identifier] > _(not immediately followed by `'`)_ +>    | RAW_LIFETIME +> +> RAW_LIFETIME :\ +>       `'r#` [IDENTIFIER_OR_KEYWORD][identifier] r[lex.token.life.intro] Lifetime parameters and [loop labels] use LIFETIME_OR_LABEL tokens. Any From df53c5e330011ac165a80ee635305f20de9d6a5b Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Sun, 8 Sep 2024 06:21:54 +0300 Subject: [PATCH 2/6] Don't allow `'r#kw'` --- src/tokens.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tokens.md b/src/tokens.md index 711b39a65..3b6be971a 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -768,6 +768,7 @@ r[lex.token.life.syntax] > > RAW_LIFETIME :\ >       `'r#` [IDENTIFIER_OR_KEYWORD][identifier] +> _(not immediately followed by `'`)_ r[lex.token.life.intro] Lifetime parameters and [loop labels] use LIFETIME_OR_LABEL tokens. Any From 5c0c306bd82c3f26fff4b75936eb7e41274d36ef Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 30 Oct 2024 00:49:25 +0000 Subject: [PATCH 3/6] update --- src/tokens.md | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/src/tokens.md b/src/tokens.md index 3b6be971a..84b7e0aa1 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -758,16 +758,16 @@ r[lex.token.life.syntax] >       `'` [IDENTIFIER_OR_KEYWORD][identifier] > _(not immediately followed by `'`)_\ >    | `'_` -> _(not immediately followed by `'`)_ +> _(not immediately followed by `'`)_\ >    | RAW_LIFETIME > > LIFETIME_OR_LABEL :\ >       `'` [NON_KEYWORD_IDENTIFIER][identifier] -> _(not immediately followed by `'`)_ +> _(not immediately followed by `'`)_\ >    | RAW_LIFETIME > > RAW_LIFETIME :\ ->       `'r#` [IDENTIFIER_OR_KEYWORD][identifier] +>    `'r#` [IDENTIFIER_OR_KEYWORD][identifier] > _(not immediately followed by `'`)_ r[lex.token.life.intro] @@ -775,6 +775,15 @@ Lifetime parameters and [loop labels] use LIFETIME_OR_LABEL tokens. Any LIFETIME_TOKEN will be accepted by the lexer, and for example, can be used in macros. +r[lex.token.life.raw.intro] +A raw lifetime is like a normal lifetime, but its identifier is prefixed by `r#`. (Note that the `r#` prefix is not included as part of the actual lifetime.) + +r[lex.token.life.raw.allowed] +Unlike a normal lifetime, a raw lifetime may be any strict or reserved keyword. + +> **Edition differences**: Raw lifetimes are accepted in the 2021 +> edition or later. In earlier additions the token `'r#lt` is lexed as `'r # lt`. + ## Punctuation r[lex.token.punct] @@ -855,7 +864,8 @@ r[lex.token.reserved-prefix.syntax] > **Lexer 2021+**\ > RESERVED_TOKEN_DOUBLE_QUOTE : ( IDENTIFIER_OR_KEYWORD _Except `b` or `c` or `r` or `br` or `cr`_ | `_` ) `"`\ > RESERVED_TOKEN_SINGLE_QUOTE : ( IDENTIFIER_OR_KEYWORD _Except `b`_ | `_` ) `'`\ -> RESERVED_TOKEN_POUND : ( IDENTIFIER_OR_KEYWORD _Except `r` or `br` or `cr`_ | `_` ) `#` +> RESERVED_TOKEN_POUND : ( IDENTIFIER_OR_KEYWORD _Except `r` or `br` or `cr`_ | `_` ) `#`\ +> RESERVED_TOKEN_LIFETIME : `'` (IDENTIFIER_OR_KEYWORD _Except `r`_ | _) `#` r[lex.token.reserved-prefix.intro] Some lexical forms known as _reserved prefixes_ are reserved for future use. @@ -869,6 +879,9 @@ Note that raw identifiers, raw string literals, and raw byte string literals may r[lex.token.reserved-prefix.strings] Similarly the `r`, `b`, `br`, `c`, and `cr` prefixes used in raw string literals, byte literals, byte string literals, raw byte string literals, C string literals, and raw C string literals are not interpreted as reserved prefixes. +r[lex.token.reserved-prefix.life] +Source input which would otherwise be lexically interpreted as a non-raw lifetime (or a keyword or `_`) which is immediately followed by a `#` character (without intervening whitespace) is identified as a reserved lifetime prefix. + r[lex.token.reserved-prefix.edition2021] > **Edition differences**: Starting with the 2021 edition, reserved prefixes are reported as an error by the lexer (in particular, they cannot be passed to macros). > @@ -881,6 +894,7 @@ r[lex.token.reserved-prefix.edition2021] > lexes!{continue 'foo} > lexes!{match "..." {}} > lexes!{r#let#foo} // three tokens: r#let # foo +> lexes!{'prefix #lt} > ``` > > Examples accepted before the 2021 edition but rejected later: @@ -889,6 +903,7 @@ r[lex.token.reserved-prefix.edition2021] > lexes!{a#foo} > lexes!{continue'foo} > lexes!{match"..." {}} +> lexes!{'prefix#lt} > ``` [Inferred types]: types/inferred.md From f04cd638be4ccc2a204bc5efd2842636e8ece084 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Thu, 14 Nov 2024 08:25:52 -0800 Subject: [PATCH 4/6] Clarify raw lifetime cannot be certain keywords Update for https://github.com/rust-lang/rust/pull/132363 --- src/tokens.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/tokens.md b/src/tokens.md index 84b7e0aa1..7568f02e3 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -767,7 +767,7 @@ r[lex.token.life.syntax] >    | RAW_LIFETIME > > RAW_LIFETIME :\ ->    `'r#` [IDENTIFIER_OR_KEYWORD][identifier] +>    `'r#` [IDENTIFIER_OR_KEYWORD][identifier] *Except `crate`, `self`, `super`, `Self`* > _(not immediately followed by `'`)_ r[lex.token.life.intro] @@ -779,7 +779,7 @@ r[lex.token.life.raw.intro] A raw lifetime is like a normal lifetime, but its identifier is prefixed by `r#`. (Note that the `r#` prefix is not included as part of the actual lifetime.) r[lex.token.life.raw.allowed] -Unlike a normal lifetime, a raw lifetime may be any strict or reserved keyword. +Unlike a normal lifetime, a raw lifetime may be any strict or reserved keyword except the ones listed above for `RAW_LIFETIME`. > **Edition differences**: Raw lifetimes are accepted in the 2021 > edition or later. In earlier additions the token `'r#lt` is lexed as `'r # lt`. From f490b7ca26f9d09a8ce0f4ce99f3156fc97ed785 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Thu, 14 Nov 2024 19:17:51 -0800 Subject: [PATCH 5/6] Add rule name for raw lifetime edition differences --- src/tokens.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tokens.md b/src/tokens.md index 7568f02e3..e99982703 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -781,6 +781,7 @@ A raw lifetime is like a normal lifetime, but its identifier is prefixed by `r#` r[lex.token.life.raw.allowed] Unlike a normal lifetime, a raw lifetime may be any strict or reserved keyword except the ones listed above for `RAW_LIFETIME`. +r[lex.token.life.raw.edition2021] > **Edition differences**: Raw lifetimes are accepted in the 2021 > edition or later. In earlier additions the token `'r#lt` is lexed as `'r # lt`. From baa3d73527763aea23f3c43ac676e7bae230b542 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Thu, 14 Nov 2024 19:57:33 -0800 Subject: [PATCH 6/6] Document r#_ and 'r#_ as reserved These are rejected by the lexer. --- src/identifiers.md | 6 ++++++ src/tokens.md | 7 +++++++ 2 files changed, 13 insertions(+) diff --git a/src/identifiers.md b/src/identifiers.md index f34ae482e..a10a3f9f3 100644 --- a/src/identifiers.md +++ b/src/identifiers.md @@ -14,6 +14,8 @@ r[ident.syntax] > > IDENTIFIER :\ > NON_KEYWORD_IDENTIFIER | RAW_IDENTIFIER +> +> RESERVED_RAW_IDENTIFIER : `r#_` r[ident.unicode] @@ -71,6 +73,10 @@ r[ident.raw.allowed] Unlike a normal identifier, a raw identifier may be any strict or reserved keyword except the ones listed above for `RAW_IDENTIFIER`. +r[ident.raw.reserved] +It is an error to use the RESERVED_RAW_IDENTIFIER token `r#_` in order to avoid confusion with the [_WildcardPattern_]. + +[_WildcardPattern_]: patterns.md#wildcard-pattern [`extern crate`]: items/extern-crates.md [`no_mangle`]: abi.md#the-no_mangle-attribute [`path` attribute]: items/modules.md#the-path-attribute diff --git a/src/tokens.md b/src/tokens.md index e99982703..212d678df 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -769,6 +769,9 @@ r[lex.token.life.syntax] > RAW_LIFETIME :\ >    `'r#` [IDENTIFIER_OR_KEYWORD][identifier] *Except `crate`, `self`, `super`, `Self`* > _(not immediately followed by `'`)_ +> +> RESERVED_RAW_LIFETIME : `'r#_` +> _(not immediately followed by `'`)_ r[lex.token.life.intro] Lifetime parameters and [loop labels] use LIFETIME_OR_LABEL tokens. Any @@ -781,6 +784,9 @@ A raw lifetime is like a normal lifetime, but its identifier is prefixed by `r#` r[lex.token.life.raw.allowed] Unlike a normal lifetime, a raw lifetime may be any strict or reserved keyword except the ones listed above for `RAW_LIFETIME`. +r[lex.token.life.raw.reserved] +It is an error to use the RESERVED_RAW_LIFETIME token `'r#_` in order to avoid confusion with the [placeholder lifetime]. + r[lex.token.life.raw.edition2021] > **Edition differences**: Raw lifetimes are accepted in the 2021 > edition or later. In earlier additions the token `'r#lt` is lexed as `'r # lt`. @@ -946,6 +952,7 @@ r[lex.token.reserved-prefix.edition2021] [numeric types]: types/numeric.md [paths]: paths.md [patterns]: patterns.md +[placeholder lifetime]: lifetime-elision.md [question]: expressions/operator-expr.md#the-question-mark-operator [range]: expressions/range-expr.md [rangepat]: patterns.md#range-patterns