From a403aa175c563653286e1991bd5701613b141372 Mon Sep 17 00:00:00 2001 From: Oliver Schneider Date: Fri, 1 Jun 2018 17:34:45 +0200 Subject: [PATCH 1/5] Add some explanation of lowering ids --- src/high-level-overview.md | 3 ++- src/lowering.md | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 src/lowering.md diff --git a/src/high-level-overview.md b/src/high-level-overview.md index 60b9e80ea..733c523c9 100644 --- a/src/high-level-overview.md +++ b/src/high-level-overview.md @@ -105,7 +105,7 @@ take: 3. **Lowering to HIR** - Once name resolution completes, we convert the AST into the HIR, or "[high-level intermediate representation]". The HIR is defined in - `src/librustc/hir/`; that module also includes the lowering code. + `src/librustc/hir/`; that module also includes the [lowering] code. - The HIR is a lightly desugared variant of the AST. It is more processed than the AST and more suitable for the analyses that follow. It is **not** required to match the syntax of the Rust language. @@ -139,3 +139,4 @@ take: [query model]: query.html [high-level intermediate representation]: hir.html +[lowering]: lowering.html \ No newline at end of file diff --git a/src/lowering.md b/src/lowering.md new file mode 100644 index 000000000..335513342 --- /dev/null +++ b/src/lowering.md @@ -0,0 +1,35 @@ +# Lowering + +The lowering step converts AST to [HIR](hir.html). +This means many structures are removed if they are irrelevant +for type analysis or similar syntax agnostic analyses. Examples +of such structures include but are not limited to + +* Parenthesis + * Removed without replacement, the tree structure makes order explicit +* `for` loops and `while (let)` loops + * Converted to `loop` + `match` and some `let` bindings +* `if let` + * Converted to `match` +* Universal `impl Trait` + * Converted to generic arguments (but with some flags, to know that the user didn't write them) +* Existential `impl Trait` + * Converted to a virtual `existential type` declaration + +Lowering needs to uphold several invariants in order to not trigger the +sanity checks in `src/librustc/hir/map/hir_id_validator.rs`: + +1. A `HirId` must be used if created. So if you use the `lower_node_id`, + you *must* use the resulting `NodeId` or `HirId` (either is fine, since + any `NodeId`s in the `HIR` are checked for existing `HirId`s) +2. Lowering a `HirId` must be done in the scope of the *owning* item. + This means you need to use `with_hir_id_owner` if you are creating parts + of another item than the one being currently lowered. This happens for + example during the lowering of existential `impl Trait` +3. A `NodeId` that will be placed into a HIR structure must be lowered, + even if its `HirId` is unused. Calling + `let _ = self.lower_node_id(node_id);` is perfectly legitimate. +4. If you are creating new nodes that didn't exist in the `AST`, you *must* + create new ids for them. This is done by calling the `next_id` method, + which produces both a new `NodeId` as well as automatically lowering it + for you so you also get the `HirId`. \ No newline at end of file From c9e937f27562de33e502499cea71ff4f01302281 Mon Sep 17 00:00:00 2001 From: Oliver Schneider Date: Thu, 5 Jul 2018 09:30:05 +0200 Subject: [PATCH 2/5] Address review --- src/SUMMARY.md | 1 + src/hir.md | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index f60fee488..a41f78a1a 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -20,6 +20,7 @@ - [Macro expansion](./macro-expansion.md) - [Name resolution](./name-resolution.md) - [The HIR (High-level IR)](./hir.md) + - [Lowering AST to HIR](./lowering.md) - [The `ty` module: representing types](./ty.md) - [Type inference](./type-inference.md) - [Trait solving (old-style)](./traits/resolution.md) diff --git a/src/hir.md b/src/hir.md index 2a11531ee..0d6a6fbff 100644 --- a/src/hir.md +++ b/src/hir.md @@ -3,7 +3,8 @@ The HIR – "High-Level Intermediate Representation" – is the primary IR used in most of rustc. It is a compiler-friendly representation of the abstract syntax tree (AST) that is generated after parsing, macro expansion, and name -resolution. Many parts of HIR resemble Rust surface syntax quite closely, with +resolution (see [Lowering](./lowering.md) for how the HIR is created). +Many parts of HIR resemble Rust surface syntax quite closely, with the exception that some of Rust's expression forms have been desugared away. For example, `for` loops are converted into a `loop` and do not appear in the HIR. This makes HIR more amenable to analysis than a normal AST. From a56776b77dc5489a70e7a1a03e296e1a55b7401f Mon Sep 17 00:00:00 2001 From: Oliver Schneider Date: Thu, 5 Jul 2018 09:34:37 +0200 Subject: [PATCH 3/5] Add some details about `DefId` creation --- src/lowering.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/src/lowering.md b/src/lowering.md index 335513342..cc2a7b384 100644 --- a/src/lowering.md +++ b/src/lowering.md @@ -32,4 +32,16 @@ sanity checks in `src/librustc/hir/map/hir_id_validator.rs`: 4. If you are creating new nodes that didn't exist in the `AST`, you *must* create new ids for them. This is done by calling the `next_id` method, which produces both a new `NodeId` as well as automatically lowering it - for you so you also get the `HirId`. \ No newline at end of file + for you so you also get the `HirId`. + +If you are creating new `DefId`s, since each `DefId` needs to have a corresponding +`NodeId`, it is adviseable to add these `NodeId`s to the `AST` so you don't have +to generate new ones during lowering. This has the advantage of creating a +way to find the `DefId` of something via its `NodeId`. If lowering needs this +`DefId` in multiple places, you can't generate a new `NodeId` in all those places +because you'd also get a new `DefId` then. With a `NodeId` from the `AST` this is +not an issue. + +Having the `NodeId` also allows the `DefCollector` to generate the `DefId`s instead +of lowering having to do it on the fly. Centralizing the `DefId` generation in one +place makes it easier to refactor and reason about. \ No newline at end of file From ad24ff08ea5e60423a97e8121f58e1c58cfc1736 Mon Sep 17 00:00:00 2001 From: Oliver Schneider Date: Fri, 6 Jul 2018 14:25:34 +0200 Subject: [PATCH 4/5] Satisfy tidy checks --- src/hir.md | 12 ++++++------ src/lowering.md | 23 ++++++++++++----------- src/name-resolution.md | 12 ++++++------ 3 files changed, 24 insertions(+), 23 deletions(-) diff --git a/src/hir.md b/src/hir.md index 0d6a6fbff..0fd2fcff5 100644 --- a/src/hir.md +++ b/src/hir.md @@ -1,13 +1,13 @@ # The HIR -The HIR – "High-Level Intermediate Representation" – is the primary IR used in -most of rustc. It is a compiler-friendly representation of the abstract syntax -tree (AST) that is generated after parsing, macro expansion, and name +The HIR – "High-Level Intermediate Representation" – is the primary IR used +in most of rustc. It is a compiler-friendly representation of the abstract +syntax tree (AST) that is generated after parsing, macro expansion, and name resolution (see [Lowering](./lowering.md) for how the HIR is created). Many parts of HIR resemble Rust surface syntax quite closely, with -the exception that some of Rust's expression forms have been desugared away. For -example, `for` loops are converted into a `loop` and do not appear in the HIR. -This makes HIR more amenable to analysis than a normal AST. +the exception that some of Rust's expression forms have been desugared away. +For example, `for` loops are converted into a `loop` and do not appear in +the HIR. This makes HIR more amenable to analysis than a normal AST. This chapter covers the main concepts of the HIR. diff --git a/src/lowering.md b/src/lowering.md index cc2a7b384..eddc00af9 100644 --- a/src/lowering.md +++ b/src/lowering.md @@ -12,7 +12,8 @@ of such structures include but are not limited to * `if let` * Converted to `match` * Universal `impl Trait` - * Converted to generic arguments (but with some flags, to know that the user didn't write them) + * Converted to generic arguments + (but with some flags, to know that the user didn't write them) * Existential `impl Trait` * Converted to a virtual `existential type` declaration @@ -34,14 +35,14 @@ sanity checks in `src/librustc/hir/map/hir_id_validator.rs`: which produces both a new `NodeId` as well as automatically lowering it for you so you also get the `HirId`. -If you are creating new `DefId`s, since each `DefId` needs to have a corresponding -`NodeId`, it is adviseable to add these `NodeId`s to the `AST` so you don't have -to generate new ones during lowering. This has the advantage of creating a -way to find the `DefId` of something via its `NodeId`. If lowering needs this -`DefId` in multiple places, you can't generate a new `NodeId` in all those places -because you'd also get a new `DefId` then. With a `NodeId` from the `AST` this is -not an issue. +If you are creating new `DefId`s, since each `DefId` needs to have a +corresponding `NodeId`, it is adviseable to add these `NodeId`s to the +`AST` so you don't have to generate new ones during lowering. This has +the advantage of creating a way to find the `DefId` of something via its +`NodeId`. If lowering needs this `DefId` in multiple places, you can't +generate a new `NodeId` in all those places because you'd also get a new +`DefId` then. With a `NodeId` from the `AST` this is not an issue. -Having the `NodeId` also allows the `DefCollector` to generate the `DefId`s instead -of lowering having to do it on the fly. Centralizing the `DefId` generation in one -place makes it easier to refactor and reason about. \ No newline at end of file +Having the `NodeId` also allows the `DefCollector` to generate the `DefId`s +instead of lowering having to do it on the fly. Centralizing the `DefId` +generation in one place makes it easier to refactor and reason about. \ No newline at end of file diff --git a/src/name-resolution.md b/src/name-resolution.md index 5095b750a..bba3142fc 100644 --- a/src/name-resolution.md +++ b/src/name-resolution.md @@ -36,9 +36,9 @@ hierarchy, it's types vs. values vs. macros. ## Scopes and ribs A name is visible only in certain area in the source code. This forms a -hierarchical structure, but not necessarily a simple one ‒ if one scope is part -of another, it doesn't mean the name visible in the outer one is also visible in -the inner one, or that it refers to the same thing. +hierarchical structure, but not necessarily a simple one ‒ if one scope is +part of another, it doesn't mean the name visible in the outer one is also +visible in the inner one, or that it refers to the same thing. To cope with that, the compiler introduces the concept of Ribs. This is abstraction of a scope. Every time the set of visible names potentially changes, @@ -54,9 +54,9 @@ example: When searching for a name, the stack of ribs is traversed from the innermost outwards. This helps to find the closest meaning of the name (the one not shadowed by anything else). The transition to outer rib may also change the -rules what names are usable ‒ if there are nested functions (not closures), the -inner one can't access parameters and local bindings of the outer one, even -though they should be visible by ordinary scoping rules. An example: +rules what names are usable ‒ if there are nested functions (not closures), +the inner one can't access parameters and local bindings of the outer one, +even though they should be visible by ordinary scoping rules. An example: ```rust fn do_something(val: T) { // <- New rib in both types and values (1) From 92880cb8cace5828cbbe606a59d0e7655215ab69 Mon Sep 17 00:00:00 2001 From: Oliver Schneider Date: Fri, 6 Jul 2018 14:39:24 +0200 Subject: [PATCH 5/5] md -> html --- src/hir.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/hir.md b/src/hir.md index 0fd2fcff5..40a14dc25 100644 --- a/src/hir.md +++ b/src/hir.md @@ -3,7 +3,7 @@ The HIR – "High-Level Intermediate Representation" – is the primary IR used in most of rustc. It is a compiler-friendly representation of the abstract syntax tree (AST) that is generated after parsing, macro expansion, and name -resolution (see [Lowering](./lowering.md) for how the HIR is created). +resolution (see [Lowering](./lowering.html) for how the HIR is created). Many parts of HIR resemble Rust surface syntax quite closely, with the exception that some of Rust's expression forms have been desugared away. For example, `for` loops are converted into a `loop` and do not appear in