Skip to content
This repository was archived by the owner on Apr 1, 2025. It is now read-only.

Multi component repl #373

Merged
merged 40 commits into from
Nov 1, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
0aa6eff
:fire: the project-wide .ghci file.
robrix Oct 30, 2019
f799b3b
Ignore dist-repl.
robrix Oct 30, 2019
73badce
Add a script to run ghci.
robrix Oct 30, 2019
c53e18e
Add a .ghci file configured to set up the project loading &c.
robrix Oct 30, 2019
f71ad89
Use dist-repl import paths.
robrix Oct 30, 2019
3e0a968
Set the output, o, hi, and stub dirs to dist-repl.
robrix Oct 30, 2019
e58bb88
Enable StrictData.
robrix Oct 30, 2019
f9bbcb3
Comment the .ghci file
robrix Oct 30, 2019
df9e7fa
Note how we came by this.
robrix Oct 30, 2019
186ed4b
Turn off -Werror in the repl.
robrix Oct 30, 2019
f8b1195
Add back in the pretty-printing stuff but tell people to put it in th…
robrix Oct 30, 2019
cb7e6b6
Spacing.
robrix Oct 30, 2019
b6a9682
Propose some more flags.
robrix Oct 30, 2019
5d3c4aa
Propose some language extensions.
robrix Oct 30, 2019
08f10ee
Add some setup instructions.
robrix Oct 30, 2019
b6b00a8
Note why these aren’t in script/repl.
robrix Oct 30, 2019
9beb6d5
:fire: -j.
robrix Oct 30, 2019
5d361fc
Comments.
robrix Oct 30, 2019
b74256f
Move the .ghci file so it doesn’t affect `cabal repl`.
robrix Oct 30, 2019
3205826
Move the proposed additions to the .ghci file.
robrix Oct 30, 2019
d99fe09
Usage for script/repl.
robrix Oct 30, 2019
105c036
Add the app component’s path.
robrix Oct 30, 2019
381728a
Note why we’re using a separate directory.
robrix Oct 30, 2019
abd8216
$0
robrix Oct 30, 2019
47b0dac
Build both static & dynamic files so we don’t have to live in a separ…
robrix Oct 30, 2019
ade8713
Clarify why these things are in this file.
robrix Oct 30, 2019
733a0a4
Revert "Build both static & dynamic files so we don’t have to live in…
robrix Oct 30, 2019
df54afc
Merge branch 'play-nice-with-ghcide' into multi-component-repl
robrix Oct 30, 2019
7b6e3a5
Rename .ghci to .ghci.default.
robrix Nov 1, 2019
3710150
Uncomment the proposed .ghci file.
robrix Nov 1, 2019
2214adb
Rename ghci.default to ghci.sample.
robrix Nov 1, 2019
2d04c23
Ignore .ghci.
robrix Nov 1, 2019
a637e6e
:fire: a redundant link.
robrix Nov 1, 2019
51884e6
Fix a typo.
robrix Nov 1, 2019
5c8bc78
Remove some stuff.
robrix Nov 1, 2019
66c1abd
Promote the threadscope section out of the ghci section.
robrix Nov 1, 2019
6863b55
Simplify the threadscope steps.
robrix Nov 1, 2019
055c842
Rewrite the ghci section.
robrix Nov 1, 2019
d030f02
Ignore pretty-simple before loading it to avoid multiple loaded copies.
robrix Nov 1, 2019
7683376
Enable pretty-printing immediately.
robrix Nov 1, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 0 additions & 31 deletions .ghci

This file was deleted.

31 changes: 31 additions & 0 deletions .ghci.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
-- Consider copying this to your ~/.ghc/ghci.conf file:

-- Pretty-printing
:set -ignore-package pretty-simple -package pretty-simple
:def! pretty \ _ -> pure ":set -interactive-print Text.Pretty.Simple.pPrint"
:def! no-pretty \ _ -> pure ":set -interactive-print System.IO.print"
:def! r \_ -> pure ":reload\n:pretty"

-- Turn on some language extensions you use a lot
:seti -XFlexibleContexts -XOverloadedStrings -XTypeApplications

-- Break on errors
:seti -fbreak-on-error

-- Automatically show the code around breakpoints
:set stop :list

-- Use a cyan lambda as the prompt
:set prompt "\ESC[1;36m\STXλ \ESC[m\STX"

-- Better errors
:set -ferror-spans -freverse-errors -fprint-expanded-synonyms

-- Path-local ghci history
:set -flocal-ghci-history

-- Better typed holes
:set -funclutter-valid-hole-fits -fabstract-refinement-hole-fits -frefinement-level-hole-fits=2

-- Enable pretty-printing immediately
:pretty
70 changes: 70 additions & 0 deletions .ghci.semantic
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
-- GHCI settings for semantic, collected by running cabal repl -v and checking out the flags cabal passes to ghc.
-- These live here instead of script/repl for ease of commenting.
-- These live here instead of .ghci so cabal repl remains unaffected.

-- Basic verbosity
:set -v1

-- No optimizations
:set -O0

-- Compile to object code
:set -fwrite-interface -fobject-code
Comment on lines +11 to +12
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m being a bit opinionated in this file:

  1. I think we want to be able to quit the repl and relaunch it quickly.
  2. I think we don’t want it to load all of the code automatically, since we’re working in various projects.
  3. I think we especially don’t want it to import all of the code automatically.

I think those are fair statements for working on a variety of tasks, but if your particular workflow is made arduous by this, it’s worth revisiting.


-- Write build products to dist-repl (so that we don’t clobber 'cabal build' outputs)
:set -outputdir dist-repl/build/x86_64-osx/ghc-8.6.5/semantic-0.8.0.0/build
:set -odir dist-repl/build/x86_64-osx/ghc-8.6.5/semantic-0.8.0.0/build
:set -hidir dist-repl/build/x86_64-osx/ghc-8.6.5/semantic-0.8.0.0/build
:set -stubdir dist-repl/build/x86_64-osx/ghc-8.6.5/semantic-0.8.0.0/build

-- Look for autogen’d files in dist-repl
:set -idist-repl/build/x86_64-osx/ghc-8.6.5/semantic-0.8.0.0/build/autogen
Comment on lines +14 to +21
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured out that you can convince ghci to build .dyn_o and .dyn_hi files alongside its usual .o and .hi files, and that this is enough to deal with the build errors I was seeing (cf haskell/cabal#6059). However, while this works well enough in fused-effects, it still ends up doing a lot of rebuilding (cf haskell/cabal#3565) in semantic, and the repl builds take longer too due to the added codegen.

Ultimately, I think we’re better off using a separate dist dir for the time being, tho it breaks my very heart.


-- Load all our sources… remember to keep this up to date when we add new packages!
-- But don’t add semantic-source, it’s important that we get that from hackage.
:set -isemantic-analysis/src
:set -isemantic-ast/src
:set -isemantic-core/src
:set -isemantic-java/src
:set -isemantic-json/src
:set -isemantic-python/src
:set -isemantic-tags/src
:set -iapp
:set -isrc
:set -ibench
:set -itest

-- Default language mode & extensions
:set -XHaskell2010
:set -XStrictData

-- Warnings for compiling .hs files
:set -Weverything
:set -Wno-all-missed-specialisations
:set -Wno-implicit-prelude
:set -Wno-missed-specialisations
:set -Wno-missing-import-lists
:set -Wno-missing-local-signatures
:set -Wno-monomorphism-restriction
:set -Wno-name-shadowing
:set -Wno-safe
:set -Wno-unsafe
:set -Wno-star-is-type
-- Bonus: silence “add these modules to your .cabal file” warnings for files we :load
:set -Wno-missing-home-modules
Comment on lines +53 to +54
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one hits you for every :loaded file if you don’t turn it off.


-- Don’t fail on warnings when in the repl
:set -Wwarn

-- Warnings for code written in the repl
:seti -Weverything
:seti -Wno-all-missed-specialisations
:seti -Wno-implicit-prelude
:seti -Wno-missed-specialisations
:seti -Wno-missing-import-lists
:seti -Wno-missing-local-signatures
:seti -Wno-monomorphism-restriction
:seti -Wno-name-shadowing
:seti -Wno-safe
:seti -Wno-unsafe
:seti -Wno-star-is-type
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ profiles
cabal.project.local*
dist
dist-newstyle
dist-repl
.ghci
.ghc.environment.*
.ghci_history

Expand Down
82 changes: 21 additions & 61 deletions docs/💡ProTip!.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,19 +148,27 @@ _Voilà!_ You’re now looking at the source code for the datatype generated fro

## GHCi

The Haskell interactive repl (GHCi) allows you to quickly typecheck your work and test out ideas interactively. It’s always worth having a repl open, but we’ve particularly tuned some workflows, e.g. semantic assignment development, for the repl.
The Haskell interactive repl (GHCi) allows you to quickly typecheck your work and test out ideas interactively. It’s always worth having a repl open, and we’ve particularly tuned some workflows for the repl.

[pretty-printing]: pretty-printing
Full docs for ghci can be found in the [user’s guide][ghci user’s guide].

[ghci user’s guide]: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html

### Configuration

We configure `ghci` with defaults & macros for use with `semantic` via the [`.ghci` file][] at the project root, and you can further customize its behaviour via the `~/.ghci` file.
### Multiple components

Full docs for ghci can be found in the [user’s guide][ghci user’s guide].
`semantic` consists of multiple packages and components, which makes it somewhat challenging to load into `ghci` using e.g. `cabal repl`. To help that, we provide [`script/repl`][] to automate working with multiple components & packages. Unlike when using `cabal repl`, after loading you will need to explicitly `:load` (at least some of) the sources you need to work with. For example, when working in the main `semantic` package, almost everything can be loaded with `:load Semantic.CLI`, since the `Semantic.CLI` module ultimately depends on just about everything else in the repo.

[ghci user’s guide]: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html
[`.ghci` file]: https://github.com/github/semantic/blob/master/.ghci
This script is also set up to store intermediate build products in a separate `dist-repl` dir to avoid colliding with normal builds.

[`script/repl`]: https://github.com/github/semantic/blob/master/script/repl


### Configuration

`ghci` can be configured with scripts containing Haskell statements and repl commands. By default, the `~/.ghc/ghci.conf` file will be loaded, as well as a `.ghci` file in the working directory, if any. We don’t currently provide such for use with `semantic`, but we do provide a [`.ghci.sample`][] file which we suggest copying to `~/.ghc/ghci.conf` for better typed holes, pretty-printing via `pretty-simple`, and a simple prompt.

[`.ghci.sample`]: https://github.com/github/semantic/blob/master/.ghci.sample


### Managing history
Expand All @@ -178,9 +186,9 @@ maxHistorySize: Nothing

### Pretty-printing

By default, GHCi prints the results of expressions using their `Show` instances, which can be particularly difficult to read for large recursive structures like `Term`s and `Diff`s. The project’s [`.ghci` file][] provides `:pretty` & `:no-pretty` macros which respectively enable & disable colourized, pretty-printed formatting of result values instead. These macros depend on the the `pretty-show` & `hscolour` packages.
By default, GHCi prints the results of expressions using their `Show` instances, which can be particularly difficult to read for large recursive structures like `Term`s and `Diff`s. The project’s [`.ghci.sample`][] file provides `:pretty` & `:no-pretty` macros which respectively enable & disable colourized, pretty-printed formatting of result values instead. These macros depend on the the `pretty-simple` package.

Since `:reload`ing resets local bindings, the [`.ghci` file][] also provides a convenient `:r` macro which reloads and then immediately re-enables `:pretty`.
Since `:reload`ing resets local bindings, the file also provides a convenient `:r` macro which reloads and then immediately re-enables `:pretty`.

You can use `:pretty` & `:no-pretty` like so:

Expand All @@ -203,60 +211,12 @@ You can use `:pretty` & `:no-pretty` like so:
```


### Working in Assignment

When working in assignment, some setup is required. This macro automates that by automatically importing the necessary modules and outputs an example command. If you provide the language you are working with as an optional parameter, the example command is formatted for that language's specific needs (parser, example file extension, etc.).

The macro is defined as:

```
:{
assignmentExample lang = case lang of
"Python" -> mk "py" "python"
"Go" -> mk "go" "go"
"Ruby" -> mk "rb" "ruby"
"JavaScript" -> mk "js" "typescript"
"TypeScript" -> mk "ts" "typescript"
"Haskell" -> mk "hs" "haskell"
"Markdown" -> mk "md" "markdown"
"JSON" -> mk "json" "json"
_ -> mk "" ""
where mk fileExtension parser = putStrLn ("example: fmap (() <$) . runTask . parse " ++ parser ++ "Parser =<< Semantic.Util.blob \"example." ++ fileExtension ++ "\"") >> return ("import Parsing.Parser\nimport Semantic.Task\nimport Semantic.Util")
:}

:def assignment assignmentExample
```

And is invoked in GHCi like:

```
λ :assignment Python
```

The output produces a one line expression assuming the syntax to assign is in a file named `example` with the relevant programming language extension:

```haskell
quieterm <$> parseFile pythonParser "example.py"
```


### Inspecting TreeSitter ASTs

Inspecting the parse tree from TreeSitter can be helpful for debugging. In GHCi, the command below allows viewing the TreeSitter production name of each node in the TreeSitter AST:

```haskell
import TreeSitter.Java
fmap nodeSymbol <$> parseFile javaASTParser "example.java"
```


### Using Threadscope
## Using Threadscope

Threadscope is a tool for profiling the multi-threaded performance of Haskell programs. It allows us to see how work is shared across processors and identify performance issues related to garbage collection or bottlenecks in our processes.

To install threadscope:

1. Download a prebuilt binary from https://github.com/haskell/ThreadScope/releases .
2. `chmod a+x` the result of extracting the release.
3. `brew install gtk+ gtk-mac-integration`.
4. profit.
1. Download a prebuilt binary from https://github.com/haskell/ThreadScope/releases
2. `chmod a+x` the result of extracting the release
3. `brew install gtk+ gtk-mac-integration`
15 changes: 15 additions & 0 deletions script/repl
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash
# Usage: script/repl [ARGS...]
# Run a repl session capable of loading all of the packages and their individual components. Any passed arguments, e.g. module names or flags, will be passed to ghci.

set -e

cd $(dirname "$0")/..

repl_builddir=dist-repl

if [[ ! -d $repl_builddir ]]; then
echo "$repl_builddir does not exist, first run 'cabal repl --builddir=$repl_builddir', exit, and then re-run $0"
else
cabal exec --builddir=$repl_builddir ghci -- -ghci-script=.ghci.semantic $@
fi