`dart build cli` promised layout

[dcharkes]

Current proposal: #60730 (comment)

What do we promise as layout for dart build exe?

If we change the directory layout later, we would break users that rely on the results. So we need to specify what users can expect as an output.

cc @mkustermann @bkonyi

Prerequisite for:

• [cli] Introduce dart install (replacing dart pub global activate to use) #60889

[dcharkes]

I've been giving this some thought, and I think we should align dart build more with flutter build and more with what's expected on the various target OSes.

Moreover, I believe we have (at least) two very different Dart apps:

• CLI apps
• UI app (such as https://github.com/halildurmus/win32/tree/main/examples/tetris on Windows and once os dialogs fail when called via dart:ffi on macOS (thread pinning Dart standalone) #38315 is addressed similar on macOS. I'm not aware of Linux apps atm, but they probably exist.)

Going at OS conventions, CLI apps and UI apps are treated quite differently:

• CLI apps are supposed to have a bin/ directory which is put on the users path or the executables in the bin/ directory are symlinked from a system bin/ directory.
  • On macOS, all executables and dylibs need to be signed individually.
  • Using a bin/ lib/ include/ data/ (or resources/) that's similar on all OSes is okay.
  • macOS/Linux do not use .exe suffix, Windows does.
• UI apps have a double clickable icon.
  • On macOS the format is the .app bundle. And this is the recommended (and only?) way to sign and notarize these as one unit and get them in the store. This is how Flutter macOS apps are built.
  • On Windows it's most common to have the MyApp.exe (note capitalization) in the root of the bundle, the dylibs right next to it, and a Data/ directory. (This is if you don't package things for the store, but just for a downloadable zip.)
  • Linux has a less well-defined way of doing things. (We should probably simply follow what Flutter does for Linux/)

It's conceivable that we might have more formats later:

• Dart code packaged in a Framework to be embedded in macOS/iOS (similar to flutter build <os>-framework)

So, I believe we should structure dart build in a similar fashion to flutter build with subcommands for the different target formats for the different target OSes.

$ dart build --help
Build an executable application or portable directory bundle.

Global options:
  -h, --help              Print this usage information.
  -v, --verbose           Noisy logging, including all shell commands executed.
                          If used with "--help", shows hidden options. (Use "-vv" to force verbose
                          logging in those cases.)
  --version               Reports the version of this tool.

Usage: dart build <subcommand> [arguments]
-h, --help  Print this usage information.

Available subcommands:
  cli                     Build one or more command-line executables into a portable directory bundle.
  macos                   Build a macOS desktop application bundle (.app).
  macos-framework         Produces .xcframeworks for a Dart project and its plugins for integration into existing macOS Xcode projects.
  ...

Now, it's rather common that bin/ directories contain more than one executable. So I believe the default for dart build cli should be a portable directory bundle with all entry points: all dart files in the packages' bin/ directory.

Having OS-compliant dart compile cli bundles with a bin/ dir will aid in creating a dart install (#60889).

Having subcommands of dart build means we promise a layout per subcommand (why this issue was filed).

We'll deprecate dart build -f exe. It will be superseded by dart build cli (but this will follow proper OS conventions such as not appending .exe on non-Windows).

We'll deprecate dart build -f aotsnapshot. We can introduce dart build bundle which should support running the provided snapshot with the dylibs and data assets in the directory. (flutter build bundle does something similar.)

I think we should hold off for building dart build macos (.app-bundle) for now, it's quite a lot of code in flutter_tools to achieve this. But we should design dart build so that we can.

Any feedback welcome! Anything we forgot to consider? Any terminology to improve? @goderbauer @mkustermann @bkonyi @mit-mit @mosuem (I believe @stuartmorgan-g and @jmagman might have made some of these design decisions for Flutter. Feel free to cc other people who were part of designing flutter build.)

[mkustermann]

CLI apps
UI apps have a double clickable icon.

My gut feeling is that the standalone Dart SDK is probably not the way to build UI apps. If someone makes a new UI framework, mots likely they want to customize the way

• development (e.g. hot reload) works
• runtime (e.g. running the application code on the main OS thread, having a custom event loop)
• deployment
• possibly specialized UI apis (similar to dart:ui)
• possibly custom asset types

So I think a popular UI framework will make a custom SDK for their developers. We see that already with flutter build/run & jaspr build/run.

Also putting any UI specifics into core Dart SDK would mean the Dart team has to maintain those pieces, I don't think we want this.

Very intentionally we've make the new dart hooks support a layered architecture where it's rather easy for new SDKs to build upon the basic building blocks and extend where needed.

It's indeed an interesting question whether a dart build should produce a folder structure containing binaries for all entrypoints in bin/ or only one. For duality with flutter run (which can only run one entrypoint) the flutter build also builds only one entrypoint (IIRC selected via -t). I'd probably tend to do the same and only build one entrypoint - which makes tree shaking also cleaner to do. It's future-proof to allow specifying multiple entrypoints if we ever want to support that.

[dcharkes]

Yes, I'm not convinced either we should have support for UI apps in dart build. We should definitely not build it now. However, the current way we structured the command only has a single way of laying out apps, so we paint ourselves in a corner. If any OS ever comes up with two ways of doing CLI apps we also need subcommands. And if we want to support something like the xcframeworks for add-to-app of Dart components we also need subcommands.

I'd probably tend to do the same and only build one entrypoint - which makes tree shaking also cleaner to do.

It's going to massively inflate the total binary size if we don't support multiple entry points but the user wants multiple entry points. Though maybe the tree-shaking currently doesn't support reasoning about multiple entry points. It's probably fine to start with only supporting a single entry-point, but the API and output format should be designed so we can have multiple entry points.

[dcharkes]

So I think a popular UI framework will make a custom SDK for their developers. We see that already with flutter build/run & jaspr build/run.

For custom SDKs we need the SDK authors to invoke package:hooks_runner themselves in order to specify the supported asset types, and to consume them in the right way.

(It would be somewhat tempting to use dart build bundle to produce a directory structure with a lib/ containing the dylibs and a lib/app.aotsnapshot with the aotsnapshot. And have the custom embedder simply invoke dart build bundle instead of dart compile aotsnapshot and invoke the hooks manually. But, this would mean the asset types would be inherited from the Dart SDK.)

The place where dart build bundle or dart build macos-framework would make more sense is if the "Dart engine" (cc @iinozemtsev) is used to add standalone Dart to an existing app (Android/iOS/...) We should hold off with adding something like this until we have a use case.

cc @bkonyi @jakemac53 @srawlins Any thoughts? (Dart CLI sync)

Concrete list of changes:

• deprecate dart build -f exe and introduce dart build cli
  • change the directory layout to bin/ and lib/
  • change the executable name to have no .exe on Linux and Windows
• deprecate dart build -f aotsnapshot and provide no replacement (yet)

Possible follow up changes:

• dart build cli multiple entry points (so that all executables are available in bin/)
• --mode=jit ([cli] Introduce dart install (replacing dart pub global activate to use) #60889)
• other formats besides cli: macos-framework, bundle for other use cases than CLI apps. For example add2app.

[iinozemtsev]

just a wild idea, maybe it makes sense to describe build layouts in a yaml file and have some predefined layouts.yaml in Dart SDK? then dart build can take a layout name, and users can define their own layouts in a project

[dcharkes]

I think we should also change the default output directory. (dart compile exe and dart build -f exe build right next to the Dart file, but that's a source directory.)

A propose align with flutter, and output into build/.

pubspec.yaml
bin/
  foo.dart
  bar.dart
build/ # Mirrors `flutter build`.
  cli/ # assuming we'll have other subcommands
    <os>-<arch>-<...>/ # assuming we'll start supporting cross compilation at some point. (If we get more config options, we can encode them in this directory name)
      bundle/
        bin/
          foo # foo.exe on Windows
          bar # bar.exe on Windows
        lib/ # contains dylibs
      debug-symbols/
lib/
  src/