Repo alignment - How to enable xplat workflow for easy contribution

[7sharp9]

@dsyme says:

We would like to make it possible to contribute to core components that underlie the OSX and Linux experiences for F# by contributing to this repo (rather than the historical way of routing these requests into the fsharp/fsharp and fsharp/FSharp.Compiler.Service repos. We can call this "repo alignment".

Separately, people have discussed splitting this repo - putting the "Visual F# IDE Tools" (vsintegration and setup directories) in a separate repo, and perhaps putting "FSharp.Core" in a separate directory too. For now we will assume that this will be done eventually

Here's a list of things we can reasonably unify:

• Mono friendly fixes and hacks from fsharp/fsharp
• Mono friendly testing from fsharp/fsharp (or replacement)
• "make install" or equivalent from fsharp/fsharp
• local builds of nuget packages for FSharp.Core nuget (from fsharp/fsharp)
• local builds of nuget packages for FSharp.Compiler.Tools nuget (from fsharp/fsharp)
• code additions in Fsharp.Compiler.Service (notably some fixes and Exprs.fs) (from fsharp/FSharp.Compiler.Service)
• local buidls of FSharp.Compiler.Service DLL (from fsharp/FSharp.Compiler.Service)
• some extra testing (from fsharp/FSharp.Compiler.Service)
• local builds of nuget packages for FSharp.Compiler.Service
• type provider authoring SDK from type provider starter pack
• compiling FSharp.Compiler.Service as a Fable component, a highly valuable long term addition to the F# ecosystem that we don't want to bit-rot
• make sure documentation in this repo is not skewed towards Visual Studio and Windows. There should be a strong path where neither Visual Studio nor Windows are assumed or even mentioned. OSX and Linux contributors should feel "first class"
• rename this repo to "fsharp" (even if it continues to hold the setup and vsintegrationdirectories). fsharp/fsharp could then be a mirror of this repo
• Move tests\service* tests into FSharp.Compiler.Unittests.dll instead of VS-only tests since they don't need any VS bits to run
• enable running FSharp.Compiler.Unittests.dll on Mono under Jenkins
• enable running FSharp.Tests.FSharpSuite.DrivingCoreCLR on Mono under Jenkins
• make sure the documentation on this repo is appropriate for FSharp.Compiler.Service and fsc.exe contributors on Mono/Linux/macOS
• make sure the testing in this repo is appropriate for contributors on Mono/Linux/macOS
• remove or minimize the use of CMD scripting such as build.cmd

Various problem with this plan:

• This repo becomes a logjam containing everything related to both F# core engineering and the Visual F# IDE Tools.
• The Windows-specific nature ("feel") of this repo will still hang around while it continues to contain the Windows-specific Visual F# IDE Tools
• The "make install" business currently uses a bunch of makefiles, and adds things like policy files for Mono's unification of FSharp.Core. It's not at all clear we want to pollute this repo with that
• Supporting the Fable bootstrap of Fsharp.Compiler.Service adds a very new and different dimension to this repo that may blow the minds of contributors, and leave them unable to debug issues with Fable.
• Do the new binaries and packages get signed, and if so using which strong names

Aside: this does not mean that Microsoft Visual F# Tools team become publishers of any of the alternative packagings of F# - and indeed the final "tagging" and "publishing" of these components may still be done in other repos - see this description for the actual tagging and release repos, including those used by Xamarin. This is normal. But contributions would flow into this repo.

---

@7sharp9 writes:

As a starting point I want to help with the type provider type input testing and implementation, as well as many other features but doing so means porting code to and from FCS and also building and installing the compiler into osx/mono.

• I can now build and run from this repo with the resource issue: SqlDataConnection Suppresses SQLMetal Warnings #720 I think
• To get tooling support I will need to build visualfsharp and use it in place of FCS in ionide/xs/
• I also need to be able to install the compiler with a script, Im assuming theres one described in the readme somewhere.

So I would need two build target outputs for this repo, one for an FCS compatible build, and one for a compiler build, and a script to install into mono.

[vasily-kirichenko]

I'm +1000 for this. Enabling @7sharp9 to contribute worth any effort needed to make FSharp.LanguageService.Compiler usable from Ionide/XS (this means to make the API optionally public I guess?)

[cartermp]

Agreed, this is important.

[7sharp9]

At the moment Im trying to figure out how to replace the compiler installed in mono with the one just built in release, I cant see a script for that in this repo.

[7sharp9]

I added some notes here on trying to get that fork running the TP sample: colinbull@c295f8a

[dsyme]

@7sharp9 @vasily-kirichenko @KevinRansom @cartermp @brettfo I've added a lengthy description of all the work I know of to be done in the area of "repo unification" up above in the issue description, plus some additional comments

@KevinRansom @cartermp @brettfo You guys are the repo owners - if you don't want these things coming in to this repo, then you should say :)

[KevinRansom]

We are eager for the repos to be aligned. And are willing to make the necessary changes to make that happen. So long as our product pipeline is not broken by any changes we are good.

[cartermp]

Agreed with what @KevinRansom said. Let's do it.

[dsyme]

@KevinRansom Thanks, that's great to know.

p.s. That said, I fear you may have a heart attack (joke!! I hope not!! :) ) when you see a PR proposing to integrate the machine-wide policy files for FSharp.Core installed as part of Mono... The Mono packagings of F# have always used those to unify FSharp.Core to the latest version - for better or worse.... And I assume there is code that depends on that. We can perhaps reassess that, but probably best not to do that as part of unification.

So I suggest we put those under an explicit mono directory :) And in general aim to keep each of the above items fairly separable.

[dsyme]

@vasily-kirichenko

...make FSharp.LanguageService.Compiler usable from Ionide/XS ...

Let's aim instead for building FSharp.Compiler.Service in this repo.

For now let's assume FSharp.LanguageService.Compiler remains a special version of this component used only by the Visual F# Tools. This is because historically there was huge pressure from Visual Studio engineering to make FSharp.LanguageService.Compiler smaller - trimming out unnecessary parts. This is one reason I didn't integrate Exprs.fs - which is unused by FLSC. Size minimization was to reduce overall address space usage in the 32-bit devenv.exe Visual Studio process. It's possible this issue will come back to haunt us again one day, so for now just assume that the Visual F# Tools need their own special build of this component.

[smoothdeveloper]

Disclaimer: not directly related to mono support in this repository.

Can we move the current scripts under scripts/misc and plan for a comprehensive layout for future build scripts under scripts/build?

My feeling is that we should start splitting the behemoth build.cmd into smaller independant scripts, each put in scripts/build, and have the top level build.cmd handling the main parts (setting env variables for the build conditionals, calling other .cmd, and doing the main build).

This way when we start introducing FAKE script, we can focus on a smaller set of behaviour to replicate.

individual scripts I could see:

• dotnet core stuff
• vssdk stuff
• ngen stuff
• testing stuff

I know this will make it a bit harder (but how much harder? not so much) to deal with small changes, but I don't see how the community can put effort in a big bang PR to change the whole thing at once, that will need long lead time for you guys to review / assess, while changing each small script or the top level one, one by one is definitely doable.

[dsyme]

@smoothdeveloper Let's keep this thread on topic please, about repo unification

[dsyme]

@7sharp9 I've updated the issue description with a list of all the problems that will arise because of repo unification.

Specifically, if we do this unification we should go into it with our eyes open: it's important that you be aware of the inevitable problems listed above (e.g. the mess that is the Windows-specific build.cmd, or the fact that the Visual F# IDE Tools are part of this repo with all that brings in biasing it towards Windows...) . We can fix these things, but it will require work.

The point is - don't assume unification is some kind of panacea. It will bring new problems of a different kind.