Skip to content

Conversation

@forki
Copy link
Contributor

@forki forki commented Oct 20, 2017

  • restore from fake script since cmd had path issues
  • stackoverflow exception in ProjectCracker refactored to much more useful type load exception - thanks @0x53A and @matthid for support
  • Copy msbuild dependencies to Output
  • Fake installs dotnet core autimatically if needed
  • Created Clean target in build.fsx
  • made tests culture independent
  • disabled test that are dependent on MSBuild 14 runtime
  • execute flag on build.sh thanks @kjnilsson
  • better error messages in many places
  • if a project appears more than once in a deps tree we only parse it once
  • tests on Ubuntu disabled since they were already disabled in original FCS
  • corresponding FCS PR: [Test only] Apply changes from VF# Jenkins build activation fsharp/fsharp-compiler-docs#829

Remark: If we can get a CI server with MSBuild14 installed then we could enable 8 more tests

@KevinRansom
Copy link
Contributor

I will try my best ...

@forki forki closed this Oct 20, 2017
@forki forki reopened this Oct 20, 2017
@KevinRansom
Copy link
Contributor

@forki -- Tip: if you want to ran a specific test use:
"@dotnet-bot test Windows_NT Release_fcs Build please"

@forki
Copy link
Contributor Author

forki commented Oct 20, 2017

@KevinRansom I'm not sure but somehow other builds went red as well. I'm deeply sorry, but there was no way to test in pr and now I have to leave. If things are red on master apart from FCS build then please revert my stuff and I retry on monday

@forki
Copy link
Contributor Author

forki commented Oct 20, 2017

If only FCS ist Red then WE can probably ignore for now

@cartermp
Copy link
Contributor

It seems that only FCS is red here, so I think it's safe to ignore until this gets worked out

@KevinRansom
Copy link
Contributor

@forki @cartermp I ain't bothered ... I was assuming if there was an _fcs red, I was going to consider things green ... until the _fcs stuff is working.

Anyway ... broken builds is how we know work is happening ...
You know what they say ... "If it ain't broke ... we aint trying hard enough"

Hmmmm!!!! do they say really that Kevin, or did you just make that up?

@cartermp
Copy link
Contributor

@forki
Copy link
Contributor Author

forki commented Oct 20, 2017

OK the weird thing is: this looks as if my last groovy change never happened. Wtf

@forki
Copy link
Contributor Author

forki commented Oct 20, 2017

Seriously it should have run ./fcs/build.cmd and not with \
And it should also have started an Ubuntu version. Why isn't it?

@KevinRansom
Copy link
Contributor

@dotnet-bot test this please

@forki
Copy link
Contributor Author

forki commented Oct 21, 2017

#3790 did not run the fcs build. And this one runs outdated version. @KevinRansom do you understand this?

@KevinRansom
Copy link
Contributor

@forki I think it was this: PR:

https://github.com/Microsoft/visualfsharp/pull/3781/files

I think the first } is I=in the wrong place.

The CI will not update the groovy script if it has syntax errors, which makes sense if you think about it a bit.

       if (os == 'Windows_NT') { 
        } 
            configurations = ['Debug', 'Release_ci_part1', 'Release_ci_part2', 'Release_ci_part3', 'Release_net40_no_vs', 'Release_fcs' ];         else 
        { 
            // Linux 
            configurations = ['Release', 'Release_fcs' ]; 
         } 

@KevinRansom
Copy link
Contributor

@dotnet-bot test this please

@KevinRansom
Copy link
Contributor

There you go fixed it.

@forki
Copy link
Contributor Author

forki commented Oct 21, 2017

Thank you Kevin. Now I think I can try to make it work

@forki forki closed this Oct 21, 2017
@forki forki reopened this Oct 21, 2017
@forki
Copy link
Contributor Author

forki commented Oct 21, 2017

Ok progress. It now fails on the unit tests. Any ideas @vasily-kirichenko @dsyme @ncave @enricosada @Krzysztof-Cieslak

@0x53A
Copy link
Contributor

0x53A commented Oct 21, 2017

05:45:32 Finished Target: Build.NetFx
05:45:32 Starting Target: Test.NetFx (==> Build.NetFx)
05:45:32 ..\packages\NUnit.Console.3.0.0\tools\nunit3-console.exe "--noheader" "--timeout=1200000" "--work=/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45" "/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45/FSharp.Compiler.Service.Tests.dll" 
05:45:32 mono  "..\packages\NUnit.Console.3.0.0\tools\nunit3-console.exe" "--noheader" "--timeout=1200000" "--work=/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45" "/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45/FSharp.Compiler.Service.Tests.dll" 
05:45:32 Cannot open assembly '..\packages\NUnit.Console.3.0.0\tools\nunit3-console.exe': No such file or directory.
05:45:32 Running build failed.

https://github.com/Microsoft/visualfsharp/blob/f375d8d56cb69f7143254462fea2aa1c3cd12262/fcs/build.fsx#L90-L96

maybe the relative path is the issue? If the script is executed in the root, then it would try to look outside the repo.

I always use the pattern

let cd = __SOURCE_DIRECTORY__

ToolPath = cd @@ @"..\packages\NUnit.Console.3.0.0\tools\nunit3-console.exe"

in my build scripts.

@forki
Copy link
Contributor Author

forki commented Oct 21, 2017 via email

@0x53A
Copy link
Contributor

0x53A commented Oct 21, 2017

I'd appreciate a pull request against my fork. Please please please

done. Looking through the build script, relative paths are used in a few other places, so I'm no longer sure this is the issue, but at least it would now print the full path it it is looking for.


The Windows build fails with

05:37:00 1) Error : FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- 2nd level references
05:37:00 System.Exception : Could not load file or assembly 'Microsoft.Build.Utilities.v12.0, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
05:37:00    at <StartupCode$FSharp-Compiler-Service-ProjectCracker>.$ProjectCracker.convert@19(DateTime loadedTimeStamp, FSharpRef`1 logMap, ProjectOptions opts) in D:\j\w\release_fcs_w---49a16363\fcs\FSharp.Compiler.Service.ProjectCracker\ProjectCracker.fs:line 22
05:37:00    at Microsoft.FSharp.Compiler.SourceCodeServices.ProjectCracker.GetProjectOptionsFromProjectFileLogged(String projectFileName, FSharpOption`1 properties, FSharpOption`1 loadedTimeStamp, FSharpOption`1 enableLogging) in D:\j\w\release_fcs_w---49a16363\fcs\FSharp.Compiler.Service.ProjectCracker\ProjectCracker.fs:line 86
05:37:00    at FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- 2nd level references() in D:\j\w\release_fcs_w---49a16363\tests\service\ProjectOptionsTests.fs:line 180

Just judging from the file name, Microsoft.Build.Utilities.v12.0 is probably a component of Visual Studio 2013. Maybe this version is not installed on the buildserver? It probably only has VS2017, and maybe 2015.

Best solution would be a nuget package providing it.


Edit: the dll is checked in

so it must probably "only" be copied to the output path. Or the directory with it needs to be added to the dll search directories.

@forki
Copy link
Contributor Author

forki commented Oct 21, 2017 via email

@0x53A
Copy link
Contributor

0x53A commented Oct 21, 2017

The Ubuntu error is now different ...

07:34:34 Finished Target: Build.NetFx
07:34:34 Starting Target: Test.NetFx (==> Build.NetFx)
07:34:34 /mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/fcs/..\packages\NUnit.Console.3.0.0\tools\nunit3-console.exe "--noheader" "--timeout=1200000" "--work=/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45" "/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45/FSharp.Compiler.Service.Tests.dll" 
07:34:34 mono  "/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/fcs/..\packages\NUnit.Console.3.0.0\tools\nunit3-console.exe" "--noheader" "--timeout=1200000" "--work=/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45" "/mnt/j/w/Microsoft_visualfsharp/master/release_fcs_ubuntu14.04_prtest/Release/fcs/net45/FSharp.Compiler.Service.Tests.dll" 
07:34:34 
07:34:34 Unhandled Exception:
07:34:34 System.IO.FileNotFoundException: Could not load file or assembly 'nunit.engine.api, Version=3.0.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' or one of its dependencies.
07:34:34 File name: 'nunit.engine.api, Version=3.0.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'
07:34:34 
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'nunit.engine.api, Version=3.0.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' or one of its dependencies.
07:34:34 File name: 'nunit.engine.api, Version=3.0.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'
07:34:34 WARNING: The runtime version supported by this application is unavailable.
07:34:34 Using default runtime: v4.0.30319
07:34:34 Running build failed.
07:34:34 Error:
07:34:34 
NUnit test failed (1).

This may be a hint:

07:34:34 WARNING: The runtime version supported by this application is unavailable.
07:34:34 Using default runtime: v4.0.30319

Maybe need to update mono?

@0x53A
Copy link
Contributor

0x53A commented Oct 21, 2017

After setting <Private>true</Private>, the error on Windows is now different:

08:03:32 1) Error : FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- 2nd level references
08:03:32 System.Runtime.Serialization.SerializationException : Expecting element 'root' from namespace ''.. Encountered 'None'  with name '', namespace ''. 
08:03:32    at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
08:03:32    at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
08:03:32    at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(XmlDictionaryReader reader)
08:03:32    at Microsoft.FSharp.Compiler.SourceCodeServices.ProjectCracker.GetProjectOptionsFromProjectFileLogged(String projectFileName, FSharpOption`1 properties, FSharpOption`1 loadedTimeStamp, FSharpOption`1 enableLogging) in D:\j\w\release_fcs_w---49a16363\fcs\FSharp.Compiler.Service.ProjectCracker\ProjectCracker.fs:line 82
08:03:32    at FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- 2nd level references() in D:\j\w\release_fcs_w---49a16363\tests\service\ProjectOptionsTests.fs:line 180

@0x53A
Copy link
Contributor

0x53A commented Oct 21, 2017

AHA

Process is terminated due to StackOverflowException.

So the ProjectCrackerTool SOs.

@forki

@forki
Copy link
Contributor Author

forki commented Oct 21, 2017 via email

@forki forki closed this Oct 21, 2017
@forki forki force-pushed the patch-36 branch 2 times, most recently from c2d5d2f to 83cfd78 Compare October 23, 2017 15:24
@forki forki changed the title [WIP] Evaluate if FCS build works Make FCS build work Oct 23, 2017
@forki
Copy link
Contributor Author

forki commented Oct 23, 2017

giphy

After 142 commits, 1136 builds and a lot of help from the community this now finally green and ready for review.

I'm happy to answer all questions.

@cartermp
Copy link
Contributor

@forki are you feeling okay? 😄

For real though, good work.

@Krzysztof-Cieslak
Copy link
Contributor

@forki, what is the meaning of life?

Copy link
Contributor

@KevinRansom KevinRansom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Krzysztof-Cieslak
@forki spent all weekend on this, he may not be qualified to answer that question.

fcs/build.fsx Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this line correct?
+#endifests yet")

looks odd

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes merge errors

@forki
Copy link
Contributor Author

forki commented Oct 24, 2017

This is ready. The last error is fixed by groovy change in https://github.com/Microsoft/visualfsharp/pull/3805/files

@forki
Copy link
Contributor Author

forki commented Oct 24, 2017

OMG It's green.

brb Rebasing.....

@KevinRansom
Copy link
Contributor

@dotnet-bot test Windows_NT Release_ci_part2 Build please

Copy link
Contributor

@KevinRansom KevinRansom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KevinRansom
Copy link
Contributor

@forki,

I will merge it when I get in tomorrow.

Thanks for this, nice job.

@eriawan
Copy link
Member

eriawan commented Oct 24, 2017

@forki

Just a suggestion, on your commit, these lines:
https://github.com/Microsoft/visualfsharp/pull/3788/files#diff-12575809f4907caaad73a959140e137d#L66-L109

  <Choose>
    <When Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.6')">
      <ItemGroup>
        <Reference Include="Microsoft.Build">
          <HintPath>..\packages\Microsoft.Build\lib\net45\Microsoft.Build.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>
  <Choose>
    <When Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.6')">
      <ItemGroup>
        <Reference Include="Microsoft.Build.Engine">
          <HintPath>..\packages\Microsoft.Build.Engine\lib\net45\Microsoft.Build.Engine.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>
  <Choose>
    <When Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.6')">
      <ItemGroup>
        <Reference Include="Microsoft.Build.Framework">
          <HintPath>..\packages\Microsoft.Build.Framework\lib\net45\Microsoft.Build.Framework.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>
  <Choose>
    <When Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.6')">
      <ItemGroup>
        <Reference Include="Microsoft.Build.Utilities.Core">
          <HintPath>..\packages\Microsoft.Build.Utilities.Core\lib\net45\Microsoft.Build.Utilities.Core.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>

can be simplified to:

  <Choose>
    <When Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.6')">
      <ItemGroup>
        <Reference Include="Microsoft.Build">
          <HintPath>..\packages\Microsoft.Build\lib\net45\Microsoft.Build.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
        <Reference Include="Microsoft.Build.Engine">
          <HintPath>..\packages\Microsoft.Build.Engine\lib\net45\Microsoft.Build.Engine.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
        <Reference Include="Microsoft.Build.Framework">
          <HintPath>..\packages\Microsoft.Build.Framework\lib\net45\Microsoft.Build.Framework.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
        <Reference Include="Microsoft.Build.Utilities.Core">
          <HintPath>..\packages\Microsoft.Build.Utilities.Core\lib\net45\Microsoft.Build.Utilities.Core.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>

The main reason is the condition of Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.6')" is literally the same or applicable for those Reference includes. It is also simpler 🙂

@forki
Copy link
Contributor Author

forki commented Oct 24, 2017

@eriawan these lines are auto-generated by paket. we don't want to touch them!

@eriawan
Copy link
Member

eriawan commented Oct 24, 2017

these lines are auto-generated by paket. we don't want to touch them!

@forki ah, ok. thanks for letting me know. 👍

@KevinRansom
Copy link
Contributor

@forki

I will merge this shortly, we are in escrow, and have been trying to figure out what we needed from master and would be allowed to merge for the shipping branch.

We ended up going with everything, which is nice :-) When we have completed that, I will pull this and the other fcs pr.

Kevin

@forki
Copy link
Contributor Author

forki commented Oct 24, 2017 via email

@KevinRansom KevinRansom merged commit 4c19c42 into dotnet:master Oct 24, 2017
@dsyme
Copy link
Contributor

dsyme commented Oct 25, 2017

@forki What parts were you concerned about for review? I glanced over and it all looked reasonable - or at least localized to FCS and especially the project cracker

@forki
Copy link
Contributor Author

forki commented Oct 25, 2017 via email

KevinRansom pushed a commit that referenced this pull request Oct 26, 2017
* Put nupkgs into artifacts (#3806)

* Make FCS build work on Jenkins (#3788)
KevinRansom added a commit that referenced this pull request Oct 27, 2017
* Merge dev15.5 to dev15.6 (#3825)

* don't update project info if the source file collection is empty (#3792)

* install templates VSIX to a unique directory (#3804)

* P2p references (#3777)

* P2p references

* Fix test

* test fix

* go faster stripes

* new project works better

* Re-add debug assert for sourcefiles

* Parameterise rc location (#3744)

* Fix issues

* Merge master to dev15.6 (#3826)

* Put nupkgs into artifacts (#3806)

* Make FCS build work on Jenkins (#3788)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants