Skip to content

Commit e02d857

Browse files
jonpryorjonpryor
authored
[build] Remove globalPackagesFolder override (#948)
Context: dec2e39 How does one run NUnit unit-tests? `dotnet test` is an answer -- and perhaps we should consider fully migrating to it -- but for now we currently use `nunit3-console.exe`. How do we find the path to `nunit3-console.exe`? *This* is why we set the [`globalPackagesFolder`][0] property in `NuGet.config` in commit dec2e39: `globalPackagesFolder` tells NuGet where to place downloaded NuGet packages. By setting `globalPackagesFolder`, we knew where the `NUnit.ConsoleRunner` artifacts were installed, and could construct a path to `nunit3-console.exe`. The problem with `globalPackagesFolder` is that it's duplication: the `packages` folder is our own special copy of our NuGet packages, unshared by anything else on the system. This increases disk space requirements, and slows down provisioning, as those artifacts need to be re-downloaded whenever the repo is `git clean`ed. There is an alternative: when the [`%(PackageReference.GeneratePathProperty)`][1] property is True, a `$(Pkg[NuGet_Package_Name])` property is created, which contains the path to where the NuGet package is installed. Thus, instead of using a relative path that partially hard-codes NuGet paths: packages\nunit.consolerunner\3.12.0\tools\nunit3-console.exe we can instead use: $(PkgNUnit_ConsoleRunner)/tools/nunit3-console.exe …in theory. In practice, `$(PkgNUnit_ConsoleRunner)` isn't set until after `@(PackageReference)` is processed and (lots of) other MSBuild targets are run. Targets which *aren't* run when our `RunTests` MSBuild target is run. Allow `RunNUnitTests.targets` to use `$(PkgNUnit_ConsoleRunner)` by adding a new `Java.Interop.BootstrapTasks.targets` file, which which will create `bin/Build$(Configuration)/PackagePaths.props` as part of the `Java.Interop.BootstrapTasks.csproj` build. `RunNUnitTests.targets` can then import `PackagePaths.props`, obtaining access to `$(PkgNUnit_ConsoleRunner)`. A "philosophical" problem arises, around *how to build* `PackagePaths.csproj` from `make prepare`. We effectively have two separate and overlapping build systems: the original `make` system, and a `dotnet build`-based system: % make prepare % make all % make run-all-tests % make prepare-core % dotnet build -t:Prepare % dotnet build # run various tests via YAML… *Begin* unifying these systems, by updating `make prepare` to run `msbuild -target:Prepare Java.Interop.sln`, and removing make targets which overlap with the `Prepare` MSBuild target. This allows `make prepare` to create `PackagePaths.props`, and reduces the amount of Makefile code that we need to maintain. Additionally, this helps speed up `make prepare`; when `globalPackagesFolder` is used, ~573MB of data must be extracted, and all NuGet packages downloaded: # commit f91b077 % time make prepare … make prepare 37.11s user 15.57s system 85% cpu 1:01.58 total % du -sh packages 573M packages Compare to the approach taken here: % time make prepare … make prepare 14.21s user 5.32s system 83% cpu 23.344 total `make prepare` now takes ~38% of the time, and reduces disk usage by 573MB. Quasi-related: Rename the `<GenerateVersionFile/>` task to `<ReplaceFileContents/>`. This better matches what the task actually does. Remove `jdk.mk` and `jdk.targets`. These are no longer needed. Move `Java.Runtime.Environment.dll.config` generation *out of* `Makefile`, and into `Java.Runtime.Environment.targets`. This helps with build system unification. [0]: https://docs.microsoft.com/en-us/nuget/Consume-Packages/managing-the-global-packages-and-cache-folders [1]: https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#generatepathproperty
1 parent 7dc270d commit e02d857

File tree

13 files changed

+84
-107
lines changed

13 files changed

+84
-107
lines changed

‎Directory.Build.props‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,10 @@
3939
Project="$(_OutputPath)MonoInfo.props"
4040
Condition="Exists('$(_OutputPath)MonoInfo.props')"
4141
/>
42+
<Import
43+
Project="$(_OutputPath)PackagePaths.props"
44+
Condition="Exists('$(_OutputPath)PackagePaths.props')"
45+
/>
4246
<PropertyGroup>
4347
<!-- Workaround: https://github.com/dotnet/sdk/issues/19050 -->
4448
<DisableImplicitNamespaceImports>true</DisableImplicitNamespaceImports>

‎Makefile‎

Lines changed: 7 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ PTESTS = \
3838
ATESTS = \
3939
bin/Test$(CONFIGURATION)/Android.Interop-Tests.dll
4040

41-
BUILD_PROPS = bin/Build$(CONFIGURATION)/JdkInfo.props bin/Build$(CONFIGURATION)/MonoInfo.props
41+
BUILD_PROPS = bin/Build$(CONFIGURATION)/MonoInfo.props
4242

4343
all: $(DEPENDENCIES) $(TESTS)
4444

@@ -52,32 +52,20 @@ run-all-tests:
5252

5353
include build-tools/scripts/msbuild.mk
5454

55-
prepare:: $(BUILD_PROPS) src/Java.Runtime.Environment/Java.Runtime.Environment.dll.config
55+
prepare:: $(BUILD_PROPS)
5656

57-
prepare:: prepare-bootstrap
58-
$(MSBUILD) $(MSBUILD_FLAGS) /t:Restore Java.Interop.sln
57+
prepare:: prepare-core
58+
$(MSBUILD) $(MSBUILD_FLAGS) -target:Prepare
59+
$(MSBUILD) $(MSBUILD_FLAGS) -target:Restore
5960

60-
prepare-bootstrap: prepare-external bin/Build$(CONFIGURATION)/Java.Interop.BootstrapTasks.dll
61-
62-
bin/Build$(CONFIGURATION)/Java.Interop.BootstrapTasks.dll: build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks.csproj \
63-
external/xamarin-android-tools/src/Xamarin.Android.Tools.AndroidSdk/Xamarin.Android.Tools.AndroidSdk.csproj \
64-
$(wildcard build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks/*.cs)
65-
$(MSBUILD) $(MSBUILD_FLAGS) /restore "$<"
66-
67-
prepare-external $(PREPARE_EXTERNAL_FILES):
68-
git submodule update --init --recursive
69-
(cd external/xamarin-android-tools && $(MAKE) prepare)
70-
nuget restore
71-
72-
prepare-core: bin/Build$(CONFIGURATION)/MonoInfo.props src/Java.Runtime.Environment/Java.Runtime.Environment.dll.config
61+
prepare-core: bin/Build$(CONFIGURATION)/MonoInfo.props
7362

7463
clean:
7564
-$(MSBUILD) $(MSBUILD_FLAGS) /t:Clean
7665
-rm -Rf bin/$(CONFIGURATION) bin/Build$(CONFIGURATION) bin/Test$(CONFIGURATION)
77-
-rm src/Java.Runtime.Environment/Java.Runtime.Environment.dll.config
7866

7967
include build-tools/scripts/mono.mk
80-
include build-tools/scripts/jdk.mk
68+
-include bin/Build$(CONFIGURATION)/JdkInfo.mk
8169

8270
JAVA_RUNTIME_ENVIRONMENT_DLLMAP_OVERRIDE = Java.Runtime.Environment.Override.dllmap
8371
ifeq ($(wildcard $(JAVA_RUNTIME_ENVIRONMENT_DLLMAP_OVERRIDE)),)
@@ -86,10 +74,6 @@ else
8674
JAVA_RUNTIME_ENVIRONMENT_DLLMAP_OVERRIDE_CMD = '/@JAVA_RUNTIME_ENVIRONMENT_DLLMAP@/ {' -e 'r $(JAVA_RUNTIME_ENVIRONMENT_DLLMAP_OVERRIDE)' -e 'd' -e '}'
8775
endif
8876

89-
src/Java.Runtime.Environment/Java.Runtime.Environment.dll.config: src/Java.Runtime.Environment/Java.Runtime.Environment.dll.config.in \
90-
bin/Build$(CONFIGURATION)/JdkInfo.props
91-
sed -e 's#@JI_JVM_PATH@#$(JI_JVM_PATH)#g' -e 's#@OS_NAME@#$(DLLMAP_OS_NAME)#g' -e $(JAVA_RUNTIME_ENVIRONMENT_DLLMAP_OVERRIDE_CMD) < $< > $@
92-
9377
JAVA_INTEROP_LIB = libjava-interop$(NATIVE_EXT)
9478
NATIVE_TIMING_LIB = libNativeTiming$(NATIVE_EXT)
9579

‎NuGet.Config‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,4 @@
1111
<!-- For XliffTasks -->
1212
<add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" protocolVersion="3" />
1313
</packageSources>
14-
<config>
15-
<add key="globalPackagesFolder" value="packages" />
16-
</config>
1714
</configuration>

‎build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks.csproj‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
<ItemGroup>
1010
<PackageReference Include="Microsoft.Build.Framework" />
1111
<PackageReference Include="Microsoft.Build.Utilities.Core" />
12+
<PackageReference Include="NUnit.ConsoleRunner" GeneratePathProperty="True" />
1213
</ItemGroup>
1314

1415
<ItemGroup>
@@ -18,4 +19,6 @@
1819
</ProjectReference>
1920
</ItemGroup>
2021

22+
<Import Project="Java.Interop.BootstrapTasks.targets" />
23+
2124
</Project>

‎build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks.targets‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
<Project>
2+
3+
<Target Name="_CreatePackagePathsProps"
4+
AfterTargets="AfterBuild"
5+
Inputs="$(MSBuildThisFileFullPath);$(MSBuildThisFileDirectory)Java.Interop.BootstrapTasks.csproj"
6+
Outputs="$(_OutputPath)PackagePaths.props">
7+
<ItemGroup>
8+
<_Path Include="PkgNUnit_ConsoleRunner" Destination="$(PkgNUnit_ConsoleRunner)" />
9+
</ItemGroup>
10+
<Message Text="PkgNUnit_ConsoleRunner=$(PkgNUnit_ConsoleRunner)" />
11+
<ItemGroup>
12+
<_Line Include="&lt;Project&gt;" />
13+
<_Line Include=" &lt;PropertyGroup&gt;" />
14+
<_Line Include="@(_Path->' &lt;%(Identity) Condition=&quot; %27%24(%(Identity))%27 == %27%27 &quot;&gt;%(Destination)&lt;/%(Identity)&gt;', '
15+
')" />
16+
<_Line Include=" &lt;/PropertyGroup&gt;" />
17+
<_Line Include="&lt;/Project&gt;" />
18+
</ItemGroup>
19+
<WriteLinesToFile
20+
File="$(_OutputPath)PackagePaths.props"
21+
Lines="@(_Line)"
22+
Overwrite="True"
23+
/>
24+
<Touch Files="$(_OutputPath)PackagePaths.props" />
25+
</Target>
26+
27+
</Project>

‎build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks/GenerateVersionFile.cs‎ renamed to ‎build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks/ReplaceFileContents.cs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,15 @@
77

88
namespace Java.Interop.BootstrapTasks
99
{
10-
public class GenerateVersionFile : Task
10+
public class ReplaceFileContents : Task
1111
{
12-
public ITaskItem InputFile { get; set; }
12+
public ITaskItem TemplateFile { get; set; }
1313
public ITaskItem OutputFile { get; set; }
1414

1515
public ITaskItem [] Replacements { get; set; }
1616
public override bool Execute ()
1717
{
18-
string text = File.ReadAllText (InputFile.ItemSpec);
18+
string text = File.ReadAllText (TemplateFile.ItemSpec);
1919
foreach (var replacement in Replacements)
2020
{
2121
text = text.Replace (replacement.ItemSpec, replacement.GetMetadata ("Replacement"));

‎build-tools/scripts/RunNUnitTests.targets‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
11
<?xml version="1.0" encoding="utf-8"?>
22
<Project DefaultTargets="RunTests" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
3+
<Import
4+
Condition=" Exists('..\..\bin\Build$(Configuration)\PackagePaths.props') "
5+
Project="..\..\bin\Build$(Configuration)\PackagePaths.props"
6+
/>
37
<PropertyGroup>
48
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
59
<_TopDir>$(MSBuildThisFileDirectory)..\..</_TopDir>
610
<_Runtime Condition=" '$(RUNTIME)' != '' ">$(RUNTIME)</_Runtime>
711
<_Runtime Condition=" '$(RUNTIME)' == '' And '$(OS)' != 'Windows_NT' ">mono --debug</_Runtime>
8-
<_NUnit>$(_Runtime) packages\nunit.consolerunner\3.12.0\tools\nunit3-console.exe</_NUnit>
12+
<_NUnit>$(_Runtime) $(PkgNUnit_ConsoleRunner)/tools/nunit3-console.exe</_NUnit>
913
<_Run Condition=" '$(RUN)' != '' ">--run=&quot;$(RUN)&quot;</_Run>
1014
</PropertyGroup>
1115
<Import

‎build-tools/scripts/VersionInfo.targets‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
33

44
<UsingTask AssemblyFile="$(MSBuildThisFileDirectory)..\..\bin\Build$(Configuration)\Java.Interop.BootstrapTasks.dll"
5-
TaskName="Java.Interop.BootstrapTasks.GenerateVersionFile" />
5+
TaskName="Java.Interop.BootstrapTasks.ReplaceFileContents" />
66
<PropertyGroup>
77
<GitDefaultBranch>main</GitDefaultBranch>
88
<GitThisAssembly>false</GitThisAssembly>
@@ -20,8 +20,8 @@
2020
<Replacements Include="@COMMIT@" Replacement="$(GitCommit)"/>
2121
<Replacements Include="@BRANCH@" Replacement="$(GitBranch)"/>
2222
</ItemGroup>
23-
<GenerateVersionFile
24-
InputFile="$(MSBuildThisFileDirectory)Version.props.in"
23+
<ReplaceFileContents
24+
TemplateFile="$(MSBuildThisFileDirectory)Version.props.in"
2525
OutputFile="$(MSBuildThisFileDirectory)..\..\bin\Build$(Configuration)\Version.props"
2626
Replacements="@(Replacements)"
2727
/>

‎build-tools/scripts/jdk.mk‎

Lines changed: 0 additions & 49 deletions
This file was deleted.

‎build-tools/scripts/jdk.targets‎

Lines changed: 0 additions & 22 deletions
This file was deleted.

0 commit comments

Comments
 (0)