Skip to content

Commit 6f9fc19

Browse files
teo-tsirpanisteo-tsirpanis
authored
Overhaul the NuGet packaging infrastructure and the way the Core binaries get acquired. (#146)
* Start writing an MSBuild script that creates NuGet packages for the core. For now it downloads the artifacts from GitHub Releases. * Support creating the native packages. * Support creating the metapackage. * (REMOVE BEFORE MERGE) Add a temporary MyGet feed for the TileDB native packages. * Use the TileDB Embedded page for the project URL in the native packages. * Use the `TileDB.Native` NuGet package and remove the nuspec of `TileDB.CSharp`. * Set `UseCurrentRuntimeIdentifier` in the executable projects. * Restore the template packages. And add a property to them that seems to save time. * Remove the ValueSeparator hack. * Remove the "Install/Download tiledb" steps from CI. And disable fail-fast in the Run-Tests action matrix. * Remove all CI jobs except of Run-Tests. Now that we don't use a nuspec to pack the C# library there are little reasons to specifically test it with a packed NuGet package. The Release job also went away; it would upload the 0.0.0-local package. We need a dedicated release workflow. * Remove RollForward from TileDB.CSharp. It does not apply to libraries. * Update package metadata and move them to a dedicated file. I had originally removed the nuspec files from the native packages but changed my mind for aesthetic reasons since they generate an empty .NET Standard 2.0 dependency group. * Add support for generating development editions of the native packages. They have SemVer 2.0 versions that identify the branch. * Enable Central Package Management with package source mapping. And support changing the native package under a different name in development builds. * Add release notes in the packages. * Refactor the nightly build workflow to use native development NuGet packages. The core is built in separate jobs and not once for each .NET version. * Update the repository README and add a package README. * Require `Version` and `VersionTag` to be specified. Make the version tag optional for development builds. And fail if no native binaries were found in development builds. * Add a README describing the native NuGet package generator. * Demand Core version 2.12.x. * Warn when building an RID-agnostic executable that uses `TileDB.CSharp`. * Remove the RID-specific build instructions in the metapackage's description. This package is not intended to be directly referenced by the average developer, at least not without `TileDB.CSharp`. * Remove the MyGet feed; we are now using the official TileDB binaries from NuGet. * Use MSBuild's custom item separator feature instead of a `string.Replace` hack.
1 parent 5c217a9 commit 6f9fc19

33 files changed

+464
-920
lines changed

‎.editorconfig‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ indent_style = space
55
indent_size = 4
66
insert_final_newline = true
77

8-
[*.{xml,*proj,props,targets,yml}]
8+
[*.{xml,*proj,props,targets,yml,nuspec}]
99
indent_size = 2
1010

1111
[*.cs]

‎.github/scripts/download_tiledb.sh‎

Lines changed: 0 additions & 18 deletions
This file was deleted.

‎.github/scripts/install_tiledb.sh‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎.github/workflows/nightly.yml‎

Lines changed: 87 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,90 +1,135 @@
11
name: Nightly-Build
22

3+
env:
4+
TILEDB_NIGHTLY_BUILD: 1
5+
36
on:
47
schedule:
58
- cron: "15 2 * * *"
69
workflow_dispatch:
710

811
jobs:
9-
Build:
12+
Build-Native:
1013
strategy:
1114
fail-fast: false
1215
matrix:
13-
os: [ubuntu-22.04, ubuntu-latest, macos-latest, windows-latest]
16+
os: [ubuntu-latest, macos-latest, windows-latest]
1417
include:
15-
- os: ubuntu-22.04
16-
HOST: linux
17-
BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
18+
- BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
1819
- os: ubuntu-latest
19-
HOST: linux
20-
BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
20+
platform: linux-x86_64
2121
- os: macos-latest
22-
HOST: osx
23-
BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
22+
platform: macos-x86_64
2423
- os: windows-latest
25-
HOST: win
24+
platform: windows-x86_64
2625
BOOTSTRAP: ../bootstrap.ps1 -EnableS3 -EnableSerialization
2726
tag: [release-2.12, dev]
28-
dotnet: ['net5.0', 'net6.0']
2927
runs-on: ${{ matrix.os }}
30-
3128
steps:
32-
- name: Checkout TileDB-CSharp
33-
uses: actions/checkout@v3
34-
with:
35-
path: tiledb-csharp
36-
3729
- name: Checkout TileDB
3830
uses: actions/checkout@v3
3931
with:
4032
repository: TileDB-Inc/TileDB
4133
ref: ${{ matrix.tag }}
42-
path: tiledb
34+
35+
- name: Build TileDB
36+
run: |
37+
mkdir -p build
38+
cd build
39+
${{ matrix.BOOTSTRAP }}
40+
cmake --build . --config Release
41+
cmake --build . --config Release --target install-tiledb
42+
cd ../
43+
mkdir -p artifacts/${{ matrix.platform }}
44+
mv dist/* artifacts/${{ matrix.platform }}
45+
46+
# We combine the binaries of the many platforms per tag by uploading them to the same artifact.
47+
- name: Upload native artifacts
48+
uses: actions/upload-artifact@v3
49+
with:
50+
name: tiledb-native-${{ matrix.tag }}
51+
path: artifacts/
52+
53+
Pack-NuGet:
54+
needs: Build-Native
55+
strategy:
56+
fail-fast: false
57+
matrix:
58+
tag: [release-2.12, dev]
59+
runs-on: ubuntu-latest
60+
steps:
61+
- name: Checkout TileDB-CSharp
62+
uses: actions/checkout@v3
4363

4464
# GitHub runners come with several versions of .NET preinstalled; Remove them to target version
4565
- name: Remove existing .NET versions
4666
shell: bash
47-
run: |
48-
rm -rf $DOTNET_ROOT
67+
run: rm -rf $DOTNET_ROOT
4968

5069
- name: Set up .NET SDK from global.json
5170
uses: actions/setup-dotnet@v3
52-
with:
53-
global-json-file: tiledb-csharp/global.json
5471

5572
- name: Display dotnet versions
5673
run: dotnet --info
5774

58-
- name: Build TileDB
59-
run: |
60-
mkdir -p tiledb/build
61-
cd tiledb/build
62-
${{ matrix.BOOTSTRAP }}
63-
cmake --build . --config Release
64-
cmake --build . --config Release --target install-tiledb
65-
cd ../../
66-
cp -r tiledb/dist/lib/* tiledb-csharp/sources/TileDB.CSharp/runtimes/${{ matrix.HOST }}-x64/native/
75+
- name: Download native artifacts
76+
uses: actions/download-artifact@v3
77+
with:
78+
name: tiledb-native-${{ matrix.tag }}
79+
path: scripts/nuget/temp
6780

68-
- name: Copy Windows tiledb.dll
69-
if: matrix.HOST == 'win'
70-
run: |
71-
cp tiledb/dist/bin/tiledb.dll tiledb-csharp/sources/TileDB.CSharp/runtimes/${{ matrix.HOST }}-x64/native/
81+
- name: Build native NuGet packages
82+
shell: bash
83+
run: dotnet pack ./scripts/nuget/GenerateNuGetPackages.proj -p:DevelopmentBuild=true -p:VersionTag=${{ matrix.tag }}
84+
85+
- name: Upload native NuGet packages
86+
uses: actions/upload-artifact@v3
87+
with:
88+
name: tiledb-nuget-${{ matrix.tag }}
89+
path: scripts/nuget/packages/
90+
91+
Test-NuGet:
92+
needs: Pack-NuGet
93+
strategy:
94+
fail-fast: false
95+
matrix:
96+
os: [ubuntu-22.04, ubuntu-latest, macos-latest, windows-latest]
97+
tag: [release-2.12, dev]
98+
dotnet: ['net5.0', 'net6.0']
99+
runs-on: ${{ matrix.os }}
100+
steps:
101+
- name: Checkout TileDB-CSharp
102+
uses: actions/checkout@v3
103+
104+
# GitHub runners come with several versions of .NET preinstalled; Remove them to target version
105+
- name: Remove existing .NET versions
106+
shell: bash
107+
run: rm -rf $DOTNET_ROOT
108+
109+
- name: Set up .NET SDK from global.json
110+
uses: actions/setup-dotnet@v3
111+
112+
- name: Display dotnet versions
113+
run: dotnet --info
114+
115+
- name: Download native NuGet packages
116+
uses: actions/download-artifact@v3
117+
with:
118+
name: tiledb-nuget-${{ matrix.tag }}
119+
path: packages
72120

73121
- name: Build TileDB-CSharp
74-
run: |
75-
dotnet build tiledb-csharp/sources/TileDB.CSharp/TileDB.CSharp.csproj /p:Platform=x64 -c Release
122+
run: dotnet build sources/TileDB.CSharp/TileDB.CSharp.csproj -c Release
76123

77124
- name: Test TileDB-CSharp
78-
run: |
79-
dotnet test tiledb-csharp/tests/TileDB.CSharp.Test/TileDB.CSharp.Test.csproj -c Release -f ${{ matrix.dotnet }}
125+
run: dotnet test tests/TileDB.CSharp.Test/TileDB.CSharp.Test.csproj -c Release -f ${{ matrix.dotnet }}
80126

81127
- name: Run examples
82128
shell: bash
83-
run: |
84-
find tiledb-csharp/examples/ -name *.csproj -execdir dotnet run \;
129+
run: find examples/ -name *.csproj | xargs -I{} dotnet run --project {} -f ${{ matrix.dotnet }}
85130

86131
Create-Issue:
87-
needs: Build
132+
needs: Test-NuGet
88133
if: failure()
89134
runs-on: ubuntu-latest
90135
steps:

‎.github/workflows/tiledb-csharp.yml‎

Lines changed: 2 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@ name: TileDB-CSharp
22

33
on:
44
push:
5-
tags: [ '*' ]
65
branches: [ main ]
76
pull_request:
87
branches: [ main ]
@@ -11,6 +10,7 @@ on:
1110
jobs:
1211
Run-Tests:
1312
strategy:
13+
fail-fast: false
1414
matrix:
1515
# Will be checking following versions
1616
dotnet: ['net5.0', 'net6.0']
@@ -21,11 +21,6 @@ jobs:
2121
# Checks out repository
2222
- uses: actions/checkout@v3
2323

24-
# Install tiledb
25-
- name: Install tiledb
26-
shell: bash
27-
run: ./.github/scripts/install_tiledb.sh
28-
2924
- name: Remove existing .NET versions
3025
shell: bash
3126
run: |
@@ -40,7 +35,7 @@ jobs:
4035
# DotNet build
4136
- name: Dotnet build for TileDB.CSharp
4237
run: |
43-
dotnet build /p:Platform=x64 -c Release sources/TileDB.CSharp
38+
dotnet build -c Release sources/TileDB.CSharp
4439
4540
# DotNet test
4641
- name: Test TileDB.CSharp
@@ -51,96 +46,3 @@ jobs:
5146
shell: bash
5247
run: |
5348
find examples/ -name *.csproj | xargs -I{} dotnet run --project {} -f ${{ matrix.dotnet }}
54-
55-
Stage-Release-Candidate:
56-
needs: Run-Tests
57-
runs-on: ubuntu-latest
58-
steps:
59-
# Checks out repository
60-
- uses: actions/checkout@v3
61-
62-
- name: Set up .NET SDK from global.json
63-
uses: actions/setup-dotnet@v3
64-
65-
- name: Display dotnet versions
66-
run: dotnet --info
67-
68-
# Download tiledb
69-
- name: Download tiledb
70-
run: ./.github/scripts/download_tiledb.sh
71-
72-
# DotNet build
73-
- name: Dotnet build for TileDB.CSharp
74-
run: |
75-
dotnet build /p:Platform=x64 -c Release sources/TileDB.CSharp
76-
77-
# DotNet pack
78-
- name: Dotnet pack for TileDB.CSharp
79-
run: |
80-
dotnet pack ./sources/TileDB.CSharp/TileDB.CSharp.csproj -p:Version=0.0.0-local -c Release
81-
82-
- name: Archive nuget artifact
83-
uses: actions/upload-artifact@v3
84-
with:
85-
name: TileDB NuGet Package
86-
path: sources/TileDB.CSharp/lib/TileDB.CSharp.*.nupkg
87-
88-
Test-NuGet-Release:
89-
needs: Stage-Release-Candidate
90-
strategy:
91-
matrix:
92-
os: [ubuntu-latest, macos-latest, windows-latest]
93-
dotnet: ['net5.0', 'net6.0']
94-
runs-on: ${{ matrix.os }}
95-
steps:
96-
- name: Checkout TileDB-CSharp repository
97-
uses: actions/checkout@v3
98-
99-
- name: Download TileDB.CSharp NuGet artifact
100-
uses: actions/download-artifact@v3
101-
with:
102-
name: TileDB NuGet Package
103-
path: packages
104-
105-
# GitHub runners come with several versions of .NET preinstalled; Remove them to target version
106-
- name: Remove existing .NET versions
107-
shell: bash
108-
run: |
109-
rm -rf $DOTNET_ROOT
110-
111-
- name: Set up .NET SDK from global.json
112-
uses: actions/setup-dotnet@v3
113-
114-
- name: Display dotnet versions
115-
run: dotnet --info
116-
117-
- name: Setup NuGet
118-
uses: nuget/setup-nuget@v1
119-
120-
# Run tests using NuGet release candidate
121-
- name: Test TileDB.CSharp
122-
run: |
123-
dotnet test -c Release tests/TileDB.CSharp.Test.NuGet -f ${{ matrix.dotnet }}
124-
125-
Release:
126-
# Only run this job if a tag was provided
127-
if: startsWith(github.ref, 'refs/tags/')
128-
needs: Test-NuGet-Release
129-
runs-on: ubuntu-latest
130-
steps:
131-
- uses: actions/download-artifact@v3
132-
with:
133-
name: TileDB NuGet Package
134-
135-
- name: Create Release
136-
id: create_release
137-
uses: softprops/action-gh-release@v1
138-
env:
139-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
140-
with:
141-
files: ./*.nupkg
142-
tag_name: ${{ github.ref_name }}
143-
name: ${{ github.ref_name }}
144-
body: ${{ steps.github_release.outputs.changelog }}
145-
draft: false
146-
prerelease: false

‎.gitignore‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,6 @@
4444

4545
TileDB-CSharp.sln.DotSettings.user
4646

47-
build
48-
4947
TileDBConfig.cmake
5048
TileDbTargets*.cmake
5149
tiledb.pc
@@ -54,3 +52,7 @@ tiledb.pc
5452

5553
.DS_Store
5654
include
55+
scripts/nuget/temp
56+
*.binlog
57+
*.nupkg
58+
packages

‎Directory.Build.props‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,4 @@
22
<PropertyGroup>
33
<LanguageVersion>latest</LanguageVersion>
44
</PropertyGroup>
5-
<ItemGroup>
6-
<PackageReference Include="SonarAnalyzer.CSharp" Version="8.48.0.56517" PrivateAssets="all" />
7-
</ItemGroup>
85
</Project>

0 commit comments

Comments
 (0)