Skip to content

Improve Windows build setup #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 31, 2023
Merged

Improve Windows build setup #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e99ac3b
Improve Windows build setup
jasagredo Jul 26, 2023
4928334
Bring back pacman packages
jasagredo Jul 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ This action will install the system dependencies we have:

- libsodium
- libsecp256k1
- pkg-config
- openssl
- libblst
- libsystemd (on Linux only)

See [iohk-nix/releases](https://github.com/input-output-hk/iohk-nix/releases/tag/latest)

Expand All @@ -19,13 +22,12 @@ See [iohk-nix/releases](https://github.com/input-output-hk/iohk-nix/releases/tag

## haskell action

This action will install ghc and cabal as needed
This action will set ghc and cabal as needed

```
- name: Install Haskell
uses: input-output-hk/actions/haskell@latest
with:
ghc-version: ${{ matrix.ghc }}
cabal-version: 3.10.1.0
pacman-packages: ... # more pacman package to install
```
```
48 changes: 32 additions & 16 deletions base/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'Setup IOG librararies'
description: 'Installs libsodum, libsecp256k1 and libblst'
name: 'Setup IOG libraries'
description: 'Installs a set of libraries our projects usually depend on'
inputs:
use-sodium-vrf:
description: Use libsodium with VRF extension
Expand Down Expand Up @@ -28,24 +28,38 @@ runs:
popd
rm -fR __prep__

- name: "[darwin] update environemnt variables"
- name: "[darwin] update environment variables"
if: runner.os == 'macOS'
shell: bash
run: |
echo 'PKG_CONFIG_PATH=/usr/local/opt/cardano/lib/pkgconfig:$PKG_CONFIG_PATH' >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=/usr/local/lib" >> $GITHUB_ENV
echo "PKG_CONFIG_PATH=/usr/local/opt/cardano/lib/pkgconfig:/usr/local/opt/[email protected]/lib/pkgconfig:$PKG_CONFIG_PATH" >> $GITHUB_ENV

# There are three pkg-config packages available for mingw, and each one has their problems:
#
# - `pacman -S pkgconf` (usually pulled by installing `base-devel`): can't interpret the `PKG_CONFIG_PATH` in a semicolon format.
# For `PKG_CONFIG_PATH=C:/pathA;C:/pathB` it searches in `C`, `/pathA;C` and `/pathB`.
#
# - `pacman -S mingw-w64-x86_64-pkgconf`: when queried for multiple modversions (as cabal does), it will return only one.
#
# - `pacman -S mingw-w64-x86_64-pkg-config`: will strip out `C:/mingw64/lib` and `C:/mingw64/include` from the returned values
# unless provided with `PKG_CONFIG_ALLOW_SYSTEM_(CFLAGS|LIBS)` as I do here.
- name: "[windows] Install library dependencies"
if: runner.os == 'Windows'
shell: C:/msys64/usr/bin/bash.exe -e '{0}'
env:
MSYSTEM: MSYS
MSYSTEM: MINGW64
# do not ever change to $HOME by yourself.
CHERE_INVOKING: 1
# do we want to inherit the path?
# MSYS2_PATH_TYPE: inherit
run: |
/usr/bin/pacman --noconfirm -S \
mingw-w64-x86_64-pkg-config
mv /c/Strawberry/perl/bin/pkg-config /c/Strawberry/perl/bin/perl-pkg-config
mv /c/Strawberry/perl/bin/pkg-config.bat /c/Strawberry/perl/bin/perl-pkg-config.bat

/usr/bin/pacman --noconfirm -S mingw-w64-x86_64-pkg-config mingw-w64-x86_64-openssl
echo "PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1" >> $GITHUB_ENV
echo "PKG_CONFIG_ALLOW_SYSTEM_LIBS=1" >> $GITHUB_ENV

mkdir __prep__
pushd __prep__
Expand All @@ -58,20 +72,22 @@ runs:
popd
rm -fR __prep__

- name: "[windows] update environemnt variables"
# MSYS will inherit the PATH from PWSH, but not the other way round, and setting the path
# to "PATH=...:$PATH" in bash results in all the paths starting with `/c/` instead of `C:\` which
# makes some actions (like `actions/checkout`) fail as it cannot find executables.
- name: "[windows] update environment variables"
if: runner.os == 'Windows'
shell: pwsh
run: |
# Add the `bin` folder, which contains all the .dlls on windows, so we
# can find them.
"PATH=C:\\msys64\mingw64\opt\cardano\bin;C:\msys64\mingw64\bin;{0}" -f $env:PATH >> $env:GITHUB_ENV
"PKG_CONFIG_PATH=C:\\msys64\mingw64\opt\cardano\lib\pkgconfig;{0}" -f $env:PKG_CONFIG_PATH >> $env:GITHUB_ENV

"PATH=C:\\msys64\\mingw64\\opt\\cardano\\bin;C:\\msys64\\mingw64\\bin;{0}" -f $env:PATH >> $env:GITHUB_ENV
"PKG_CONFIG_PATH=C:\\msys64\\mingw64\\opt\\cardano\\lib\\pkgconfig;C:\\msys64\\mingw64\\lib\\pkgconfig;C:\\msys64\\mingw64\\share\\pkgconfig;C:\\msys64\\usr\\lib\\pkgconfig;C:\\msys64\\usr\\share\\pkgconfig;{0}" -f $env:PKG_CONFIG_PATH >> $env:GITHUB_ENV

- name: "[linux] Install library dependencies"
if: runner.os == 'Linux'
shell: bash
run: |
sudo apt-get -y install libsystemd-dev

mkdir __prep__
pushd __prep__
curl -sL ${{ inputs.url-prefix }}/debian.${{ inputs.use-sodium-vrf && 'libsodium-vrf' || 'libsodium' }}.deb > libsodium.deb
Expand All @@ -83,9 +99,9 @@ runs:
popd
rm -fR __prep__

- name: "[linux] update environemnt variables"
- name: "[linux] update environment variables"
if: runner.os == 'Linux'
shell: bash
run: |
echo 'PKG_CONFIG_PATH=/usr/local/opt/cardano/lib/pkgconfig:$PKG_CONFIG_PATH' >> $GITHUB_ENV
echo 'LD_LIBRARY_PATH=/usr/local/opt/cardano/lib:$LD_LIBRARY_PATH' >> $GITHUB_ENV
echo "PKG_CONFIG_PATH=/usr/local/opt/cardano/lib/pkgconfig:$PKG_CONFIG_PATH" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=/usr/local/opt/cardano/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
82 changes: 10 additions & 72 deletions haskell/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'Setup libsecp256k1'
description: 'Installs libsecp256k1'
name: 'Setup Haskell toolchain versions'
description: 'Setup Haskell toolchain versions'
inputs:
ghc-version:
description: Version of GHC to install
Expand All @@ -21,84 +21,22 @@ outputs:
runs:
using: "composite"
steps:
- name: "Install pkg-config"
if: runner.os == 'macOS'
shell: bash
run: brew install pkg-config

- name: Setup PKG_CONFIG_PATH
shell: bash
run: |
if [ ${{ runner.os }} == "macOS" ]; then
# OpenSSL is installed in a non-standard location in MacOS. See
# https://github.com/actions/virtual-environments/blob/main/images/macos/macos-11-Readme.md
echo "PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/local/opt/[email protected]/lib/pkgconfig" | tee -a "$GITHUB_ENV"
elif [ ${{ runner.os }} == "Linux" ]; then
echo "PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/local/lib/pkgconfig" | tee -a "$GITHUB_ENV"
fi

- name: Setup LD_LIBRARY_PATH
shell: bash
run: |
if [ ${{ runner.os }} != "Windows" ]; then
# OpenSSL is installed in a non-standard location in MacOS. See
# https://github.com/actions/virtual-environments/blob/main/images/macos/macos-11-Readme.md
echo "LD_LIBRARY_PATH=/usr/local/lib" | tee -a "$GITHUB_ENV"
fi

# For some unknown reason, the pacman setup must come before ghc installation.
# It appears as if PATHEXT is set _after_ ghcup install ghc/cabal, and
# as such we'd need pacman.exe instead.
- name: "WIN: Install System Dependencies via pacman (msys2)"
if: runner.os == 'Windows' && inputs.pacman-packages != ''
shell: pwsh
run: |
# ghcup should be installed on current GHA Windows runners. Let's use ghcup to run
# pacman, to install the necessary dependencies, ...
ghcup run -- pacman --noconfirm -S ${{ inputs.pacman-packages }}

- name: Setup GHC (Windows)
if: inputs.ghc-version != '' && runner.os == 'Windows'
shell: pwsh
run: ghcup install ghc --set ${{ inputs.ghc-version }}

- name: Setup GHC (Posix)
- name: Setup GHC
if: inputs.ghc-version != ''
shell: bash
run: ghcup install ghc --set ${{ inputs.ghc-version }}

- name: Setup cabal (Windows)
if: inputs.cabal-version != '' && runner.os == 'Windows'
shell: pwsh
run: ghcup install cabal --set ${{ inputs.cabal-version }}

- name: Setup cabal (Posix)
- name: Setup cabal
if: inputs.cabal-version != ''
shell: bash
run: ghcup install cabal --set ${{ inputs.cabal-version }}

- name: "WIN: fixup cabal config"
if: runner.os == 'Windows'
shell: pwsh
- name: "WIN: Install System Dependencies via pacman (msys2)"
if: runner.os == 'Windows' && inputs.pacman-packages != ''
shell: 'C:/msys64/usr/bin/bash.exe -e {0}'
run: |
# make sure cabal knows about msys64, and mingw64 tools. Not clear why C:/cabal/config is empty
# and C:/cabal doesn't even exist. The ghcup bootstrap file should have create it in the image:
# See https://github.com/haskell/ghcup-hs/blob/787edc17af4907dbc51c85e25c490edd8d68b80b/scripts/bootstrap/bootstrap-haskell#L591
# So we'll do it by hand here for now.
#
# We'll _not_ add extra-include-dirs, or extra-lib-dirs, and rely on what's shipped with GHC.
# https://github.com/msys2/MINGW-packages/issues/10837#issuecomment-1047105402
# https://gitlab.haskell.org/ghc/ghc/-/issues/21111
# if we _do_ want them, this would be the lines to add below

$ghcMingwDir = Join-Path -Path $(ghc --print-libdir) `
-ChildPath ../mingw/x86_64-*-mingw32/lib/ `
-Resolve

cabal user-config -a "extra-prog-path: C:/msys64/mingw64/bin, C:/msys64/usr/bin" `
-a "extra-include-dirs: C:/msys64/mingw64/include" `
-a ("extra-lib-dirs: {0}, C:/msys64/mingw64/lib" -f $ghcMingwDir) `
-f init
printf "::warning::pacman packages will be installed via the input-output-hk/actions/haskell workflow.\nHowever there is no need to do it here and in general it is better to just do it on your workflow using \`/usr/bin/pacman -S -noconfirm %s\`.\nWe might remove this feature soon so please do as advised.\n\nA full step you can copy-paste into your workflow would look as follows:\n\`\`\`\n - name: \"WIN: Install System Dependencies via pacman (msys2)\"\n if: runner.os == 'Windows'\n shell: 'C:/msys64/usr/bin/bash.exe -e {0}'\n run: |\n /usr/bin/pacman -S -noconfirm %s\n\`\`\`\n" "$(echo "${{ inputs.pacman-packages }}" | head -c -1)" "$(echo "${{ inputs.pacman-packages }}" | head -c -1)"
/usr/bin/pacman --noconfirm -S ${{ inputs.pacman-packages }}

# Unify the computation of the cabal store directory to a single step. This makes referencing the cabal
# store in later steps easier.
Expand Down Expand Up @@ -135,4 +73,4 @@ runs:
else
echo "cabal-store=C:\\cabal\\store" | tee -a "$GITHUB_OUTPUT"
fi
fi
fi