Start publishing Unified Builds to BAR

[premun]

Depends on #4157

• Figure out if packages are placed in a common directory in the UB build -> packages are now under artifacts/packages/<config>/Shipping/** and assets are under artifacts/assets/<config>.
• Figure out if we want to use /src/arcade/eng/common or /eng/common. I think the first as the other doesn't get updated really -> we will use /eng/common
• Create and Publish VerticalManifests during unified builds #4198
• Create and publish a MergedManifest for unified builds #4199
• Add a Publish using Darc stage that will publish a build to a darc channel

The UB build: https://dnceng.visualstudio.com/internal/_build?definitionId=1330

Depends On

• Resolve dual 'source of truth' for publishing info #4101

[premun]

There's no urgency with that task yet as we don't have assets in the build yet but maybe we could publish an empty BAR build from the VMR so that we can trigger fake subscriptions.

It might be better to start publishing this in the INT Maestro BAR then.

[ViktorHofer]

Figure out if we want to use /src/arcade/eng/common or /eng/common. I think the first as the other doesn't get updated really

We should use /eng/common. src/arcade/eng/common, src/installer/eng/common and /eng/common are always in sync (identical).

Figure out if packages are placed in a common directory in the UB build

I'm working on that right now.

[ViktorHofer]

Depends on #4157

[ViktorHofer]

FYI @akoeplinger

[dkurepa]

Currently, every repo uses the following publishing flow:

flowchart TB
subgraph emsdk
    subgraph "win x64 leg"
        winBuild[Build] --> winSign[Sign]
        winSign --> winAssetManifest[Create Asset Manifest]
        winAssetManifest --> winPublish[Publish to Artifacts]
    end
    subgraph "linux x64 leg"
        linuxBuild[Build] --> linuxSign[Sign]
        linuxSign --> linuxAssetManifest[Create Asset Manifest]
        linuxAssetManifest --> linuxPublish[Publish to Artifacts]
    end
    subgraph "macOS x64 leg"
        macOSBuild[Build] --> macOSSign[Sign]
        macOSSign --> macOSAssetManifest[Create Asset Manifest]
        macOSAssetManifest --> macOSPublish[Publish to Artifacts]
    end
    subgraph bar["Publish to Bar"]
        mergedManifest["Create Merged Manifest"] --> publishBar["Publish build to bar, using Merged Manifest"]
    end
    winPublish --> bar
    linuxPublish --> bar
    macOSPublish --> bar
end

Loading

• Every build leg creates and publishes packages as artifacts and creates its own Asset Manifest
• After all build legs are finished, a MergedManifest is created using all Asset Manifests

Things will work a bit differently for Unified builds:

flowchart TB
subgraph UnifiedBuild
    subgraph win-x64
        subgraph winEmsdk["emsdk"]
            emsdkWinBuild["Build"] --> emsdkWinPublish[Publish produced bits and AssetManifest]
        end
        subgraph winRuntime["runtime"]
            runtimeWinBuild["Build"] --> runtimeWinPublish[Publish produced bits and AssetManifest]
        end
        subgraph winAspnetcore["aspnetcore"]
            aspWinBuild["Build"] --> aspWinPublish[Publish produced bits and AssetManifest]
        end
        emsdkWinPublish --> winMergeManifests[Merge Windows Asset Manifests into a VerticalManifest]
        runtimeWinPublish --> winMergeManifests
        aspWinPublish --> winMergeManifests
    end
    subgraph linux-x64
        subgraph linuxEmsdk["emsdk"]
            emsdkLinuxBuild["Build"] --> emsdkLinuxPublish[Publish produced bits and AssetManifest]
        end
        subgraph linuxRuntime["runtime"]
            runtimeLinuxBuild["Build"] --> runtimeLinuxPublish[Publish produced bits and AssetManifest]
        end
        subgraph linuxAspnetcore["aspnetcore"]
            aspLinuxBuild["Build"] --> aspLinuxPublish[Publish produced bits and AssetManifest]
        end
        emsdkLinuxPublish --> linuxMergeManifests[Merge Linux Asset Manifests into a VerticalManifest]
        runtimeLinuxPublish --> linuxMergeManifests
        aspLinuxPublish --> linuxMergeManifests
    end
    winMergeManifests --> mergedManifest["Generate Merged Manifest"]
    linuxMergeManifests --> mergedManifest
    mergedManifest --> barPublish["Publish Unified Build to BAR"]
end

Loading

• Every leg will build its own vertical.
• In a vertical, every repo will be built using arcade, so every repo will have its own AssetManifest
• After every vertical build, we'll create a VerticalAssetManifest, which will combine all AssetManifests, and also add additional data if needed
• After all legs are done, we'll create a MergedManifest by merging all of the VerticalAsseetManifests and use it to create a BAR build

[ViktorHofer]

cc @mmitche