Preserve attributes for files cached with Pipeline Caching

[johnterickson]

Pipeline Caching cache entries currently don't include file attributes (e.g. executable bit, mtime) nor preserve symbolic links.

[willsmythe]

The probable solution here is to tar/zip the contents of the cache prior to upload (and when downloading). In some scenarios (like caches with a large number of small files), this should also help improve cache restore/save performance.

[willsmythe]

Created a separate issue (#10925) to track cache content packing using tar/7z. Although packing is likely a solution for preserving file attributes, we should track these as separate requirements for now.

[wichert]

For what it's worth: this issue is currently resulting in electron builds failing on non-windows hosts if you use a cache. The resulting error is cryptic:

[command]/bin/bash --noprofile --norc /Users/vsts/agent/2.154.3/work/_temp/1e16c5e1-16da-4beb-8cbe-5423ac597cdb.sh
yarn run v1.16.0
$ electron-builder --mac
  • electron-builder version=20.41.0
  • artifacts will be published if draft release exists reason=CI detected
  • loaded configuration file=package.json ("build" field)
  • rebuilding native production dependencies platform=darwin arch=x64
  • rebuilding native dependency name=keytar
  • packaging       platform=darwin arch=x64 electron=5.0.2 appOutDir=dist/mac
Error: spawn /Users/vsts/agent/2.154.3/work/1/s/node_modules/app-builder-bin/mac/app-builder EACCES
    at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
    at onErrorNT (internal/child_process.js:415:16)
    at process._tickCallback (internal/process/next_tick.js:63:19)
From previous event:
 …  lot more tracebacks ...

The root cause is node_modules/app-builder-bin/mac/app-builder no longer being executable. A workaround is to manually fix permissions for a cache hit:

  - task: CacheBeta@0
    inputs:
      key: |
        yarn
        $(Agent.OS)
        $(Build.SourcesDirectory)/yarn.lock
      path: $(YARN_CACHE_FOLDER)
      cacheHitVar: CACHE_RESTORED
    displayName: Cache Yarn packages

  - script: |
      yarn --frozen-lockfile
    displayName: Install dependencies

  - script: |
      chmod +x node_modules/app-builder-bin/mac/app-builder
      chmod +x node_modules/7zip-bin/mac/7za
    condition: and(eq(variables.CACHE_RESTORED, 'true'), ne(variables['Agent.OS'], 'Windows_NT'))
    displayName: Correct file permissions