Support for multiple patterns with different settings for each

[dannyfallon]

We're currently uploading to S3 via a Grunt plugin. It allows considerable flexibility over the current S3 plugin that I'm interested in seeing if I can get some consensus on. Consider the following example of our setup:

module.exports = function(grunt) {
  return {
    release: {
      files: [
        {
          cwd: 'dist/assets/',
          dest: 'ember/',
          expand: true,
          src: ['*.css'],
          params: {
            ContentType: 'text/css; charset=utf8'
          }
        },
        {
          cwd: 'dist/assets/',
          dest: 'ember/',
          expand: true,
          src: ['*.js'],
          params: {
            ContentType: 'application/javascript; charset=utf8'
          }
        },
        {
          cwd: 'dist/assets/',
          dest: 'ember/',
          expand: true,
          src: ['*.gz'],
        },
        {
          cwd: 'dist/assets/',
          dest: 'ember/assets',
          expand: true,
          src: ['**/*.!(js|css|gz)', '!assetMap.json', '!failed-*.png', "!passed-*.png"]
        }
      ],
      options: {
        access: 'public-read',
        bucket: '...',
        region: '...',
        uploadConcurrency: 16,
        sslEnabled: true
      }
    }
  };
};

Firstly, pattern arrays are supported. This has a few advantages, but what we use it here for is negation. I see from the gzip plugin that support for an ignorePattern option was added, is this how you intend to proceed with other plugins? I see the pattern array with matching left to right as being more powerful and flexible than the current approach, if only because it allows multiple patterns and won't require extra parameters to be passed through promises.

The second thing I see here is that we're able to specify specific content-types and destinations for each pattern group. Is this something you could see a use for?

I've tried to sidestep this problem entirely by hooking a private plugin into the didBuild step, moving the JS/CSS in the <distDir>/assets directory out to the parent directory but alas each Pipeline's returning context is merged with what exists already, meaning you can't delete existing context - in this case I can't delete the <distDir>/assets/*.{js,css} files and later plugins bomb out when they try to access the non-existent files.

I could probably achieve what I desire with a custom upload client but that'd just be private and I'm interested in contributing if there's some agreement, though given the time pressure I'm under I reckon a custom upload client is going to happen anyways 😄

[lukemelia]

@dannyfallon have you explored the "alias" feature of ember-cli-deploy? It allows you to have multiple instances of the same plugin in your pipeline, each with its own configuration. I think this may provide you with the flexibility you need.

[dannyfallon]

Thanks for that, I wasn't aware about it but I think this may be the answer to my problem It won't handle the negation but I'm cool with that, it's just a tiny few assets. I'll let you know how I get on, really appreciate the pointer here

[lukemelia]

Sounds good. Also, I'm definitely open to finding a better pattern-matching primitive that we can offer across the plugin ecosystem. The pattern array one is interesting.

[dannyfallon]

So this didn't quite work out but I've solved my problem another way without having to create a custom upload client or use aliasing. I created a copier plugin that copied assets/*.{js,css,js.gz,css.gz} to distDir and use a single negation glob pattern to upload everything that matched with !assets/*.{js,css,js.gz,css.gz}.

The plugin does not have an option to strip the path before uploading - it always uses filePath (e.g. /assets/application.css) when uploading. This means that even using aliases and specifying a different prefix for my JS/CSS resulted it the files being uploaded to <prefix>/assets/application.css, not <prefix>/application.css.

I'm not sure what your thoughts are on that option @lukemelia, it appears to me that such an option coupled with aliasing would have prevented me writing my copier plugin but maybe it's undesirable as a wider feature

[lukemelia]

Closing for lack of activity.