Mixins

[bajtos]

LoopBack provides two kinds of mixins which we need to support in loopback-boot too:

• A regular LoopBack model - mixing process copies all properties and methods to the target model.
• A function - mixing process calls the function after the model has been attached.

Mixin as a model - NOT IMPLEMENTED

loopback-boot already has mechanism for defining arbitrary models, we should reuse this mechanism for defining mixins too. Benefits: developers can apply their knowledge about how to create models to create mixins too. Tools like yo loopback:model and Studio will support mixin definitions OOTB.

// common/models/timestamp-mixin.json
{
  "name": "TimeStampMixin",
  "properties": {
    "timestamp": "Date"
  }
}

// common/models/timestamp-mixin.js
module.exports = function(TimeStampMixin) {
  // setup auto-update of timestamp on save
}

To support this kind of mixins, we need to extend addAllBaseModels() and sortByInheritance() to take into account mixed-in models in addition to base models. Once that is done, model-like mixins will work both on the server and in the browserified client.

Note that this is can be implemented independently (and should be).

Mixin as a function

This is the part where we need to come up with a new convention.

Ultimately, I'd like to see a solution similar to what we have for models, where the compiler understands relations between model definitions and mixin definitions, and can build optimised instructions to load only those mixins that are actually used in the app.

Another thing to consider is the new component architecture as outlined here.

I am trying to come up with a small solution that can be incrementally extended in next PRs.

How about this:

• compiler takes two new options: mixinScripts that contains a list of source files to load, normalizeNames that defines the algorithm for converting file names into mixin/model names.
• mixin names are built from file names by the compiler, using normalizeNames
• executor is extended to define mixins per instructions from the compiler

Usage:

boot(app, {
  appRootDir: __dirname,
  mixins: [ './mixins/timestamp.js' ]
});

The next step is to implement clever auto-loading of mixins from directories, as described earlier. That way the usage would become

// explicit
boot(app, {
  appRootDir: __dirname,
  mixinSources: [ './mixins' ]
});

// relying on the default mixinSources value
boot(app, __dirname)

---

To summarise the next steps as I see them now:

1. Support mixins defined as models
2. Support mixins defined as functions via mixins option
3. Support mixins defined as functions via mixinSources option

---

Random notes:

• It seems to me that the setup is not handled correctly by the current mixing architecture. Imagine a TimeStampMixin as outlined above that adds a setup method to configure hooks to update the timestamp. Such setup method would override any setup method provided by the target class. This is most likely a problem of Model hooks in general as the same applies to beforeSave.
• How can a Model that is being mixed in access the mixin options provided by the app? It seems to me these options are discarded when the mixin is a Model instead of a function.

This issue is a follow-up for the unfinished pull request #33, the discussion there contains useful bits of information too.

[fabien]

@bajtos @raymondfeng please take care to support the following function signature for mixins:

module.exports = function(Model, options) { }

Otherwise, I am fine with mixinScripts and mixinSources. Too bad my previous work never got merged in - it has served me well all this time, including the two referenced issues above. It all just worked ...

In the meantime I have developed many (+30) useful mixins, full fledged & configureable, which I was hoping to share with the community. But given how this thing got dragged out all these months - overcomplicating things - I wasn't very inclined to put more effort in that, and focus on my own work instead. My original PR merge was long overdue ...

Reading the above, you don't seem to get what a proper mixin would be: a simple function that would take the model and some options. Your random notes aren't even valid concerns when you strictly stick with mixins functions.

Instead, the focus has been too much on the 'mixin as a model' idea, which quite frankly, is worthless IMO.

[bajtos]

@fabien I am sorry for your resentment. Let me provide some more information to explain my position.

We went quite a long way to modify loopback in such way that will allow us to build IDE-like tools - see generator-loopback and strong-arc. To do that, we need the users to specify as much metadata as possible in a declarative way (i.e. via JSON files). The proposed approach "mixin as a function" does not support this use case, as the tooling would have to parse and understand possibly arbitrary javascript code.

Now I am well aware of the benefits of defining models and mixins code-first and always try to provide good support for developers preferring code over configuration. See e.g. my work on lib/registry.js which exposes loopback-boot internals via a public API.

It would be great if we could come up with a solution that would allow developers to define mixins as functions and provide enough metadata for our tooling at the same time. An example of such metadata is a list of properties added by a mixin.

What I would not like is to end up with an ecosystem of mixins that don't work with the standard loopback tooling.

[fabien]

@bajtos thanks for elaborating, it explains your reasoning much better than before. It is quite a lofty goal, which is really a polished-up solution for your (company's) tooling. I believe the benefits of having this working in a more basic manner right now, outweighs waiting for something that you're trying to achieve on the long-term, tailored to your proprietary IDE toolchain ...

There not as much resentment, as there is disappointment that people haven't really grasped the full potential of this simple addition yet. And those that have, cannot really use it at the moment. Moving on, I just went ahead with what I had implemented, and I can tell you that it's one of the most important additions to Juggler/LB in general.

(Here's just a small list of some of the ones I have in use, > 3000 LOC: alias.js, branches.js, cascade-delete.js, cloning.js, collection.js, conflicts.js, entry.js, flags.js, graph.js, grouping.js, hierarchy.js, import.js, includes.js, mongo-escape.js, object-id.js, ownership.js, protect.js, query.js, references.js, reorder.js, revisions.js, scoping.js, time-stamps.js, transform.js, tree.js, urlify.js, uuid.js, versions.js, virtuals.js, ...)

I disagree with the fact that a mixin should be declarative in itself, to the point where tooling should be able to parse it - it's java/xml config-hell all over again, if taken to an extreme. And it severely limits what a mixin can do. In fact, it was introduced be able to do almost anything that you cannot already do declaratively.

It's sole purpose is being dynamic, which means that added properties and relationships usually have dynamic names too - these are not known upfront, and the whole mixin can operate differently, depending on the options passed to it, or the Model operated on.

The elegance and simplicity of a simple function cannot be underestimated, and I think instead of trying to aim for a declarative way, we should treat mixins as other applications would treat 'plugins'. A black box with a known API, a good description of what it does and possibly other metadata to inform the developer of it's use and options.

I have mixins that setup properties, hooks, relationships, remote methods, and most are a combination of all of these together. This is what makes them so powerful.

In other words, let's try to develop a standard to declare this metadata alongside the function definition:

// mixins/my-mixin.js

module.exports = { 
  name: 'MyMixin', 
  description: 'This plugin provides ...',
  options: {
    key1: 'description of key', key2: 'description of key' // option metadata declaration
  } 
};

module.exports.mixin = function(Model, options) {
   // augment Model here, based on options
}

If we can do that, we're mostly there. Wouldn't you agree?

[bajtos]

In other words, let's try to develop a standard to declare this metadata alongside the function definition:

For the tooling, it's best to have the metadata in a JSON file, so that the tools don't have to execute arbitrary code inside their process.

Extending your example, we can have two files:

// mixins/my-mixin.json
{ 
  "name": "MyMixin",
  "description": "This plugin provides ...",
  "options": {
    "key1": "description of key", 
    "key2": "description of key" // option metadata declaration
  } 
}

// mixins/my-mixin.js
module.exports.mixin = function(Model, options) {
   // augment Model here, based on options
}

I have an idea. If most mixins don't need any extra metadata, then we can make mixins/my-mixin.json an optional thing. It means loopback-boot must be able to infer mixin name from the filename, support .js files without .json metadata, and it all depends on the assumption that there are no dependencies between mixins that would have to be handled by loopback-boot.

That should satisfy your requirements (code-first approach, very little or no hand-written metadata) as well as requirements from our tooling.

Thoughts?

[fabien]

Hi @bajtos sorry for the late reply - yes, I would be pleased with this approach. It's a great idea to follow the .js/.json metadata approach as we do with models already. Do you want me to look into it and implement this?

[bajtos]

Do you want me to look into it and implement this?

Yes please, that would be great.

[clarkbw]

// mixins/my-mixin.json
{ 
  "module": "my-npm-mixin-module",
  "options": {
    "key1": "description of key", 
    "key2": "description of key"
  } 
}

Not sure if this is still progressing but I wanted to add a request for using npm modules for mixins. It would be nice if there was an attribute in the json file that could require a module. This would allow community developers to publish mixin modules on npm instead of copying the JS files around. I imagine the instructions would tell people to copy the JSON file into their mixins directory to load the mixin and then use the model config per model.

I created a prototype of the classic TimeStamp mixin to demonstrate how this could work, though I haven't mentioned the mixins directory yet.

[bajtos]

Using npm modules for mixins. It would be nice if there was an attribute in the json file that could require a module. This would allow community developers to publish mixin modules on npm instead of copying the JS files around.

Yes, the implementation must support loading mixins from node_modules.

[robotiko]

Hi fabien,
I just read about those mixins that you mentioned.
I was wondering if they are in a public repo to have a look "for inspiration" while the pr comes to place.
Any stats/ timeseries mixins? sounds really interesting.

Thxs

David

[fabien]

@bajtos I rebased/resolved any conflicts here: https://github.com/fabien/loopback-boot/tree/feature/mixin-loading-update

Next up would be having the two files, an optional mixin.json and a mandatory mixin.js file per Mixin.

@robotiko I'd have to extract the code to be more isolated first - hopefully soon, though

[fabien]

Added loading metadata from *.json files. @bajtos let me know what you think - perhaps we can standardize some of the metadata keys (I've followed npm's package.json as a guideline).

Regarding loading mixins from node_modules - yes, this should be possible, however, I think it should be possible to load multiple mixins from a module, as well as having just a single mixin wrapped as a module. Note that it's now easy to load mixins, by setting mixinSources.