Enhancement: allow for custom sort order of kernel specs

[rolweber]

Currently, kernel specs are sorted by their display name. The sorting logic is encapsulated in kernelselector.js and tricky to customize. Provide some way to allow for a custom sort order. Example use cases:

1. In the Data Science Experience, notebooks for Apache Spark services can choose from several programming languages and combine these with different Spark versions. We want the higher Spark versions listed before the lower ones.
   Disclaimer: I'm a developer working on the Data Sciene Experience.
2. ENH: Highlight some kernels in the UI while allowing access to more kernels #2514, originally a post on the Jupyter mailing list, asks for "favorite kernels" being listed first, with the option of hiding other kernels in a submenu. The list of favorite kernels is defined by the operator of the notebook server, not by the users.
   https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/jupyter/qhDpFTuxzgs

Implementation options could be to customize the sorting logic, or to add metadata to kernel specs. We observed that with notebook 5.0, the kernel specs are sometimes loaded before custom.js, so replacing the sorting logic at that point is a problem. Also, there are two different implementations of the sorting logic in kernelselector.js.
Adding an optional "sort key" attribute to kernel specs, which defaults to the display name, would allow hosting services to customize the sort order of the provided kernels. In cases where the notebook server is backend by a kernel gateway, the sort order would be under control of the kernel gateway operator.

[dsblank]

I like the idea of a "sort key" especially if that key would automatically give a sub-menu for all kernels that have the same sort key. For example, I have three kernels for Python that appear in different locations. I could imagine changing the sort key in the kernel spec file to give different arrangement of the menu. For example, change the sort key to be course names.

You could also consider making the sort key a delimited string indicating the menu structure (here, based on courses):

"Data Structures/Fall 2017/Python2"
"Data Structures/Fall 2017/Python3"

Or (here, based on project):

"Calysto/Scheme"
"Calysto/Python"
"Calysto/Processing"

[rolweber]

Yes, I was also thinking of a delimiter in there. But that's a second step beyond my proposal. The kernel list is also used in a drop-down list of a dialog, when loading a notebook with an unknown kernel spec. We don't have sub-menus there. I'd rather solve the sorting problem before addressing the sub-menu problem :-)

Edit: On second thought, it's not such a good idea to encode a menu structure in a sort key. The submenus will need a display name of their own, and the sort key is not the place to store it.

[takluyver]

I'm proposing some changes to how kernel discovery work which may affect this:
jupyter/jupyter_client#261

In particular, if we go forwards with that scheme, kernels will be namespaced by how they're found, e.g. spec/python3 or conda/myenv.

[rolweber]

@takluyver: Thanks for the pointer. AIUI, the namespace will apply to the identifier of the kernel spec. The sort order is currently based on the display name of kernel specs, not on the identifier. Unless the identifier is used as a default for missing display names, your changes will not impact the sort order. But surely somebody's going to propose a sub-menu for each kernel finder :-)

[takluyver]

Yep, it does work on the identifier rather than the display name. I just thought that as you were discussing namespacing kernels, you'd probably want to be aware of that. :-)

[rolweber]

We'd have to deal with identical sort keys. Display name and kernel spec name can serve as tie breakers.

What about kernel specs without a sort key? Should the display name be used directly as a sort key, or should all kernel specs without a sort key be sorted together, with the display name as a tie breaker among them? I guess the first approach makes more sense, because people might use sort keys to bring selected kernels to the front, or to bring others to the end of the list. To allow for both, kernel specs without a sort key must sort among those with a sort key.