Deferred Service Builder

[gnunicorn]

Right now, the ServiceBuilder creates the client, backend and all else right at the beginning and executes the configured functions directly. This has some implicatations, we do not want:

1. you can out the ServiceBuilder into an unexpected/unrecommended State by calling a service method multiple times with different callbacks. As these callbacks typically have side-effects, this can lead to inconsistent and unreliable cases–even to race conditions if, for example, an authoring mechanism was initiated.
2. some of these functions actually conflict and shouldn''t both be called. However right now, we have no way of preventing that.
3. It is up to the user to call the functions in the right order, which is unreliable and an unnecessary foodgun – especially if we had to change that order.
4. We can't provide sane defaults – like the longest chain selector or the light client transaction pool, both are the same for all known usages of our node-template – without requiring to the do 1 and cause unexpected side effects.
5. Certain aspects, like the transaction pool or externality extensions, are actually created during client creation and thus have a hacky API so we can change them after the fact, see Allow node to add externalities extensions #4583.

Instead the pattern should only keep the references to the fns and execute all in one go on .build only. This would allow us to provide defaults, enforce the order in which set up takes place and would ensure any fn is only called once.

Further more this might allow us to clean up the api, so you can have one global builder and then call build_light and build_full on them rather than having to specify two builder...

[tomaka]

The original idea behind the ServiceBuilder is that the callbacks would be completely removed.
It doesn't make sense to me that the user has to call a function and pass a closure, only for this closure to be immediately called by that function and that's it.

After all, the parameters that the builder passes to the stack are very arbitrary. If for instance you want to use an implementation of a TxPool that requires a reference to the network, you would need to modify the ServiceBuilder to pass the NetworkService to the closure that creates the TxPool.

I'm not sure how feasible that is, but my original idea was to create everything on the stack, and then pass that to the ServiceBuilder as values. Without having any closure/callback involved.

[cecton]

This is mine now

[cecton]

I got an idea. Let me know if that sounds good:

• I realized that what the ServiceBuilder will do after the change is: a. create a struct with whatever you pass to it; b. provide functions to add all the builders; c. and finally allow you to build() the service or client and the queue (which is the relevant part).
• The idea of @tomaka to remove all the builders makes perfect sense to me.
• @gnunicorn told me we should actually drop the concept of client.

This makes me conclude that I can probably drop the entire ServiceBuilder and replace it by a few functions (build_service, build_for_import, build_client). This will also simplify greatly the typing.

[cecton]

So I have to drop it for now to focus on cumulus... I did some progress as I found out that the builder itself don't actually help much and can be entirely removed instead of making it defer the calls to the with_* methods.

This is actually more flexible AND it doesn't reduce at all the readability nor the simplicity in the service.rs files that were using it.

[gnunicorn]

I suppose the only way to actually reduce the code for it would be by using something like derive_builder.

[tomaka]

To give my two cents: I don't even know why there is a Service struct at all.

The Service as it exists is a complete pot-pourri of multiple design attempts.

As far as I know, it was initially supposed to be the container that contains all the sub-components, and you would create it in a kind-of-declarative way with a macro.

During the six months or so when I worked on it, to me it was supposed to coordinate the various components of Substrate (#2536), but that is obviously not the case, as these components "coordinate each other" in a spaghetti-ish way at the moment (which also contributes to all the issues with unbounded channels that we have, but that's off-topic). Actually reworking that without rewriting Substrate is too big of a task at this point.

Now, almost the only thing that sc-service does is plug the Legos together at initialization. Once initialization is done, the only reason why the service exists at all is to pass to the informant and to detect whether essential tasks have crashed (both of which can be very easily implemented differently).