Add magic command and QSharpCallable method.

[cgranade]

This PR adds support for calling into the new sparse simulator from Q# notebooks and Q# + Python applications.

This PR Provides Grounds for:

• quantum-docs-private: Added the SparseSimulator to the docs

Related (but CI-independent) PR(s):

• Q#RT: Enabled listing the sparse simulator in the command line help

[bettinaheim]

Could we also add a magic command simulate_fullstate that resolves to the quantum simulator? The simulate command for now can stay as it is, but it would be good to have the option to treat it as something that resolves to a default choice later on.

[cgranade]

I'd rather not, but instead would want to find the right way to add a keyword argument to .simulate itself for the simulator and remove all of the other method calls.

For using Q# notebooks directly, we can use --simulator as a parameter name to disambiguate the simulator choice from inputs to the Q# entry point itself, but for Python we need to do something different to disambiguate. For example, we could consider def simulate(self, op, __simulator=None, **kwargs) since the __ prefix is reserved in Q# and cannot be listed as an entry point input. Using that, the full-state simulator would be usable as RunFoo.simulate("full-state", x=3) and the sparse simulator would be usable as RunFoo.simulate("sparse", x=3). That would also allow custom third-party simulators to be added more easily without modifying existing APIs, reducing the challenges run into by https://github.com/qsharp-community/chp-sim. That would also make it easier to add a new config setting qsharp.config['simulator.default'] that users could use to work with default simulator preferences.

[bettinaheim]

I am open to how exactly the commands should look like. I believe the following points would be important:

• We should make room for having a command where the user doesn't explicitly specify the simulator but we instead make a default choice for them (mirroring how standalone executables also work).
• The command that is currently most widely used (%simulate) should resolve to our default choice of simulator.
• If possible it would be great to align all commands such that we don't need to do any changes to iqsharp to enable a new simulator; ideally, that would allow also third party simulators to be used with iqsharp, but it is already and improvement if all simulator in the Microsoft.Quantum.Simulators package are automatically available via iqsharp.

[cgranade]

• If possible it would be great to align all commands such that we don't need to do any changes to iqsharp to enable a new simulator; ideally, that would allow also third party simulators to be used with iqsharp, but it is already and improvement if all simulator in the Microsoft.Quantum.Simulators package are automatically available via iqsharp.

In earlier runtime versions, individual simulators needed much more customization than is currently needed, such that this is much more plausible than before. In particular, much of the sparse simulator work resulted in a significant reduction of code volume in IQ# by removing the override that the %simulate command used to need in order to provide visualizations for the full-state simulator.

That said, even with the previous approach of a new command for each simulator, third-party simulators such as https://github.com/qsharp-community/chp-sim were able to use IQ#'s functionality for dynamic loading of new magic commands to add new simulators. For example, using the chp-sim package, a user could run %package QSharpCommunity.Simulators.Chp, at which point IQ# automatically discovered the new simulator command, added it to documentation in the %lsmagic command, tab completion, and so forth. I want to make sure that as we consolidate simulator logic into a single command, and as we make it easier to enable third party simulators, we also maintain the current user experience, wherein dynamically loaded simulators are easily discoverable at runtime. Perhaps something like %simulate --list-available to list currently known simulators?

[bettinaheim]

Do you want to make a complete list for how all simulator related magic commands could look like? Also, I didn't fully realize that the functionality to discover available simulator was added. Given that, what is the reason for wanting to revise the current API further?

[cgranade]

Also, I didn't fully realize that the functionality to discover available simulator was added.

It is, but only through having a proliferation of distinct magic commands. Consolidating the API into a single magic command with an argument for different simulators is much more maintainable and I'd argue usable as well. Deferring past this PR for now, in any case.

[kuzminrobin]

@cgranade,
My Docs PR depends on this one (search for "xref-not-found" in my PR). Can this PR be merged any time soon?
Are there any other actions required (like publishing the API docs online) to resolve the "xref-not-found" in my PR?