Customizing fpm with plugins

[awvwgk]

For commands like fpm doc or fpm fmt the available pool of tools is quite large and heterogeneous, if we want to support such functionality it must allow some kind of customization

A non-comprehensive list of tools for fpm doc:

• doxygen
  • requires Doxyfile as input
• ford
  • markdown file with yaml header as input, can have any name
• ufpp
  • actually a preprocessor which can also extract documentation

For fpm fmt:

• lfortran fmt
  • actually an LLVM compiler + formatting
• findent
  • can only take stdin and only writes to stdout
• fprettify
  • can recursively run over the source tree

All of those tools come with specific requirements and limitations. Also, I doubt there would be a perfect choice we could all agree on for those functionalities. One possibility is to collect all common tools and add native support for them in fpm, which might be cumbersome and if one has a personal set of tools for this task, they cannot use it with fpm in an automated way.

My proposal it to allow defining those functions as “plugins” instead:

[plugin]
[plugin.fmt]  # Adds `fmt` command to fpm
program = "lfortran"
# Special variables like @INPUT@ and @OUTPUT@ are replaced by the filenames
command = ["fmt", "--no-color", "@INPUT@"]
# Standard input and standard output can be captured if necessary 
output.capture = true
# Probably needs a better way to define the interaction with the source...
scope = ["src/", "app/"]

Those could be defined locally in a fpm.toml, globally in a .fpmrc.toml or fetched like any other package. Note that installing the actual tool would be in many cases beyond the scope of fpm.

[LKedward]

I like this idea of plugins to keep things modular but extensible. This will allow the fpm codebase to remain focused and robust. Regarding plugins, it would be good if they could be fetched like other packages to simplify the workflow.

[certik]

Copying my text from #313, which I am closing in favor of this issue:

This was first mentioned here:

#309 (reply in thread)

The idea is that people are already writing many 3rd party tools for everything from formatting, automatic documentation lookups, pre-processors, etc. Many of these tools require to understand the structure of the project, which fpm knows and can provide to them, so that lfortran fmt can format your whole project, it would get the list of files from fpm.

It could go the other way too, so that fman can register a new command fpm man, which would look up things, or lfortran can register fpm fmt. Similar to how git can be extended by simply providing a tool git-something in a $PATH and git something becomes available.

The way various Jupyter kernels register with jupyter is that they simply create a file in $PREFIX/share/jupyter/kernels/fortran/kernel.json, which creates a fortran kernel. So in the same way, fman can install a file $PREFIX/share/fpm/plugins/fman/plugin.json which has information where to find fman and anything else needed for fpm to be able to call it. The way this works with Jupyter is that you just install lfortran (say using conda install lfortran) and it will automatically appear in Jupyter as an optional kernel, users do not have to do anything, it just works.

Let's brainstorm this more.