Use fire without editing code directly

[jtratner]

Currently you have to add something like this to the end of your file just to mess around with fire:

if __name__ == '__main__':
    fire.Fire()

and then run with

python path/to/code.py

But it'd be neat if instead you could just do

fire path/to/code.py

and that does equivalent of exec appending the ifmain statement to the end.

Not sure how to do it - I know kernprof does some magic to inject into builtins before exec'ing code. (or maybe you get a code object back from exit) - but this would make fire even better as a debugging tool as well.

[dbieber]

Cool idea.

Let's think through some of the behavior specifics.

• What happens if the file already has a main()? What if it's already using Fire or some other CLI library? Probably we don't want __name__ == '__main__' to be True when we run code.py, but inevitably if code.py has top-level code then it will be executed.
• Python supports 'python -m' for running a module. Do we want to support 'fire -m' as well?
• With regular Fire CLIs, we need to separate flags from the fire command with an isolated '--'. Here we could potentially allow the fire flags to preceed the filename. e.g. fire -i code.py or fire -h code.py command. Is this something we'd want?
• I have a design doc coming for updating tab completion; we'll have to think about how this would play alongside tab completion.

[jtratner]

Great point on executing ifmain by accident. This set me on to a neat idea - what if you could just pass in any fully qualified python module/function/class?

I snagged the AppEngine mapreduce's for_name to do the resolution and voila:

›› python fire/cli.py os.path basename a/b/c
c
›› python fire/cli.py os.path isfile a/b/c                                                                                                              
False
›› python fire/cli.py os.path exists a/b/c                                                                                                              
False
›› python fire/cli.py os.path exists fire/cli.py                                                                                                        
True

branch here with initial doodle - https://github.com/jtratner/python-fire/tree/run-on-any-fully-qualified-name

If we could get it to work so that --interactive ended up in the namespace of the module (rather than caller namespace) that'd make this incredibly useful.

I'll think about your other questions and get back to you later (have to get a deliverable together).

[jtratner]

›› python fire/cli.py requests.get https://www.google.com - headers items
["Date", "Wed, 22 Mar 2017 21:29:50 GMT"]
["Expires", "-1"]
["Cache-Control", "private, max-age=0"]
["Content-Type", "text/html; charset=ISO-8859-1"]
["P3P", "CP=\"This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info.\""]
["Content-Encoding", "gzip"]
["Server", "gws"]
["X-XSS-Protection", "1; mode=block"]
["X-Frame-Options", "SAMEORIGIN"]
[...]
["Transfer-Encoding", "chunked"]

[dbieber]

That's nice 👍 .

I think we'd want to support 'fire filepath.py' as well, as in your original idea.

If we could get it to work so that --interactive ended up in the namespace of the module (rather than caller namespace) that'd make this incredibly useful.

Interesting. So, this would behave as if 'fire.Fire()' were appended to the module you're running fire against, as in your original description, yes?

Haven't tried this fully yet, but I think something like this could work:

import fire
import imp
module = imp.load_source('module', '/path/to/file.py')
from module import *
del imp
fire.Fire(module)  # or fire.Fire()?

[dbieber]

Here's a working first pass for supporting file paths:

import fire

import imp
import sys
module = imp.load_source('module', sys.argv[1])
from module import *

if __name__ == '__main__':
  fire.Fire(module, command=' '.join(sys.argv[2:]))  # or fire.Fire()?

We'll want to replace 'command' with 'argv'.

[jtratner]

The issue of __main__ namespace getting mangled is pretty big, e.g., it makes pickling really annoying and sometimes causes weird issues with imports.

Just to clarify what I mean about namespaces:

fire -m pandas -- --interactive
IPython shell!
$ read_csv('whatever')
$ import pandas as pd
$ assert read_csv is pd.read_csv 
$ assert DataFrame.__module__ != '__main__'

Under the hood would be essentially the same as what you have, but using import_module instead.

import importlib
module = importlib.import_module(name)
fire.Fire(module)

Which would then cause all of the module's locals (or maybe __dict__ to be in namespace).

So then you could end up with something like:

fire -m numpy genfromtxt 'myfile.csv' --delimiter=',' - sum

[jtratner]

Python supports 'python -m' for running a module. Do we want to support 'fire -m' as well?

Could be a good way to simplify loading, but I'm not really sure what you'd gain there / what it would mean. (would that mean you run the module's main function?) Ideally I'd like fire to always import modules and classes with namespaces intact.

With regular Fire CLIs, we need to separate flags from the fire command with an isolated '--'. Here we could potentially allow the fire flags to precede the filename. e.g. fire -i code.py or fire -h code.py command. Is this something we'd want?

That would simplify the user experience a little - easy to understand "fire options come before specifying what to operate on".

I have a design doc coming for updating tab completion; we'll have to think about how this would play alongside tab completion.

Good question - seems like it should be able to work the same way. It would be cool to be able to "teach" fire how to handle nested completion, e.g. I say:

fire some_module <method1> <args> - <method 2> - <method 3>

and the completion script somehow dumps completions for methods1 2 and 3 ;)

[jtratner]

Interesting. So, this would behave as if 'fire.Fire()' were appended to the module you're running fire against, as in your original description, yes?

Agreed. Exactly how I'd expect it to work. (and we could either automagically determine what user meant on CLI or have different flags depending on what you are referencing)

[jtratner]

How do you feel about this as precedence for loading modules vs. loading classes / methods vs. loading files (pandas vs. pandas.read_csv vs. pandas.DataFrame)?

1. check for a module, if True pass to fire.Fire and stop
2. check if it's a class or method, if True pass to fire.Fire and stop
3. assume it's a file, if exists, try to load with load_source
4. error out

[dbieber]

... and the completion script somehow dumps completions for methods1 2 and 3 ;)

Yes, would love to get to that point. I put a draft of the tab completion doc at #32.

Need to think more about the precedence idea. On the one hand, it's nice if a tool like this 'just works'. On the other hand, it's good to conform to standards set by tools like python and ipython. If we do go the precedence route, I suspect that we'd want file paths to take precedence over module names, but that's a detail.

[nealmcb]

Fire shares similarities with hug: Embrace the APIs of the future, which also generates RESTful web interfaces. Hug supports both a -m module option and a -f file option for starting up a web server, and a command-line option via the __main__ trick. It's probably worthwhile digging in to the implications of how they did it all as you design this feature in fire, which certainly sounds like a great idea to me.

Exploring how it works with some existing python modules is probably a good idea. E.g. I was delighted to to see @dbieber's quick implementation work like this:

$ cd /usr/lib/python3.4 
$ dbieberfire random.py
Type:        module
String form: <module 'module' from 'random.py'>
File:        /usr/lib/python3.4/random.py
Line:        1
Docstring:   Random variable generators.
.... more documentation....

$ dbieberfire random.py Random
....yet more specific documentation...

$ dbieberfire random.py Random gauss 50 5
53.698054343119814

$ dbieberfire calendar.py month 1957 6
     June 1957
Mo Tu We Th Fr Sa Su
                1  2
 3  4  5  6  7  8  9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

But trying to use it for b85encode ran into the question of how to create a "bytes" input parameter

$ dbieberfire base64.py b85encode xyzzy
...
TypeError: memoryview: str object does not have the buffer interface

[nealmcb]

OK, I figured out that the join operation in @dbieber's version is what was breaking the use of arguments of type bytes, by turning them back into strings.

The attached python script (with a .txt prefix to keep github happy, but you should rename it fire) will load files (with -f option) and modules (with -m option), and leave bytes strings alone.

fire.txt

So now this works:

$ fire -m base64 b85encode "b'Hello world'" - decode                                                                 
NM&qnZy<MXa%^M
$ fire -m base64 b85decode "b'NM&qnZy<MXa%^M'" - decode                                                              
Hello world

[dbieber]

Glad you found the bytes workaround 👍 .

For sharing code like fire.txt, consider using a gist or pastebin in the future instead of attaching a text file.

Python question: is there a way to do the equivalent of "from X import *" inside a Python function? I know the naive approach gives a syntax error, since "import *" is only allowed at the module scope.

Design thought: I think we'll want a sensible default so that people don't have to specify -m or -f. I'm inclined to take the precedence approach (first check if it's a file path that exists. if not, check if it describes a Python object using the for_name approach that jtratner described.) Will have to think more carefully about ways in which this might violate the user's expectations though.

On the subject of Hug, one of my low-priority plans for Fire is to experiment with adding a --serve flag to Fire CLIs. Much like how Fire CLIs expose an object or a whole program as a CLI, a Fire Server would expose an object or a whole program over HTTP. I can open a separate issue if you want to discuss this further or are interested in exploring the idea / contributing.