2020 Summer Internship Project: Lightning Support for Bitcoin Standup

[ChristopherA]

Several interns were interested in adding Lightning to Bitcoin Standup, and ultimately to FullyNoded 1 & FullyNoded 2. There was also interest by one intern in possibly some limited FullyNoded 1 like functionality to BitcoinStandup for Android, which could be a start toward a Fully Noded 2 for Android as well.

The biggest problem with this project is that it is too big and insufficient scoped to an internship project. In fact, it is so big if it was fully funded by our Patron's that I'd still have reservations about the large scope.

There are some sub-projects that might be good internship projects, but one of the first issues is which version of Lightning?

@shannona and I always planned to add C-Lighting to our Learning Bitcoin from the Command Line course. This version of Lightning we felt was well aligned with the goal of the course to teach how Bitcoin really worked, in ways that they could leverage people getting jobs in the industry. I still feel that C-Lightning is more the "industrial" and "cash register" version of Lightning for use in bigger projects. There are some advantages also that is is written in C++ and uses LibWally which are used by other Blockchain Commons projects.

However, I believe Lighting Labs lnd tends to be more of a leader in new approaches. In particular, there is something called LSATs that is of interest to a couple of Patron's of Blockchain Commons that could lead to real paid work from Blockchain Commons.

So I think the first step is to look at both of these implementations from the perspective of the command-line course, and from the perspective of advances like LSATs (for instance, maybe LSATs are on the C-Lightning roadmap), and post it to our Research repo.

It is also relatively trivial to fork one of the linux standup scripts to also install lightning. A challenge is that this require a full node (and likely tx-index). This is not inexpensive to host. So another project is to discover the cheapest way to host the bitcoin data on a VPS service separate from the VPS itself. For instance, on Linode a single VPS that can hold a full node is $160 a month, but that includes 32GB RAM and SSD which is overkill.

Once that is addressed in Bitcoin Standup linux scripts, then we can try out both Lightning implementations.

Other ideas? Again, we want to scope things such that the intern leading the project can complete something meaningful in 8-12 weeks.

-- Christopher Allen

[javiervargas]

Proposal

Research and document about Lightning Network's c-lightning and LND implementations taking into account the focus of the Learning Bitcoin (and Lightning) from the Command Line course and the focus of new developments. The research will delve into topics such as the ease of backups, the use of WatchTowers, API's for developers, the universe of related LAPPS and new ongoing developments such as LSAT in LND.

Investigate and document the best way to run a c-lightning node in terms of cost and storage since c-lightning requires a full node with txindex activated. C-lightning can work using a full remote node accessed by JSON-RPC. LND can work with Neutrino or a pruned node if the blocks are kept until March 2018 minimum. Document about the possibility of having both implementations on the same server using the same bitcoind server.

As this research phase has progressed we could detail more in the tasks.

J.

[jodobear]

It is also relatively trivial to fork one of the linux standup scripts to also install lightning. A challenge is that this require a full node (and likely tx-index).

BTCPay uses pruned node by default. It needs to be at least 3 months (~25GB) for lightning to work. I think we can very well incorporate functioning lightning with standup scripts and pruned node.

[watersnake1]

@javiervargas I think I'd be pretty helpful on this project as well. It looks like we'll need to do the following during the setup to add support for lightning:

Prompt user for their choice of C-lightning or LND
Check if the dependencies of the selected implementation are present on the system
Install any missing dependencies
Download lightning
Compile and install lightning

Stop Tor if it is running
Add HiddenServicePort entry in .torrc specifying lightning's port
restart Tor
create .conf file
Configure proxy at localhost:9050 in the lightning conf file (tor proxy)
start lightning
if the user has selected lnd, then they will be prompted by lnd to create and unlock a new wallet because lnd is being run for the first time.

Future

Going forward, I think the following additions would be helpful:

1. Generate QuickConnect QR codes for the lightning node. This may involve modifying the standards of QuickConnect
2. Give Standup an ncurses menu similar to the one used in Raspiblitz. Adding support to Lightning would introduce at least one choice to the user, and this interface might be more inviting / easier to understand to users.

I'd like to volunteer to lead this project.
-Christian

[ChristopherA]

Give Standup an ncurses menu

I'd like to see the 2nd version of a Lighting script also support MacOS directly (i.e. not use our MacOS app), and may work for older Macs) as it is very close to Linux. There is something called osascript on the Mac wish can display simple dialogues and simple selection.

osascript -e 'display dialog "Hello from bash!"'

-- Christopher Allen

[ChristopherA]

@Fonta1n3 What feedback do we already have from other parties using QuickConnect?

Do you have GitHub handles for our primary contacts for each of those people who implemented it on their platform? Can you /cc them on this issue so we can define additional QuickConect requirements?

-- Christopher Allen

[Fonta1n3]

@Fonta1n3 What feedback do we already have from other parties using QuickConnect?

Do you have GitHub handles for our primary contacts for each of those people who implemented it on their platform? Can you /cc them on this issue so we can define additional QuickConect requirements?

-- Christopher Allen

Everyone seems to be happy with it, “no news is good news”.

LND already has LNDConnect so I would suggest just using that for LND.

Afaik C-Lightning can work with QuickConnect as is. What additional params would be needed?

You can add as many optional params as you’d like to QuickConnect, if http auth is not required the client can simply omit them or they can be passed as empty strings. Its really just http auth with an onion url and optional params for different clients.

I do not have their Github handles unfortunately and tbh have minimal ongoing dialog as it just seems to get the job done.

[watersnake1]

Thanks for the input @Fonta1n3
Here's my initial list of tasks that will have to get done

Tasks

Initial

• Study BTC-StandUp Linux Scripts (including linode)
• Study BTC-StandUp MacOS Scripts
• Create lightning branch on standup

Development

• User client selection: user is presented with choice of using C-lightning or LND
• Script checks system and installs missing dependencies
• Script downloads, compiles and installs selected lightning client
• Script checks if Tor is running, and stops Tor if it is
• Script reconfigures .torrc to handle lightning's port and restarts
• If using LND, generate an LND QR
• If using C-lightning, generate a QuickConnect QR
• Start bitcoind and lightning

Future

• Install local price / historical data server for FN2 (dependant on completion of milestones of fn-server)

[javiervargas]

It is also relatively trivial to fork one of the linux standup scripts to also install lightning. A challenge is that this require a full node (and likely tx-index).

BTCPay uses pruned node by default. It needs to be at least 3 months (~25GB) for lightning to work. I think we can very well incorporate functioning lightning with standup scripts and pruned node.

Lnd team confirms that pruning is not fully supported by lnd. You need to make sure you prune only back to the first mainnet channel. When node syncing the graph you'll run with issues because it cannot find the blocks of all channels. Maybe you could try --asumechanvalid to skip the verification but it’s not recommended for safety reasons as well and it will cause lnd run slowly.

[ChristopherA]

you need to make sure you prune only back to the first mainnet channel.

BTW, this is one of the reasons why our emerging specifications for seeds and recovery all optionally support birthdate, so that you can know when to prune to.

-- Christopher Allen

[watersnake1]

Since @jodobear has already moved forward on implementing for btc standup lightning, I will not be doing the same according to the above list and instead focusing on the price server for now. Am still very interested in lending a hand on this project if it is needed!

[shannona]

I'm closing out Summer 2020 issues.

There has been great progress on this, and we've got Lightning and a few other things on the verge of release in our Bitcoin Standup Scripts. See Scripts PR 15. Thanks to @jodobear