Close-transaction not in the mem pool, funds locked in, channel in "waiting close" forever

[Smaart020]

Background

I have a Node running Umbrel.
About 1/2 months ago I tried closing a Lightning channel.
Ever since that time, it's been stuck in Waiting close state.
When looking for info in RTL it does not show an Transaction ID.

Your environment

Running the most updated version of Umbrel 0.3.14, not sure about the LND version. Do not know how to figure this out.

When running lncli pendingchannels this returns:
{
"total_limbo_balance": "0",
"pending_open_channels": [
],
"pending_closing_channels": [
],
"pending_force_closing_channels": [
],
"waiting_close_channels": [
{
"channel": {
"remote_node_pub": "03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053",
"channel_point": "21ce5b2d1259ce91405d5a9531e8fb5ac1157a3d078e4e1834404fea911d6eed:1",
"capacity": "500000",
"local_balance": "0",
"remote_balance": "0",
"local_chan_reserve_sat": "0",
"remote_chan_reserve_sat": "0",
"initiator": "INITIATOR_LOCAL",
"commitment_type": "STATIC_REMOTE_KEY"
},
"limbo_balance": "0",
"commitments": {
"local_txid": "",
"remote_txid": "",
"remote_pending_txid": "",
"local_commit_fee_sat": "0",
"remote_commit_fee_sat": "0",
"remote_pending_commit_fee_sat": "0"
}
}
]
}

Expected behaviour

I expected the transaction to end in the mem pool and the channel to be closed.

Actual behaviour

Tell us what happens instead

• the closing transaction is not being processed.

Things I have tried:

• a full reinstall and recovery from seed, and channel backup

• Was advised to add: minrelaytxfee=0.000001 to bitcoin.conf, and restart my node.

• Using Chantools I got stuck at this step:
  umbrel@umbrel:~ $ docker exec lnd lncli create Input wallet password: [lncli] inappropriate ioctl for device

• Tried this force close:
  umbrel@umbrel:~ $ docker exec lnd lncli closechannel --force 21ce5b2d1259ce91405d5a9531e8fb5ac1157a3d078e4e1834404fea911d6eed 1 [lncli] rpc error: code = Unknown desc = cannot close channel with state: ChanStatusLocalDataLoss|ChanStatusRestored

So far I am stuck with this channel (500000sats) that is waiting to be closed. Any help/tips is appriciated

[guggero]

Instead of reinstalling and recovering from seed you should have tried the force close first. Now that you have used the channel backup you are reliant on the remote peer for the recovery. Try connecting to the node manually with lncli connect 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@188.166.81.82:9735 and check what lncli pendingchannels states.

[Smaart020]

Thank you Guggero! I tried connecting, this is what is being returned:

umbrel@umbrel:~ $ docker exec lnd lncli connect 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@188.166.81.82:9735
[lncli] rpc error: code = Unknown desc = already connected to peer: 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@188.166.81.82:9735

Whenever I try the lncli pendingchannels, it returns the same as I posted above.

[Roasbeef]

Running the most updated version of Umbrel 0.3.14, not sure about the LND version. Do not know how to figure this out.

If you run lncli getinfo (assuming things were compiled "correctly") you should see the lnd version there.

[guggero]

Can you post the latest logs, filtered by 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053 and 21ce5b2d1259ce91405d5a9531e8fb5ac1157a3d078e4e1834404fea911d6eed? The peer should force close the channel from their end, not sure why it doesn't.

[Smaart020]

@Roasbeef ,
Here's my LND Version info:
"version": "0.12.1-beta commit=v0.12.1-beta"

@guggero , Can you point me to the specific log file I should check? I found multiple log files?

[Roasbeef]

Logs get rotated, check the latest one, and if nothing is found there, check the prior ones (assuming you've kept them around).

[Smaart020]

Just checked the logs, this is is being returned on the items @guggero suggested to filter by:

2021-08-16 08:44:31.287 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): requesting new chans from height=0 and 696040 blocks after
2021-08-16 08:44:31.966 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=8000
2021-08-16 08:44:32.084 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=7999
2021-08-16 08:44:32.205 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=7997
2021-08-16 08:44:32.368 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=7999
2021-08-16 08:44:32.509 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=7999
2021-08-16 08:44:32.702 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=7956
2021-08-16 08:44:32.816 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=8000
2021-08-16 08:44:33.023 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=8000
2021-08-16 08:44:33.026 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): buffering chan range reply of size=676
2021-08-16 08:44:33.026 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): filtering through 64626 chans
2021-08-16 08:44:33.178 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): starting query for 338 new chans
2021-08-16 08:44:33.178 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): querying for 338 new channels
2021-08-16 08:44:34.614 [INF] DISC: GossipSyncer(03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053): no more chans to query

The log does not say anything about 21ce5b2d1259ce91405d5a9531e8fb5ac1157a3d078e4e1834404fea911d6eed

[guggero]

Thanks, unfortunately there isn't any useful info in that block. Can you set the debuglevel=debug in your config, restart lnd then grep again please??

[Smaart020]

I just added, debuglevel=debug at the start of the config file. restarted my node and now the whole logfile is empty whenever I check it with the command sudo nano lnd.log.

When I login to the Umbrel webinterface and requestt logfiles this is returned in the LND part:

Attaching to lnd
lnd | 2021-08-18 08:54:14.772 [DBG] BTCN: Attempting to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@188.166.81.82:9735 (reqid 4)
lnd | 2021-08-18 08:54:14.773 [INF] DISC: Attempting to bootstrap with: Authenticated Channel Graph
lnd | 2021-08-18 08:54:14.785 [DBG] BTCN: Attempting to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3)
lnd | 2021-08-18 08:54:14.838 [INF] DISC: Obtained 3 addrs to bootstrap network with
lnd | 2021-08-18 08:54:17.840 [DBG] SRVR: Waiting 2s before trying to locate bootstrap peers (attempt #1)
lnd | 2021-08-18 08:54:19.841 [INF] DISC: Attempting to bootstrap with: Authenticated Channel Graph
lnd | 2021-08-18 08:54:19.945 [INF] DISC: Obtained 3 addrs to bootstrap network with
lnd | 2021-08-18 08:54:22.946 [DBG] SRVR: Waiting 4s before trying to locate bootstrap peers (attempt #2)
lnd | 2021-08-18 08:54:26.946 [INF] DISC: Attempting to bootstrap with: BOLT-0010 DNS Seed: [[nodes.lightning.directory soa.nodes.lightning.directory] [lseed.bitcoinstats.com ]]
lnd | 2021-08-18 08:54:39.266 [DBG] BTCN: Failed to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3): socks connect tcp 10.21.21.11:9050->10.21.21.11:57126: unknown error general SOCKS server failure
lnd | 2021-08-18 08:54:39.267 [DBG] BTCN: Retrying connection to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3) in 5s
lnd | 2021-08-18 08:54:44.267 [DBG] BTCN: Attempting to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3)
lnd | 2021-08-18 08:54:51.759 [DBG] BTCN: Failed to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3): socks connect tcp 10.21.21.11:9050->10.21.21.11:57126: unknown error general SOCKS server failure
lnd | 2021-08-18 08:54:51.759 [DBG] BTCN: Retrying connection to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3) in 10s
lnd | 2021-08-18 08:55:01.760 [DBG] BTCN: Attempting to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3)
lnd | 2021-08-18 08:55:13.330 [DBG] BTCN: Failed to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3): socks connect tcp 10.21.21.11:9050->10.21.21.11:57126: unknown error general SOCKS server failure
lnd | 2021-08-18 08:55:13.330 [DBG] BTCN: Retrying connection to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3) in 15s
lnd | 2021-08-18 08:55:19.793 [DBG] RPCS: [walletbalance] Total balance=0 BTC (confirmed=0 BTC, unconfirmed=0 BTC)
lnd | 2021-08-18 08:55:19.852 [DBG] RPCS: [listchannels] fetched 0 channels from DB
lnd | 2021-08-18 08:55:19.860 [DBG] RPCS: [pendingchannels]
lnd | 2021-08-18 08:55:19.920 [DBG] RPCS: [listchannels] fetched 0 channels from DB
lnd | 2021-08-18 08:55:20.002 [DBG] RPCS: [pendingchannels]
lnd | 2021-08-18 08:55:20.085 [DBG] RPCS: [ListPayments]
lnd | 2021-08-18 08:55:28.330 [DBG] BTCN: Attempting to connect to 03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053@10.21.21.11:57126 (reqid 3)
lnd | 2021-08-18 08:55:39.721 [DBG] RPCS: [walletbalance] Total balance=0 BTC (confirmed=0 BTC, unconfirmed=0 BTC)
lnd | 2021-08-18 08:55:39.873 [DBG] RPCS: [listchannels] fetched 0 channels from DB
lnd | 2021-08-18 08:55:39.884 [DBG] RPCS: [pendingchannels]
lnd | 2021-08-18 08:55:39.897 [DBG] RPCS: [listchannels] fetched 0 channels from DB
lnd | 2021-08-18 08:55:40.043 [DBG] RPCS: [ListPayments]
lnd | 2021-08-18 08:55:40.045 [DBG] RPCS: [pendingchannels]

[guggero]

Try the manual connection again to the 188.166.81.82 IP address. Does that still give you the "already connected" error?
What is your current output of lncli pendingchannels?

[Smaart020]

Try the manual connection again to the 188.166.81.82 IP address. Does that still give you the "already connected" error?

• Whenever I login to RTL and try to connect to the 188.166.81.82 IP adres is says already connected to peer.
• Should I try to open a new channel ( and should I re-add balance?)

What is your current output of lncli pendingchannels?
{
"total_limbo_balance": "0",
"pending_open_channels": [
],
"pending_closing_channels": [
],
"pending_force_closing_channels": [
],
"waiting_close_channels": [
{
"channel": {
"remote_node_pub": "03e691f81f08c56fa876cc4ef5c9e8b727bd682cf35605be25d48607a802526053",
"channel_point": "21ce5b2d1259ce91405d5a9531e8fb5ac1157a3d078e4e1834404fea911d6eed:1",
"capacity": "500000",
"local_balance": "0",
"remote_balance": "0",
"local_chan_reserve_sat": "0",
"remote_chan_reserve_sat": "0",
"initiator": "INITIATOR_LOCAL",
"commitment_type": "STATIC_REMOTE_KEY"
},
"limbo_balance": "0",
"commitments": {
"local_txid": "",
"remote_txid": "",
"remote_pending_txid": "",
"local_commit_fee_sat": "0",
"remote_commit_fee_sat": "0",
"remote_pending_commit_fee_sat": "0"
}
}
]
}

[guggero]

This is quite strange. Your peer should go ahead and force-close the channel from their end.
Can you grep your recent log by the channel point (21ce5b2d1259ce91405d5a9531e8fb5ac1157a3d078e4e1834404fea911d6eed) please, to see if there's a hint in there?