Unable to upgrade from 0.4.2 to 0.5

[hikedev]

Background

Unable to upgrade from 0.4.2 to 0.5. (not resolved)
Unable to start 0.4.2 anymore. (resolved)

Your environment

• 0.4.2 Version commit=7cf5ebe2650b6798182e10be198c7ffc1f1d6e19
• Version v0.5-beta-rc2 commit=fb95858afce6d7129758e7677bcb1552c5be2c51
• Version v0.5-beta commit=3b2c807288b1b7f40d609533c1e96a510ac5fa6d
• lnd-linux-amd64
• bitcoind mainnet

Actual behavior

Update from 0.4.2 to Version v0.5-beta-rc2 commit=fb95858afce6d7129758e7677bcb1552c5be2c51 results in this error log.

2018-09-15 23:21:55.229 [INF] LTND: Version 0.5.0-beta commit=fb95858afce6d7129758e7677bcb1552c5be2c51
2018-09-15 23:21:55.229 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2018-09-15 23:21:55.229 [INF] CHDB: Checking for schema update: latest_version=6, db_version=0
2018-09-15 23:21:55.229 [INF] CHDB: Performing database schema migration
2018-09-15 23:21:55.229 [INF] CHDB: Applying migration #1
2018-09-15 23:21:55.229 [INF] CHDB: Populating new node update index bucket
2018-09-15 23:21:55.239 [INF] CHDB: Populating new edge update index bucket
2018-09-15 23:21:55.665 [INF] CHDB: Migration to node and edge update indexes complete!
2018-09-15 23:21:55.665 [INF] CHDB: Applying migration #2
2018-09-15 23:21:55.665 [INF] CHDB: Migrating invoice database to new time series format
2018-09-15 23:21:55.665 [INF] CHDB: Migration to invoice time series index complete!
2018-09-15 23:21:55.665 [INF] CHDB: Applying migration #3
2018-09-15 23:21:55.665 [INF] CHDB: Migrating invoice database to new outgoing payment format
2018-09-15 23:21:55.665 [INF] CHDB: Migration to outgoing payment invoices complete!
2018-09-15 23:21:55.665 [INF] CHDB: Applying migration #4
2018-09-15 23:21:55.804 [INF] CHDB: Migration of edge policies complete!
2018-09-15 23:21:55.804 [INF] CHDB: Applying migration #5
2018-09-15 23:21:55.804 [INF] CHDB: Migrating database to support payment statuses
2018-09-15 23:21:55.804 [INF] CHDB: Marking all known circuits with status InFlight
2018-09-15 23:21:55.804 [INF] CHDB: Marking all existing payments with status Completed
2018-09-15 23:21:55.804 [INF] CHDB: Migration of payment statuses complete!
2018-09-15 23:21:55.804 [INF] CHDB: Applying migration #6
2018-09-15 23:21:55.804 [INF] CHDB: Migrating database to properly prune edge update index
2018-09-15 23:21:55.807 [INF] CHDB: Unable to apply migration #6
2018-09-15 23:21:55.808 [ERR] LTND: unable to open channeldb: unable to remove existing edge update index: delete bucket: incompatible value
2018-09-15 23:21:55.808 [INF] LTND: Shutdown complete
unable to remove existing edge update index: delete bucket: incompatible value

Than I tried v0.5-beta commit=3b2c807288b1b7f40d609533c1e96a510ac5fa6d, no further error logs given.

2018-09-16 14:26:17.292 [INF] LTND: Version 0.5.0-beta commit=3b2c807288b1b7f40d609533c1e96a510ac5fa6d
2018-09-16 14:26:17.292 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2018-09-16 14:26:17.292 [INF] CHDB: Checking for schema update: latest_version=6, db_version=0
2018-09-16 14:26:17.292 [INF] CHDB: Performing database schema migration
2018-09-16 14:26:17.292 [INF] CHDB: Applying migration #1
2018-09-16 14:26:17.293 [INF] CHDB: Populating new node update index bucket
2018-09-16 14:26:17.298 [INF] CHDB: Populating new edge update index bucket
2018-09-16 14:26:17.298 [INF] CHDB: Unable to apply migration #1
2018-09-16 14:26:17.299 [ERR] LTND: unable to open channeldb: unable to update edge indexes: EOF
2018-09-16 14:26:17.300 [INF] LTND: Shutdown complete
unable to update edge indexes: EOF

If I want to return to 0.4.2, the software shuts down when I want to unlock the wallet. No further error log than this.

2018-09-16 14:45:47.738 [INF] LTND: Version 0.4.2-beta commit=7cf5ebe2650b6798182e10be198c7ffc1f1d6e19
2018-09-16 14:45:47.738 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2018-09-16 14:45:47.739 [INF] CHDB: Checking for schema update: latest_version=0, db_version=0
2018-09-16 14:45:47.760 [INF] RPCS: password RPC server listening on 127.0.0.1:10009
2018-09-16 14:45:47.760 [INF] RPCS: password gRPC proxy started at 127.0.0.1:8080
2018-09-16 14:45:47.760 [INF] LTND: Waiting for wallet encryption password. Use lncli create to create wallet, or lncli unlock to unlock already created wallet.

Additional

Some commands in my lnd.conf don't work anymore with 0.5

debughtlc=true will result in loadConfig: debug-htlc mode cannot be used on bitcoin mainnet

bitcoind.zmqpath will result in unknown command in config

bitcoind.zmqpubrawblock & bitcoind.zmqpubrawtx have to be different ports in 0.5, with 0.4.2 it worked when they have been on the same port in the bitcoin.conf.

0.5 did only start by removing those lines(bitcoind.zmqpubrawblock & bitcoind.zmqpubrawtx) from the lnd.conf and start with
--bitcoind.zmqpubrawblock=tcp://127.0.0.1:18501 --bitcoind.zmqpubrawtx=tcp://127.0.0.1:18502

[cfromknecht]

@hikedev it's not clear to me why you are unable to start on 0.4.2. The db applies all migrations in one db transaction, so erroring out should not have modified the db state. Did you revert the changes to zmq ports in your bitcoin.conf as well?

debughtlc=true will result in loadConfig: debug-htlc mode cannot be used on bitcoin mainnet

Yep debughtlc is for testing, just remove it from your conf

0.5 did only start by removing those lines(bitcoind.zmqpubrawblock & bitcoind.zmqpubrawtx) from the lnd.conf and start with

This was documented in the release notes, though it has now been made more prominent

[hikedev]

@cfromknecht I got 0.4.2 running again thanks there was some copy-paste error in the bitcoin.conf.

With 0.5.0 I still run into the same upgrade issues. Maybe it will be the best to close all channels, transfer the coins and start from scratch with 0.5.0.

Just strange for everyone else it seems the database schema migration works.

[cfromknecht]

@hikedev starting a fresh node is definitely an option, though if you're looking to keep your existing channels and nodeid then I'm happy to help get 0.5 stood up. I probably won't take too long to identify the cause and patch it. Would be nice to also get a fix in for other users reporting similar issues.

[hikedev]

@cfromknecht Ok, I still got my 0.4.2 node. Any specific thing I should do? I will be around at the #lnd IRC channel the next days.

[cfromknecht]

@hikedev sure thing, ping me on IRC, bitconner