Runtime: out of memory: cannot allocate 4194304-byte block (1491501056 in use) fatal error: out of memory

[Fishh40]

System information

CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz 3.30 GHz
RAM: 16 GB
System: 64-bit
Free space C-drive (Windows): ~5 GB
Free space D-drive (Execution client): ~275 GB
Free space E-drive (Consensus client): ~800 GB

Geth version: v1.10.23
OS & Version: Windows 10 Pro
Commit hash : no developer - only running ETH validator

Expected behaviour

After updating to Geth v1.10.23 I want to run the execution client. I first deleted the old v1.10.21 geth.exe files, and then reïnstalled the v1.10.23. Running this should work and sync to the existing database (that is on the D-drive) when using this command: "geth.exe --datadir "D:\Execution". This seems not to be the case...

Actual behaviour

So if we try to use the command you can see above, we get a fews blocks in, but within the minute we get the error: "runtime: out of memory: cannot allocate 4194304-byte block (1491501056 in use) fatal error: out of memory"

There is nothing running in the background except a timesync app, all the rest is closed. Execution node uses 5-6GB RAM of the 16 GB available. No other parameters go into abnormal zones.

Steps to reproduce the behaviour

The issue persists even after restarting device or execution node and running the command: "geth.exe --datadir "D:\Execution"

Backtrace

I discussed this with the Prysm Team, and here we got something strange out of the log. This makes us think that geth is making my machine as 32-bit and not 64-bit. It feels like you aren't accounting for windows machines correctly

WARN [09-07|11:17:55.010] Lowering memory allowance on 32bit arch available=16351 addressable=2048

Log from execution client (full log in attachment):
log.txt

C:\Users\TV_Storage>geth.exe --datadir "D:\Execution"
INFO [09-07|11:17:54.952] Starting Geth on Ethereum mainnet...
INFO [09-07|11:17:54.998] Bumping default cache on mainnet         provided=1024 updated=4096
INFO [09-07|11:17:55.005] Maximum peer count                       ETH=50 LES=0 total=50
WARN [09-07|11:17:55.010] Lowering memory allowance on 32bit arch  available=16351 addressable=2048
WARN [09-07|11:17:55.010] Sanitizing cache to Go's GC limits       provided=4096 updated=682
INFO [09-07|11:17:55.011] Set global gas cap                       cap=50,000,000
INFO [09-07|11:17:55.013] Allocated trie memory caches             clean=102.00MiB dirty=170.00MiB
INFO [09-07|11:17:55.013] Allocated cache and file handles         database=D:\Execution\geth\chaindata cache=340.00MiB handles=8192
INFO [09-07|11:17:56.405] Found legacy ancient chain path          location=D:\Execution\geth\chaindata\ancient
INFO [09-07|11:17:56.415] Opened ancient database                  database=D:\Execution\geth\chaindata\ancient readonly=false
INFO [09-07|11:17:56.415]
INFO [09-07|11:17:56.415] ---------------------------------------------------------------------------------------------------------------------------------------------------------
INFO [09-07|11:17:56.415] Chain ID:  1 (mainnet)
INFO [09-07|11:17:56.415] Consensus: Beacon (proof-of-stake), merging from Ethash (proof-of-work)
INFO [09-07|11:17:56.415]
INFO [09-07|11:17:56.415] Pre-Merge hard forks:
INFO [09-07|11:17:56.415]  - Homestead:                   1150000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/homestead.md)
INFO [09-07|11:17:56.415]  - DAO Fork:                    1920000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/dao-fork.md)
INFO [09-07|11:17:56.415]  - Tangerine Whistle (EIP 150): 2463000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/tangerine-whistle.md)
INFO [09-07|11:17:56.415]  - Spurious Dragon/1 (EIP 155): 2675000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md)
INFO [09-07|11:17:56.415]  - Spurious Dragon/2 (EIP 158): 2675000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md)
INFO [09-07|11:17:56.415]  - Byzantium:                   4370000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/byzantium.md)
INFO [09-07|11:17:56.415]  - Constantinople:              7280000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/constantinople.md)
INFO [09-07|11:17:56.415]  - Petersburg:                  7280000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/petersburg.md)
INFO [09-07|11:17:56.415]  - Istanbul:                    9069000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/istanbul.md)
INFO [09-07|11:17:56.415]  - Muir Glacier:                9200000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/muir-glacier.md)
INFO [09-07|11:17:56.415]  - Berlin:                      12244000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/berlin.md)
INFO [09-07|11:17:56.415]  - London:                      12965000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/london.md)
INFO [09-07|11:17:56.415]  - Arrow Glacier:               13773000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/arrow-glacier.md)
INFO [09-07|11:17:56.425]  - Gray Glacier:                15050000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/gray-glacier.md)
INFO [09-07|11:17:56.425]
INFO [09-07|11:17:56.425] Merge configured:
INFO [09-07|11:17:56.425]  - Hard-fork specification:    https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/paris.md
INFO [09-07|11:17:56.425]  - Network known to be merged: false
INFO [09-07|11:17:56.425]  - Total terminal difficulty:  58750000000000000000000
INFO [09-07|11:17:56.425]  - Merge netsplit block:       <nil>
INFO [09-07|11:17:56.425] ---------------------------------------------------------------------------------------------------------------------------------------------------------
INFO [09-07|11:17:56.425]
INFO [09-07|11:17:59.557] Disk storage enabled for ethash caches   dir=D:\Execution\geth\ethash count=3
INFO [09-07|11:17:59.558] Disk storage enabled for ethash DAGs     dir=C:\Users\TV_Storage\AppData\Local\Ethash count=2
INFO [09-07|11:17:59.562] Initialising Ethereum protocol           network=1 dbversion=8
INFO [09-07|11:17:59.566] Loaded most recent local header          number=15,471,793 hash=9f5093..668bea td=57,946,766,311,083,637,150,481 age=2d20h16m
INFO [09-07|11:17:59.566] Loaded most recent local full block      number=15,471,793 hash=9f5093..668bea td=57,946,766,311,083,637,150,481 age=2d20h16m
INFO [09-07|11:17:59.566] Loaded most recent local fast block      number=15,471,793 hash=9f5093..668bea td=57,946,766,311,083,637,150,481 age=2d20h16m
INFO [09-07|11:17:59.566] Loaded last fast-sync pivot marker       number=15,383,507
WARN [09-07|11:17:59.567] Head state missing, repairing            number=15,471,793 hash=9f5093..668bea snaproot=f7fe6e..265040
INFO [09-07|11:18:01.692] Loaded most recent local header          number=15,471,793 hash=9f5093..668bea td=57,946,766,311,083,637,150,481 age=2d20h16m
INFO [09-07|11:18:01.692] Loaded most recent local full block      number=15,470,315 hash=60968d..5232b8 td=57,928,235,318,341,406,366,306 age=3d1h55m
INFO [09-07|11:18:01.692] Loaded most recent local fast block      number=15,471,793 hash=9f5093..668bea td=57,946,766,311,083,637,150,481 age=2d20h16m
INFO [09-07|11:18:01.692] Loaded last fast-sync pivot marker       number=15,383,507
WARN [09-07|11:18:01.692] Enabling snapshot recovery               chainhead=15,470,315 diskbase=15,471,580
WARN [09-07|11:18:01.872] Snapshot is not continuous with chain    snaproot=552c67..4b46fc chainroot=5a0b18..0d3b16
INFO [09-07|11:18:01.872] Loaded local transaction journal         transactions=0 dropped=0
INFO [09-07|11:18:01.872] Regenerated local transaction journal    transactions=0 accounts=0
WARN [09-07|11:18:01.872] Switch sync mode from snap sync to full sync
WARN [09-07|11:18:01.872] Chain pre-merge, sync via PoW (ensure beacon client is ready)
INFO [09-07|11:18:01.872] Gasprice oracle is ignoring threshold set threshold=2
WARN [09-07|11:18:01.872] Old unclean shutdowns found              count=5
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T00:51:47+0200 age=10h26m14s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T00:55:26+0200 age=10h22m35s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T00:59:12+0200 age=10h18m49s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T10:22:41+0200 age=55m20s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T10:23:32+0200 age=54m29s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T10:24:59+0200 age=53m2s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T10:25:56+0200 age=52m5s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T10:33:52+0200 age=44m9s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T10:37:46+0200 age=40m15s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T10:40:22+0200 age=37m39s
WARN [09-07|11:18:01.872] Unclean shutdown detected                booted=2022-09-07T11:05:59+0200 age=12m2s
WARN [09-07|11:18:01.872] Engine API enabled                       protocol=eth
INFO [09-07|11:18:01.882] Starting peer-to-peer node               instance=Geth/v1.10.23-stable-d901d853/windows-386/go1.18.5
INFO [09-07|11:18:01.925] New local node record                    seq=1,660,991,818,534 id=570ec3877388dab1 ip=192.168.0.250 udp=30303 tcp=30303
INFO [09-07|11:18:01.927] Started P2P networking                   self=enode://f52a4ec4853bbf11c57b972d6dc3e223a5fc48afcb0a0043b25e8b12ab7ad1fe0d595b91ac4ebfcef7e9c965fac0efff8c7a3f4b26739988577b7910321b364d@192.168.0.250:30303
INFO [09-07|11:18:01.927] Mapped network port                      proto=tcp extport=30303 intport=30303 interface=NAT-PMP(192.168.100.1)
INFO [09-07|11:18:01.927] IPC endpoint opened                      url=\\.\pipe\geth.ipc
INFO [09-07|11:18:01.929] Loaded JWT secret file                   path=D:\Execution\geth\jwtsecret crc32=0xf4485ae9
INFO [09-07|11:18:01.932] WebSocket enabled                        url=ws://127.0.0.1:8551
INFO [09-07|11:18:01.932] HTTP server started                      endpoint=127.0.0.1:8551 auth=true prefix= cors=localhost vhosts=localhost
INFO [09-07|11:18:01.973] Mapped network port                      proto=udp extport=30303 intport=30303 interface=NAT-PMP(192.168.100.1)
INFO [09-07|11:18:11.935] Looking for peers                        peercount=0 tried=17 static=0
INFO [09-07|11:18:22.004] Looking for peers                        peercount=1 tried=29 static=0
INFO [09-07|11:18:30.277] Block synchronisation started
WARN [09-07|11:18:30.278] Synchronisation failed, retrying         err="peer is unknown or unhealthy"
INFO [09-07|11:18:32.417] Looking for peers                        peercount=1 tried=27 static=0
WARN [09-07|11:18:36.902] Merge is configured, but no beacon client seen. Please ensure you have one available before the transition arrives! eta=1w5d14h
INFO [09-07|11:18:42.921] Looking for peers                        peercount=1 tried=29 static=0
INFO [09-07|11:18:52.962] Looking for peers                        peercount=1 tried=33 static=0
INFO [09-07|11:19:03.070] Looking for peers                        peercount=2 tried=30 static=0
INFO [09-07|11:19:13.177] Looking for peers                        peercount=1 tried=30 static=0
INFO [09-07|11:19:23.264] Looking for peers                        peercount=1 tried=31 static=0
INFO [09-07|11:19:33.273] Looking for peers                        peercount=1 tried=36 static=0
INFO [09-07|11:19:43.396] Looking for peers                        peercount=2 tried=27 static=0
INFO [09-07|11:19:53.973] Looking for peers                        peercount=1 tried=26 static=0
INFO [09-07|11:20:03.977] Looking for peers                        peercount=1 tried=32 static=0
INFO [09-07|11:20:14.004] Looking for peers                        peercount=1 tried=35 static=0
INFO [09-07|11:20:24.029] Looking for peers                        peercount=2 tried=43 static=0
INFO [09-07|11:20:34.123] Looking for peers                        peercount=1 tried=33 static=0
INFO [09-07|11:20:44.261] Looking for peers                        peercount=1 tried=34 static=0
INFO [09-07|11:20:52.362] Downloader queue stats                   receiptTasks=0 blockTasks=190 itemSize=54.50B throttle=8192
INFO [09-07|11:20:52.509] Importing sidechain segment              start=15,471,743 end=15,471,806
INFO [09-07|11:20:54.319] Looking for peers                        peercount=3 tried=32 static=0
ERROR[09-07|11:20:55.086] Expired request does not exist           peer=f6c8601ba82e05ba1bb68ccfd440fc76a6058ccdbe2e824e6dbf7accd85beb5c
ERROR[09-07|11:20:55.086] Expired request does not exist           peer=f6c8601ba82e05ba1bb68ccfd440fc76a6058ccdbe2e824e6dbf7accd85beb5c
INFO [09-07|11:20:55.949] Extend chain                             add=1428 number=15,471,743 hash=128abe..0654a8
runtime: out of memory: cannot allocate 4194304-byte block (1495629824 in use)
fatal error: out of memory

[holiman]

crash

goroutine 3156 [running]:
runtime.throw({0x2168e9b, 0xd})
        runtime/panic.go:992 +0x64 fp=0x5278521c sp=0x52785208 pc=0x1019bc4
runtime.(*mcache).refill(0x4c402d8, 0x31)
        runtime/mcache.go:164 +0x1eb fp=0x52785240 sp=0x5278521c pc=0xff5a6b
runtime.(*mcache).nextFree(0x4c402d8, 0x31)
        runtime/malloc.go:886 +0x81 fp=0x52785260 sp=0x52785240 pc=0xfec751
runtime.mallocgc(0x1c0, 0x1fbc320, 0x1)
        runtime/malloc.go:1085 +0x3d8 fp=0x527852a8 sp=0x52785260 pc=0xfeccc8
runtime.makeslice(0x1fbc320, 0x1c0, 0x1c0)
        runtime/slice.go:103 +0x4f fp=0x527852bc sp=0x527852a8 pc=0x102ef3f
runtime.makeslice64(0x1fbc320, 0x1c0, 0x1c0)
        runtime/slice.go:117 +0x6f fp=0x527852d0 sp=0x527852bc pc=0x102f01f
github.com/ethereum/go-ethereum/rlp.(*Stream).Bytes(0x30e62000)
        github.com/ethereum/go-ethereum/rlp/decode.go:620 +0x89 fp=0x527852fc sp=0x527852d0 pc=0x1166319
github.com/ethereum/go-ethereum/rlp.decodeByteSlice(0x30e62000, {0x1f7a3e0, 0xbb0f50, 0x197})
        github.com/ethereum/go-ethereum/rlp/decode.go:336 +0x25 fp=0x52785330 sp=0x527852fc pc=0x1163435
github.com/ethereum/go-ethereum/rlp.makeStructDecoder.func1(0x30e62000, {0x206ce20, 0xbb0f30, 0x199})
        github.com/ethereum/go-ethereum/rlp/decode.go:394 +0x894 fp=0x52785400 sp=0x52785330 pc=0x1164e34
github.com/ethereum/go-ethereum/rlp.(*Stream).Decode(0x30e62000, {0x1ff14a0, 0xbb0f30})
        github.com/ethereum/go-ethereum/rlp/decode.go:886 +0x1c7 fp=0x52785474 sp=0x52785400 pc=0x11676e7
github.com/ethereum/go-ethereum/rlp.DecodeBytes({0xbd6500, 0x261, 0x261}, {0x1ff14a0, 0xbb0f30})
        github.com/ethereum/go-ethereum/rlp/decode.go:97 +0x143 fp=0x527854ac sp=0x52785474 pc=0x1160183
github.com/ethereum/go-ethereum/core/types.(*LogForStorage).DecodeRLP(0xbf5800, 0x18914960)
        github.com/ethereum/go-ethereum/core/types/log.go:120 +0x9e fp=0x527855e8 sp=0x527854ac pc=0x11f349e
github.com/ethereum/go-ethereum/rlp.decodeDecoder(0x18914960, {0x20d7d00, 0xbf5800, 0x199})
        github.com/ethereum/go-ethereum/rlp/decode.go:517 +0x8c fp=0x52785608 sp=0x527855e8 pc=0x116617c
github.com/ethereum/go-ethereum/rlp.makeSimplePtrDecoder.func1(0x18914960, {0x2022d00, 0xbd5ad8, 0x196})
        github.com/ethereum/go-ethereum/rlp/decode.go:440 +0x115 fp=0x52785640 sp=0x52785608 pc=0x1165485
github.com/ethereum/go-ethereum/rlp.decodeSliceElems(0x18914960, {0x1f6d8e0, 0xab72f4, 0x197}, 0x185620b0)
        github.com/ethereum/go-ethereum/rlp/decode.go:304 +0x248 fp=0x527856b4 sp=0x52785640 pc=0x1162698
github.com/ethereum/go-ethereum/rlp.decodeListSlice(0x18914960, {0x1f6d8e0, 0xab72f4, 0x197}, 0x185620b0)
        github.com/ethereum/go-ethereum/rlp/decode.go:281 +0x18a fp=0x527856ec sp=0x527856b4 pc=0x1161fda
github.com/ethereum/go-ethereum/rlp.makeListDecoder.func3(0x18914960, {0x1f6d8e0, 0xab72f4, 0x197})
        github.com/ethereum/go-ethereum/rlp/decode.go:266 +0x42 fp=0x5278570c sp=0x527856ec pc=0x1161d72
github.com/ethereum/go-ethereum/rlp.makeStructDecoder.func1(0x18914960, {0x206cea0, 0xab72e0, 0x199})
        github.com/ethereum/go-ethereum/rlp/decode.go:394 +0x894 fp=0x527857dc sp=0x5278570c pc=0x1164e34
github.com/ethereum/go-ethereum/rlp.(*Stream).Decode(0x18914960, {0x1f4fbc0, 0xab72e0})
        github.com/ethereum/go-ethereum/rlp/decode.go:886 +0x1c7 fp=0x52785850 sp=0x527857dc pc=0x11676e7
github.com/ethereum/go-ethereum/rlp.DecodeBytes({0xa9b800, 0x7fa, 0x7fa}, {0x1f4fbc0, 0xab72e0})
        github.com/ethereum/go-ethereum/rlp/decode.go:97 +0x143 fp=0x52785888 sp=0x52785850 pc=0x1160183
github.com/ethereum/go-ethereum/core/types.decodeStoredReceiptRLP(0xbfe1a0, {0xa9b800, 0x7fa, 0x7fa})
        github.com/ethereum/go-ethereum/core/types/receipt.go:314 +0x6f fp=0x52785aa8 sp=0x52785888 pc=0x11f490f
github.com/ethereum/go-ethereum/core/types.(*ReceiptForStorage).DecodeRLP(0xbfe1a0, 0x1c260050)
        github.com/ethereum/go-ethereum/core/types/receipt.go:303 +0x5d fp=0x52785ad0 sp=0x52785aa8 pc=0x11f47ed
github.com/ethereum/go-ethereum/rlp.decodeDecoder(0x1c260050, {0x20ec120, 0xbfe1a0, 0x199})
        github.com/ethereum/go-ethereum/rlp/decode.go:517 +0x8c fp=0x52785af0 sp=0x52785ad0 pc=0x116617c
github.com/ethereum/go-ethereum/rlp.makeSimplePtrDecoder.func1(0x1c260050, {0x2022d60, 0x7f8fd3b4, 0x196})
        github.com/ethereum/go-ethereum/rlp/decode.go:440 +0x115 fp=0x52785b28 sp=0x52785af0 pc=0x1165485
github.com/ethereum/go-ethereum/rlp.decodeSliceElems(0x1c260050, {0x1f6d960, 0xbd5390, 0x197}, 0x4cac3b20)
        github.com/ethereum/go-ethereum/rlp/decode.go:304 +0x248 fp=0x52785b9c sp=0x52785b28 pc=0x1162698
github.com/ethereum/go-ethereum/rlp.decodeListSlice(0x1c260050, {0x1f6d960, 0xbd5390, 0x197}, 0x4cac3b20)
        github.com/ethereum/go-ethereum/rlp/decode.go:281 +0x18a fp=0x52785bd4 sp=0x52785b9c pc=0x1161fda
github.com/ethereum/go-ethereum/rlp.makeListDecoder.func3(0x1c260050, {0x1f6d960, 0xbd5390, 0x197})
        github.com/ethereum/go-ethereum/rlp/decode.go:266 +0x42 fp=0x52785bf4 sp=0x52785bd4 pc=0x1161d72
github.com/ethereum/go-ethereum/rlp.(*Stream).Decode(0x1c260050, {0x1f47c40, 0xbd5390})
        github.com/ethereum/go-ethereum/rlp/decode.go:886 +0x1c7 fp=0x52785c68 sp=0x52785bf4 pc=0x11676e7
github.com/ethereum/go-ethereum/rlp.DecodeBytes({0xbe2000, 0x11667, 0x12000}, {0x1f47c40, 0xbd5390})
        github.com/ethereum/go-ethereum/rlp/decode.go:97 +0x143 fp=0x52785ca0 sp=0x52785c68 pc=0x1160183
github.com/ethereum/go-ethereum/core/rawdb.ReadRawReceipts({0x383ba910, 0x15e80730}, {0x1b, 0x6a, 0x6d, 0xd3, 0xda, 0xcc, 0xe9, 0x39, ...}, ...)
        github.com/ethereum/go-ethereum/core/rawdb/accessors_chain.go:611 +0xc8 fp=0x52785d3c sp=0x52785ca0 pc=0x131d468
github.com/ethereum/go-ethereum/core/rawdb.ReadReceipts({0x383ba910, 0x15e80730}, {0x1b, 0x6a, 0x6d, 0xd3, 0xda, 0xcc, 0xe9, 0x39, ...}, ...)
        github.com/ethereum/go-ethereum/core/rawdb/accessors_chain.go:631 +0x60 fp=0x52785e14 sp=0x52785d3c pc=0x131d6b0
github.com/ethereum/go-ethereum/core.(*BlockChain).collectLogs(0x15ff2500, {0x1b, 0x6a, 0x6d, 0xd3, 0xda, 0xcc, 0xe9, 0x39, 0x5, ...}, ...)
        github.com/ethereum/go-ethereum/core/blockchain.go:1966 +0xbf fp=0x52785e9c sp=0x52785e14 pc=0x1550cbf
github.com/ethereum/go-ethereum/core.(*BlockChain).reorg(0x15ff2500, 0x3b2d9cc0, 0x2bdc98b0)
        github.com/ethereum/go-ethereum/core/blockchain.go:2135 +0x1a46 fp=0x527863c0 sp=0x52785e9c pc=0x15528d6
github.com/ethereum/go-ethereum/core.(*BlockChain).writeBlockAndSetHead(0x15ff2500, 0x2bdc98b0, {0x35af3000, 0x14a, 0x200}, {0x28f6b500, 0x228, 0x2a0}, 0x27bd2000, 0x0)
        github.com/ethereum/go-ethereum/core/blockchain.go:1357 +0x1b1 fp=0x527864a4 sp=0x527863c0 pc=0x1548921
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0x15ff2500, {0x3015f500, 0x40, 0x40}, 0x0, 0x1)
        github.com/ethereum/go-ethereum/core/blockchain.go:1712 +0x2d55 fp=0x52786d38 sp=0x527864a4 pc=0x154c905
github.com/ethereum/go-ethereum/core.(*BlockChain).insertSideChain(0x15ff2500, 0x2acf9770, 0x191338c8)
        github.com/ethereum/go-ethereum/core/blockchain.go:1908 +0x1c0f fp=0x527871d0 sp=0x52786d38 pc=0x15504df
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0x15ff2500, {0x1787fac8, 0x1, 0x1}, 0x1, 0x1)
        github.com/ethereum/go-ethereum/core/blockchain.go:1542 +0x1305 fp=0x52787a64 sp=0x527871d0 pc=0x154aeb5
github.com/ethereum/go-ethereum/core.(*BlockChain).InsertChain(0x15ff2500, {0x1787fac8, 0x1, 0x1})
        github.com/ethereum/go-ethereum/core/blockchain.go:1441 +0xb0f fp=0x52787c70 sp=0x52787a64 pc=0x1549a6f
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).importBlockResults(0x3015e100, {0x1787f998, 0x1, 0x1})
        github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1549 +0x406 fp=0x52787e6c sp=0x52787c70 pc=0x16662b6
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFullSyncContent(0x3015e100, 0x15c96300, 0x0)
        github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1516 +0x172 fp=0x52787fb0 sp=0x52787e6c pc=0x16658b2
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func9()
        github.com/ethereum/go-ethereum/eth/downloader/downloader.go:622 +0x2f fp=0x52787fc8 sp=0x52787fb0 pc=0x165d7ff
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1()
        github.com/ethereum/go-ethereum/eth/downloader/downloader.go:634 +0x6c fp=0x52787ff0 sp=0x52787fc8 pc=0x165ddcc
runtime.goexit()
        runtime/asm_386.s:1326 +0x1 fp=0x52787ff4 sp=0x52787ff0 pc=0x1049701
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
        github.com/ethereum/go-ethereum/eth/downloader/downloader.go:634 +0x5f

[holiman]

This happens here, and len(newChain) is on the order of 1400 blocks:

	for i := len(newChain) - 1; i >= 1; i-- {
		// Collect reborn logs due to chain reorg
		logs := bc.collectLogs(newChain[i].Hash(), false) // <-- this line
		if len(logs) > 0 {
			rebirthLogs = append(rebirthLogs, logs)
		}
	}

[holiman]

This case was somewhat improved by #24616, but apparently it can still be problematic

[holiman]

Hah: https://github.com/ethereum/go-ethereum/pull/24616/files#r839537977

I pointed that out already back then, but we didn't do anything about it

[MariusVanDerWijden]

I wonder if its really necessary to copy each log within collectLogs? L1966

[holiman]

Oh, here's another reason why this happened:

WARN [09-07|11:17:55.010] Lowering memory allowance on 32bit arch  available=16351 addressable=2048
WARN [09-07|11:17:55.010] Sanitizing cache to Go's GC limits       provided=4096 updated=682

It's a 32-bit system, with only 2Gb memory

[holiman]

RAM: 16 GB
System: 64-bit

Maybe you downloaded the wrong binary?

EDIT: after re-reading your issue description, I finally noticed that you had also seen and mentioned this thing. Note to self to read more properly from the beginning :)

[holiman]

Geth/v1.10.23-stable-d901d853/windows-386/go1.18.5

You have most definitely downloaded the wrong version.
The one you want is
https://gethstore.blob.core.windows.net/builds/geth-windows-amd64-1.10.23-d901d853.exe

But I think the one you have is:
https://gethstore.blob.core.windows.net/builds/geth-windows-386-1.10.23-d901d853.exe

[holiman]

Closing this. Feel free to reopen if the issue persists or anything is unclear

[Fishh40]

@holiman Maybe add a better error message for this? So it is clear this is the issue and can be fixed easily? Would make this more fool proof. ;)

But thanks anyway, this was the fix! I'll have to read better...