Unhandled exception when shutting down database process

[gsalgado]

What is wrong?

Occasionally get an unhandled exception when I Ctrl+C trinity on the terminal:

^C    INFO  06-04 14:58:59        main  Keyboard Interrupt: Stopping                                                                                                            
    INFO  06-04 14:58:59         ipc  Sent SIGINT to process 27652, waiting 5 seconds for it to terminate                                                                       
Process ForkProcess-1:                                                                                                                                                          
Traceback (most recent call last):                                                                                                                                              
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap                                                                                                 
    self.run()                                                                                                                                                                  
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run                                                                                                         
    self._target(*self._args, **self._kwargs)                                                                                                                                   
  File "/home/salgado/src/py-evm/trinity/utils/profiling.py", line 27, in inner                                                                                                 
    return fn(*args, **kwargs)                                                                                                                                                  
  File "/home/salgado/src/py-evm/trinity/utils/logging.py", line 115, in inner
    return fn(*args, **inner_kwargs)
  File "/home/salgado/src/py-evm/trinity/main.py", line 195, in run_database_process
    serve_chaindb(chain_config, base_db)
  File "/home/salgado/src/py-evm/trinity/chains/__init__.py", line 187, in serve_chaindb
    server.serve_forever()  # type: ignore
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 167, in serve_forever
    sys.exit(0)
SystemExit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 261, in _bootstrap
    util._exit_function()
  File "/usr/lib/python3.6/multiprocessing/util.py", line 322, in _exit_function
    _run_finalizers()
  File "/usr/lib/python3.6/multiprocessing/util.py", line 262, in _run_finalizers
    finalizer()
  File "/usr/lib/python3.6/multiprocessing/util.py", line 186, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 191, in _finalize_join
    thread.join()
  File "/usr/lib/python3.6/threading.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

Again, it looks like the kind of thing that would be caused by the multiprocessing module:

If the SIGINT signal generated by Ctrl-C arrives while the main thread is blocked by a call to BoundedSemaphore.acquire(),
Lock.acquire(), RLock.acquire(), Semaphore.acquire(), Condition.acquire() or Condition.wait() then the call will be immediately
interrupted and KeyboardInterrupt will be raised.

[carver]

Related to #863

[gsalgado]

Looks like this has been fixed in #1174

[gsalgado]

Although this seemed to have been fixed before #1310, ISTM that was only because we were sleeping for a long time waiting for the never-ending subtask. I'm seeing this again; will investigate

[gsalgado]

Fixed in #1309