Panic on empty *.arrows file

[krumpira]

Hi, thank you for this great product.
I had a situation today where Parseable v1.2.0 was running, but no new events were ingested, and it panicked every minute with this backtrace:

thread '<unnamed>' panicked at server/src/utils/arrow/reverse_reader.rs:133:13:
range start index 1 out of range for slice of length 0
stack backtrace:
   0:     0x56201fdf8156 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h9cca0343d66d16a8
   1:     0x56201fe25d90 - core::fmt::write::h4311bce0ee536615
   2:     0x56201fdf4bff - std::io::Write::write_fmt::h0685c51539d0a0cd
   3:     0x56201fdf7f34 - std::sys_common::backtrace::print::h2fb8f70628a241ed
   4:     0x56201fdf97b7 - std::panicking::default_hook::{{closure}}::h05093fe2e3ef454d
   5:     0x56201fdf9519 - std::panicking::default_hook::h5ac38aa38e0086d2
   6:     0x56201fdf9c48 - std::panicking::rust_panic_with_hook::hed79743dc8b4b969
   7:     0x56201fdf9b22 - std::panicking::begin_panic_handler::{{closure}}::ha437b5d58f431abf
   8:     0x56201fdf8656 - std::sys_common::backtrace::__rust_end_short_backtrace::hd98e82d5b39ec859
   9:     0x56201fdf9874 - rust_begin_unwind
  10:     0x56201d065685 - core::panicking::panic_fmt::hc69c4d258fe11477
  11:     0x56201d065c52 - core::slice::index::slice_start_index_len_fail::hdaca407d67a34612
  12:     0x56201d10c20c - parseable::utils::arrow::reverse_reader::get_reverse_reader::h67f5559b08d527a2
  13:     0x56201d8b2dec - parseable::utils::arrow::merged_reader::MergedReverseRecordReader::try_new::h312c19569974cb9c
  14:     0x56201d5a132d - parseable::storage::staging::convert_disk_files_to_parquet::ha6890a3dc1dd6d1c
  15:     0x56201d5183a8 - parseable::storage::object_storage::ObjectStorage::sync::{{closure}}::h23130b858cc033f4
  16:     0x56201d2d62c7 - parseable::sync::object_store_sync::{{closure}}::{{closure}}::{{closure}}::{{closure}}::{{closure}}::he20a50aef03c915e
  17:     0x56201db22aa5 - <clokwerk::async_scheduler::AsyncSchedulerFuture as core::future::future::Future>::poll::h93925fcc6b55d1ab
  18:     0x56201d606363 - <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll::hffb07a0dd0622bef
  19:     0x56201d2a100f - tokio::runtime::scheduler::current_thread::Context::enter::hd87acca89d038162
  20:     0x56201d68dcea - tokio::runtime::context::set_scheduler::h891511bffc0c301c
  21:     0x56201d2a019b - tokio::runtime::runtime::Runtime::block_on::hf61dd9224cc6d678
  22:     0x56201d3105b5 - std::panicking::try::h85b8a6e58ec7686f
  23:     0x56201d48ce74 - std::sys_common::backtrace::__rust_begin_short_backtrace::hd14ba080d3e68409
  24:     0x56201d65df5c - core::ops::function::FnOnce::call_once{{vtable.shim}}::h3fee5fe4d9b8eb5e
  25:     0x56201fdfda55 - std::sys::pal::unix::thread::Thread::new::thread_start::h40e6fd3f8ce15a14
  26:     0x7f56d335c134 - <unknown>
  27:     0x7f56d33dba40 - clone
  28:                0x0 - <unknown>

After trial and error I discovered that the issue is present only when a *.arrows file with size 0 bytes is present in the staging directory.
After deleting the 0 byte (empty) arrows file, Parseable continued working normally.

[nikhilsinhaparseable]

@krumpira thanks for trying out Parseable and reporting the issue.
We will analyse the issue and get back to you!

[krumpira]

Your responsiveness and quick fixing is impressive. Thank you.