FIRESTORE INTERNAL ASSERTION FAILED: Dangling query-target reference found

[gautier-gdx]

[REQUIRED] Step 1: Describe your environment

Xcode version: 12.0.1
Firebase SDK version: 6.33.0
Firebase Component: Firestore
Component version: 6.33.0
Installation method: CocoaPods

[REQUIRED] Step 2: Describe the problem

So far this issue is only happening in production but it's happening in about 2% of all cases (about 25k crashes so far) So it starts to be a big problem for us

basically this is an internal error, probably related to levelDB
Here is the crash log

Fatal Exception: NSInternalInconsistencyException FIRESTORE INTERNAL ASSERTION FAILED: Dangling query-target reference found: [query_target: canonical_id=apps/App|f:|ob:asc target_id=14336] points to [target: target_id=14336]; seeking there found [target: target_id=26618]

> Fatal Exception: NSInternalInconsistencyException
> 0  CoreFoundation                 0x184a4b4e4 (Missing)
> 1  libobjc.A.dylib                0x18476e26c (Missing)
> 2  CoreFoundation                 0x18495b59c (Missing)
> 3  Foundation                     0x184e0ed98 (Missing)
> 4  Wizz                           0x100d00538 firebase::firestore::util::ObjcThrowHandler(firebase::firestore::util::ExceptionType, char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 59 (exception_apple.mm:59)
> 5  Wizz                           0x100d001e0 firebase::firestore::util::Throw(firebase::firestore::util::ExceptionType, char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 90 (exception.cc:90)
> 6  Wizz                           0x100d50530 firebase::firestore::util::internal::FailAssertion(char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*) + 42 (hard_assert.cc:42)
> 7  Wizz                           0x100d6a01c firebase::firestore::local::LevelDbTargetCache::GetTarget(firebase::firestore::core::Target const&) + 171 (leveldb_target_cache.cc:171)
> 8  Wizz                           0x100d788b8 std::__1::__function::__func<std::__1::enable_if<!(std::is_same<void, decltype(fp0())>::value), decltype(fp0())>::type firebase::firestore::local::Persistence::Run<firebase::firestore::local::LocalStore::AllocateTarget(firebase::firestore::core::Target)::$_12>(absl::lts_2020_02_25::string_view, firebase::firestore::local::LocalStore::AllocateTarget(firebase::firestore::core::Target)::$_12)::'lambda'(), std::__1::allocator<std::__1::enable_if<!(std::is_same<void, decltype(fp0())>::value), decltype(fp0())>::type firebase::firestore::local::Persistence::Run<firebase::firestore::local::LocalStore::AllocateTarget(firebase::firestore::core::Target)::$_12>(absl::lts_2020_02_25::string_view, firebase::firestore::local::LocalStore::AllocateTarget(firebase::firestore::core::Target)::$_12)::'lambda'()>, void ()>::operator()()
> 9  Wizz                           0x100d627b4 firebase::firestore::local::LevelDbPersistence::RunInternal(absl::lts_2020_02_25::string_view, std::__1::function<void ()>) + 2620 (memory:2620)
> 10 Wizz                           0x100d740b8 firebase::firestore::local::LocalStore::AllocateTarget(firebase::firestore::core::Target)
> 11 Wizz                           0x100db89f4 firebase::firestore::core::SyncEngine::Listen(firebase::firestore::core::Query)
> 12 Wizz                           0x100cfd150 firebase::firestore::core::EventManager::AddQueryListener(std::__1::shared_ptr<firebase::firestore::core::QueryListener>)
> 13 Wizz                           0x100d26c80 std::__1::__function::__func<firebase::firestore::core::FirestoreClient::ListenToQuery(firebase::firestore::core::Query, firebase::firestore::core::ListenOptions, std::__1::shared_ptr<firebase::firestore::core::EventListener<firebase::firestore::core::ViewSnapshot> >&&)::$_10, std::__1::allocator<firebase::firestore::core::FirestoreClient::ListenToQuery(firebase::firestore::core::Query, firebase::firestore::core::ListenOptions, std::__1::shared_ptr<firebase::firestore::core::EventListener<firebase::firestore::core::ViewSnapshot> >&&)::$_10>, void ()>::operator()() + 4520 (memory:4520)
> 14 Wizz                           0x100ce1518 firebase::firestore::util::AsyncQueue::ExecuteBlocking(std::__1::function<void ()> const&) + 957 (atomic:957)
> 15 Wizz                           0x100dc5788 firebase::firestore::util::Task::ExecuteAndRelease() + 1846 (functional:1846)
> 16 libdispatch.dylib              0x18470b6d8 (Missing)
> 17 libdispatch.dylib              0x1846b4cf0 (Missing)
> 18 libdispatch.dylib              0x1846b57cc (Missing)
> 19 libdispatch.dylib              0x1846bf170 (Missing)
> 20 libsystem_pthread.dylib        0x1847625a4 (Missing)
> 21 libsystem_pthread.dylib        0x184765874 (Missing)

[wu-hui]

Hi @gautier-gdx

Thank you for the report and sorry for the issue. One thing I want to know is does this only happen with 6.33.0? Have you been using earlier releases and observed no such issues?

Thanks!

[gautier-gdx]

@wu-hui Thanks for the reply, yes indeed it's affecting us since 6.29 I believe

[wu-hui]

So the one version that did not have this issue is 6.28, can I assume that?

[gautier-gdx]

Yes I think so, one other hint I can give is that it seems to happen more often on iOS 14 devices but it's also happening on older versions

The problem is getting bigger each days thanks to iOS 14 adoptions

[gautier-gdx]

HI @wu-hui, did you had the chance to dig into this issue ?

Best

[wu-hui]

Hi @gautier-gdx

Yes. I have tried to diff the code between those versions, unfortunately nothing obvious jumps out.

This issue might be related to underlying OS change, especially if there are optimizations done to file system, which will affect LevebDb, leading to missing writes.

If you can reproduce it with your App on device, and share the leveldb file with us, that might help us figure out why this is happening.

In the meantime, the assertion we did there could be just a warning, our SDK should still be functioning after seeing this. It's just it is still important to figure out why this is happening.

I'll make a change to make the assertion a warning there.

Thanks.

[gautier-gdx]

@wu-hui Thanks,

The assertion being just a warning would explain why we haven't suceed to reproduce the crash on our end for months.

I will keep you updated if the crashes starts to drop after updating with the fix

[morganchen12]

Hey @gautier-gdx, have you been able to measure a difference in the crash volume after updating Firestore?

[gautier-gdx]

Hi @morganchen12, Indeed it has completely disappeared, thanks a lot !