From e9383f5d52b2e3d0261b29439a0b8196ca7d558a Mon Sep 17 00:00:00 2001 From: rumackaaron Date: Mon, 4 Oct 2021 16:49:16 -0400 Subject: [PATCH 1/5] Description of CHNG as-of estimation problems --- docs/api/covidcast-signals/chng.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/api/covidcast-signals/chng.md b/docs/api/covidcast-signals/chng.md index 5375c4b57..ce9383681 100644 --- a/docs/api/covidcast-signals/chng.md +++ b/docs/api/covidcast-signals/chng.md @@ -185,6 +185,10 @@ Note that due to local differences in health record-keeping practices, estimates are not always comparable across locations. We are currently working on adjustments to correct this spatial bias. +Indicator values for as-of dates before 2021-02-21 are merely estimates, as +these indicators were not yet available in real time. Backfill behavior of +these estimates is erratic and not indicative of current backfill behavior. + ## Qualifying Conditions We receive data on the following six categories of counts: From 66b31a6e1097f1c9972fb824b2a22451a2ff6b0f Mon Sep 17 00:00:00 2001 From: Katie Mazaitis Date: Fri, 8 Oct 2021 17:00:08 -0400 Subject: [PATCH 2/5] Include pipeline pause in limitations update for CHNG --- docs/api/covidcast-signals/chng.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/api/covidcast-signals/chng.md b/docs/api/covidcast-signals/chng.md index ce9383681..26b6734dc 100644 --- a/docs/api/covidcast-signals/chng.md +++ b/docs/api/covidcast-signals/chng.md @@ -17,6 +17,8 @@ grand_parent: COVIDcast Epidata API ## Overview +**Notice: This data source was paused on 2021-10-04 so that we can resolve some problems with the data pipeline. [Additional details on this pause are available below](#pipeline-pause).** + This data source is based on Change Healthcare claims data that has been de-identified in accordance with HIPAA privacy regulations. Change Healthcare is a healthcare technology company that aggregates data from many healthcare providers. @@ -189,6 +191,20 @@ Indicator values for as-of dates before 2021-02-21 are merely estimates, as these indicators were not yet available in real time. Backfill behavior of these estimates is erratic and not indicative of current backfill behavior. +### Pipeline pause + +Starting on October 4, 2021, a problem with the `chng` pipeline began causing it +to mark some days of data as deleted in their most recent version. These +spurious deletions affected all regions and `chng` signals from July 31 to +August 3, 2021, and the affected date range would continue to grow by one day +each day if we allowed the pipeline to continue running. + +On October 8, 2021, we paused the `chng` pipeline, and it will remain inactive +until we can identify and implement a fix. In the meantime, the versions with +the deletion markings have been removed, so that default (latest) queries and +queries with as-of set to 20211004 or later will return the +next-most-recently-updated value for these dates. + ## Qualifying Conditions We receive data on the following six categories of counts: From b5f01122968512b4c9e96cfa4baff6252789f542 Mon Sep 17 00:00:00 2001 From: Katie Mazaitis Date: Fri, 8 Oct 2021 17:36:17 -0400 Subject: [PATCH 3/5] Apply suggestions from review Co-authored-by: Alex Reinhart --- docs/api/covidcast-signals/chng.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api/covidcast-signals/chng.md b/docs/api/covidcast-signals/chng.md index 26b6734dc..cbf8aa97a 100644 --- a/docs/api/covidcast-signals/chng.md +++ b/docs/api/covidcast-signals/chng.md @@ -187,7 +187,7 @@ Note that due to local differences in health record-keeping practices, estimates are not always comparable across locations. We are currently working on adjustments to correct this spatial bias. -Indicator values for as-of dates before 2021-02-21 are merely estimates, as +Indicator values for issue dates before 2021-02-21 are merely estimates, as these indicators were not yet available in real time. Backfill behavior of these estimates is erratic and not indicative of current backfill behavior. From 87eeae7f137623ad6223f5e5f521351e357e61e8 Mon Sep 17 00:00:00 2001 From: Katie Mazaitis Date: Mon, 11 Oct 2021 08:58:25 -0400 Subject: [PATCH 4/5] Spelling changes from code review Co-authored-by: Alex Reinhart --- docs/api/covidcast-signals/chng.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/api/covidcast-signals/chng.md b/docs/api/covidcast-signals/chng.md index cbf8aa97a..1e73629b7 100644 --- a/docs/api/covidcast-signals/chng.md +++ b/docs/api/covidcast-signals/chng.md @@ -191,7 +191,7 @@ Indicator values for issue dates before 2021-02-21 are merely estimates, as these indicators were not yet available in real time. Backfill behavior of these estimates is erratic and not indicative of current backfill behavior. -### Pipeline pause +### Pipeline Pause Starting on October 4, 2021, a problem with the `chng` pipeline began causing it to mark some days of data as deleted in their most recent version. These @@ -202,7 +202,7 @@ each day if we allowed the pipeline to continue running. On October 8, 2021, we paused the `chng` pipeline, and it will remain inactive until we can identify and implement a fix. In the meantime, the versions with the deletion markings have been removed, so that default (latest) queries and -queries with as-of set to 20211004 or later will return the +queries with as-of set to 2021-10-04 or later will return the next-most-recently-updated value for these dates. ## Qualifying Conditions From 1270f12b65d58ae4b1ece88874828e0801450ff2 Mon Sep 17 00:00:00 2001 From: Katie Mazaitis Date: Mon, 11 Oct 2021 17:04:38 -0400 Subject: [PATCH 5/5] chng docs: issue link Co-authored-by: Alex Reinhart --- docs/api/covidcast-signals/chng.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/api/covidcast-signals/chng.md b/docs/api/covidcast-signals/chng.md index 1e73629b7..4e00aaf82 100644 --- a/docs/api/covidcast-signals/chng.md +++ b/docs/api/covidcast-signals/chng.md @@ -189,7 +189,9 @@ adjustments to correct this spatial bias. Indicator values for issue dates before 2021-02-21 are merely estimates, as these indicators were not yet available in real time. Backfill behavior of -these estimates is erratic and not indicative of current backfill behavior. +these estimates is erratic and not indicative of current backfill behavior. For more +information on this effect and to track updates as we develop a fix, please see +[covidcast-indicators issue #1289: CHNG historical issues are wrong before 2021-02-21](https://github.com/cmu-delphi/covidcast-indicators/issues/1289). ### Pipeline Pause