Modify all pipelines to list signal names in a uniform location

[krivard]

Changing signal names from their wip_ to their public versions turns out to be an important part of the release schedule, so let's make it easier to catch these updates in PRs. Maybe params.json.template? Or a separate file?

• safegraph
• ght
• emr-hosp
• combo cases/deaths
• quidel covid tests
• doctor-visits [blocked?]
• quidel flu tests [blocked; wait for merge to main]
• jhu [blocked; wait for geo revision]
• usafacts [blocked; wait for geo revision]
• combo nmf_day_doc_fbc_fbs_ght [blocked; not in this codebase]
• python template
• fb-survey [blocked; wait for merge to main]
• corona data scraper (cds)

[krivard]

emr-hosp is proving difficult to run -- V will follow up if it doesn't come unstuck

[krivard]

Currently on safegraph; it runs but doesn't generate any output.

[krivard]

Ready for PR!

One failing test for CDC-COVIDnet; chasing down with @eujing.

[krivard]

Proposal #1:
SIGNALNAMES={'safegraph':['completely_home', 'part_time_work', ..]}
COMPLETELY_HOME=0

...
SIGNALNAMES['safegraph'][COMPLETELY_HOME]

confirmed/deaths
incidence/cumulative
num/prop
raw/smoothed

Proposal #2:
class SIGNALNAMES(object):
  COMPLETELY_HOME="completely_home"
  THREE_SIX_HOURS="part_time_work"

..

df[COMPLETELY_HOME] = // math

Proposal #3:
class SIGNALNAMES(object):
    COMPLETELY_HOME(self):
        if self.wip: return "wip_"+_COMPLETELY_HOME
        return _COMPLETELY_HOME
    THREE_SIX_HOURS="part_time_work"


Proposal #4:
Check the meta API to see if a signal is already in production, and if not default to WIP status unless params tells you not to
* still has to list all signals in params at least once for their initial production load

--
{prefix:true|false}
{prefix:true|false|list()} -- list prefixes only the signals listed

• easy to turn WIP status on and off for completely new data sources
• easy to turn WIP status on and off for new signals on existing data sources
• no signal name strings used anywhere in the code except in a central index for each indicator

[krivard]

Use meta API to automatically detect when a signal is new and should be prefixed with wip_.

safegraph almost done, will review and then move on to the next indicator.

[krivard]

safegraph is done; cleaning up the commit history.

wip policy:

• if params.json param is true
• if signal is not in metadata
• if signal is listed in params.json param

cdc-covidnet is next.

[krivard]

cvc-covidnet is not a production indicator, and approaches signal names differently but that's been resolved.

PR for safegraph in #168

google-health is next.

[krivard]

ght is done except for tests. Will also fix the dict.keys() test. When complete, PR.

emr-hosp next (finish up)

Adapt the python template to use this new naming scheme?

[krivard]

Incorporating safegraph changes that came up during review.

[krivard]

Updating the indicator template.

[krivard]

ght is failing tests because the existing tests don't know what to do with WIP signals.

Retain the original test that only does public signals, and add a new test for WIP signals.

Will merge fixed cdc-covidnet branch.