Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
167 commits
Select commit Hold shift + click to select a range
5944402
emr hospitalization indicator
mariajahja May 29, 2020
28746ba
Cut 7-day average signal name down to fit within the 32-character limit
krivard Jun 9, 2020
5593c38
separate out data loading and add start/end day parameters
mariajahja Jun 12, 2020
eb207ba
add load data to init
mariajahja Jun 12, 2020
63726ca
revert spacing back to 4
mariajahja Jun 12, 2020
fa98e32
states -> state
mariajahja Jun 12, 2020
35d0b18
update template with new parameters
mariajahja Jun 12, 2020
726bead
synthetic data for testing
mariajahja Jun 12, 2020
12bacf0
fix end_date -> drop_date
mariajahja Jun 12, 2020
f188931
Initial commit for covidnet
eujing Jun 12, 2020
bc59132
fix most egregious pylint format errors
mariajahja Jun 12, 2020
c257510
config and loaddata tests
mariajahja Jun 12, 2020
4f3d5e5
extract combination outside update_sensor for testing; add tests
mariajahja Jun 12, 2020
6de0e7c
sensor tests
mariajahja Jun 12, 2020
eece60b
smoother tests
mariajahja Jun 13, 2020
eeaa95b
add write csv tests
mariajahja Jun 13, 2020
61975bf
adjust imports for consistency
mariajahja Jun 13, 2020
04df10a
Fixed setup.py install error
eujing Jun 15, 2020
ceae6fa
update template
mariajahja Jun 15, 2020
33a79bd
ensure smoothed num <= smoothed den (for synthetic data), deescalate …
mariajahja Jun 15, 2020
731b7f6
Fixed bad output path and mp Pool usage with pytest
eujing Jun 15, 2020
6283c9c
Initial unit tests
eujing Jun 15, 2020
f0faa73
initial commit
huisaddison Jun 16, 2020
96101be
initial commit
huisaddison Jun 16, 2020
4a2fc8a
Merge branch 'dev/safegraph' of github.com:cmu-delphi/covidcast-indic…
huisaddison Jun 16, 2020
e565ac6
update pointer
huisaddison Jun 16, 2020
e8999df
update readme
huisaddison Jun 16, 2020
dd9daf7
Updated and fixed template errors
eujing Jun 16, 2020
0505c2a
Added test for run
eujing Jun 16, 2020
4c004a8
remove symlink, use params.json.template instead
huisaddison Jun 16, 2020
29306a6
Added basic type annotations
eujing Jun 16, 2020
26f9db9
Use temp dirs for test cleanup
eujing Jun 16, 2020
8004e50
Added required static files
eujing Jun 16, 2020
f01eaac
Review updates
eujing Jun 16, 2020
d4204c0
Updated tests for review changes
eujing Jun 16, 2020
00bc931
expand documentation
huisaddison Jun 16, 2020
cb7fa9c
Merge branch 'dev/safegraph' of github.com:cmu-delphi/covidcast-indic…
huisaddison Jun 16, 2020
c8d6545
clarify that aws cli must be installed
huisaddison Jun 16, 2020
3ede32a
add tests
huisaddison Jun 16, 2020
4c67693
clarify use of subprocess
huisaddison Jun 16, 2020
df7ca20
promote to official signals for the PR
huisaddison Jun 16, 2020
408c45f
Merge pull request #79 from cmu-delphi/cdc-covidnet-package
krivard Jun 17, 2020
9a3d7d5
restructure update sensor: added SensorUpdator class, modularizing up…
jsharpna Jun 17, 2020
ca46f7d
added test code for SensorUpdator
jsharpna Jun 17, 2020
8214a7b
add usafacts
Jun 18, 2020
51b217b
added geo_reindex, unit test
jsharpna Jun 18, 2020
aae6dc7
added update sensor test, debugged update sensor
jsharpna Jun 18, 2020
1bf25f1
modified sensors idx, made groupby for loop
jsharpna Jun 18, 2020
2263105
EMRHospSensor: added gauss_smooth, backfill uses numpy only, fit uses…
jsharpna Jun 18, 2020
01e8d8a
clarify status of API key pair
huisaddison Jun 18, 2020
f84d0cb
some documentation
jsharpna Jun 19, 2020
8b6d2a3
update
Jun 19, 2020
170a10e
update geo.py for unallocated
Jun 19, 2020
23302c9
add unallocated
Jun 19, 2020
e9fb201
disburse NYC unallocated equally to NYC counties
Jun 19, 2020
33531be
delete witespaces
Jun 19, 2020
05c9485
fixed error for test_pull
Jun 19, 2020
09a9237
change params[base_url] from dict to str
Jun 19, 2020
63bc981
change base_url from dict to str
Jun 19, 2020
3fa6358
modify unittest
Jun 19, 2020
279b603
add .gitignore
Jun 19, 2020
7e69677
add .gitignore
Jun 19, 2020
2607332
Merge pull request #70 from cmu-delphi/fix/long-signal-names
krivard Jun 19, 2020
05a3db9
Merge pull request #96 from cmu-delphi/master
krivard Jun 19, 2020
36625e6
update unittests
Jun 19, 2020
da9d711
delete output files
Jun 19, 2020
f594560
update unittests
Jun 19, 2020
e2d8be1
update DETAILS
Jun 19, 2020
54b8562
bug fix
huisaddison Jun 19, 2020
f7f6f29
rename prop_... to ..._prop signals
huisaddison Jun 19, 2020
63f437c
revert the population for counties in NYC
Jun 22, 2020
6cc6397
fixed bug, module runs
jsharpna Jun 22, 2020
8af01ae
Merge pull request #81 from cmu-delphi/dev/safegraph
krivard Jun 22, 2020
0592d17
Merge pull request #75 from cmu-delphi/emr-hosp-package
mariajahja Jun 24, 2020
4efcf62
change filenames for 7dav_ signals
Jun 25, 2020
de7c4f0
change filenames for 7dav_ signals
Jun 25, 2020
12bb07b
update unittest
Jun 25, 2020
d3e9e51
update params for tests
Jun 25, 2020
bbf498b
update params for tests
Jun 25, 2020
f2f79f3
update small test data
Jun 25, 2020
9dd6601
Update DETAILS.md
huisaddison Jun 26, 2020
663c8f7
options to include standard errors under obfuscation
mariajahja Jun 26, 2020
46a3d19
update unit tests for se option
mariajahja Jun 26, 2020
c29b3f7
revert filenames for 7dav_ signals
Jun 27, 2020
c81788f
Add Sir Complains-a-Lot
capnrefsmmat Jun 29, 2020
20fc002
Explicitly depend on covidcast
capnrefsmmat Jun 29, 2020
7e5e1e6
Merge pull request #97 from cmu-delphi/hosp-admit-stable
krivard Jun 29, 2020
0b9e94c
change filenames
Jun 29, 2020
f175221
Install Jenkins machinery for automating indicator pipelines
korlaxxalrok Jun 5, 2020
6cb7310
Documentation for new jenkins/ansible deployment workflow
korlaxxalrok Jun 22, 2020
26f2c11
detect unexpected columns
Jul 1, 2020
5ff04c0
Add handling of unassigned cases/deaths to jhu
Jun 8, 2020
6815308
Update NYC population information in JHU DETAILS.md
jingjtang Jun 22, 2020
464fb61
Promote wip 7dav signals to their production names.
Jun 25, 2020
2cee045
Merge pull request #126 from cmu-delphi/dev/rebase-jhu-deploy
krivard Jul 1, 2020
f21de2c
Merge pull request #111 from cmu-delphi/fix/safegraph-documentation
krivard Jul 1, 2020
646374b
Mention API documentation step in the contribution guide
capnrefsmmat Jul 1, 2020
55b3f70
New cases/deaths indicator for the map that uses jhu for Puerto Rico …
Jun 30, 2020
8e1ffd5
Merge pull request #127 from cmu-delphi/readme-docs
krivard Jul 2, 2020
bb93296
Minor text addition to test CI/CD
korlaxxalrok Jul 6, 2020
98cf286
Merge pull request #128 from cmu-delphi/rebase/usafacts-plus-jhu
krivard Jul 6, 2020
ec7d349
Merge pull request #133 from cmu-delphi/test-run-usafacts
korlaxxalrok Jul 6, 2020
fb37e23
Add basic functionality to process only the latest (ie yesterday's) u…
korlaxxalrok Jul 10, 2020
1411b4b
Cleanup in order to start running usafacts in producion (#142)
korlaxxalrok Jul 13, 2020
b67c26a
add code
Jul 15, 2020
332ec54
add unittests
Jul 15, 2020
9aef490
update tests
Jul 15, 2020
28ba922
fix a bug and add test_pull
Jul 15, 2020
71d5c68
update unittest
Jul 15, 2020
c2e07cc
udpate code
Jul 16, 2020
3c3855e
fixed errors in README
Jul 16, 2020
c59f068
update unittests
Jul 16, 2020
5a096ce
udpate params template file
Jul 16, 2020
d04bac9
udpate readme file
Jul 16, 2020
b5f77d7
Add "wip" in the name of smoothed signal
jingjtang Jul 16, 2020
86b0789
modify code
Jul 16, 2020
7741ff6
add gitignore
Jul 16, 2020
861d6a1
move two dependencies into setup.py
Jul 16, 2020
e7ad84e
add detailed description
Jul 16, 2020
9468c97
udpate description
Jul 16, 2020
dd4ce38
add whitespace to DETAILS
Jul 16, 2020
239144a
Update DETAILS.md
jingjtang Jul 16, 2020
e433e52
Fixed a bug in setup.py
jingjtang Jul 16, 2020
8080715
Fixed a bug in the setup.py
jingjtang Jul 16, 2020
fb52548
update code
Jul 16, 2020
fa0b940
udpate unittests
Jul 16, 2020
821f93b
created folders for tests
Jul 16, 2020
9eba114
Delete .gitignore
jingjtang Jul 16, 2020
4cf437b
update unittests
Jul 16, 2020
c633b0e
fixed an error
Jul 16, 2020
5dd497f
Update setup.py
jingjtang Jul 16, 2020
a32a4f1
Removed commented lines
jingjtang Jul 16, 2020
36ea688
Update DETAILS.md
jingjtang Jul 17, 2020
9accf11
Merge pull request #89 from cmu-delphi/run-usafacts
krivard Jul 21, 2020
7579adf
Merge pull request #115 from cmu-delphi/sir-complainsalot
krivard Jul 21, 2020
b13f895
[usafacts] corrects the package name in setup.py
krivard Jul 21, 2020
f71c66b
[usafacts] Expand definition of "latest"
krivard Jul 21, 2020
41b78d5
Merge pull request #156 from cmu-delphi/dev/usafacts-upload-last-7
krivard Jul 21, 2020
667a572
Add Jenkins integration/deployment files
korlaxxalrok Jul 6, 2020
1ecce3f
Add usafacts production params file
korlaxxalrok Jul 6, 2020
39f1467
Minor text addition to test CI/CD (#134)
korlaxxalrok Jul 6, 2020
4ef7e47
Update start date and dir for production (#145)
korlaxxalrok Jul 13, 2020
5076b4d
Refactor to use non-imaginary date functions
krivard Jul 22, 2020
38ee4de
Appease the linter
krivard Jul 22, 2020
53aeb39
Merge pull request #160 from cmu-delphi/dev/usafacts-upload-last-7
krivard Jul 23, 2020
fae9e1d
Merge pull request #154 from cmu-delphi/quidel_covidtest
krivard Jul 25, 2020
57e5c69
Initial diff upload utility function
eujing Jul 28, 2020
cee1cbb
Split up functions into diff, archive, replace
eujing Jul 30, 2020
15e63f4
Create commits in a specified branch instead of active branch
eujing Jul 30, 2020
fefc1fd
Abstracted out archiving
eujing Jul 31, 2020
7c1aecc
Support for S3 bucket archiving
eujing Aug 3, 2020
8a49e17
Added S3 archive diff unit tests
eujing Aug 4, 2020
cf4b86b
Rename deploy_cases_and_deaths to combo_cases_and_deaths and refactor…
krivard Jul 6, 2020
c392bc3
Make default combo_cases_and_deaths behavior robust to skipped days. …
krivard Jul 7, 2020
b9ab3f4
A complete issue includes smoothed data on the specified date range a…
krivard Aug 5, 2020
4e80fa5
Completed passing tests and linter for combo cases and deaths
krivard Aug 5, 2020
0087f4d
Updated unit tests and docs
eujing Aug 5, 2020
7f93a5b
Updated JHU example
eujing Aug 5, 2020
eecb91e
Update JHU params.json.template
eujing Aug 5, 2020
3fffd54
Merge pull request #184 from cmu-delphi/refactor/combo_cases_and_deaths
krivard Aug 6, 2020
2e4facd
Rearranged imports
eujing Aug 6, 2020
e472089
Added more docs
eujing Aug 6, 2020
8cac767
Added some NaNs to tests to verify behavior
eujing Aug 10, 2020
65dfc43
Merge pull request #185 from cmu-delphi/diff-uploads
krivard Aug 10, 2020
23d7ae0
Merge remote-tracking branch 'origin/deploy-jhu' into main
eujing Aug 10, 2020
288ec3c
Fixed jhu tests to mock s3 too
eujing Aug 10, 2020
975dd54
Initial ansible modifications without vault
eujing Aug 11, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 19 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,15 @@ Each subdirectory contained here that is named after an indicator has specific d

**tl;dr**

- Create your new indicator branch from `main`.
- Build it using the appropriate template, following the guidelines in the included README.md and REVIEW.md files.
- Make some stuff!
- When your stuff works, push your `dev-*` branch to remote for review.
- Consult with a platform engineer for the remaining production setup needs. They will create a branch called `deploy-*` for your indicator.
- Initiate a pull request against this new branch.
- If your peers like it and Jenkins approves, deploy your changes by merging the PR.
- Rejoice!
1. Create your new indicator branch from `main`.
2. Build it using the appropriate template, following the guidelines in the included README.md and REVIEW.md files.
3. Make some stuff!
4. When your stuff works, push your `dev-*` branch to remote for review.
5. Consult with a platform engineer for the remaining production setup needs. They will create a branch called `deploy-*` for your indicator.
6. Initiate a pull request against this new branch.
7. Following [the source documentation template](https://github.com/cmu-delphi/delphi-epidata/blob/main/docs/api/covidcast-signals/_source-template.md), create public API documentation for the source. You can submit this as a pull request against the delphi-epidata repository.
8. If your peers like the code, the documentation is ready, and Jenkins approves, deploy your changes by merging the PR.
9. Rejoice!

### Starting out

Expand All @@ -44,6 +45,15 @@ Once you have something that runs locally and passes tests you set up your remot
git push -u origin dev-my-feature-branch
```

You can then set draft public API documentation for people who would fetch this
data from the API. Public API documentation is kept in the delphi-epidata
repository, and there is a [template Markdown
file](https://github.com/cmu-delphi/delphi-epidata/blob/main/docs/api/covidcast-signals/_source-template.md)
that outlines the features that need to be documented. You can create a pull
request to add a new file to `docs/api/covidcast-signals/` for your source. Our
goal is to have public API documentation for the data at the same time as it
becomes available to the public.

### Setting up for review and deployment

Once you have your branch set up you should get in touch with a platform engineer to pair up on the remaining production needs. These include:
Expand All @@ -55,7 +65,7 @@ Once you have your branch set up you should get in touch with a platform enginee

Once all the last mile configuration is in place you can create a pull request against the correct `deploy-*` branch to initiate the CI/CD pipeline which will build, test, and package your indicator for deployment.

If everything looks ok, platform engineering has validated the last mile, and the pull request is accepted, you can merge the PR. Deployment will start automatically.
If everything looks ok, you've drafted source documentation, platform engineering has validated the last mile, and the pull request is accepted, you can merge the PR. Deployment will start automatically.

Hopefully it'll be a full on :tada:, after that :crossed_fingers:

Expand Down
1 change: 1 addition & 0 deletions _delphi_utils_python/delphi_utils/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

from __future__ import absolute_import

from .archive import ArchiveDiffer, GitArchiveDiffer, S3ArchiveDiffer
from .export import create_export_csv
from .utils import read_params

Expand Down
Loading