Minor fixes and update changelog for release 0.2.0

CHANGELOG.md

@@ -1,9 +1,27 @@
 # Changelog
 
-Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and 
+Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
 [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.
 
-## 0.1.0 - 2022-06-02
+## [0.2.0] - 2022-07-08
+
++ Add - Adopt black formatting into code base
++ Add - GitHub Issue templates
++ Add - Code of conduct
++ Add - Jupytext paired python scripts
++ Add - Example data for trials, events, blocks, recordings, and alignments
++ Add - Analysis schema
++ Add - Raster and PSTH plots
++ Add - Video tutorial link to README
++ Add - Images for README
++ Add - Jupyter notebooks for downstream analysis, electrode localization, and NWB export
++ Add - Electrode localization schema activation script
++ Update - README for ephys modes, element-event, and element-electrode-localization
++ Update - Docker and Compose files for element-event and element-electrode-localization
++ Update - Pipeline script for ephys mode and element-event
++ Update - Pytests
+
+## [0.1.0] - 2022-06-02
 
 + Update - Docker and Compose files
 + Add - NWB export pytests
@@ -17,7 +35,7 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
 + Update - Notebooks
 + Update - Move instructions to [datajoint-elements/install.md](
      https://github.com/datajoint/datajoint-elements/blob/main/install.md).
-+ Update - Docker and Compose files for new base image and added options to install 
++ Update - Docker and Compose files for new base image and added options to install
 specific forks for tests.
 
 ## 0.1.0a2 - 2021-04-12
@@ -28,3 +46,6 @@ specific forks for tests.
 ## 0.1.1 - 2021-03-26
 
 + Add - Version
+
+[0.2.0]: https://github.com/datajoint/workflow-array-ephys/releases/tag/0.2.0
+[0.1.0]: https://github.com/datajoint/workflow-array-ephys/releases/tag/0.1.0

README.md

@@ -1,27 +1,29 @@
 # DataJoint Workflow - Array Electrophysiology
 
-Workflow for extracellular array electrophysiology data acquired with a polytrode probe 
-(e.g. [Neuropixels](https://www.neuropixels.org), Neuralynx) using the [SpikeGLX](https://github.com/billkarsh/SpikeGLX) or 
+Workflow for extracellular array electrophysiology data acquired with a polytrode probe
+(e.g. [Neuropixels](https://www.neuropixels.org), Neuralynx) using the [SpikeGLX](https://github.com/billkarsh/SpikeGLX) or
 [OpenEphys](https://open-ephys.org/gui) acquisition software and processed with [MATLAB-based Kilosort](https://github.com/MouseLand/Kilosort) or [python-based Kilosort](https://github.com/MouseLand/pykilosort) spike sorting software.
 
 A complete electrophysiology workflow can be built using the DataJoint Elements.
+
 + [element-lab](https://github.com/datajoint/element-lab)
 + [element-animal](https://github.com/datajoint/element-animal)
 + [element-session](https://github.com/datajoint/element-session)
 + [element-array-ephys](https://github.com/datajoint/element-array-ephys)
 
 This repository provides demonstrations for:
+
 1. Set up a workflow using DataJoint Elements (see
 [workflow_array_ephys/pipeline.py](workflow_array_ephys/pipeline.py))
 2. Ingestion of data/metadata based on a predefined file structure, file naming
 convention, and directory lookup methods (see
 [workflow_array_ephys/paths.py](workflow_array_ephys/paths.py)).
 3. Ingestion of clustering results.
-4. Export of `no_curation` schema to NWB and DANDI (see (notebooks/09-NWB-export.ipynb)[notebooks/09-NWB-export.ipynb]
+4. Export of `no_curation` schema to NWB and DANDI (see [notebooks/09-NWB-export.ipynb](notebooks/09-NWB-export.ipynb)).
 
 See the [Element Array Electrophysiology documentation](https://elements.datajoint.org/description/array_ephys/) for the background information and development timeline.
 
-For more information on the DataJoint Elements project, please visit https://elements.datajoint.org.  This work is supported by the National Institutes of Health.
+For more information on the DataJoint Elements project, please visit <https://elements.datajoint.org>.  This work is supported by the National Institutes of Health.
 
 ## Workflow architecture
 
@@ -31,52 +33,58 @@ The electrophysiology workflow presented here uses components from 4 DataJoint E
 [element-session](https://github.com/datajoint/element-session),
 [element-array-ephys](https://github.com/datajoint/element-array-ephys)) assembled
 together to form a fully functional workflow. Note that element-array-ephys offers three
-schema options, selected via the DataJoint config file, with 
+schema options, selected via the DataJoint config file, with
 `dj.config['custom']['ephys_mode']`
+
 + `acute` probe insertion, with curated clustering
 + `chronic` probe insertion, with curated clustering
 + `no-curation`, acute probe insertion with kilosort triggered clustering and supported NWB export
++ `precluster`, acute probe insertion with pre-processing steps prior to clustering and curated clustering
 
-![element-array-ephys](images/attached_array_ephys_element.svg)
+![](https://raw.githubusercontent.com/datajoint/workflow-array-ephys/main/images/attached_array_ephys_element.svg)
 
-Optionally, this can be used in conjunction with 
+Optionally, this can be used in conjunction with
 [element-event](https://github.com/datajoint/element-event)
 and [element-electrode-localization](https://github.com/datajoint/element-electrode-localization/).
 
-![element-event_attached](images/attached_trial_analysis.svg)
+![](https://raw.githubusercontent.com/datajoint/workflow-array-ephys/main/images/attached_trial_analysis.svg)
 
-![element-electrode-localization_attached](images/attached_electrode_localization.svg)
+![](https://raw.githubusercontent.com/datajoint/workflow-array-ephys/main/images/attached_electrode_localization.svg)
 
 ## Installation instructions
 
-The installation instructions can be found at the 
+The installation instructions can be found at the
 [DataJoint Elements documentation](https://elements.datajoint.org/usage/install/).
 
 ## Interacting with the DataJoint workflow
 
-Please refer to the workflow-specific 
-[Jupyter notebooks](/notebooks) 
+Please refer to the workflow-specific
+[Jupyter notebooks](/notebooks)
 for an in-depth explanation of how to ...
+
 1. Run the workflow ([03-process.ipynb](notebooks/03-process.ipynb)).
 2. Explore the data ([05-explore.ipynb](notebooks/05-explore.ipynb)).
-3. Examine trialized analyses, and establish downstream analyses 
+3. Examine trialized analyses, and establish downstream analyses
 ([07-downstream-analysis.ipynb](notebooks/07-downstream-analysis.ipynb))
-4. Locate probes within the 
-[Common Coordinate Framework](https://www.sciencedirect.com/science/article/pii/S0092867420304025) 
+4. Locate probes within the
+[Common Coordinate Framework](https://www.sciencedirect.com/science/article/pii/S0092867420304025)
 ([08-electrode-localization.ipynb](notebooks/08-electrode-localization.ipynb))
 5. Export to NWB and DANDI ([09-NWB-export.ipynb](notebooks/09-NWB-export.ipynb))
 
+See our YouTube tutorial for a walkthrough of the schemas and functions:
+  [![YouTube tutorial](https://img.youtube.com/vi/KQlGYOBq7ow/0.jpg)](https://www.youtube.com/watch?v=KQlGYOBq7ow)
 
 ## Citation
 
 If your work uses DataJoint and DataJoint Elements, please cite the respective Research Resource Identifiers (RRIDs) and manuscripts.
 
-+ DataJoint for Python or MATLAB
-    + Yatsenko D, Reimer J, Ecker AS, Walker EY, Sinz F, Berens P, Hoenselaar A, Cotton RJ, Siapas AS, Tolias AS. DataJoint: managing big scientific data using MATLAB or Python. bioRxiv. 2015 Jan 1:031658. doi: https://doi.org/10.1101/031658
 
-    + DataJoint ([RRID:SCR_014543](https://scicrunch.org/resolver/SCR_014543)) - DataJoint for `<Select Python or MATLAB>` (version `<Enter version number>`)
+* DataJoint for Python or MATLAB
+  + Yatsenko D, Reimer J, Ecker AS, Walker EY, Sinz F, Berens P, Hoenselaar A, Cotton RJ, Siapas AS, Tolias AS. DataJoint: managing big scientific data using MATLAB or Python. bioRxiv. 2015 Jan 1:031658. doi: <https://doi.org/10.1101/031658>
+
+  + DataJoint ([RRID:SCR_014543](https://scicrunch.org/resolver/SCR_014543)) - DataJoint for `<Select Python or MATLAB>` (version `<Enter version number>`)
 
 + DataJoint Elements
-    + Yatsenko D, Nguyen T, Shen S, Gunalan K, Turner CA, Guzman R, Sasaki M, Sitonic D, Reimer J, Walker EY, Tolias AS. DataJoint Elements: Data Workflows for Neurophysiology. bioRxiv. 2021 Jan 1. doi: https://doi.org/10.1101/2021.03.30.437358
+  + Yatsenko D, Nguyen T, Shen S, Gunalan K, Turner CA, Guzman R, Sasaki M, Sitonic D, Reimer J, Walker EY, Tolias AS. DataJoint Elements: Data Workflows for Neurophysiology. bioRxiv. 2021 Jan 1. doi: <https://doi.org/10.1101/2021.03.30.437358>
 
-    + DataJoint Elements ([RRID:SCR_021894](https://scicrunch.org/resolver/SCR_021894)) - Element Array Electrophysiology (version `<Enter version number>`)
+  + DataJoint Elements ([RRID:SCR_021894](https://scicrunch.org/resolver/SCR_021894)) - Element Array Electrophysiology (version `<Enter version number>`)

notebooks/00-data-download-optional.ipynb

@@ -4,6 +4,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "# Download example data\n",
+    "\n",
     "This workflow will need Ephys data collected from either SpikeGLX or OpenEphys and the output from kilosort2. We provided an example dataset to be downloaded to run through the pipeline. This notebook walks you through the process to download the dataset."
    ]
   },
@@ -41,19 +43,12 @@
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "e55f4f77",
    "metadata": {},
    "outputs": [],
    "source": [
     "import os\n",
-    "import djarchive_client\n",
-    "client = djarchive_client.client()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "To browse the datasets that are available in djarchive:"
+    "import djarchive_client"
    ]
   },
   {
@@ -62,14 +57,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "list(client.datasets())"
+    "client = djarchive_client.client()"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Each of the datasets have different versions associated with the version of workflow package. To browse the revisions:"
+    "To browse the datasets that are available in djarchive:"
    ]
   },
   {
@@ -78,14 +73,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "list(client.revisions())"
+    "list(client.datasets())"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "To download the dataset, let's prepare a root directory, for example in `/tmp`:"
+    "Each of the datasets have different versions associated with the version of workflow package. To browse the revisions:"
    ]
   },
   {
@@ -94,36 +89,23 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "os.mkdir('/tmp/test_data')"
+    "list(client.revisions())"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Get the dataset revision with the current version of the workflow:"
+    "To download the dataset, let's prepare a root directory, for example in `/tmp`:"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'0_1_0a2'"
-      ]
-     },
-     "execution_count": 1,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
-    "from workflow_array_ephys import version\n",
-    "revision = version.__version__.replace('.', '_')\n",
-    "revision"
+    "os.mkdir(\"/tmp/test_data\")"
    ]
   },
   {
@@ -139,7 +121,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "client.download('workflow-array-ephys-test-set', target_directory='/tmp/test_data', revision=revision)"
+    "client.download(\n",
+    "    \"workflow-array-ephys-benchmark\",\n",
+    "    target_directory=\"/tmp/test_data\",\n",
+    "    revision='v1',\n",
+    ")"
    ]
   },
   {
@@ -184,12 +170,12 @@
  ],
  "metadata": {
   "jupytext": {
-   "formats": "ipynb,py_scripts//py"
+   "formats": "ipynb,py"
   },
   "kernelspec": {
-   "display_name": "ephys_workflow_runner",
+   "display_name": "Python 3.10.4 64-bit ('python3p10')",
    "language": "python",
-   "name": "ephys_workflow_runner"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -201,7 +187,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.0"
+   "version": "3.10.4"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "ff52d424e56dd643d8b2ec122f40a2e279e94970100b4e6430cb9025a65ba4cf"
+   }
   }
  },
  "nbformat": 4,