Merge pull request #345 from ComputationalCryoEM/develop

Develop - BETA

Author: garrettwrong

.bumpversion.cfg

@@ -0,0 +1,24 @@
+[bumpversion]
+current_version = 0.6.0
+commit = True
+tag = True
+
+[bumpversion:file:setup.py]
+search = version='{current_version}'
+replace = version='{new_version}'
+
+[bumpversion:file:README.md]
+search = v{current_version}
+replace = v{new_version}
+
+[bumpversion:file:src/aspire/__init__.py]
+search = __version__ = '{current_version}'
+replace = __version__ = '{new_version}'
+
+[bumpversion:file:docs/source/conf.py]
+search = release = version = '{current_version}'
+replace = release = version = '{new_version}'
+
+[bumpversion:file:docs/source/index.rst]
+search = v{current_version}
+replace = v{new_version}

.git-blame-ignore-revs

@@ -0,0 +1,2 @@
+# Initial Black autoformatting
+fad41cbb60b15108da2d74f3125541d8eadf8c83

.gitattributes

@@ -0,0 +1,5 @@
+# See https://git-scm.com/docs/gitattributes for details on how to use this file
+
+# Among a bunch of other stuff we can
+#     add files or directory that should be ignored by tools like `git grep` like so
+tutorials/notebooks/*.ipynb binary

.gitignore

@@ -93,3 +93,5 @@ venv.bak/
 # all logs
 *.log
 
+# Generated documentation
+docs/source/

.travis.yml

@@ -1,32 +1,42 @@
-sudo: required
 language: python
-python:
-  - "3.6"
+
+# manylinux2010 wheels are not found on xenial
+dist: bionic
+
+os:
+  - linux
+
+arch:
+  - amd64
+
+stages:
+  - check
+  - test
+  - deploy
+
+jobs:
+  include:
+    - python: 3.6
+      env: TOXENV=clean,py36-stable,coveralls,report
+    - python: 3.6
+      env: TOXENV=py36-dev
+    - python: 3.7
+      env: TOXENV=py37-stable
+    - python: 3.7
+      env: TOXENV=py37-dev
+    - python: 3.8
+      env: TOXENV=py38-stable
+    - python: 3.8
+      env: TOXENV=py38-dev
+    - python: 3.6
+      env: TOXENV=check
+    - python: 3.6
+      env: TOXENV=docs
+
 install:
-  - sudo apt-get update
-  # We do this conditionally because it saves us some downloading if the
-  # version is the same.
-  - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then
-      wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -O miniconda.sh;
-    else
-      wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
-    fi
-  - bash miniconda.sh -b -p $HOME/miniconda
-  - export PATH="$HOME/miniconda/bin:$PATH"
-  - hash -r
-  - conda config --set always_yes yes --set changeps1 no
-  - conda update -q conda
-  # Useful for debugging any issues with conda
-  - conda info -a
-
-  - conda env create -n test-environment -f environment.yml
-  - source activate test-environment
-  - python setup.py install
-  
-script: py.test --cov=aspire tests
-
-after_success:
-  - coveralls
+  - pip install -U tox tox-travis
+
+script: tox -v --skip-missing-interpreters false -e $TOXENV
 
 before_deploy:
   - cd docs && sphinx-apidoc -f -o ./source ../src -H Modules && make html && touch build/html/.nojekyll

CODEOWNERS

@@ -0,0 +1,38 @@
+# This is a comment.
+# Each line is a file pattern followed by one or more owners.
+
+# These owners will be the default owners for everything in
+# the repo. Unless a later match takes precedence,
+# @global-owner1 and @global-owner2 will be requested for
+# review when someone opens a pull request.
+# *       @global-owner1 @global-owner2
+*        @janden
+
+# Order is important; the last matching pattern takes the most
+# precedence. When someone opens a pull request that only
+# modifies JS files, only @js-owner and not the global
+# owner(s) will be requested for a review.
+# *.js    @js-owner
+
+# You can also use email addresses if you prefer. They'll be
+# used to look up users just like we do for commit author
+# emails.
+# *.go [email protected]
+
+# In this example, @doctocat owns any files in the build/logs
+# directory at the root of the repository and any of its
+# subdirectories.
+# /build/logs/ @doctocat
+
+# The `docs/*` pattern will match files like
+# `docs/getting-started.md` but not further nested files like
+# `docs/build-app/troubleshooting.md`.
+# docs/*  [email protected]
+
+# In this example, @octocat owns any file in an apps directory
+# anywhere in your repository.
+# apps/ @octocat
+
+# In this example, @doctocat owns any file in the `/docs`
+# directory in the root of your repository.
+# /docs/ @doctocat

CODING_GUIDELINES.md

@@ -6,30 +6,99 @@ This document contains the coding guidelines for the ASPIRE Python project.
 ## Coding Style
 
 In order to keep the code consistent, please read and follow [PEP 8 Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008)
-and [PEP 257 Docstring Conventions](https://www.python.org/dev/peps/pep-0257/) guidelines. [This webpage](https://realpython.com/documenting-python-code/) is a good guide for documenting Python code. 
-For automatic checking of style problems, [Pycharm from Jet Brains](https://www.jetbrains.com/pycharm/) 
-is a good candidate which has the professional version for academic users with free license fee. 
+and [PEP 257 Docstring Conventions](https://www.python.org/dev/peps/pep-0257/) guidelines. [This webpage](https://realpython.com/documenting-python-code/) is a good guide for documenting Python code.
+For an editor with some automatic checking of style problems, [Pycharm from Jet Brains](https://www.jetbrains.com/pycharm/)
+is a good candidate which has the professional version for academic users with free license fee.
 
+In the future components of PEP8 and PEP257 will be checked programmatically.  Documentation of a standard tool to perform
+these operations locally will be provided in Google Doc to developers at that time.
 
-## Best Practices
+## Good Practices Reading
 
-For beginners in scientific computing, some general guides are suggested in Greg Wilson's papers as below:
-
- 1. [Good Enough Practices in Scientific Computing]( https://doi.org/10.1371/journal.pcbi.1005510) 
- 2. [Best Practices for Scientific Computing]( https://doi.org/10.1371/journal.pbio.1001745) 
+Some thoughts on Scientific Computing are discussed in Greg Wilson and collaborators' paper as below:
 
+ 1. [Good Enough Practices in Scientific Computing]( https://doi.org/10.1371/journal.pcbi.1005510)
+ 2. [Best Practices for Scientific Computing]( https://doi.org/10.1371/journal.pbio.1001745)
 
 ## Source Code Control
 
-As an open source software, we use Git and GitHub for our source code control. 
-The following links are useful tutorials. 
-1. Git tutorials from [Atalassian](https://www.atlassian.com/git/tutorials) and [Git-SCM](https://git-scm.com/docs/gittutorial)
-2. GitHub guides from [GitHub](https://guides.github.com/) 
+As an open source software, we use Git with GitHub for our source code control.
+The following links are useful tutorials.
+1. Git tutorials from [Atlassian](https://www.atlassian.com/git/tutorials) and [Git-SCM](https://git-scm.com/docs/gittutorial)
+2. GitHub guides from [GitHub](https://guides.github.com/)
 
-## Git Workflow Changes
+## Git Workflow
 
-Our project follows [Gitflow Workflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow).
-Please submit any PRs against the `develop` branch.
+Our project follows a reduced [Gitflow Workflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow).
+Please submit any Pull Requests (PR) against the `develop` branch.
 
 ![Gitflow Diagram](https://wac-cdn.atlassian.com/dam/jcr:61ccc620-5249-4338-be66-94d563f2843c/05%20(2).svg?cdnVersion=357)
 
+The basic idea is that we have `feature` branches,  `develop` as an integration focused branch, and `master` for releases.
+We do not currently have a need for a dedicated release or staging branch; in our case `develop` also serves this purpose.
+Generally feature branches should branch off of the latest `develop` branch.
+
+Developers are welcome to maintain their own fork, but this is not required.
+
+When `develop` is at a milestone where we want to cut a release,
+following the controls below we will commit with a
+[PEP440](https://www.python.org/dev/peps/pep-0440/)
+compliant `Major.Minor` version scheme
+and complete a reviewed merge to `master`.
+Once in `master` a formal Git (version) tag can be applied to the commit,
+and at this point the release would be a candidate to upload to PyPI.
+
+The process for these steps will be documented in a Google Doc.
+
+###  Git Workflow Controls
+
+To facilitate effective integration, merges into `develop` must occur as a pull request that is approved by two reviewers.
+One reviewer can be anyone, but should probably be a developer related to the effort if that is applicable.
+The second reviewer is a designated codeowner described in the `CODEOWNERS` file.
+More information on codeowners can be [found on GitHub.](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners)
+
+Additionally, all outstanding requested changes and comments should be resolved
+by the requesting reviewer before merging.
+Pushing new changes should trigger a re-review process
+of changed files to prevent oversights.  
+
+Code should be passing integration and unit tests as defined in code on all
+supported platforms by the evolving Continuous Integration (CI) systems,
+or be explicitly exempted by a codeowner.
+This is both for code quality and so CI maintenance is not overlooked.
+
+All of these controls will be managed automatically by the GitHub server.
+
+Releases and merges to `master` should be coordinated between developers and the project PI.
+This will be documented with an additional required review from the PI when merging into `master`.
+
+### Git Workflow Courtesy
+
+Contributors are responsible for keeping their working repositories up to date with the main GitHub repo.
+This is particularly important during active development,
+as it will reduce the likelihood and extent of code conflicts
+when new features are implemented.
+
+If a piece of work is known to change files that will effect other developers' active work, they should be invited to the review or tagged,
+so that the most effective order of integration can be established.  In the case of your own work having integration
+concerns, again, inviting other developers to discuss might be helpful.
+
+In the case a codeowner will not be available, such as (sickeness, vacation, travel, and so on)
+and an emergency patch can not be delayed, a simple explicit consensus among
+the remaining developers should suffice.
+
+Do not push or merge over another working branch unless explicitly permitted to do so by the authors.
+Instead, when working on a shared branch submit a Pull Request to the other party.
+
+## Project Collaboration
+
+We will for a time attempt using
+[GitHub project boards](https://help.github.com/en/github/managing-your-work-on-github/about-project-boards)
+in conjunction with
+[Issues](https://help.github.com/en/github/managing-your-work-on-github/about-issues)
+to document tasks in various stages.
+Generally this will look like a board collecting related tasks representing a conceptual project or
+actual milestone that comes out of our meetings.  For example, an event, release, or large feature set
+with many components all are reasonable project boards.  This is for planning purposes and to facilitate coordination
+in the remote environment. If it helps we can keep doing it so long as the time and effort is accounted for.
+

MANIFEST.in

@@ -0,0 +1,30 @@
+include *.md
+include *.rst
+include *.yml
+include .bumpversion.cfg
+include .flake8
+include .git-blame-ignore-revs
+include .isort.cfg
+include CODEOWNERS
+include LICENSE
+include pytest.ini
+include tox.ini
+include logging.conf
+include config.ini
+recursive-include docs *.bat
+recursive-include docs *.bib
+recursive-include docs *.py
+recursive-include docs *.rst
+recursive-include docs Makefile
+recursive-include tests *.mrc
+recursive-include tests *.mrcs
+recursive-include tests *.npy
+recursive-include tests *.py
+recursive-include tests *.star
+recursive-include tutorials *.dontrun
+recursive-include tutorials *.ipynb
+recursive-include tutorials *.mat
+recursive-include tutorials *.mrc
+recursive-include tutorials *.npy
+recursive-include tutorials *.py
+prune docs/source

README.md

@@ -4,9 +4,9 @@
 [![Travis Build Status](https://travis-ci.org/ComputationalCryoEM/ASPIRE-Python.svg?branch=master)](https://travis-ci.org/ComputationalCryoEM/ASPIRE-Python)
 [![Coverage Status](https://coveralls.io/repos/github/ComputationalCryoEM/ASPIRE-Python/badge.svg?branch=master)](https://coveralls.io/github/ComputationalCryoEM/ASPIRE-Python?branch=master)
 
-# ASPIRE - Algorithms for Single Particle Reconstruction
+# ASPIRE - Algorithms for Single Particle Reconstruction - v0.6.0
 
-This is the Python version to supersede the [Matlab ASPIRE](https://github.com/PrincetonUniversity/aspire). 
+This is the Python version to supersede the [Matlab ASPIRE](https://github.com/PrincetonUniversity/aspire).
 
 ASPIRE is an open-source software package for processing single-particle cryo-EM data to determine three-dimensional structures of biological macromolecules. The package includes advanced algorithms based on rigorous mathematics and recent developments in
 statistics and machine learning. It provides unique and improved solutions to important computational challenges of the cryo-EM
@@ -21,49 +21,53 @@ For more information about the project, algorithms, and related publications ple
 For end-users
 -------------
 
-ASPIRE is a pip-installable package that works on Linux/Mac/Windows, and requires Python 3.6. The simplest option is to use Anaconda 64-bit for your platform with a minimum of Python 3.6 and pip, and then use `pip` to install `aspire` in that environment.
+ASPIRE is a pip-installable package that works on Linux/Mac/Windows, and requires Python 3.6. The simplest option is to use Anaconda 64-bit for your platform with a minimum of Python 3.6 and `pip`, and then use `pip` to install `aspire` in that environment.
 
 ```
 conda create -n aspire_env python=3.6 pip
 conda activate aspire_env
 pip install aspire
 ```
 
-The final step above should install any dependent packages from `pip` automatically.
+The final step above should install any dependent packages from `pip` automatically. To see what packages are required, browse `setup.py`.
 
 Note that this step installs the base `aspire` package for you to work with, but not the unit tests/documentation. If you need to install ASPIRE for development purposes, read on.
 
 For developers
 --------------
 
-After cloning this repo, the simplest option is to use Anaconda 64-bit for your platform, and use the provided `environment.yml` file to build a Conda environment to run ASPIRE.
+After cloning this repo, the simplest option is to use Anaconda 64-bit for your platform, and use the provided `environment.yml` file to build a Conda environment to run ASPIRE. This is very similar to above except you will be based off of your local checkout, and you are free to rename `aspire_dev` used in the commands below. The `pip` line will install aspire in a locally editable mode, and is equivalent to `python setup.py develop`:
 
 ```
 cd /path/to/git/clone/folder
-conda env create -f environment.yml
-conda activate aspire
-```
 
-### Make sure everything works
+# Create's the conda environment and installs base dependencies.
+conda env create -f environment.yml --name aspire_dev
 
-Once ASPIRE is installed, make sure the unit tests run correctly on your platform by doing:
-```
-cd /path/to/git/clone/folder
-python setup.py test
+# Enable the environment
+conda activate aspire_dev
+
+# Install the aspire package in a locally editable way:
+pip install -e .
 ```
 
-Tests currently take around 2 minutes to run. If some tests fail, you may realize that `python setup.py test` produces too much information.
-You may want to re-run tests using:
+If you prefer not to use Anaconda, or want to manage environments yourself, you should be able to use `pip` with Python >= 3.6.
+Please see the full documentation for details.
+
+You may optionally install additional packages:
+
 ```
-cd /path/to/git/clone/folder
-PYTHONPATH=./src pytest tests
+# Additional GPU packages (reqires CUDA)
+pip install -e ".[gpu]"
+# Additional developer tools
+pip install -e ".[dev]"
 ```
-This provides a cleaner output to analyze.
 
-### Install
+### Make sure everything works
+
+Once ASPIRE is installed, make sure the unit tests run correctly on your platform by doing:
 
-If the tests pass, install the ASPIRE package for the currently active Conda environment:
 ```
 cd /path/to/git/clone/folder
-python setup.py install
+pytest
 ```