diff --git a/README.md b/README.md index 43df6fa..e7c7b03 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,9 @@ _Not sure which Endpoint to use? We have a [page](search-workflows.md) for that_ - [Software Requirements](requirements.md) A list of all software requirements for Pelias ### Pelias project development +- [Developer guide](/development/guide.md). Guide to submitting changes to Pelias code. +- [Project infrastructure](/development/project_infra.md). Guide to Pelias build tools, continuous integration, Docker image, etc. +- [Contributor Status](/development/contributor.md). Repeat contributors to Pelias are granted additional permissions, explained here! - [Release notes](release-notes.md). See notable changes in Pelias over time - [Development roadmap](development/roadmap.md). Plans for future improvements to Pelias. Read this to see what's coming and how you can help diff --git a/development/contributor.md b/development/contributor.md new file mode 100644 index 0000000..83ade59 --- /dev/null +++ b/development/contributor.md @@ -0,0 +1,16 @@ +# Pelias Contributor + +We want to make it as easy as possible for people to contribute to the Pelias +project. + +For repeat contributors, we generally extend an invitation to join the [Pelias +contributors GitHub team](https://github.com/orgs/pelias/teams/contributors). + +Joining this team allows you to create new branches and pull requests +_directly_ in repositories that are a part of the Pelias org. Hopefully this +saves some time and hassle! + +Because contributors can create branches in Pelias repositories, our [project +infrastruture](./project_infra.md) will also ensure that Docker images are +automatically built and generated for those changes. This can greatly speed +development in combination with our [Docker projects](https://github.com/pelias/docker). diff --git a/development/guide.md b/development/guide.md new file mode 100644 index 0000000..d9bdce4 --- /dev/null +++ b/development/guide.md @@ -0,0 +1,118 @@ +# Pelias Developer Guide + +So, you want to help out with code changes to Pelias? Great! + +This guide is meant to help you make effective, efficient changes that can be +accepted into the Pelias project. + +## General Guidelines + +### Use our test suites to help you + +All Pelias modules come with a test suite to help ensure your changes do not break existing +functionality. We encourage you to extend the test suites for new features or bug fixes you submit. + +Generally, these test suites fall into two categories: unit tests and functional (sometimes called +end-to-end or integration) tests. + +Unit tests should run in just a few seconds, while functional tests may take a few minutes. + +You can always run just a projects unit tests with: + +``` +npm test +``` + +You can run the functional tests with one of the following, depending on the repository + +``` +npm run functional +npm run integration +npm run end-to-end +``` + +The functional tests may take a few minutes to run, which is why we've separated them out. + +### Take advantage of our pre-commit hooks + +Most of our repositories use the [pre-commit](https://www.npmjs.com/package/pre-commit) Node.js +module to run various checks before allowing you to commit your code. Please follow these pre-commit hooks! + +Currently our pre-commit hooks generally test: + +- That the unit tests pass +- That there are no missing or extraneous NPM dependencies +- That the code passes our linter rules. + +Note that if you really need to, you can bypass them with `git commit --no-verify`. This can be +useful if you want to test something out real quick, but plan to clean it up later before submitting +the change. + +### Releases are driven by commit messages + +We manage Pelias releases through commit message contents using [Semantic Release](https://github.com/semantic-release/semantic-release). + +If you'd like, you can help us make that task easier by following our [release +tagging guide](./tagging_releases.md). But don't worry too much about it, the core +team can always make any needed tags before merging your pull request. + +## Best practices for Pull Requests + +### Reach out to us first before doing a lot of work + +Nothing makes us more sad than saying no to a pull request that someone worked hard on. + +If you're ever in doubt about whether or not making a change is worthwhile, ask us first. Open an +issue, email the core team, etc. + +We'll discuss what makes sense and why, give you context about our ideas for +the project going forward that might influence your work, and discuss any plans +you have before you proceed. + +We're also happy to review "draft" or unfinished PRs before they are done, so we can give input on +whether or not a particular approach is looking promising. + +### Keep your pull requests small + +We always appreciate small pull requests that focus on a specific change. This makes it easier to +test and review. + +Whenever possible, we support your efforts to split one large pull request into several smaller +ones. + +### Rebase against master + +If other changes have been made while your Pull Request is in progress, please occasionally update +your PR to rebase against those new changes. + +This helps ensure that we can review your changes effectively, that your changes will merge as +intended when the time comes, and, **most importantly**, that the project commit history is clean so +that its easy to figure out how and why changes came in when working in the future. + +You can generally do this with `git rebase origin master`. + +### Allow changes to your pull requests + +GitHub allows maintainers to make changes to a pull request created by another user. Please [enable +this +functionality](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork) as we often want to make little changes to your pull request rather than go back and forth over them. + +## Commonly used tools + +As a large project with many smaller components, we try to keep the Pelias +development process as consistent as possible. + +This section lists various tools and utilities that are common in Pelias. Where +possible, use these tools. + +If one of these tools isn't suitable for a particular purpose, ideally we would +settle on a new option for the entire project. + +| Functionality | Pelias Standard | +| --- | --- | +| Javascript Linting | [JSHint](https://github.com/jshint/jshint/) | +| Test framework | [Tape](https://github.com/substack/tape) | +| Node.js stream library | [through2](https://github.com/rvagg/through2) | +| General utility library | [lodash](https://github.com/lodash/lodash) | +| SQLite library | [better-sqlite3](https://github.com/JoshuaWise/better-sqlite3) | + diff --git a/development/project_infra.md b/development/project_infra.md new file mode 100644 index 0000000..7f5f44a --- /dev/null +++ b/development/project_infra.md @@ -0,0 +1,24 @@ +# Pelias Project Infrastructure + +The Pelias project utilizes lots of outside services to aid in development. + +## Continuous Integration + +All branches and pull requests opened on Pelias projects run our test suites through TravisCI. + +This includes both unit tests, which run quickly, and the longer-running functional tests. + +## Docker images + +Any branch created in a Pelias GitHub repository will have several [Docker images created](https://github.com/pelias/ci-tools#build-docker-images). + +The images will be tagged in the following formats, including the Pelias repository, the branch name, the date the image was built, and the full git commit hash. + +``` +pelias/repository:branch +pelias/repository:branch-date-commit +``` + +## Future work: full end to end testing + +In the future, we hope to build systems for running complete end to end tests of a small Pelias build using our [docker projects](https://github.com/pelias/docker). For some ideas, see our discussion on [city acceptance tests](https://github.com/pelias/pelias/issues/718). diff --git a/development/tagging_releases.md b/development/tagging_releases.md new file mode 100644 index 0000000..169f531 --- /dev/null +++ b/development/tagging_releases.md @@ -0,0 +1,82 @@ +## Tagging releases with Semantic Release + +We use the wonderful [Semantic Release](https://github.com/semantic-release/semantic-release) +project to manage releasing new versions of all our projects. + +Semantic Release looks for a [certain commit message format](https://github.com/semantic-release/semantic-release#commit-message-format) to decide when to release. + +To help you decide which one to use, we'll list our general guidelines below. Feel free to include +these in your commit messages if you feel confident. If not, the core team will take care of it. + +The general format is: + +``` +type(component): Message +``` + +Where `type` is one of the below commit message types, `component` is a description of what area of +the codebase affects, and `Message` is a normal commit message. + +#### Chores + +[Chores](https://blog.carbonfive.com/2020/02/24/what-are-these-chores-doing-in-my-backlog/) are +anything that don't affect the behavior of Pelias code, but help keep the project a healthy and +efficient place to work. + +Another way to look at chores is they are changes to the codebase that would not warrant a new +release at all. + +Good examples include: + +- Fixing typos in documentation +- Updating settings in our CI infrastructure +- Removing dead or unused code + +A commit message for a chore looks like this: + +``` +chore(CI): Remove deprecated Travis CI configuration option +``` + +#### Bug fixes + +Bug fixes correct existing errors in the behavior of code. + +A bugfix commit looks something like this: + +``` +fix(query): Ensure Elasticsearch queries handle empty value +``` + +#### Features + +Features are the good stuff! They add new behavior that helps the project or improve how it behaves. + +A feature commit usually looks like this: + +``` +feat(sql): Improve performance of SQL queries +``` + +#### Breaking Changes + +Any of the above types of commits can also be marked as a breaking change. Breaking changes are used +to signal to users or modules that depend on the software that a backwards-incompatible change has +been made. In Pelias, this will trigger a new _major version_ release. + +Breaking changes are created by ensuring that the term `BREAKING CHANGE:` occurs in the _body_ of +the commit message. This means you can't trigger a breaking change with something like `git commit +-m "my commit message"`, you have to run `git commit` and let it open your editor to write the +commit message. + +Also, note the colon at the end of the string. We've often missed breaking changes because we only +wrote `BREAKING CHANGE`. + +A breaking change commit looks like this: + + +``` +feat(services): Add support for libpostal service + +BREAKING CHANGE: this project now requires the Libpostal Service to be present in order to work +```