From db76291f5b74c41862dac3f8039555f6e0352f87 Mon Sep 17 00:00:00 2001 From: Julian Simioni Date: Fri, 13 Mar 2020 14:22:22 -0700 Subject: [PATCH 1/2] WIP --- README.md | 3 + development/guide.md | 172 +++++++++++++++++++++++++++++++++++ development/project_infra.md | 9 ++ 3 files changed, 184 insertions(+) create mode 100644 development/guide.md create mode 100644 development/project_infra.md diff --git a/README.md b/README.md index 43df6fa..e7c7b03 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,9 @@ _Not sure which Endpoint to use? We have a [page](search-workflows.md) for that_ - [Software Requirements](requirements.md) A list of all software requirements for Pelias ### Pelias project development +- [Developer guide](/development/guide.md). Guide to submitting changes to Pelias code. +- [Project infrastructure](/development/project_infra.md). Guide to Pelias build tools, continuous integration, Docker image, etc. +- [Contributor Status](/development/contributor.md). Repeat contributors to Pelias are granted additional permissions, explained here! - [Release notes](release-notes.md). See notable changes in Pelias over time - [Development roadmap](development/roadmap.md). Plans for future improvements to Pelias. Read this to see what's coming and how you can help diff --git a/development/guide.md b/development/guide.md new file mode 100644 index 0000000..a2abbeb --- /dev/null +++ b/development/guide.md @@ -0,0 +1,172 @@ +# Pelias Developer Guide + +So, you want to help out with code changes to Pelias? Great! + +This guide is meant to help you make effective, efficient changes that can be +accepted into the Pelias project. + +## General Guidelines + +### Use our test suites to help you + +All Pelias modules come with a test suite to help ensure your changes do not break existing +functionality. We encourage you to extend the test suites for new features or bug fixes you submit. + +Generally, these test suites fall into two categories: unit tests and functional (sometimes called +end-to-end or integration) tests. + +Unit tests should run in just a few seconds, while functional tests may take a few minutes. + +You can always run just a projects unit tests with: + +``` +npm test +``` + +You can run the functional tests with one of the following, depending on the repository + +``` +npm run functional +npm run integration +npm run end-to-end +``` + +The functional tests may take a few minutes to run, which is why we've separated them out. + +### Take advantage of our pre-commit hooks + +Most of our repositories use the [pre-commit](https://www.npmjs.com/package/pre-commit) Node.js +module to run various checks before allowing you to commit your code. Please follow these pre-commit hooks! + +Currently our pre-commit hooks generally test: + +- That the unit tests pass +- That there are no missing or extraneous NPM dependencies +- That the code passes our linter rules. + +Note that if you really need to, you can bypass them with `git commit --no-verify`. This can be +useful if you want to test something out real quick, but plan to clean it up later before submitting +the change. + +### Releases are driven by commit messages + +We use the wonderful [Semantic Release](https://github.com/semantic-release/semantic-release)) +project to manage releasing new versions of all our projects. + +Semantic Release looks for a [certain commit message format](https://github.com/semantic-release/semantic-release#commit-message-format) to decide when to release. + +To help you decide which one to use, we'll list our general guidelines below. Feel free to include +these in your commit messages if you feel confident. If not, the core team will take care of it. + +The general format is: + +``` +type(component): Message +``` + +Where `type` is one of the below commit message types, `component` is a description of what area of +the codebase affects, and `Message` is a normal commit message. + +#### Chores + +[Chores](https://blog.carbonfive.com/2020/02/24/what-are-these-chores-doing-in-my-backlog/) are +anything that don't affect the behavior of Pelias code, but help keep the project a healthy and +efficient place to work. + +Another way to look at chores is they are changes to the codebase that would not warrant a new +release at all. + +Good examples include: + +- Fixing typos in documentation +- Updating settings in our CI infrastructure +- Removing dead or unused code + +A commit message for a chore looks like this: + +``` +chore(CI): Remove deprecated Travis CI configuration option +``` + +#### Bug fixes + +Bug fixes correct existing errors in the behavior of code. + +A bugfix commit looks something like this: + +``` +fix(query): Ensure Elasticsearch queries handle empty value +``` + +#### Features + +Features are the good stuff! They add new behavior that helps the project or improve how it behaves. + +A feature commit usually looks like this: + +``` +feat(sql): Improve performance of SQL queries +``` + +#### Breaking Changes + +Any of the above types of commits can also be marked as a breaking change. Breaking changes are used +to signal to users or modules that depend on the software that a backwards-incompatible change has +been made. In Pelias, this will trigger a new _major version_ release. + +Breaking changes are created by ensuring that the term `BREAKING CHANGE:` occurs in the _body_ of +the commit message. This means you can't trigger a breaking change with something like `git commit +-m "my commit message"`, you have to run `git commit` and let it open your editor to write the +commit message. + +Also, note the colon at the end of the string. We've often missed breaking changes because we only +wrote `BREAKING CHANGE`. + +A breaking change commit looks like this: + + +``` +feat(services): Add support for libpostal service + +BREAKING CHANGE: this project now requires the Libpostal Service to be present in order to work +``` + +## Best practices for Pull Requests + +#### Reach out to us first before doing a lot of work + +Nothing makes us more sad than saying no to a pull request that someone worked hard on. + +If you're ever in doubt about whether or not making a change is worthwhile, ask us first. Open an +issue, email the core team, etc. + +We'll discuss with you what makes sense and why, give you context about our ideas for the project +going forward that might influence your work, and discuss any plans you have before you proceed. + +We're also happy to review "draft" or unfinished PRs before they are done, so we can give input on +whether or not a particular approach is looking promising. + +#### Keep your pull requests small + +We always appreciate small pull requests that focus on a specific change. This makes it easier to +test and review. + +Whenever possible, we support your efforts to split one large pull request into several smaller +ones. + +#### Rebase against master + +If other changes have been made while your Pull Request is in progress, please occasionally update +your PR to rebase against those new changes. + +This helps ensure that we can review your changes effectively, that your changes will merge as +intended when the time comes, and, **most importantly**, that the project commit history is clean so +that its easy to figure out how and why changes came in when working in the future. + +You can generally do this with `git rebase origin master`. + +#### Allow changes to your pull requests + +GitHub allows maintainers to make changes to a pull request created by another user. Please [enable +this +functionality](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork) as we often want to make little changes to your pull request rather than go back and forth over them. diff --git a/development/project_infra.md b/development/project_infra.md new file mode 100644 index 0000000..a7a26e2 --- /dev/null +++ b/development/project_infra.md @@ -0,0 +1,9 @@ +# Pelias Project Infrastructure + +The Pelias project utilizes lots of outside services to aid in development. + +## Continuous Integration + +## Docker images + +## Future work: full end to end testing From ea1eb1d6566718ac52b7cb173790610ac19903fa Mon Sep 17 00:00:00 2001 From: Julian Simioni Date: Mon, 23 Mar 2020 13:51:07 -0700 Subject: [PATCH 2/2] WIP 2 --- development/contributor.md | 16 +++++ development/guide.md | 116 +++++++++----------------------- development/project_infra.md | 15 +++++ development/tagging_releases.md | 82 ++++++++++++++++++++++ 4 files changed, 144 insertions(+), 85 deletions(-) create mode 100644 development/contributor.md create mode 100644 development/tagging_releases.md diff --git a/development/contributor.md b/development/contributor.md new file mode 100644 index 0000000..83ade59 --- /dev/null +++ b/development/contributor.md @@ -0,0 +1,16 @@ +# Pelias Contributor + +We want to make it as easy as possible for people to contribute to the Pelias +project. + +For repeat contributors, we generally extend an invitation to join the [Pelias +contributors GitHub team](https://github.com/orgs/pelias/teams/contributors). + +Joining this team allows you to create new branches and pull requests +_directly_ in repositories that are a part of the Pelias org. Hopefully this +saves some time and hassle! + +Because contributors can create branches in Pelias repositories, our [project +infrastruture](./project_infra.md) will also ensure that Docker images are +automatically built and generated for those changes. This can greatly speed +development in combination with our [Docker projects](https://github.com/pelias/docker). diff --git a/development/guide.md b/development/guide.md index a2abbeb..d9bdce4 100644 --- a/development/guide.md +++ b/development/guide.md @@ -50,103 +50,29 @@ the change. ### Releases are driven by commit messages -We use the wonderful [Semantic Release](https://github.com/semantic-release/semantic-release)) -project to manage releasing new versions of all our projects. +We manage Pelias releases through commit message contents using [Semantic Release](https://github.com/semantic-release/semantic-release). -Semantic Release looks for a [certain commit message format](https://github.com/semantic-release/semantic-release#commit-message-format) to decide when to release. - -To help you decide which one to use, we'll list our general guidelines below. Feel free to include -these in your commit messages if you feel confident. If not, the core team will take care of it. - -The general format is: - -``` -type(component): Message -``` - -Where `type` is one of the below commit message types, `component` is a description of what area of -the codebase affects, and `Message` is a normal commit message. - -#### Chores - -[Chores](https://blog.carbonfive.com/2020/02/24/what-are-these-chores-doing-in-my-backlog/) are -anything that don't affect the behavior of Pelias code, but help keep the project a healthy and -efficient place to work. - -Another way to look at chores is they are changes to the codebase that would not warrant a new -release at all. - -Good examples include: - -- Fixing typos in documentation -- Updating settings in our CI infrastructure -- Removing dead or unused code - -A commit message for a chore looks like this: - -``` -chore(CI): Remove deprecated Travis CI configuration option -``` - -#### Bug fixes - -Bug fixes correct existing errors in the behavior of code. - -A bugfix commit looks something like this: - -``` -fix(query): Ensure Elasticsearch queries handle empty value -``` - -#### Features - -Features are the good stuff! They add new behavior that helps the project or improve how it behaves. - -A feature commit usually looks like this: - -``` -feat(sql): Improve performance of SQL queries -``` - -#### Breaking Changes - -Any of the above types of commits can also be marked as a breaking change. Breaking changes are used -to signal to users or modules that depend on the software that a backwards-incompatible change has -been made. In Pelias, this will trigger a new _major version_ release. - -Breaking changes are created by ensuring that the term `BREAKING CHANGE:` occurs in the _body_ of -the commit message. This means you can't trigger a breaking change with something like `git commit --m "my commit message"`, you have to run `git commit` and let it open your editor to write the -commit message. - -Also, note the colon at the end of the string. We've often missed breaking changes because we only -wrote `BREAKING CHANGE`. - -A breaking change commit looks like this: - - -``` -feat(services): Add support for libpostal service - -BREAKING CHANGE: this project now requires the Libpostal Service to be present in order to work -``` +If you'd like, you can help us make that task easier by following our [release +tagging guide](./tagging_releases.md). But don't worry too much about it, the core +team can always make any needed tags before merging your pull request. ## Best practices for Pull Requests -#### Reach out to us first before doing a lot of work +### Reach out to us first before doing a lot of work Nothing makes us more sad than saying no to a pull request that someone worked hard on. If you're ever in doubt about whether or not making a change is worthwhile, ask us first. Open an issue, email the core team, etc. -We'll discuss with you what makes sense and why, give you context about our ideas for the project -going forward that might influence your work, and discuss any plans you have before you proceed. +We'll discuss what makes sense and why, give you context about our ideas for +the project going forward that might influence your work, and discuss any plans +you have before you proceed. We're also happy to review "draft" or unfinished PRs before they are done, so we can give input on whether or not a particular approach is looking promising. -#### Keep your pull requests small +### Keep your pull requests small We always appreciate small pull requests that focus on a specific change. This makes it easier to test and review. @@ -154,7 +80,7 @@ test and review. Whenever possible, we support your efforts to split one large pull request into several smaller ones. -#### Rebase against master +### Rebase against master If other changes have been made while your Pull Request is in progress, please occasionally update your PR to rebase against those new changes. @@ -165,8 +91,28 @@ that its easy to figure out how and why changes came in when working in the futu You can generally do this with `git rebase origin master`. -#### Allow changes to your pull requests +### Allow changes to your pull requests GitHub allows maintainers to make changes to a pull request created by another user. Please [enable this functionality](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork) as we often want to make little changes to your pull request rather than go back and forth over them. + +## Commonly used tools + +As a large project with many smaller components, we try to keep the Pelias +development process as consistent as possible. + +This section lists various tools and utilities that are common in Pelias. Where +possible, use these tools. + +If one of these tools isn't suitable for a particular purpose, ideally we would +settle on a new option for the entire project. + +| Functionality | Pelias Standard | +| --- | --- | +| Javascript Linting | [JSHint](https://github.com/jshint/jshint/) | +| Test framework | [Tape](https://github.com/substack/tape) | +| Node.js stream library | [through2](https://github.com/rvagg/through2) | +| General utility library | [lodash](https://github.com/lodash/lodash) | +| SQLite library | [better-sqlite3](https://github.com/JoshuaWise/better-sqlite3) | + diff --git a/development/project_infra.md b/development/project_infra.md index a7a26e2..7f5f44a 100644 --- a/development/project_infra.md +++ b/development/project_infra.md @@ -4,6 +4,21 @@ The Pelias project utilizes lots of outside services to aid in development. ## Continuous Integration +All branches and pull requests opened on Pelias projects run our test suites through TravisCI. + +This includes both unit tests, which run quickly, and the longer-running functional tests. + ## Docker images +Any branch created in a Pelias GitHub repository will have several [Docker images created](https://github.com/pelias/ci-tools#build-docker-images). + +The images will be tagged in the following formats, including the Pelias repository, the branch name, the date the image was built, and the full git commit hash. + +``` +pelias/repository:branch +pelias/repository:branch-date-commit +``` + ## Future work: full end to end testing + +In the future, we hope to build systems for running complete end to end tests of a small Pelias build using our [docker projects](https://github.com/pelias/docker). For some ideas, see our discussion on [city acceptance tests](https://github.com/pelias/pelias/issues/718). diff --git a/development/tagging_releases.md b/development/tagging_releases.md new file mode 100644 index 0000000..169f531 --- /dev/null +++ b/development/tagging_releases.md @@ -0,0 +1,82 @@ +## Tagging releases with Semantic Release + +We use the wonderful [Semantic Release](https://github.com/semantic-release/semantic-release) +project to manage releasing new versions of all our projects. + +Semantic Release looks for a [certain commit message format](https://github.com/semantic-release/semantic-release#commit-message-format) to decide when to release. + +To help you decide which one to use, we'll list our general guidelines below. Feel free to include +these in your commit messages if you feel confident. If not, the core team will take care of it. + +The general format is: + +``` +type(component): Message +``` + +Where `type` is one of the below commit message types, `component` is a description of what area of +the codebase affects, and `Message` is a normal commit message. + +#### Chores + +[Chores](https://blog.carbonfive.com/2020/02/24/what-are-these-chores-doing-in-my-backlog/) are +anything that don't affect the behavior of Pelias code, but help keep the project a healthy and +efficient place to work. + +Another way to look at chores is they are changes to the codebase that would not warrant a new +release at all. + +Good examples include: + +- Fixing typos in documentation +- Updating settings in our CI infrastructure +- Removing dead or unused code + +A commit message for a chore looks like this: + +``` +chore(CI): Remove deprecated Travis CI configuration option +``` + +#### Bug fixes + +Bug fixes correct existing errors in the behavior of code. + +A bugfix commit looks something like this: + +``` +fix(query): Ensure Elasticsearch queries handle empty value +``` + +#### Features + +Features are the good stuff! They add new behavior that helps the project or improve how it behaves. + +A feature commit usually looks like this: + +``` +feat(sql): Improve performance of SQL queries +``` + +#### Breaking Changes + +Any of the above types of commits can also be marked as a breaking change. Breaking changes are used +to signal to users or modules that depend on the software that a backwards-incompatible change has +been made. In Pelias, this will trigger a new _major version_ release. + +Breaking changes are created by ensuring that the term `BREAKING CHANGE:` occurs in the _body_ of +the commit message. This means you can't trigger a breaking change with something like `git commit +-m "my commit message"`, you have to run `git commit` and let it open your editor to write the +commit message. + +Also, note the colon at the end of the string. We've often missed breaking changes because we only +wrote `BREAKING CHANGE`. + +A breaking change commit looks like this: + + +``` +feat(services): Add support for libpostal service + +BREAKING CHANGE: this project now requires the Libpostal Service to be present in order to work +```