-
Notifications
You must be signed in to change notification settings - Fork 78
Feature/workspaces #152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gautambaghel
wants to merge
12
commits into
main
Choose a base branch
from
feature/workspaces
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,014
−17
Open
Feature/workspaces #152
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
9a55f49
feat: adding workspace tools
gautambaghel c747e8b
adding tests
gautambaghel 95007a2
Merge branch 'main' into feature/workspaces
gautambaghel a6a19ce
stateless fix
gautambaghel 93022e1
Merge branch 'main' into feature/workspaces
gautambaghel 0ad9704
fixing folder struct
gautambaghel 8d989ea
updating tool responses to JSON instead of markdown
gautambaghel f5fab0f
adding id to project
gautambaghel f1dac1b
Merge branch 'main' into feature/workspaces
gautambaghel 1c361c0
Apply suggestion from @mayank-sisodiya
gautambaghel 2d1ac36
implementing fixes, search -> list for tools
gautambaghel 565f705
adding project_id for search/list criteria
gautambaghel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
# Running tests | ||
|
||
The HCP Terraform and TFE tools for the `terraform-mcp-server` relies on acceptance tests against their respective APIs. terraform-mcp-server is tested against HCP Terraform by our CI environment, and against Terraform Enterprise prior to release or otherwise as needed. | ||
|
||
## 1. (Optional) Create repositories for policy sets and registry modules | ||
|
||
If you are planning to run the full suite of tests or work on policy sets or registry modules, you'll need to set up repositories for them in GitHub. | ||
|
||
Your policy set repository will need the following: | ||
1. A policy set stored in a subdirectory `policy-sets/foo` | ||
1. A branch other than `main` named `policies` | ||
|
||
Alternatively, you can start with this [example repository for policy sets](https://github.com/hashicorp/test-policy-set) by forking the repository to your GitHub account, then setting `GITHUB_POLICY_SET_IDENTIFIER` to the forked repository identifier `your-github-handle/test-policy-set`. | ||
|
||
Your registry module repository will need to be a [valid module](https://developer.hashicorp.com/terraform/cloud-docs/registry/publish-modules#preparing-a-module-repository). | ||
It will need the following: | ||
1. To be named `terraform-<PROVIDER>-<NAME>` | ||
1. At least one valid SemVer tag in the format `x.y.z` | ||
[terraform-random-module](https://github.com/caseylang/terraform-random-module) is a good example repo. | ||
|
||
## 2. Set up environment variables (ENVVARS) | ||
|
||
You'll need to have environment variables setup in your environment to run the tests. There are different options to facilitate setting up environment variables, using the tool [envchain](https://github.com/sorah/envchain) is one option: | ||
1. Install envchain - [refer to the envchain README for details](https://github.com/sorah/envchain#installation) | ||
2. Pick a namespace for storing your environment variables, such as: `terraform-mcp-server`. Then, for each environment variable you need to set, run the following command: | ||
```sh | ||
envchain --set YOUR_NAMESPACE_HERE ENVIRONMENT_VARIABLE_HERE | ||
``` | ||
**OR** | ||
|
||
Set all of the environment variables at once with the following command: | ||
```sh | ||
envchain --set YOUR_NAMESPACE_HERE TFE_ADDRESS TFE_TOKEN OAUTH_CLIENT_GITHUB_TOKEN GITHUB_POLICY_SET_IDENTIFIER | ||
``` | ||
|
||
### Required ENVVARS | ||
|
||
Tests are run against an actual backend so they require a valid backend address and token: | ||
|
||
1. `TFE_ADDRESS` - URL of a HCP Terraform or Terraform Enterprise instance to be used for testing, including scheme. Example: `https://tfe.local` | ||
1. `TFE_TOKEN` - A [user API token](https://developer.hashicorp.com/terraform/cloud-docs/users-teams-organizations/users#tokens) for the HCP Terraform or Terraform Enterprise instance being used for testing. | ||
|
||
**Note:** Alternatively, you can set `TFE_HOSTNAME` which serves as a fallback for `TFE_ADDRESS`. It will only be used if `TFE_ADDRESS` is not set and will resolve the host to an `https` scheme. Example: `tfe.local` => resolves to `https://tfe.local` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see any occurrence of TFE_HOSTNAME is this repo, where is this fallback handled? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
## 3. Make sure run queue settings are correct | ||
|
||
In order for the tests relating to queuing and capacity to pass, FRQ (fair run queuing) should be | ||
enabled with a limit of 2 concurrent runs per organization on the HCP Terraform or Terraform Enterprise instance you are using for testing. | ||
|
||
## 4. Run tests | ||
|
||
For most situations, it's recommended to run specific tests because it takes about 20 minutes to run all of the tests. | ||
|
||
### Running specific tests | ||
|
||
Typically, you'll want to run specific tests. The commands below use notification configurations as an example. | ||
|
||
#### With envchain: | ||
```sh | ||
$ envchain YOUR_NAMESPACE_HERE go test -run TestNotificationConfiguration -v ./... | ||
``` | ||
|
||
#### Without envchain (Using TFE_ADDRESS): | ||
```sh | ||
$ TFE_TOKEN=xyz TFE_ADDRESS=https://tfe.local ENABLE_TFE=1 go test -run TestNotificationConfiguration -v ./... | ||
``` | ||
|
||
#### Without envchain (Using TFE_HOSTNAME): | ||
```sh | ||
$ TFE_TOKEN=xyz TFE_HOSTNAME=tfe.local ENABLE_TFE=1 go test -run TestNotificationConfiguration -v ./... | ||
``` | ||
|
||
#### Using Makefile target `test` | ||
```sh | ||
TFE_TOKEN=xyz TFE_ADDRESS=https://tfe.local TESTARGS="-run TestNotificationConfiguration" make test | ||
``` | ||
|
||
### Running all tests | ||
It takes about 20 minutes to run all of the tests, so specify a larger timeout when you run the tests (_the default timeout is 10 minutes_): | ||
|
||
#### With envchain: | ||
```sh | ||
$ envchain YOUR_NAMESPACE_HERE go test ./... -timeout=30m | ||
``` | ||
|
||
#### Without envchain (Using TFE_ADDRESS): | ||
```sh | ||
$ TFE_TOKEN=xyz TFE_ADDRESS=https://tfe.local ENABLE_TFE=1 go test ./... -timeout=30m | ||
``` | ||
|
||
#### Without envchain (Using TFE_HOSTNAME): | ||
```sh | ||
$ TFE_TOKEN=xyz TFE_HOSTNAME=tfe.local ENABLE_TFE=1 go test ./... -timeout=30m | ||
``` | ||
|
||
|
||
### Running tests for HCP Terraform features that require paid plans (HashiCorp Employees) | ||
|
||
You can use the test helper `newSubscriptionUpdater()` to upgrade your test organization to a Business Plan, giving the organization access to all features in HCP Terraform. This method requires `TFE_TOKEN` to be a user token with administrator access in the target test environment. Furthermore, you **can not** have enterprise features enabled (`ENABLE_TFE=1`) in order to use this method since the API call fails against Terraform Enterprise test environments. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these the instructions to run e2e tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, this is to test with actual HCP Terraform tokens, I haven't added live testing yet (only in/out mock tests), but we should start adding live HCP TF tests to make sure tools are running they way they're supposed to
it should be similar to
tfe-go
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
local testing for now but we should add them to GitHub Actions