Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions .github/CI_TRIGGER_IMPLEMENTATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
# CI Trigger Implementation for External PRs

## Overview

This implementation solves the problem of CI not running for external contributor PRs due to GitHub Actions security restrictions that prevent access to repository secrets.

## Files Created

### 1. `.github/workflows/trigger-ci.yml`

**Purpose:** Main workflow that triggers CI for external PRs

**Trigger:** `issue_comment` event with `/run-ci` command

**Key Features:**
- Permission check: Verifies commenter has admin or write access
- Fetches external PR branch from fork
- Creates a new branch in main repo (pattern: `ci-test/{pr-number}-{timestamp}`)
- Creates a draft PR that triggers all existing CI workflows
- Comments on original PR with status
- Adds labels: `ci-test`, `automated`

**Security:**
- Only admin/write access users can trigger
- Fails gracefully with clear error message for unauthorized users
- Comments on PR to notify unauthorized attempts

### 2. `.github/workflows/cleanup-ci-test-prs.yml`

**Purpose:** Automatic cleanup of CI test PRs

**Trigger:** `workflow_run` event when "Tests" workflow completes

**Key Features:**
- Detects CI test branches (prefix: `ci-test/`)
- Comments on CI test PR with results (✅ or ❌)
- Closes the draft PR automatically
- Deletes the temporary branch
- Comments on original PR with final results
- Links to full test run

### 3. `.github/EXTERNAL_PR_CI.md`

**Purpose:** Comprehensive documentation for the feature

**Contents:**
- Problem statement
- Solution explanation
- How-to guide for admins
- Workflow details
- Security considerations
- Troubleshooting guide
- Limitations

### 4. Updated `README.md`

**Purpose:** Inform external contributors about the process

**Changes:**
- Added "For External Contributors" section
- Links to detailed documentation
- Explains that maintainers will trigger CI

## How It Works

### Flow Diagram

```
1. External Contributor submits PR
2. Admin reviews code
3. Admin comments "/run-ci" on PR
4. trigger-ci.yml workflow runs:
- Checks admin permissions ✓
- Fetches external branch
- Creates ci-test branch
- Creates draft PR
- Comments on original PR
5. All CI workflows run on draft PR
(with full access to secrets)
6. Tests complete (success or failure)
7. cleanup-ci-test-prs.yml workflow runs:
- Comments on draft PR with results
- Closes draft PR
- Deletes ci-test branch
- Comments on original PR with results
```

## Testing the Implementation

### Test Scenario 1: Authorized User

1. Create a test PR from a fork (or ask an external contributor)
2. Comment `/run-ci` on the PR as an admin
3. Expected results:
- New draft PR created with title `[CI Test] {original title}`
- Comment appears on original PR with success message
- CI workflows start running on draft PR
- After CI completes, draft PR is closed
- Original PR receives comment with results

### Test Scenario 2: Unauthorized User

1. Create a test PR from a fork
2. Comment `/run-ci` on the PR as a non-admin user
3. Expected results:
- Comment appears: "❌ Only repository admins..."
- No draft PR created
- Workflow fails with permission error

### Test Scenario 3: Not a PR Comment

1. Comment `/run-ci` on an issue (not a PR)
2. Expected results:
- Workflow doesn't run (filtered by `if` condition)

### Test Scenario 4: CI Cleanup

1. After a CI test PR completes:
2. Expected results:
- Draft PR gets comment with ✅ or ❌ status
- Draft PR is automatically closed
- Branch `ci-test/{number}-{timestamp}` is deleted
- Original PR receives comment with results link

## Security Considerations

### Why This Is Safe

1. **Permission Gating:** Only admin/write users can trigger
2. **Code Review Required:** Admins must manually review before triggering
3. **Audit Trail:** All actions are logged in PR comments
4. **Isolated Branches:** Each test uses a unique branch name
5. **Automatic Cleanup:** Temporary branches are deleted after use

### Risks to Be Aware Of

1. **Secret Exposure:** Malicious code in external PR could attempt to exfiltrate secrets
- Mitigation: Admins MUST review code before triggering
2. **Resource Usage:** Multiple CI runs increase GitHub Actions minutes
- Mitigation: Only trigger when necessary
3. **Branch Spam:** Could create many branches if used excessively
- Mitigation: Automatic cleanup workflow

## Workflow Permissions

Both workflows use these permissions:
```yaml
permissions:
contents: write # Create branches, delete branches
pull-requests: write # Create PRs, update PRs
issues: write # Create comments
```

## Integration with Existing CI

The implementation works seamlessly with existing CI:
- All existing workflows in `tests.yml` run on the draft PR
- E2E tests have access to secrets (VERCEL_LABS_TOKEN, etc.)
- Vercel deployments trigger automatically
- Results are reported back to original PR

## Future Enhancements

Potential improvements:
1. Add `/cancel-ci` command to stop running tests
2. Support for re-running specific failed jobs
3. Automatic retry on flaky test failures
4. Status checks on original PR that mirror draft PR status
5. Configurable retention period for CI branches
6. Support for multiple CI runs per PR with history

## Troubleshooting

### Common Issues

**Issue:** Branch already exists error
- **Cause:** Timestamp collision (very rare)
- **Solution:** Wait 1 second and retry `/run-ci`

**Issue:** Cannot fetch external branch
- **Cause:** Fork is private or deleted
- **Solution:** Ask contributor to make fork public

**Issue:** Draft PR not created
- **Cause:** Base branch protected, insufficient permissions
- **Solution:** Check GitHub Actions logs for specific error

## Monitoring

To monitor usage:
1. Check Actions tab for "Trigger CI for External PRs" runs
2. Search for PRs with label `ci-test`
3. Review comments from `github-actions` bot

## Maintenance

### Updating the Workflows

If you need to modify the workflows:
1. Test changes on a fork first
2. Be careful with permissions
3. Update this documentation

### Dependencies

The workflows depend on:
- `actions/checkout@v4`
- `actions/github-script@v7`
- `git` command-line tool (built-in)

## Questions?

For questions or issues with this implementation:
- Open a GitHub Discussion
- Create an issue with label `ci-automation`
- Contact the repository maintainers

103 changes: 103 additions & 0 deletions .github/EXTERNAL_PR_CI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Running CI for External Contributor PRs

## Problem

When external contributors (non-members) submit pull requests, GitHub Actions has security restrictions that prevent:

1. Vercel deployments from automatically running
2. Secret environment variables (like `VERCEL_LABS_TOKEN`, `TURBO_TOKEN`) from being injected into workflows

This means E2E tests and other CI checks that depend on these secrets will fail or not run at all.

## Solution

We've implemented a `/run-ci` command that repository admins can use to trigger CI for external PRs.

## How It Works

### For Repository Admins

When an external contributor submits a PR:

1. Review the PR code for any malicious content (this is important for security!)
2. Comment `/run-ci` on the PR
3. The workflow will:
- Verify you have admin/write permissions
- Create a new branch in the main repository based on the external PR's branch
- Create a draft PR from that branch
- Run all CI checks with full access to secrets
4. Once CI completes, you'll get a notification on the original PR with the results
5. The draft PR will be automatically closed and the branch deleted

### Workflow Details

**Trigger Workflow** (`.github/workflows/trigger-ci.yml`):
- Triggered by: PR comments containing `/run-ci`
- Permissions required: Admin or Write access
- Creates: A draft PR with the naming pattern `[CI Test] {original PR title}`
- Labels: `ci-test`, `automated`

**Cleanup Workflow** (`.github/workflows/cleanup-ci-test-prs.yml`):
- Triggered by: Completion of the "Tests" workflow
- Automatically closes CI test PRs
- Deletes the temporary CI test branches
- Comments on both the CI test PR and original PR with results

## Security Considerations

⚠️ **Important Security Notes:**

1. **Only admins/maintainers should trigger CI** - The `/run-ci` command requires admin or write permissions
2. **Review code before triggering** - Always review the PR code before running CI, as it will have access to repository secrets
3. **Malicious code risk** - External PRs could contain malicious code that attempts to exfiltrate secrets
4. **Branch protection** - The main branch should have branch protection rules enabled

## Example Usage

```markdown
Comment on PR #123:

/run-ci
```

Response:

```markdown
✅ CI test triggered by @admin-username!

CI is now running in draft PR #456. You can monitor the progress there.

Once the tests complete, you can review the results and the draft PR will be automatically closed.
```

## Branch Naming Convention

CI test branches follow the pattern:
```
ci-test/{original-pr-number}-{timestamp}
```

Example: `ci-test/123-1699876543210`

## Troubleshooting

### "Insufficient permissions" error

Only repository admins and members with write access can trigger CI. If you see this error, you don't have the required permissions.

### CI test PR not created

1. Check that the comment was on a pull request (not an issue)
2. Verify the exact text `/run-ci` was in the comment
3. Check the GitHub Actions logs for the "Trigger CI for External PRs" workflow

### Branch conflicts

If the external PR's branch has conflicts with the base branch, the CI test PR will also have those conflicts. The contributor should resolve conflicts in their original PR first.

## Limitations

1. The external contributor's branch must be accessible (public fork or within the same organization)
2. CI tests will run against the code at the time `/run-ci` was triggered. If the contributor pushes new commits, you'll need to run `/run-ci` again
3. Only one CI test can be running per PR at a time (subsequent `/run-ci` commands will create new test PRs)

Loading
Loading