Add Claude Code GitHub Workflow #906
Conversation
daniel-noland
force-pushed
the
add-claude-github-actions-1760391654541
branch
from
40dad39 to
1f788c2
Compare
|
@claude comment "potato" if you can see this message |
daniel-noland
requested review from
Fredi-raspall
and removed request for
a team
daniel-noland
added
ci
|
@claude please review this PR |
|
It seems this needs to be merged before the bot will actually issue a review. Quite annoying |
Machine generated commit to add claude pr assistant to the ci workflow. This will require some adjustment but is hopefully helpful for our review process. Signed-off-by: Daniel Noland <[email protected]>
Another machine generated commit to add claude support to the workflow. Signed-off-by: Daniel Noland <[email protected]>
Configure claude to use specific agents for more focused reviews. Signed-off-by: Daniel Noland <[email protected]>
daniel-noland
force-pushed
the
add-claude-github-actions-1760391654541
branch
from
3de2e44 to
37acfc2
Compare
|
@claude can you review this PR now? |
There was a problem hiding this comment.
Pull Request Overview
Adds GitHub Actions workflows to integrate Claude Code for on-demand assistance and automated PR reviews, plus agent configuration files to guide review behavior.
- Introduces a comment/mention-triggered workflow to run Claude Code.
- Adds an automated code review workflow for PRs with a tailored prompt and tool permissions.
- Provides agent prompt files for structured reviews (security, style, logic, docs, tests, nits, and coordinator).
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/claude.yml | New workflow to run Claude Code when @claude is mentioned in issues, PR reviews, or comments. |
| .github/workflows/claude-code-review.yml | New workflow to auto-run Claude Code for PR reviews on open/sync events. |
| .claude/agents/review-tests.md | Adds test-focused review agent prompt. |
| .claude/agents/review-style.md | Adds style-focused review agent prompt. |
| .claude/agents/review-security.md | Adds security-focused review agent prompt. |
| .claude/agents/review-pr.md | Adds coordinator agent to delegate among specific reviewers. |
| .claude/agents/review-nits.md | Adds nitpicks-focused review agent prompt. |
| .claude/agents/review-logic.md | Adds logic-focused review agent prompt. |
| .claude/agents/review-docs.md | Adds docs-focused review agent prompt. |
| .claude/agents/review-design.md | Adds design-focused review agent prompt. |
| contents: read | ||
| pull-requests: read | ||
| issues: read |
There was a problem hiding this comment.
The workflow grants only read permissions, which will prevent Claude from commenting on issues/PRs or committing changes. To enable comments and repository writes (as described in the PR), elevate permissions to at least contents: write, pull-requests: write, and issues: write.
| contents: read | |
| pull-requests: read | |
| issues: read | |
| contents: write | |
| pull-requests: write | |
| issues: write |
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| pull-requests: read |
There was a problem hiding this comment.
This workflow instructs Claude to leave feedback via gh pr comment, which requires write access on pull requests. Update permissions to pull-requests: write (contents can remain read).
| pull-requests: read | |
| pull-requests: write |
| contents: read | ||
| pull-requests: read | ||
| issues: read | ||
| id-token: write |
There was a problem hiding this comment.
id-token: write is not needed for this job and increases token capabilities unnecessarily. Remove it or set to id-token: read to adhere to least-privilege.
| id-token: write | |
| id-token: read |
| contents: read | ||
| pull-requests: read | ||
| issues: read | ||
| id-token: write |
There was a problem hiding this comment.
id-token: write appears unnecessary for this workflow (no OIDC usage). Remove it or downgrade to id-token: read to minimize permissions.
| id-token: write |
| pull-requests: read | ||
| issues: read | ||
| id-token: write | ||
| actions: read # Required for Claude to read CI results on PRs |
There was a problem hiding this comment.
actions: read is configured both at the job level and again via additional_permissions, which is redundant. Keep a single source of truth (prefer the job-level permissions) and remove the duplicate additional_permissions entry.
| You may comment on style or quality criteria which are not covered in the guide. | ||
|
|
||
| Be concise. | ||
| Reserve praise or complements for exceptional work. |
There was a problem hiding this comment.
Correct 'complements' to 'compliments' (praise).
| Reserve praise or complements for exceptional work. | |
| Reserve praise or compliments for exceptional work. |
| Do not comment on anything you consider major or blocking (that is reserved for other review steps). | ||
|
|
||
| Be concise. | ||
| Reserve praise or complements for exceptional work. |
There was a problem hiding this comment.
Correct 'complements' to 'compliments' (praise).
| Reserve praise or complements for exceptional work. | |
| Reserve praise or compliments for exceptional work. |
| - If you are unsure that the code is correct, or don't understand, ask for clarification. | ||
|
|
||
| Be concise. | ||
| Reserve praise or complements for exceptional work. |
There was a problem hiding this comment.
Correct 'complements' to 'compliments' (praise).
| Reserve praise or complements for exceptional work. | |
| Reserve praise or compliments for exceptional work. |
| Try to limit your focus to places where the problems you find are introduced by the PR you are reviewing. | ||
|
|
||
| Be concise. | ||
| Reserve praise or complements for exceptional work. |
There was a problem hiding this comment.
Correct 'complements' to 'compliments' (praise).
| Reserve praise or complements for exceptional work. | |
| Reserve praise or compliments for exceptional work. |
| - If you think the code is confusing or poorly designed, and do not have a fix you are confident in, explain what you find confusing and/or request clarification. | ||
|
|
||
| Be concise. | ||
| Reserve praise or complements for exceptional work. |
There was a problem hiding this comment.
Correct 'complements' to 'compliments' (praise).
| Reserve praise or complements for exceptional work. | |
| Reserve praise or compliments for exceptional work. |
There was a problem hiding this comment.
Looks good overall. I'm not familiar enough with the permission model to comment on Copilot's review above, but it seems worth taking a look.
Regarding PR reviews from Claude, do the workflow file allow Claude to auto-review all PRs? If so, should we restrict it to contributors from the team, to avoid someone creating tons of PRs just for the sake of burning compute? We could still ask Claude for reviews manually for external PRs
| claude-review: | ||
| # Optional: Filter by PR author | ||
| # if: | | ||
| # github.event.pull_request.user.login == 'external-contributor' || | ||
| # github.event.pull_request.user.login == 'new-developer' || | ||
| # github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR' |
There was a problem hiding this comment.
In the absence of filtering here, does this mean anyone can trigger Claude reviews by opening Pull Requests?
There was a problem hiding this comment.
Daniel is there a good way to address this before merge. I'd like to avoid a $10k bill where someone used our PR as an agent interface to Claude :)
| REPO: ${{ github.repository }} | ||
| PR NUMBER: ${{ github.event.pull_request.number }} | ||
| Please review this pull request and provide feedback on: |
There was a problem hiding this comment.
It's hard to do it without a “please”, isn't it 🙂
There was a problem hiding this comment.
We do not want to anger our future AI overlords.
.github/workflows/claude.yml
Outdated
| actions: read # Required for Claude to read CI results on PRs | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 |
There was a problem hiding this comment.
Simple fix, please fix before merge.
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 |
|
I've added the don't merge tag until we can discuss security/billing implications. |
yup, that is fine. I want to fiddle with it a bit before it hits main anyway |
We have two workflow files related to Claude, one for having Claude do Pull Requests reviews and the other for generic interaction. The latter is simply named "claude.yml", which may lead to confusion; instead, rename it as "claude-convsersations.yml". Signed-off-by: Quentin Monnet <[email protected]>
Let's beat Dependabot to it. Signed-off-by: Quentin Monnet <[email protected]>
It's not really clear why Claude should run on assignments in addition to comments and Issue creation. And then, whose permissions should we check when assigning? The Issue author? In that case it means we've already run the workflow at issue creation if it was possible. The assignee? The assigner? We can't check their "author_association" from the API, so that would make things tricky. I also fear this may create too much noise if we create the issue with a mention in the description, then assign the issue. We'll have two interventions from Claude in that case. Let's remove execution from assignments. If it is desirable to get Claude's perspective after assigning, it remains possible to mention the agent directly in a comment. Signed-off-by: Quentin Monnet <[email protected]>
Use guardrails to restrict who can run Claude-related workflows. - For code reviews, only run the workflow if the author of the Pull Request is from the owning organisation, or the owner, or a GitHub collaborator for the repository. - For @claude comments in Issue comments and descriptions or Pull Request review comments, restrict the workflow to commenters with the same association (member, owner, or collaborator). Link: https://docs.github.com/en/graphql/reference/enums#commentauthorassociation Link: https://docs.github.com/en/webhooks/webhook-events-and-payloads Signed-off-by: Quentin Monnet <[email protected]>
qmonnet
force-pushed
the
add-claude-github-actions-1760391654541
branch
from
bce7c46 to
e1e7f01
Compare
This comment was marked as outdated.
This comment was marked as outdated.
|
Scratch my previous comment, I think I got confused.
Am I correct in understanding that everyone can trigger the workflow itself, but only users with write access will be able to access the repository's secrets and use the API key to run the actual query to Claude? |
|
Another concern with the current experiment is that the location where Claude inserts its comments in the diff seems a bit approximative, sometimes right above the line or block it comments on (making these lines invisible in GitHub's discussions, given that GitHub shows the snippet above the comment), sometimes above/below but a few lines apart. I wonder if this is Claude not being accurate, or some bug in the API it uses to select the lines it comments. Maybe worth adding some guideline for that in the instructions? |
fully agree. I will try to tune the prompt for that this weekend |
|
I'm changing the status to "draft" for now, I hope you don't mind. This is to avoid the PR to show up as ready-for-review-and-maybe-merge in the list of open PRs for the repo. |
3 participants
🤖 Installing Claude Code GitHub App
This PR adds a GitHub Actions workflow that enables Claude Code integration in our repository.
What is Claude Code?
Claude Code is an AI coding agent that can help with:
How it works
Once this PR is merged, we'll be able to interact with Claude by mentioning @claude in a pull request or issue comment.
Once the workflow is triggered, Claude will analyze the comment and surrounding context, and execute on the request in a GitHub action.
Important Notes
Security
There's more information in the Claude Code action repo.
After merging this PR, let's try mentioning @claude in a comment on any PR to get started!