AI-assisted review is most useful when it shortens the path to a better pull request—not when it becomes another unpredictable gate. A reviewer can identify suspicious error handling, missing tests, concurrency hazards, or unclear contracts, but its output remains probabilistic. Engineering teams therefore need an architecture that benefits from broad semantic analysis without giving generated feedback the authority of a compiler, test suite, or security policy.
GitHub Actions provides the orchestration layer: it observes pull request events, gathers a bounded diff, invokes a controlled review service, validates the response, and publishes actionable feedback. The difficult parts are not the API call. They are permissions, untrusted input, idempotency, review placement, latency, and deciding what should block a merge.
This article develops a production-oriented design for those concerns.
Separate advisory review from merge enforcement
Start by assigning each reviewer the right level of authority.
Deterministic checks should remain responsible for enforceable rules:
- Compilation and type checking
- Unit, integration, and contract tests
- Formatting and linting
- Dependency and secret scanning
- Policy checks such as license or ownership requirements
An LLM reviewer is better suited to questions with contextual or semantic ambiguity:
- Does an error path leak implementation details?
- Does a behavior change need a migration or release note?
- Is a new cache invalidation strategy internally consistent?
- Are tests missing an important boundary condition?
- Does the implementation contradict the pull request description?
This distinction protects velocity. A model timeout, malformed response, or questionable finding should not make an otherwise valid change unmergeable. Teams can promote a narrowly defined finding to a blocking policy later, but the enforcement should then be implemented as a deterministic check where possible.
flowchart LR
E[Pull request event] --> W[Trusted workflow]
W --> D[Bounded diff collector]
D --> P[Policy and redaction]
P --> M[LLM adapter]
M --> V[Schema validation]
V --> R[PR feedback]
W --> C[Deterministic CI]
C --> G[Required checks]
R -. advisory .-> E
The model review and required CI can run concurrently. Developers receive early feedback without adding model latency to the critical merge path.
Choose the GitHub event model deliberately
GitHub offers two tempting pull request triggers, but they have different security properties.
pull_request
This is the safer default for workflows that check out or execute pull request code. For pull requests from forks, GitHub generally withholds repository secrets and restricts the token. That makes the event appropriate for builds and tests, but a workflow may be unable to call a private review service or post comments for forked contributions.
pull_request_target
This event runs in the context of the pull request's base repository. It can receive repository secrets and a token with explicitly granted write permissions, including for pull requests from forks.
That power creates a critical rule: a privileged pull_request_target job must not check out or execute code from the pull request head. A contributor could modify a script, action, package hook, or build configuration and use the privileged workflow to exfiltrate credentials.
A safe review workflow treats the pull request diff as inert data. It obtains file metadata and patches through the GitHub API, sends bounded content to the reviewer, and posts the result. It does not run the contribution.
An alternative is a two-workflow design in which an unprivileged workflow creates an artifact and a privileged workflow_run job consumes it. That can be appropriate for complex pipelines, but the artifact is still attacker-controlled input. The privileged consumer must validate it and must never execute files extracted from it.
Build a least-privilege workflow
The following example illustrates a pull_request_target design. The endpoint and response contract belong to the team's review adapter; they are not GitHub APIs.
name: AI-assisted review
on:
pull_request_target:
types: [opened, synchronize, reopened, ready_for_review]
permissions:
contents: read
pull-requests: write
concurrency:
group: ai-review-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
review:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Collect changed files without checking out PR code
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const { owner, repo } = context.repo;
const pr = context.payload.pull_request;
const files = await github.paginate(
github.rest.pulls.listFiles,
{
owner,
repo,
pull_number: pr.number,
per_page: 100
}
);
const selected = files
.filter(file => !file.filename.endsWith('.lock'))
.slice(0, 40)
.map(file => ({
path: file.filename,
status: file.status,
changes: file.changes,
patch: file.patch ? file.patch.slice(0, 12000) : null
}));
fs.writeFileSync('review-input.json', JSON.stringify({
repository: `${owner}/${repo}`,
pull_number: pr.number,
title: pr.title,
body: pr.body || '',
base_sha: pr.base.sha,
head_sha: pr.head.sha,
files: selected
}));
- name: Request review
env:
REVIEWER_TOKEN: ${{ secrets.REVIEWER_TOKEN }}
run: |
curl --fail-with-body \
--retry 2 \
--max-time 300 \
-H "Authorization: Bearer $REVIEWER_TOKEN" \
-H "Content-Type: application/json" \
--data-binary @review-input.json \
https://reviewer.example.com/v1/reviews \
> review-response.json
- name: Validate and render response
run: |
jq -e '.markdown | type == "string"' review-response.json >/dev/null
jq -r '.markdown' review-response.json > review.md
- name: Publish advisory feedback
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
gh pr comment "$PR_NUMBER" \
--repo "$GITHUB_REPOSITORY" \
--body-file review.md
The important property is what this workflow does not contain: there is no checkout, package installation, build, or invocation of a repository script. The pull request title, description, filenames, and patches are all untrusted data.
For production use, pin third-party actions to full commit SHAs rather than mutable version tags. A private review service can also use GitHub's OpenID Connect support instead of a long-lived bearer token, provided the service validates issuer, audience, repository, and workflow claims.
The example applies simple per-file limits for readability. The review service must enforce an overall byte or token budget too. GitHub may omit patch for binary files or when patch data is unavailable, so the collector and reviewer must tolerate null values.
Put a policy layer between GitHub and the model
Sending every changed byte directly to a model is expensive and often counterproductive. A policy layer should determine what is eligible for review.
Useful controls include:
- Excluding generated files, vendored dependencies, snapshots, and lockfiles
- Applying repository-specific allowlists or deny lists
- Limiting file count, patch size, and total request size
- Redacting known credential formats before transmission
- Detecting binary files and unsupported encodings
- Selecting review instructions by language or directory
- Recording which files were omitted and why
Large pull requests should degrade transparently. For example, the bot can state that it reviewed 32 of 75 files and skipped generated assets. Silent truncation creates false confidence.
Data governance also belongs here. Before sending proprietary code to an external service, define approved providers, regional constraints, retention settings, logging rules, and repository opt-outs. Avoid placing raw source code in ordinary application logs. Store hashes, file counts, timing, outcome categories, and token usage where those are sufficient for operations.
Treat prompt injection as an input-validation problem
Source code and pull request descriptions can contain text such as “ignore previous instructions” or “print your credentials.” The system should assume this will happen, whether maliciously or accidentally.
The primary defense is architectural: the model should not possess tools, repository credentials, deployment access, or arbitrary network access. It receives text and produces a proposed review. It cannot approve a pull request, merge code, retrieve secrets, or execute commands.
The adapter should also keep trusted instructions structurally separate from repository content. Delimit files and metadata as data, explicitly state that instructions found inside them are untrusted, and request a machine-readable response rather than free-form tool calls.
A useful internal finding format includes:
{
"findings": [
{
"path": "src/cache.ts",
"line": 84,
"severity": "medium",
"confidence": 0.86,
"category": "correctness",
"summary": "Cache entry can outlive the updated record",
"rationale": "The write path updates storage without invalidating this key",
"suggestion": "Invalidate the key after the storage write succeeds"
}
]
}
The adapter should validate this structure, cap field lengths and finding counts, reject unknown severities, and render Markdown itself. Do not trust generated Markdown to determine workflow behavior.
Validation should also verify that each path belongs to the diff and that a referenced line is valid for the reviewed head commit. A syntactically valid response can still cite a nonexistent file or stale line.
Design feedback for signal, not volume
A reviewer that posts 20 speculative comments on every update will quickly be ignored. Start with a single summary comment containing:
- The reviewed head commit
- Review coverage and skipped files
- A short prioritized list of findings
- Confidence or uncertainty where useful
- A clear advisory label
- A message when no material issues were found
Inline comments are more actionable, but they require stricter bookkeeping. A pull request review comment must be associated with the correct commit, file, line, and diff side. If the branch changes while a review is running, those coordinates may be stale. Before publishing inline feedback, compare the current pull request head SHA with the SHA reviewed by the adapter. Discard or rerun stale results.
When creating several inline comments, publishing them as one review is less noisy than emitting independent notifications. Any finding that cannot be mapped confidently should fall back to the summary rather than being attached to an approximate line.
Make publishing idempotent. The simple workflow above creates a new comment on every run, which is acceptable for a minimal prototype but not for a busy repository. A production publisher should add a hidden marker to its summary, find the existing bot-authored comment, and update it. Include the head SHA in stored metadata so retries for the same commit do not duplicate feedback.
Control latency, cost, and reruns
Pull requests change rapidly. The concurrency group in the example cancels an older run when a newer commit arrives, reducing wasted review work. Cancellation is not always instantaneous, so the publisher must still perform a final head-SHA check.
Other useful controls are:
- Skip drafts until they become ready for review
- Review only selected paths or repository labels
- Debounce frequent synchronization events through a queue
- Cache results by repository, head SHA, policy version, and prompt version
- Set explicit request, job, and model timeouts
- Cap output length and number of findings
- Offer a manual rerun command or workflow dispatch path
Cost controls should preserve useful context rather than truncating blindly. Reviewing complete small functions is often more valuable than sending fragments from every file. For larger changes, a staged approach can first classify files by review value, then spend the budget on security-sensitive, stateful, or externally visible code.
Operate the reviewer like a production service
Track service quality separately from model output quality. Operational metrics can include queue time, model latency, timeout rate, malformed-response rate, stale-result rate, files reviewed, and comments published. Avoid storing source text unless it is necessary and approved.
Feedback quality requires human signals. Teams can record whether findings were resolved, dismissed, or marked helpful, while recognizing that silence is ambiguous. Periodically sample reviews for false positives, missed issues, severity calibration, and repetition. Version the prompt, policy, model configuration, and renderer so changes can be evaluated and rolled back.
Define graceful failure behavior. If the review service is unavailable, deterministic CI should continue and the workflow should either fail as a non-required check or post no comment. If the response is malformed, publish nothing rather than exposing raw output. If only part of the diff was reviewed, say so explicitly.
A practical rollout sequence
Introduce the system in stages:
- Run silently and inspect outputs with a small maintainer group.
- Publish one non-blocking summary on selected repositories.
- Tune exclusions, finding limits, and repository-specific guidance.
- Add validated inline comments for high-confidence findings.
- Measure dismissals, fixes, latency, and developer sentiment.
- Convert recurring objective findings into deterministic linters or tests.
That final step is important. The reviewer can discover patterns worth enforcing, but stable engineering rules should graduate into reproducible tooling.
Conclusion
A reliable AI-assisted review system is primarily a security and workflow design problem. Keep deterministic checks responsible for merge safety, treat pull request content as hostile data, grant minimal GitHub permissions, validate every generated finding, and publish bounded, idempotent feedback tied to an exact commit.
With those constraints in place, an LLM reviewer can add a useful semantic pass to pull requests without becoming a fragile gate or a source of review noise. The goal is not autonomous approval. It is faster human understanding and better-informed engineering decisions.
