Intermediate

Step 3: Posting Review Comments

Take the structured issues from the LLM and post them as inline review comments on the GitHub PR — with severity labels, code suggestions, and a summary review.

GitHub Review API Overview

GitHub provides two ways to post comments on a PR:

• Individual comments (POST /pulls/{pull_number}/comments) — Each comment posted separately. Simple but creates many notifications.
• Review with comments (POST /pulls/{pull_number}/reviews) — All comments bundled into a single review. One notification. This is what we will use.

Building the Comment Poster

Create src/commenter.js:

// src/commenter.js
const { octokit } = require('./github');

// Severity emoji and labels
const SEVERITY_MAP = {
  critical: { emoji: '🔴', label: 'CRITICAL', event: 'REQUEST_CHANGES' },
  warning:  { emoji: '🟠', label: 'WARNING',  event: 'COMMENT' },
  suggestion: { emoji: '🔵', label: 'SUGGESTION', event: 'COMMENT' },
};

/**
 * Post all issues as a single GitHub review with inline comments.
 */
async function postReviewComments({ owner, repo, pullNumber, headSha, issues }) {
  // Build the review comments array
  const comments = issues.map(issue => formatComment(issue));

  // Determine the review event based on highest severity
  const hasCritical = issues.some(i => i.severity === 'critical');
  const reviewEvent = hasCritical ? 'REQUEST_CHANGES' : 'COMMENT';

  // Build the summary body
  const summary = buildSummary(issues);

  try {
    await octokit.pulls.createReview({
      owner,
      repo,
      pull_number: pullNumber,
      commit_id: headSha,
      event: reviewEvent,
      body: summary,
      comments: comments,
    });

    console.log(`Posted review with ${comments.length} comments`);
  } catch (error) {
    console.error('Failed to post review:', error.message);

    // Fallback: post comments individually if batch fails
    if (error.status === 422) {
      console.log('Falling back to individual comments...');
      await postIndividualComments({ owner, repo, pullNumber, headSha, issues });
    }
  }
}

/**
 * Format a single issue into a GitHub review comment object.
 */
function formatComment(issue) {
  const severity = SEVERITY_MAP[issue.severity] || SEVERITY_MAP.suggestion;

  let body = `${severity.emoji} **${severity.label}**: ${issue.title}\n\n`;
  body += `${issue.description}\n`;

  // Add suggestion in GitHub's suggestion format (renders as a diff)
  if (issue.suggestion) {
    body += `\n**Suggested fix:**\n`;
    body += `\`\`\`suggestion\n${issue.suggestion}\n\`\`\`\n`;
  }

  return {
    path: issue.file,
    line: issue.line,
    body: body,
  };
}

/**
 * Build a summary of all issues found in the review.
 */
function buildSummary(issues) {
  const critical = issues.filter(i => i.severity === 'critical').length;
  const warnings = issues.filter(i => i.severity === 'warning').length;
  const suggestions = issues.filter(i => i.severity === 'suggestion').length;

  let summary = `## AI Code Review Summary\n\n`;
  summary += `Found **${issues.length}** issue(s) in this PR:\n\n`;

  if (critical > 0) summary += `- 🔴 **${critical}** critical issue(s)\n`;
  if (warnings > 0) summary += `- 🟠 **${warnings}** warning(s)\n`;
  if (suggestions > 0) summary += `- 🔵 **${suggestions}** suggestion(s)\n`;

  summary += `\n---\n`;
  summary += `*Reviewed by AI Code Review Bot* | `;
  summary += `[Configure rules](.ai-review.yml)`;

  return summary;
}

/**
 * Fallback: post comments individually when batch review fails.
 * This can happen when line numbers do not match the diff.
 */
async function postIndividualComments({ owner, repo, pullNumber, headSha, issues }) {
  let posted = 0;
  let failed = 0;

  for (const issue of issues) {
    try {
      const comment = formatComment(issue);
      await octokit.pulls.createReviewComment({
        owner,
        repo,
        pull_number: pullNumber,
        commit_id: headSha,
        path: comment.path,
        line: comment.line,
        body: comment.body,
      });
      posted++;
    } catch (error) {
      console.warn(`Failed to post comment on ${issue.file}:${issue.line}:`,
        error.message);
      failed++;
    }
  }

  console.log(`Individual comments: ${posted} posted, ${failed} failed`);
}

module.exports = { postReviewComments, formatComment, buildSummary };

Understanding GitHub's Line Mapping

One of the trickiest parts of posting inline comments is getting the line number right. GitHub requires the line number to be in the new version of the file (the right side of the diff). Our diff parser already tracks this, but there are edge cases:

The Suggestion Format

GitHub has a special markdown syntax for code suggestions that renders an "Apply suggestion" button:

```suggestion
const result = items.filter(item => item != null);
```

When you wrap code in a suggestion code fence, GitHub renders it as a green diff with an "Apply suggestion" button. The developer can accept the fix with a single click, which creates a new commit. Our formatComment() function uses this format automatically when the LLM provides a suggestion.

Review Events Explained

When creating a review, the event field controls the review status:

• COMMENT — Posts the review without approving or requesting changes. Best for suggestions and minor warnings.
• REQUEST_CHANGES — Marks the PR as needing changes. The developer must address the issues before merging. We use this when critical issues are found.
• APPROVE — Approves the PR. We never auto-approve because AI should not be the sole approver.

Our logic: if any issue has critical severity, we request changes. Otherwise, we post as a comment. This ensures critical bugs block merging while minor suggestions do not.

Complete Pipeline Test

At this point, the full pipeline is functional. Let us trace through a complete review:

// test-pipeline.js - Full end-to-end test
require('dotenv').config();
const { parseDiff } = require('./src/diff-parser');
const { analyzeCode } = require('./src/analyzer');
const { postReviewComments, buildSummary } = require('./src/commenter');

async function testPipeline() {
  // 1. Simulate a parsed diff
  const files = [{
    filename: 'src/auth.js',
    status: 'modified',
    hunks: [{
      header: '@@ -10,5 +10,12 @@ function login(username, password) {',
      startLine: 10,
      lines: [
        { type: 'context', lineNumber: 10, content: 'async function login(username, password) {' },
        { type: 'added', lineNumber: 11, content: '  const query = `SELECT * FROM users WHERE name=\'${username}\'`;' },
        { type: 'added', lineNumber: 12, content: '  const user = await db.query(query);' },
        { type: 'added', lineNumber: 13, content: '  if (user.password === password) {' },
        { type: 'added', lineNumber: 14, content: '    return { token: jwt.sign({ id: user.id }, "secret123") };' },
        { type: 'added', lineNumber: 15, content: '  }' },
        { type: 'context', lineNumber: 16, content: '}' },
      ]
    }]
  }];

  // 2. Analyze with LLM
  console.log('Analyzing code...');
  const issues = await analyzeCode(files);
  console.log(`Found ${issues.length} issues:`);
  console.log(JSON.stringify(issues, null, 2));

  // 3. Show what the review would look like
  console.log('\n--- Review Summary ---');
  console.log(buildSummary(issues));
}

testPipeline().catch(console.error);

Run this test and you should see the LLM identify the SQL injection vulnerability, the plaintext password comparison, and the hardcoded JWT secret — all real security issues that would be caught in a code review.

What Is Next

The core pipeline is complete: webhook receives events, diff parser extracts changes, LLM analyzes code, and the commenter posts reviews. In the next lesson, we will build Step 4: Custom Rules and Config — a .ai-review.yml configuration system that lets teams customize what gets reviewed and how.