Mod Logo

Review

Review agent-generated code with full specification context using Mod's IDE tooling

Reviewing Specifications with IDE Plugin

When AI agents execute your ModSpec files, the quality of your review process determines the quality of your final implementation. This guide shows how to use Mod's IDE plugin to efficiently review agent-generated code with full specification context, ensuring every implementation meets your requirements.

Contents

  1. Review Workflow Overview
  2. IDE Plugin Features
  3. Reviewing Agent Implementation
  4. Specification Compliance Review
  5. Interactive Review Tools
  6. Collaborative Review Process

Review Workflow Overview

The Modern Review Challenge

Traditional code review focuses on implementation details without specification context. When AI agents generate code, reviewers need to understand:

  • What requirement the code is supposed to fulfill
  • Why implementation choices were made
  • How completely the specification was implemented
  • Where gaps might exist in the implementation

Spec-Driven Review Process

Mod's IDE plugin enables a new review workflow:

  1. Specification Context - See what requirement each code section implements
  2. Implementation Tracing - Navigate between spec requirements and code
  3. Compliance Checking - Verify agent implementation matches specifications
  4. Gap Analysis - Identify missing or incomplete requirements
  5. Quality Validation - Ensure code meets non-functional requirements

Review Efficiency Gains

Traditional Review: 2-4 hours per feature

  • Read code to understand intent
  • Cross-reference with separate documentation
  • Manually verify requirement coverage
  • Check for missing functionality

Spec-Driven Review: 30-60 minutes per feature

  • Instant specification context for each code section
  • Automated requirement coverage analysis
  • Real-time compliance validation
  • Interactive navigation between specs and implementation

IDE Plugin Features

Traces Sidebar

The primary interface for specification-driven review:

┌─────────────────────────────────────────┐
│ TRACES                                  │
├─────────────────────────────────────────│
│ 📋 SPEC-AUTH-001 (User Authentication) │
│ ├── ✅ WORKFLOW-AUTH-1 (100%)          │
│ ├── ✅ MODEL-USER (100%)               │
│ ├── 🟡 API-AUTH-1 (75%)                │
│ │   ├── ✅ REQ-VAL-1 Email validation   │
│ │   ├── ✅ REQ-SEC-1 Password hashing   │
│ │   ├── ❌ REQ-PERF-1 Rate limiting     │
│ │   └── ❌ REQ-LOG-1 Audit logging      │
│ ├── 🔴 COMPONENT-AUTH-1 (20%)          │
│ │   ├── ❌ REQ-UI-1 Form validation     │
│ │   ├── ❌ REQ-UI-2 Error handling      │
│ │   └── ❌ REQ-A11Y-1 Accessibility     │
│ └── ⚪ TEST-AUTH-1 (0%)                 │
└─────────────────────────────────────────┘

Status Indicators:

  • Completed (Green): All requirements implemented with high confidence
  • 🟡 Partial (Yellow): Some requirements implemented, others missing
  • 🔴 Incomplete (Red): Major requirements missing or incorrectly implemented
  • Not Started (Gray): No implementation evidence found

File Annotations

Code files display specification traceability directly in the editor:

// @spec SPEC-AUTH-001/API-AUTH-1
// @context Implements POST /auth/signup with validation
// @dependencies REQ-SEC-1 (password hashing)
export async function signup(req: Request, res: Response) {
  // @spec REQ-VAL-1: Email format validation
  if (!isValidEmail(req.body.email)) {     // ← Green dot: Spec covered
    return res.status(400).json({
      error: 'Invalid email format'
    });
  }
  
  // @spec REQ-SEC-1: Password hashing with bcrypt
  const hash = await bcrypt.hash(req.body.password, 12); // ← Green dot: Spec covered
  
  // Missing: Rate limiting (REQ-PERF-1)         // ← Red dot: Missing requirement
  // Missing: Audit logging (REQ-LOG-1)          // ← Red dot: Missing requirement
  
  const user = await User.create({
    email: req.body.email,
    password_hash: hash
  });
}

Annotation Colors:

  • 🟢 Green Dots: Code that implements specific requirements
  • 🔴 Red Dots: Missing requirements or orphaned code
  • 🟡 Yellow Dots: Partially implemented or needs review
  • 🔵 Blue Dots: Recently modified by agents

Hover Context

Instant specification details when hovering over code:

┌─────────────────────────────────────┐
│ Specification Context              │
├─────────────────────────────────────│
│ REQ-SEC-1: Password Hashing        │
│                                     │
│ Requirement:                        │
│ Passwords MUST be hashed with       │
│ bcrypt using cost factor 12         │
│                                     │
│ Implementation Status: ✅ Complete  │
│ Agent: dev-agent-v2.1              │
│ Last Modified: 2 hours ago         │
│                                     │
│ Related Requirements:               │
│ → REQ-SEC-2: Session management    │
│ → REQ-VAL-1: Input validation      │
└─────────────────────────────────────┘

Reviewing Agent Implementation

Starting a Review Session

1. Open the Specification

# IDE Plugin automatically detects and loads specs
# Navigate to: View > Traces Sidebar

2. Review Implementation Status

SPEC-AUTH-001 Implementation Overview:
├── Requirements Implemented: 15/18 (83%)
├── Code Coverage: 1,247 lines traced
├── Orphaned Code: 45 lines (3.6%)
├── Recent Agent Activity: 8 files modified
└── Review Priority: Medium (missing non-critical features)

3. Identify Review Priorities

  • 🔴 Critical gaps: Security, data integrity requirements
  • 🟡 Important missing: Performance, user experience features
  • 🟢 Nice to have: Polish, edge case handling

Agent Implementation Analysis

Reviewing Agent Choices

// Agent Implementation Review
// @spec SPEC-AUTH-001/REQ-SEC-1 Password hashing with bcrypt

// ✅ GOOD: Correct bcrypt usage with proper cost factor
const hash = await bcrypt.hash(req.body.password, 12);

// ❓ REVIEW NEEDED: Agent chose to hash in the API layer
// Alternative: Hash in User model or service layer
// Trade-offs: Performance vs separation of concerns

// ❌ MISSING: Agent didn't implement timing attack protection
// Specification requires constant-time response for invalid users
// Need to add: await bcrypt.hash('dummy', 12) for non-existent users

Agent Decision Context Mod tracks agent reasoning for implementation choices:

{
  "agent_context": {
    "requirement": "REQ-SEC-1",
    "implementation_choice": "bcrypt hashing in API endpoint", 
    "reasoning": "Follows existing codebase pattern in auth/login.ts",
    "alternatives_considered": [
      "Hash in User model (rejected: breaks API layer pattern)",
      "Hash in service layer (rejected: adds complexity)"
    ],
    "confidence": 0.85
  }
}

Common Agent Implementation Patterns

Pattern 1: Over-Implementation

// Agent implemented more than required
// @spec REQ-VAL-1: Basic email validation

// ✅ Required: Email format validation
if (!isValidEmail(req.body.email)) {
  return res.status(400).json({ error: 'Invalid email' });
}

// ❓ Agent added (not in spec): Domain blacklist checking
if (isBlockedDomain(req.body.email)) {
  return res.status(400).json({ error: 'Domain not allowed' });
}

// Review Decision: Keep extra validation, update spec to document it

Pattern 2: Under-Implementation

// Agent missed edge cases from specification
// @spec REQ-ERROR-1: Comprehensive error handling

// ✅ Implemented: Basic validation errors
if (!req.body.email) {
  return res.status(400).json({ error: 'Email required' });
}

// ❌ Missing: Network error handling (specified in REQ-ERROR-1)
// ❌ Missing: Database connection failures
// ❌ Missing: Rate limit exceeded responses

// Review Action: Agent needs to implement complete error handling

Pattern 3: Spec Interpretation Issues

// Agent misunderstood specification requirement
// @spec REQ-PERF-1: Signup completes within 2 seconds

// ❌ Agent interpreted as: Individual operations must be fast
const user = await User.create(data);        // 50ms
const session = await Session.create(data);  // 30ms  
await emailService.send(email);              // 1200ms (not awaited)

// ✅ Correct interpretation: Total user-facing response time < 2s
// Fix: Background email processing, return 201 immediately

Specification Compliance Review

Requirement Verification Matrix

Review each specification requirement systematically:

┌─────────────────────────────────────────────────────────────┐
│ SPEC-AUTH-001/API-AUTH-1: POST /auth/signup                │
├─────────────────────────────────────────────────────────────┤
│ ✅ Request Format: { email, password }                     │
│ ✅ Success Response: 201 with { user_id, message }         │
│ ❌ Error Response: 400 for validation (missing 409, 422)   │
│ ✅ Password Hashing: bcrypt with cost=12                   │
│ ❌ Rate Limiting: 5 attempts per IP per hour (missing)     │
│ ❌ Audit Logging: Log all signup attempts (missing)        │
│ ✅ Email Validation: Server-side format check             │
│ ⚠️  Performance: <2s response (needs testing)              │
└─────────────────────────────────────────────────────────────┘

Review Status: 5/8 requirements complete (63%)
Action Required: Implement missing requirements before approval

Non-Functional Requirements Review

Performance Requirements

// @spec REQ-PERF-1: Signup completes within 2 seconds (p95)

// Review Checklist:
// ✅ Database queries are optimized (User.create uses prepared statements)
// ❌ Email sending not moved to background queue
// ❓ Need load testing to verify p95 performance under realistic conditions

// Performance Test Results:
// Current: 1.8s average, 3.2s p95 (FAILS requirement)
// Bottleneck: Synchronous email sending (1.2s average)
// Fix: Implement background email queue

Security Requirements

// @spec REQ-SEC-1: Password hashing with bcrypt (cost=12)
// @spec REQ-SEC-2: Session tokens must be UUID v4
// @spec REQ-SEC-3: HTTPS required for all auth endpoints

// Security Review Checklist:
// ✅ bcrypt.hash(password, 12) - Correct implementation
// ✅ session.token = uuid.v4() - Proper token generation  
// ❌ No HTTPS enforcement in middleware (missing)
// ❌ No timing attack protection (constant-time responses)
// ❌ No input sanitization beyond validation

// Security Risk: Medium
// Critical gaps: HTTPS enforcement, timing attack protection

Reliability Requirements

// @spec REQ-REL-1: Failed logins logged for security audit
// @spec REQ-REL-2: Session cleanup job runs hourly

// Reliability Review:
// ❌ No audit logging implemented
// ❌ No session cleanup job created
// ✅ Database constraints prevent duplicate emails
// ❓ Error handling needs comprehensive testing

// Operational Risk: High
// Missing: Audit trail, session management, monitoring

Code Quality Review

Traceability Quality

// Review @spec comment quality and accuracy

// ✅ Good traceability:
// @spec SPEC-AUTH-001/REQ-SEC-1: Password hashing with bcrypt
const hash = await bcrypt.hash(req.body.password, 12);

// ❌ Poor traceability:  
const userData = processUserData(req.body); // No @spec comment - orphaned code

// ❓ Unclear traceability:
// @spec SPEC-AUTH-001: General auth implementation  
if (complexBusinessLogic()) { /* Which specific requirement? */ }

// Traceability Score: 73% (needs improvement)
// Action: Add specific @spec comments to all implementation code

Architecture Alignment

// Review agent's architectural choices against specifications

// @spec SPEC-AUTH-001/ARCHITECTURE: Clean Architecture pattern

// ✅ Agent followed layered architecture:
// - Controllers handle HTTP requests
// - Services contain business logic  
// - Repositories manage data access

// ❓ Agent created new patterns not in existing codebase:
// - Custom validation decorators (not used elsewhere)
// - Different error handling pattern (inconsistent)

// Recommendation: 
// - Keep clean architecture (aligns with spec)
// - Standardize on existing codebase patterns for consistency

Interactive Review Tools

Click-Through Navigation

Spec-to-Code Navigation

  1. Click requirement in Traces sidebar → Jump to implementation
  2. Right-click requirement → "Show All Implementations"
  3. Hover requirement → Preview implementation snippet

Code-to-Spec Navigation

  1. Click @spec comment → Open specification section
  2. Right-click code → "Show Related Requirements"
  3. Hover green/red dots → Specification context popup

Real-Time Compliance Dashboard

┌─────────────────────────────────────────────────────────┐
│ Real-Time Review Dashboard                              │
├─────────────────────────────────────────────────────────│
│ Feature: User Authentication (SPEC-AUTH-001)           │
│                                                         │
│ Overall Implementation: 67% Complete                   │
│ ████████████████████████████████░░░░░░░░░░░░░░░        │
│                                                         │
│ By Category:                                            │
│ ✅ Data Models: 2/2 (100%)                             │
│ 🟡 API Endpoints: 3/5 (60%)                           │
│ 🔴 UI Components: 1/4 (25%)                           │
│ ⚪ Security: 0/3 (0%)                                 │
│                                                         │
│ Critical Missing:                                       │
│ • Rate limiting (REQ-PERF-1)                           │
│ • HTTPS enforcement (REQ-SEC-3)                        │
│ • Audit logging (REQ-REL-1)                            │
│                                                         │
│ Recent Agent Activity:                                  │
│ • 2 hours ago: Implemented password hashing            │
│ • 1 hour ago: Added email validation                   │
│ • 30 mins ago: Created User model                      │
└─────────────────────────────────────────────────────────┘

Batch Review Operations

Review All Missing Requirements

# IDE Plugin: Review > Show Missing Requirements
# Opens split view with all unimplemented specs

Missing Requirements (5):
├── REQ-PERF-1: Rate limiting (Critical)
├── REQ-SEC-3: HTTPS enforcement (Critical)  
├── REQ-REL-1: Audit logging (Important)
├── REQ-UI-3: Accessibility support (Important)
└── REQ-TEST-1: End-to-end tests (Nice to have)

# Batch Actions:
[ Assign to Agent ] [ Create Tasks ] [ Mark as Reviewed ]

Review All Agent Changes

# IDE Plugin: Review > Show Recent Agent Activity
# Shows all code modified by agents in the last review cycle

Agent Changes (8 files):
├── src/auth/signup.ts (SPEC-AUTH-001/API-AUTH-1)
├── src/models/User.ts (SPEC-AUTH-001/MODEL-USER)
├── ⚠️  src/utils/validation.ts (Orphaned - no spec link)
├── 🔴 src/auth/middleware.ts (Incomplete - missing HTTPS)
└── ... 4 more files

# Actions per file:
[ Approve ] [ Request Changes ] [ Add Spec Links ]

Specification Gap Analysis

Automated Gap Detection

interface SpecGapAnalysis {
  totalRequirements: number;
  implementedRequirements: number;
  missingRequirements: SpecRequirement[];
  orphanedCode: OrphanedCodeSection[];
  coveragePercentage: number;
  riskLevel: 'low' | 'medium' | 'high' | 'critical';
}

// Current Analysis:
{
  totalRequirements: 23,
  implementedRequirements: 15,
  missingRequirements: [
    { id: "REQ-PERF-1", priority: "critical", effort: "2h" },
    { id: "REQ-SEC-3", priority: "critical", effort: "1h" },
    { id: "REQ-REL-1", priority: "important", effort: "4h" }
  ],
  orphanedCode: [
    { file: "utils/legacy.ts", lines: 45, riskLevel: "low" }
  ],
  coveragePercentage: 65.2,
  riskLevel: "medium" // Missing critical security requirements
}

Collaborative Review Process

Multi-Reviewer Workflow

Review Assignment by Expertise

SPEC-AUTH-001 Review Team:
├── Security Review: @security-team
│   ├── REQ-SEC-1: Password hashing ✅ Approved
│   ├── REQ-SEC-2: Session management ⏳ Pending
│   └── REQ-SEC-3: HTTPS enforcement ❌ Not implemented
├── Performance Review: @performance-team  
│   ├── REQ-PERF-1: Response times ⚠️ Needs load testing
│   └── REQ-PERF-2: Concurrency ⏳ Pending
└── UX Review: @design-team
    ├── REQ-UI-1: Form validation ❌ Not implemented
    └── REQ-A11Y-1: Accessibility ❌ Not implemented

Review Status Tracking

Review Progress Dashboard:
┌─────────────────────────────────────────┐
│ SPEC-AUTH-001 Review Status            │
├─────────────────────────────────────────│
│ Security Team: 1/3 approved (33%)      │
│ Performance Team: 0/2 approved (0%)    │  
│ Design Team: 0/2 approved (0%)         │
│ Overall: 1/7 requirements approved     │
│                                         │
│ Blockers:                               │
│ • Missing HTTPS enforcement (critical) │
│ • No performance testing completed     │
│ • UI components not implemented        │
│                                         │
│ Next Actions:                           │
│ • Assign REQ-SEC-3 to agent            │
│ • Schedule performance testing          │  
│ • Create UI implementation tasks        │
└─────────────────────────────────────────┘

Review Handoff Process

Agent → Human Review

## Agent Implementation Summary
**Agent**: dev-agent-v2.1
**Specification**: SPEC-AUTH-001 (User Authentication)
**Implementation Time**: 2.5 hours  
**Confidence**: 85%

### Completed Requirements (5/8)
✅ REQ-VAL-1: Email validation - `src/auth/signup.ts:23`
✅ REQ-SEC-1: Password hashing - `src/auth/signup.ts:31`  
✅ MODEL-USER: User data model - `src/models/User.ts`
✅ API-AUTH-1: Basic signup endpoint - `src/auth/signup.ts`
✅ WORKFLOW-AUTH-1: Core signup flow - End-to-end implementation

### Missing Requirements (3/8)
❌ REQ-PERF-1: Rate limiting (Agent Note: Needs Redis configuration)
❌ REQ-SEC-3: HTTPS enforcement (Agent Note: Middleware pattern unclear)
❌ REQ-REL-1: Audit logging (Agent Note: Logging service not available)

### Implementation Decisions Made
1. **Database Layer**: Used existing TypeORM patterns from User model
2. **Validation**: Implemented server-side only (client-side in separate task)
3. **Error Handling**: Followed existing API error response format
4. **Password Storage**: bcrypt cost=12 (exceeded minimum requirement of 10)

### Human Review Needed
1. **Architecture**: Verify signup flow integrates with existing auth system
2. **Security**: Validate bcrypt implementation meets security standards  
3. **Performance**: Load test signup endpoint before enabling rate limiting
4. **UX**: Review error messages for user-friendliness

### Agent Confidence Issues
- **REQ-PERF-1**: Uncertain about Redis configuration for rate limiting
- **Middleware Integration**: Unclear how to integrate HTTPS enforcement
- **Audit Logging**: No established logging service pattern in codebase

Human → Agent Feedback

## Review Feedback for Agent
**Reviewer**: @senior-dev
**Status**: Approved with Changes
**Review Time**: 45 minutes

### Approved Requirements ✅
- REQ-VAL-1: Email validation (Good implementation)
- REQ-SEC-1: Password hashing (Exceeds requirements - excellent)
- MODEL-USER: User model (Follows established patterns)

### Changes Requested 🔄

#### REQ-PERF-1: Rate Limiting
**Issue**: Redis dependency not configured in project
**Solution**: Use in-memory rate limiting for MVP, Redis for production
**Implementation**: Install `express-rate-limit` package, configure for 5 attempts/hour
**Agent Task**: Implement rate limiting with fallback strategy

#### REQ-SEC-3: HTTPS Enforcement  
**Issue**: Missing middleware integration
**Solution**: Add to existing `src/middleware/security.ts`
**Example Pattern**: Follow CORS middleware pattern in same file
**Agent Task**: Add HTTPS enforcement to security middleware

#### Code Quality Issues
1. **Error Messages**: Too technical for end users
   - Change: "Invalid email format" → "Please enter a valid email address"
2. **Validation**: Add password strength requirements
   - Must include: 8+ chars, uppercase, lowercase, number
3. **Testing**: Missing unit tests for edge cases
   - Add tests for: duplicate email, weak password, invalid input

### Architecture Feedback
**Good**: Clean separation between validation, hashing, and storage
**Good**: Consistent error handling with existing API patterns  
⚠️  **Improve**: Add input sanitization before validation
⚠️  **Improve**: Consider extracting validation logic to shared utility

### Next Steps
1. Agent implements requested changes
2. Security team review of HTTPS implementation  
3. Load testing after rate limiting is added
4. Final approval after all changes complete

**Estimated completion**: 3-4 hours additional agent work

With Mod's IDE plugin, specification review becomes a guided, efficient process where both human reviewers and AI agents have full context about requirements, implementation status, and quality expectations. This dramatically reduces review time while improving implementation quality and specification alignment.