Review
Review agent-generated code with full specification context using Mod's IDE tooling
Reviewing Specifications with IDE Plugin
When AI agents execute your ModSpec files, the quality of your review process determines the quality of your final implementation. This guide shows how to use Mod's IDE plugin to efficiently review agent-generated code with full specification context, ensuring every implementation meets your requirements.
Contents
- Review Workflow Overview
- IDE Plugin Features
- Reviewing Agent Implementation
- Specification Compliance Review
- Interactive Review Tools
- Collaborative Review Process
Review Workflow Overview
The Modern Review Challenge
Traditional code review focuses on implementation details without specification context. When AI agents generate code, reviewers need to understand:
- What requirement the code is supposed to fulfill
- Why implementation choices were made
- How completely the specification was implemented
- Where gaps might exist in the implementation
Spec-Driven Review Process
Mod's IDE plugin enables a new review workflow:
- Specification Context - See what requirement each code section implements
- Implementation Tracing - Navigate between spec requirements and code
- Compliance Checking - Verify agent implementation matches specifications
- Gap Analysis - Identify missing or incomplete requirements
- Quality Validation - Ensure code meets non-functional requirements
Review Efficiency Gains
Traditional Review: 2-4 hours per feature
- Read code to understand intent
- Cross-reference with separate documentation
- Manually verify requirement coverage
- Check for missing functionality
Spec-Driven Review: 30-60 minutes per feature
- Instant specification context for each code section
- Automated requirement coverage analysis
- Real-time compliance validation
- Interactive navigation between specs and implementation
IDE Plugin Features
Traces Sidebar
The primary interface for specification-driven review:
┌─────────────────────────────────────────┐
│ TRACES │
├─────────────────────────────────────────│
│ 📋 SPEC-AUTH-001 (User Authentication) │
│ ├── ✅ WORKFLOW-AUTH-1 (100%) │
│ ├── ✅ MODEL-USER (100%) │
│ ├── 🟡 API-AUTH-1 (75%) │
│ │ ├── ✅ REQ-VAL-1 Email validation │
│ │ ├── ✅ REQ-SEC-1 Password hashing │
│ │ ├── ❌ REQ-PERF-1 Rate limiting │
│ │ └── ❌ REQ-LOG-1 Audit logging │
│ ├── 🔴 COMPONENT-AUTH-1 (20%) │
│ │ ├── ❌ REQ-UI-1 Form validation │
│ │ ├── ❌ REQ-UI-2 Error handling │
│ │ └── ❌ REQ-A11Y-1 Accessibility │
│ └── ⚪ TEST-AUTH-1 (0%) │
└─────────────────────────────────────────┘Status Indicators:
- ✅ Completed (Green): All requirements implemented with high confidence
- 🟡 Partial (Yellow): Some requirements implemented, others missing
- 🔴 Incomplete (Red): Major requirements missing or incorrectly implemented
- ⚪ Not Started (Gray): No implementation evidence found
File Annotations
Code files display specification traceability directly in the editor:
// @spec SPEC-AUTH-001/API-AUTH-1
// @context Implements POST /auth/signup with validation
// @dependencies REQ-SEC-1 (password hashing)
export async function signup(req: Request, res: Response) {
// @spec REQ-VAL-1: Email format validation
if (!isValidEmail(req.body.email)) { // ← Green dot: Spec covered
return res.status(400).json({
error: 'Invalid email format'
});
}
// @spec REQ-SEC-1: Password hashing with bcrypt
const hash = await bcrypt.hash(req.body.password, 12); // ← Green dot: Spec covered
// Missing: Rate limiting (REQ-PERF-1) // ← Red dot: Missing requirement
// Missing: Audit logging (REQ-LOG-1) // ← Red dot: Missing requirement
const user = await User.create({
email: req.body.email,
password_hash: hash
});
}Annotation Colors:
- 🟢 Green Dots: Code that implements specific requirements
- 🔴 Red Dots: Missing requirements or orphaned code
- 🟡 Yellow Dots: Partially implemented or needs review
- 🔵 Blue Dots: Recently modified by agents
Hover Context
Instant specification details when hovering over code:
┌─────────────────────────────────────┐
│ Specification Context │
├─────────────────────────────────────│
│ REQ-SEC-1: Password Hashing │
│ │
│ Requirement: │
│ Passwords MUST be hashed with │
│ bcrypt using cost factor 12 │
│ │
│ Implementation Status: ✅ Complete │
│ Agent: dev-agent-v2.1 │
│ Last Modified: 2 hours ago │
│ │
│ Related Requirements: │
│ → REQ-SEC-2: Session management │
│ → REQ-VAL-1: Input validation │
└─────────────────────────────────────┘Reviewing Agent Implementation
Starting a Review Session
1. Open the Specification
# IDE Plugin automatically detects and loads specs
# Navigate to: View > Traces Sidebar2. Review Implementation Status
SPEC-AUTH-001 Implementation Overview:
├── Requirements Implemented: 15/18 (83%)
├── Code Coverage: 1,247 lines traced
├── Orphaned Code: 45 lines (3.6%)
├── Recent Agent Activity: 8 files modified
└── Review Priority: Medium (missing non-critical features)3. Identify Review Priorities
- 🔴 Critical gaps: Security, data integrity requirements
- 🟡 Important missing: Performance, user experience features
- 🟢 Nice to have: Polish, edge case handling
Agent Implementation Analysis
Reviewing Agent Choices
// Agent Implementation Review
// @spec SPEC-AUTH-001/REQ-SEC-1 Password hashing with bcrypt
// ✅ GOOD: Correct bcrypt usage with proper cost factor
const hash = await bcrypt.hash(req.body.password, 12);
// ❓ REVIEW NEEDED: Agent chose to hash in the API layer
// Alternative: Hash in User model or service layer
// Trade-offs: Performance vs separation of concerns
// ❌ MISSING: Agent didn't implement timing attack protection
// Specification requires constant-time response for invalid users
// Need to add: await bcrypt.hash('dummy', 12) for non-existent usersAgent Decision Context Mod tracks agent reasoning for implementation choices:
{
"agent_context": {
"requirement": "REQ-SEC-1",
"implementation_choice": "bcrypt hashing in API endpoint",
"reasoning": "Follows existing codebase pattern in auth/login.ts",
"alternatives_considered": [
"Hash in User model (rejected: breaks API layer pattern)",
"Hash in service layer (rejected: adds complexity)"
],
"confidence": 0.85
}
}Common Agent Implementation Patterns
Pattern 1: Over-Implementation
// Agent implemented more than required
// @spec REQ-VAL-1: Basic email validation
// ✅ Required: Email format validation
if (!isValidEmail(req.body.email)) {
return res.status(400).json({ error: 'Invalid email' });
}
// ❓ Agent added (not in spec): Domain blacklist checking
if (isBlockedDomain(req.body.email)) {
return res.status(400).json({ error: 'Domain not allowed' });
}
// Review Decision: Keep extra validation, update spec to document itPattern 2: Under-Implementation
// Agent missed edge cases from specification
// @spec REQ-ERROR-1: Comprehensive error handling
// ✅ Implemented: Basic validation errors
if (!req.body.email) {
return res.status(400).json({ error: 'Email required' });
}
// ❌ Missing: Network error handling (specified in REQ-ERROR-1)
// ❌ Missing: Database connection failures
// ❌ Missing: Rate limit exceeded responses
// Review Action: Agent needs to implement complete error handlingPattern 3: Spec Interpretation Issues
// Agent misunderstood specification requirement
// @spec REQ-PERF-1: Signup completes within 2 seconds
// ❌ Agent interpreted as: Individual operations must be fast
const user = await User.create(data); // 50ms
const session = await Session.create(data); // 30ms
await emailService.send(email); // 1200ms (not awaited)
// ✅ Correct interpretation: Total user-facing response time < 2s
// Fix: Background email processing, return 201 immediatelySpecification Compliance Review
Requirement Verification Matrix
Review each specification requirement systematically:
┌─────────────────────────────────────────────────────────────┐
│ SPEC-AUTH-001/API-AUTH-1: POST /auth/signup │
├─────────────────────────────────────────────────────────────┤
│ ✅ Request Format: { email, password } │
│ ✅ Success Response: 201 with { user_id, message } │
│ ❌ Error Response: 400 for validation (missing 409, 422) │
│ ✅ Password Hashing: bcrypt with cost=12 │
│ ❌ Rate Limiting: 5 attempts per IP per hour (missing) │
│ ❌ Audit Logging: Log all signup attempts (missing) │
│ ✅ Email Validation: Server-side format check │
│ ⚠️ Performance: <2s response (needs testing) │
└─────────────────────────────────────────────────────────────┘
Review Status: 5/8 requirements complete (63%)
Action Required: Implement missing requirements before approvalNon-Functional Requirements Review
Performance Requirements
// @spec REQ-PERF-1: Signup completes within 2 seconds (p95)
// Review Checklist:
// ✅ Database queries are optimized (User.create uses prepared statements)
// ❌ Email sending not moved to background queue
// ❓ Need load testing to verify p95 performance under realistic conditions
// Performance Test Results:
// Current: 1.8s average, 3.2s p95 (FAILS requirement)
// Bottleneck: Synchronous email sending (1.2s average)
// Fix: Implement background email queueSecurity Requirements
// @spec REQ-SEC-1: Password hashing with bcrypt (cost=12)
// @spec REQ-SEC-2: Session tokens must be UUID v4
// @spec REQ-SEC-3: HTTPS required for all auth endpoints
// Security Review Checklist:
// ✅ bcrypt.hash(password, 12) - Correct implementation
// ✅ session.token = uuid.v4() - Proper token generation
// ❌ No HTTPS enforcement in middleware (missing)
// ❌ No timing attack protection (constant-time responses)
// ❌ No input sanitization beyond validation
// Security Risk: Medium
// Critical gaps: HTTPS enforcement, timing attack protectionReliability Requirements
// @spec REQ-REL-1: Failed logins logged for security audit
// @spec REQ-REL-2: Session cleanup job runs hourly
// Reliability Review:
// ❌ No audit logging implemented
// ❌ No session cleanup job created
// ✅ Database constraints prevent duplicate emails
// ❓ Error handling needs comprehensive testing
// Operational Risk: High
// Missing: Audit trail, session management, monitoringCode Quality Review
Traceability Quality
// Review @spec comment quality and accuracy
// ✅ Good traceability:
// @spec SPEC-AUTH-001/REQ-SEC-1: Password hashing with bcrypt
const hash = await bcrypt.hash(req.body.password, 12);
// ❌ Poor traceability:
const userData = processUserData(req.body); // No @spec comment - orphaned code
// ❓ Unclear traceability:
// @spec SPEC-AUTH-001: General auth implementation
if (complexBusinessLogic()) { /* Which specific requirement? */ }
// Traceability Score: 73% (needs improvement)
// Action: Add specific @spec comments to all implementation codeArchitecture Alignment
// Review agent's architectural choices against specifications
// @spec SPEC-AUTH-001/ARCHITECTURE: Clean Architecture pattern
// ✅ Agent followed layered architecture:
// - Controllers handle HTTP requests
// - Services contain business logic
// - Repositories manage data access
// ❓ Agent created new patterns not in existing codebase:
// - Custom validation decorators (not used elsewhere)
// - Different error handling pattern (inconsistent)
// Recommendation:
// - Keep clean architecture (aligns with spec)
// - Standardize on existing codebase patterns for consistencyInteractive Review Tools
Click-Through Navigation
Spec-to-Code Navigation
- Click requirement in Traces sidebar → Jump to implementation
- Right-click requirement → "Show All Implementations"
- Hover requirement → Preview implementation snippet
Code-to-Spec Navigation
- Click @spec comment → Open specification section
- Right-click code → "Show Related Requirements"
- Hover green/red dots → Specification context popup
Real-Time Compliance Dashboard
┌─────────────────────────────────────────────────────────┐
│ Real-Time Review Dashboard │
├─────────────────────────────────────────────────────────│
│ Feature: User Authentication (SPEC-AUTH-001) │
│ │
│ Overall Implementation: 67% Complete │
│ ████████████████████████████████░░░░░░░░░░░░░░░ │
│ │
│ By Category: │
│ ✅ Data Models: 2/2 (100%) │
│ 🟡 API Endpoints: 3/5 (60%) │
│ 🔴 UI Components: 1/4 (25%) │
│ ⚪ Security: 0/3 (0%) │
│ │
│ Critical Missing: │
│ • Rate limiting (REQ-PERF-1) │
│ • HTTPS enforcement (REQ-SEC-3) │
│ • Audit logging (REQ-REL-1) │
│ │
│ Recent Agent Activity: │
│ • 2 hours ago: Implemented password hashing │
│ • 1 hour ago: Added email validation │
│ • 30 mins ago: Created User model │
└─────────────────────────────────────────────────────────┘Batch Review Operations
Review All Missing Requirements
# IDE Plugin: Review > Show Missing Requirements
# Opens split view with all unimplemented specs
Missing Requirements (5):
├── REQ-PERF-1: Rate limiting (Critical)
├── REQ-SEC-3: HTTPS enforcement (Critical)
├── REQ-REL-1: Audit logging (Important)
├── REQ-UI-3: Accessibility support (Important)
└── REQ-TEST-1: End-to-end tests (Nice to have)
# Batch Actions:
[ Assign to Agent ] [ Create Tasks ] [ Mark as Reviewed ]Review All Agent Changes
# IDE Plugin: Review > Show Recent Agent Activity
# Shows all code modified by agents in the last review cycle
Agent Changes (8 files):
├── ✅ src/auth/signup.ts (SPEC-AUTH-001/API-AUTH-1)
├── ✅ src/models/User.ts (SPEC-AUTH-001/MODEL-USER)
├── ⚠️ src/utils/validation.ts (Orphaned - no spec link)
├── 🔴 src/auth/middleware.ts (Incomplete - missing HTTPS)
└── ... 4 more files
# Actions per file:
[ Approve ] [ Request Changes ] [ Add Spec Links ]Specification Gap Analysis
Automated Gap Detection
interface SpecGapAnalysis {
totalRequirements: number;
implementedRequirements: number;
missingRequirements: SpecRequirement[];
orphanedCode: OrphanedCodeSection[];
coveragePercentage: number;
riskLevel: 'low' | 'medium' | 'high' | 'critical';
}
// Current Analysis:
{
totalRequirements: 23,
implementedRequirements: 15,
missingRequirements: [
{ id: "REQ-PERF-1", priority: "critical", effort: "2h" },
{ id: "REQ-SEC-3", priority: "critical", effort: "1h" },
{ id: "REQ-REL-1", priority: "important", effort: "4h" }
],
orphanedCode: [
{ file: "utils/legacy.ts", lines: 45, riskLevel: "low" }
],
coveragePercentage: 65.2,
riskLevel: "medium" // Missing critical security requirements
}Collaborative Review Process
Multi-Reviewer Workflow
Review Assignment by Expertise
SPEC-AUTH-001 Review Team:
├── Security Review: @security-team
│ ├── REQ-SEC-1: Password hashing ✅ Approved
│ ├── REQ-SEC-2: Session management ⏳ Pending
│ └── REQ-SEC-3: HTTPS enforcement ❌ Not implemented
├── Performance Review: @performance-team
│ ├── REQ-PERF-1: Response times ⚠️ Needs load testing
│ └── REQ-PERF-2: Concurrency ⏳ Pending
└── UX Review: @design-team
├── REQ-UI-1: Form validation ❌ Not implemented
└── REQ-A11Y-1: Accessibility ❌ Not implementedReview Status Tracking
Review Progress Dashboard:
┌─────────────────────────────────────────┐
│ SPEC-AUTH-001 Review Status │
├─────────────────────────────────────────│
│ Security Team: 1/3 approved (33%) │
│ Performance Team: 0/2 approved (0%) │
│ Design Team: 0/2 approved (0%) │
│ Overall: 1/7 requirements approved │
│ │
│ Blockers: │
│ • Missing HTTPS enforcement (critical) │
│ • No performance testing completed │
│ • UI components not implemented │
│ │
│ Next Actions: │
│ • Assign REQ-SEC-3 to agent │
│ • Schedule performance testing │
│ • Create UI implementation tasks │
└─────────────────────────────────────────┘Review Handoff Process
Agent → Human Review
## Agent Implementation Summary
**Agent**: dev-agent-v2.1
**Specification**: SPEC-AUTH-001 (User Authentication)
**Implementation Time**: 2.5 hours
**Confidence**: 85%
### Completed Requirements (5/8)
✅ REQ-VAL-1: Email validation - `src/auth/signup.ts:23`
✅ REQ-SEC-1: Password hashing - `src/auth/signup.ts:31`
✅ MODEL-USER: User data model - `src/models/User.ts`
✅ API-AUTH-1: Basic signup endpoint - `src/auth/signup.ts`
✅ WORKFLOW-AUTH-1: Core signup flow - End-to-end implementation
### Missing Requirements (3/8)
❌ REQ-PERF-1: Rate limiting (Agent Note: Needs Redis configuration)
❌ REQ-SEC-3: HTTPS enforcement (Agent Note: Middleware pattern unclear)
❌ REQ-REL-1: Audit logging (Agent Note: Logging service not available)
### Implementation Decisions Made
1. **Database Layer**: Used existing TypeORM patterns from User model
2. **Validation**: Implemented server-side only (client-side in separate task)
3. **Error Handling**: Followed existing API error response format
4. **Password Storage**: bcrypt cost=12 (exceeded minimum requirement of 10)
### Human Review Needed
1. **Architecture**: Verify signup flow integrates with existing auth system
2. **Security**: Validate bcrypt implementation meets security standards
3. **Performance**: Load test signup endpoint before enabling rate limiting
4. **UX**: Review error messages for user-friendliness
### Agent Confidence Issues
- **REQ-PERF-1**: Uncertain about Redis configuration for rate limiting
- **Middleware Integration**: Unclear how to integrate HTTPS enforcement
- **Audit Logging**: No established logging service pattern in codebaseHuman → Agent Feedback
## Review Feedback for Agent
**Reviewer**: @senior-dev
**Status**: Approved with Changes
**Review Time**: 45 minutes
### Approved Requirements ✅
- REQ-VAL-1: Email validation (Good implementation)
- REQ-SEC-1: Password hashing (Exceeds requirements - excellent)
- MODEL-USER: User model (Follows established patterns)
### Changes Requested 🔄
#### REQ-PERF-1: Rate Limiting
**Issue**: Redis dependency not configured in project
**Solution**: Use in-memory rate limiting for MVP, Redis for production
**Implementation**: Install `express-rate-limit` package, configure for 5 attempts/hour
**Agent Task**: Implement rate limiting with fallback strategy
#### REQ-SEC-3: HTTPS Enforcement
**Issue**: Missing middleware integration
**Solution**: Add to existing `src/middleware/security.ts`
**Example Pattern**: Follow CORS middleware pattern in same file
**Agent Task**: Add HTTPS enforcement to security middleware
#### Code Quality Issues
1. **Error Messages**: Too technical for end users
- Change: "Invalid email format" → "Please enter a valid email address"
2. **Validation**: Add password strength requirements
- Must include: 8+ chars, uppercase, lowercase, number
3. **Testing**: Missing unit tests for edge cases
- Add tests for: duplicate email, weak password, invalid input
### Architecture Feedback
✅ **Good**: Clean separation between validation, hashing, and storage
✅ **Good**: Consistent error handling with existing API patterns
⚠️ **Improve**: Add input sanitization before validation
⚠️ **Improve**: Consider extracting validation logic to shared utility
### Next Steps
1. Agent implements requested changes
2. Security team review of HTTPS implementation
3. Load testing after rate limiting is added
4. Final approval after all changes complete
**Estimated completion**: 3-4 hours additional agent workWith Mod's IDE plugin, specification review becomes a guided, efficient process where both human reviewers and AI agents have full context about requirements, implementation status, and quality expectations. This dramatically reduces review time while improving implementation quality and specification alignment.