Feature Coverage Testing
This document describes the Feature Coverage Matrix and testing mechanism for PromptScript formatters.
Overview
The Feature Coverage Matrix is a specification that tracks:
- Tool Capabilities - What features each classified AI target supports
- Formatter Implementation - Which features our formatters implement
- Coverage Gaps - Features supported by tools but not yet implemented
flowchart TB
subgraph "Feature Coverage System"
FM[Feature Matrix] --> |Defines| Features[Feature Specs]
Features --> |Tests| FC[Feature Coverage Tests]
FC --> |Validates| Formatters[Formatter Outputs]
Features --> |Documents| Caps[Tool Capabilities]
Features --> |Tracks| Gaps[Implementation Gaps]
end
Feature Matrix
The Feature Matrix is defined in packages/formatters/src/feature-matrix.ts.
Structure
interface FeatureSpec { id: string; // Unique identifier name: string; // Human-readable name description: string; // Feature description category: FeatureCategory; // Grouping category tools: Partial<Record<ToolName, FeatureStatus>>; // Support for classified tools testStrategy?: string; // How to test docsUrl?: Record<ToolName, string>; // Documentation links}
type FeatureStatus = | 'supported' // Tool supports, formatter implements | 'not-supported' // Tool doesn't support | 'planned' // Tool supports, not yet implemented | 'partial'; // Partially implementedThe target catalog contains 50 coding agent targets and two prompt targets. The matrix currently classifies 37 of the coding agent targets. A missing entry means the target has not been classified for that feature, not that the feature is unsupported. The formatter registry and generated platform matrix remain the source of truth for target availability.
Categories
| Category | Description |
|---|---|
output-format | Markdown, MDC, code blocks, diagrams |
file-structure | Single/multi-file output, workflows |
metadata | YAML frontmatter, descriptions |
targeting | Glob patterns, activation types |
content | Character limits, section splitting |
advanced | Context inclusion, @-mentions |
Using the Matrix
Get Features for a Tool
@promptscript/formatters is not published to npm, so the code examples on this page work inside the PromptScript repository only.
import { getToolFeatures } from '@promptscript/formatters';
const features = getToolFeatures('cursor');// Returns all features where cursor status is 'supported' or 'partial'Check Specific Feature Support
import { toolSupportsFeature } from '@promptscript/formatters';
if (toolSupportsFeature('cursor', 'yaml-frontmatter')) { // Generate with frontmatter}Get Coverage Summary
import { getFeatureCoverage } from '@promptscript/formatters';
const coverage = getFeatureCoverage('cursor');// {// tool: 'cursor',// supported: 20,// partial: 3,// planned: 2,// notSupported: 5,// total: 30,// coveragePercent: 72// }Generate Report
import { generateFeatureMatrixReport } from '@promptscript/formatters';
const markdown = generateFeatureMatrixReport();console.log(markdown);// Outputs markdown table with all features and their status per toolRunning Tests
# Run feature coverage testspnpm nx test formatters --testNamePattern="Feature Coverage"
# Run all formatter testspnpm nx test formattersTest Structure
Feature coverage tests verify that formatters correctly implement the features defined in the matrix:
packages/formatters/src/__tests__/feature-coverage.spec.tsTest Categories
-
Matrix Integrity - Validates the matrix structure
- Unique feature IDs
- Required fields present
- Valid status values
-
Coverage Summary - Tests coverage calculation
- Valid coverage for each tool
- Coverage percentages are accurate
-
Feature Implementation - Tests actual formatter output
markdown-output- Valid markdown producedcode-blocks- Fenced code blocks preservedmermaid-diagrams- Mermaid diagrams preservedyaml-frontmatter- Correct frontmatter generatedglob-patterns- Glob patterns in outputworkflows- Workflow files generatedcharacter-limit- Warning on limit exceeded
Adding a New Feature
When adding a new feature to the matrix:
-
Define in Matrix
feature-matrix.ts {id: 'new-feature',name: 'New Feature',description: 'Description of the feature',category: 'output-format',tools: {github: 'not-supported',cursor: 'supported',claude: 'not-supported',antigravity: 'planned',},testStrategy: 'How to verify this feature',} -
Add Test
feature-coverage.spec.ts describe('new-feature', () => {it.each(['github', 'cursor', 'claude', 'antigravity'] as ToolName[])('%s should handle new feature correctly',(tool) => {if (!toolSupportsFeature(tool, 'new-feature')) return;const formatter = formatters.get(tool)!;const result = formatter.format(createTestAST());// Verify feature is correctly implementedexpect(result.content).toContain('expected content');});}); -
Implement Feature
Add implementation to the relevant formatter(s).
-
Update Status
Change status from
'planned'to'supported'once implemented.
Best Practices
- Keep Matrix Updated - When implementing new features, update the matrix
- Test All Statuses - Tests should handle all feature statuses appropriately
- Document Gaps - Use
'planned'status to track intended features - Add Test Strategies - Help future developers understand how to verify features
- Include Docs URLs - Link to official tool documentation for reference
API Reference
Functions
| Function | Description |
|---|---|
getToolFeatures(tool) | Get all supported features for a tool |
getPlannedFeatures(tool) | Get features planned for a tool |
getFeaturesByCategory(cat) | Get features by category |
toolSupportsFeature(t, f) | Check if tool supports feature |
getFeatureCoverage(tool) | Get coverage summary |
getToolComparison() | Get comparison matrix |
generateFeatureMatrixReport() | Generate markdown report |
Types
| Type | Description |
|---|---|
ToolName | Union of tool names |
FeatureStatus | Feature implementation status |
FeatureSpec | Feature specification |
FeatureCategory | Feature category |
FeatureCoverageSummary | Coverage summary object |