Feature Coverage Testing¶

This document describes the Feature Coverage Matrix and testing mechanism for PromptScript formatters.

Overview¶

The Feature Coverage Matrix is a specification that tracks:

1. Tool Capabilities - What features each classified AI target supports
2. Formatter Implementation - Which features our formatters implement
3. Coverage Gaps - Features supported by tools but not yet implemented

flowchart TB
  subgraph "Feature Coverage System"
    FM[Feature Matrix] --> |Defines| Features[Feature Specs]
    Features --> |Tests| FC[Feature Coverage Tests]
    FC --> |Validates| Formatters[Formatter Outputs]

    Features --> |Documents| Caps[Tool Capabilities]
    Features --> |Tracks| Gaps[Implementation Gaps]
  end

Feature Matrix¶

The Feature Matrix is defined in packages/formatters/src/feature-matrix.ts.

Structure¶

interface FeatureSpec {
  id: string; // Unique identifier
  name: string; // Human-readable name
  description: string; // Feature description
  category: FeatureCategory; // Grouping category
  tools: Partial<Record<ToolName, FeatureStatus>>; // Support for classified tools
  testStrategy?: string; // How to test
  docsUrl?: Record<ToolName, string>; // Documentation links
}

type FeatureStatus =
  | 'supported' // Tool supports, formatter implements
  | 'not-supported' // Tool doesn't support
  | 'planned' // Tool supports, not yet implemented
  | 'partial'; // Partially implemented

The target catalog contains 48 formatters. The matrix currently classifies 37 of them. A missing entry means the target has not been classified for that feature, not that the feature is unsupported. The formatter registry and generated platform matrix remain the source of truth for target availability.

Categories¶

Using the Matrix¶

Get Features for a Tool¶

import { getToolFeatures } from '@promptscript/formatters';

const features = getToolFeatures('cursor');
// Returns all features where cursor status is 'supported' or 'partial'

Check Specific Feature Support¶

import { toolSupportsFeature } from '@promptscript/formatters';

if (toolSupportsFeature('cursor', 'yaml-frontmatter')) {
  // Generate with frontmatter
}

Get Coverage Summary¶

import { getFeatureCoverage } from '@promptscript/formatters';

const coverage = getFeatureCoverage('cursor');
// {
//   tool: 'cursor',
//   supported: 20,
//   partial: 3,
//   planned: 2,
//   notSupported: 5,
//   total: 30,
//   coveragePercent: 72
// }

Generate Report¶

import { generateFeatureMatrixReport } from '@promptscript/formatters';

const markdown = generateFeatureMatrixReport();
console.log(markdown);
// Outputs markdown table with all features and their status per tool

Running Tests¶

# Run feature coverage tests
pnpm nx test formatters --testNamePattern="Feature Coverage"

# Run all formatter tests
pnpm nx test formatters

Test Structure¶

Feature coverage tests verify that formatters correctly implement the features defined in the matrix:

packages/formatters/src/__tests__/feature-coverage.spec.ts

Test Categories¶

1. Matrix Integrity - Validates the matrix structure
2. Unique feature IDs
3. Required fields present

4. Valid status values

5. Coverage Summary - Tests coverage calculation

6. Valid coverage for each tool

7. Coverage percentages are accurate

8. Feature Implementation - Tests actual formatter output

9. markdown-output - Valid markdown produced
10. code-blocks - Fenced code blocks preserved
11. mermaid-diagrams - Mermaid diagrams preserved
12. yaml-frontmatter - Correct frontmatter generated
13. glob-patterns - Glob patterns in output
14. workflows - Workflow files generated
15. character-limit - Warning on limit exceeded

Adding a New Feature¶

When adding a new feature to the matrix:

1. Define in Matrix

// feature-matrix.ts
{
  id: 'new-feature',
  name: 'New Feature',
  description: 'Description of the feature',
  category: 'output-format',
  tools: {
    github: 'not-supported',
    cursor: 'supported',
    claude: 'not-supported',
    antigravity: 'planned',
  },
  testStrategy: 'How to verify this feature',
}

1. Add Test

// feature-coverage.spec.ts
describe('new-feature', () => {
  it.each(['github', 'cursor', 'claude', 'antigravity'] as ToolName[])(
    '%s should handle new feature correctly',
    (tool) => {
      if (!toolSupportsFeature(tool, 'new-feature')) return;

      const formatter = formatters.get(tool)!;
      const result = formatter.format(createTestAST());

      // Verify feature is correctly implemented
      expect(result.content).toContain('expected content');
    }
  );
});

1. Implement Feature

Add implementation to the relevant formatter(s).

1. Update Status

Change status from 'planned' to 'supported' once implemented.

Best Practices¶

1. Keep Matrix Updated - When implementing new features, update the matrix
2. Test All Statuses - Tests should handle all feature statuses appropriately
3. Document Gaps - Use 'planned' status to track intended features
4. Add Test Strategies - Help future developers understand how to verify features
5. Include Docs URLs - Link to official tool documentation for reference

API Reference¶

Functions¶

Types¶