mirror of https://github.com/Telecominfraproject/ols-ucentral-client.git synced 2026-01-27 02:21:38 +00:00

Files

Mike Hansen a4a153197b [OLS-915] Schema Verification Enhancement

Enables undefined property detection during make validate-schema, providing warnings about typos and vendor-specific
  properties that aren't in the schema.

Signed-off-by: Mike Hansen <mike.hansen@netexperience.com>

2026-01-13 10:22:59 -05:00

32 KiB

Raw Permalink Blame History

Configuration Testing Framework Maintenance Guide

This document provides procedures for maintaining the configuration testing framework as the software evolves.

Overview

Current Approach (December 2024): Schema-Based Property Database Generation

The framework uses the uCentral schema as the single source of truth for property databases. This ensures complete coverage of all 398 schema properties, whether implemented or not.

Key Components

Schema Files - JSON/YAML schema definitions from ols-ucentral-schema repository
Property Databases - Generated from schema, tracking implementation status:
- property-database-base.c - Proto.c parsing (102 found, 296 not implemented)
- property-database-platform-brcm-sonic.c - Platform application (141 found, 257 not implemented)

Schema-Based Workflow

ols-ucentral-schema (YAML)
    ↓
fetch-schema.sh → ols-ucentral-schema/
    ↓
extract-schema-properties.py → 398 properties
    ↓
generate-database-from-schema.py → base database
generate-platform-database-from-schema.py → platform database
    ↓
Property databases with line numbers

Quick Start: Schema-Based Regeneration
Schema Update Procedures
Property Database Regeneration
Adding New Parser Functions
Platform-Specific Updates
Version Synchronization
Testing After Updates
Troubleshooting
Legacy Approach

Quick Start: Schema-Based Regeneration

This repository includes default schema files in config-samples/, so you can regenerate property databases immediately without fetching external repositories.

Complete regeneration of both property databases from scratch:

cd tests/tools

# 1. Obtain schema (use included version OR fetch newer version)
# Option A: Use included schema (recommended for most cases)
SCHEMA_SOURCE="../../config-samples/ucentral.schema.pretty.json"

# Option B: Fetch newer schema if needed (optional)
# ./fetch-schema.sh main
# SCHEMA_SOURCE="../../ols-ucentral-schema/schema"

# 2. Extract all properties from schema
# For JSON schema file (recommended):
python3 extract-schema-properties.py ../../config-samples/ucentral.schema.pretty.json \
    2>/dev/null > /tmp/all-schema-properties.txt

# For YAML schema directory (if using ols-ucentral-schema repo):
# python3 extract-schema-properties.py ../../ols-ucentral-schema/schema ucentral.yml \
#     2>/dev/null > /tmp/all-schema-properties.txt

# 3. Generate base database (proto.c)
python3 generate-database-from-schema.py \
    ../../src/ucentral-client/proto.c \
    /tmp/all-schema-properties.txt \
    /tmp/base-database-new.c

# 4. Generate platform database (plat-gnma.c)
python3 generate-platform-database-from-schema.py \
    ../../src/ucentral-client/platform/brcm-sonic/plat-gnma.c \
    /tmp/all-schema-properties.txt \
    /tmp/platform-database-new.c

# 5. Install new databases
cp /tmp/base-database-new.c ../config-parser/property-database-base.c
cp /tmp/platform-database-new.c ../config-parser/property-database-platform-brcm-sonic.c

# 6. Test in Docker
docker exec ucentral_client_build_env bash -c \
    "cd /root/ols-nos/tests/config-parser && make clean && make test-config-full"

Result: Both databases regenerated with all schema properties, showing which are implemented (with line numbers) and which are not (line_number=0).

Property Database Regeneration

When to Regenerate

Regenerate property databases when:

New parser functions added to proto.c or platform code
Schema updated with new properties
Parser functions renamed or refactored
Starting fresh after major refactoring
Periodic audit (quarterly recommended)

Why Schema-Based?

Complete Coverage - Tracks ALL 398 schema properties
Single Source of Truth - Schema defines what's possible
Shows Gaps - Properties with line_number=0 are not yet implemented
Consistent - Same properties across all platforms
Maintainable - Automatic updates when schema changes

Base Database Generation

Generates property-database-base.c from proto.c:

cd tests/tools

# Extract schema properties from included JSON file
python3 extract-schema-properties.py ../../config-samples/ucentral.schema.pretty.json \
    2>/dev/null > /tmp/schema-props.txt

# OR if using YAML from ols-ucentral-schema repository:
# python3 extract-schema-properties.py ../../ols-ucentral-schema/schema ucentral.yml \
#     2>/dev/null > /tmp/schema-props.txt

# Generate database
python3 generate-database-from-schema.py \
    ../../src/ucentral-client/proto.c \
    /tmp/schema-props.txt \
    /tmp/base-db.c

# Install database
cp /tmp/base-db.c ../config-parser/property-database-base.c

What it does:

Searches proto.c for cJSON property access patterns
Finds line numbers where each property is parsed
Marks unimplemented properties with line_number=0
Generates complete C array with all schema properties

Platform Database Generation

Generates property-database-platform-*.c from platform code:

cd tests/tools

# Generate platform database (brcm-sonic example)
python3 generate-platform-database-from-schema.py \
    ../../src/ucentral-client/platform/brcm-sonic/plat-gnma.c \
    /tmp/schema-props.txt \
    /tmp/platform-db.c

cp /tmp/platform-db.c ../config-parser/property-database-platform-brcm-sonic.c

What it does:

Analyzes platform code for config_*_apply() functions
Maps properties to platform functions by feature area
Platform code doesn't parse JSON directly
Uses feature-based matching (poe → config_poe_port_apply)

Adding New Parser Functions

When you add a new parser function to proto.c:

// Example: New parser function
static int cfg_new_feature_parse(cJSON *obj, struct plat_cfg *cfg) {
    cJSON *item = cJSON_GetObjectItemCaseSensitive(obj, "new-property");
    // ...parse new-property...
}

Steps:

Implement the parser in proto.c
Regenerate database using schema-based approach (see Quick Start)

Verify the new property appears with correct line number:

grep "new-property" tests/config-parser/property-database-base.c
# Should show: {"path.to.new-property", PROP_CONFIGURED, "proto.c", "cfg_new_feature_parse", LINE, "..."}

Test with a config containing the new property
Commit proto.c changes and regenerated database together

Platform-Specific Updates

Platform vendors (Edgecore, Dell, etc.) maintain their own platform databases.

For Platform Developers

When adding platform-specific features:

cd tests/tools

# Regenerate YOUR platform database
python3 generate-platform-database-from-schema.py \
    ../../src/ucentral-client/platform/YOUR-PLATFORM/plat-YOUR.c \
    /tmp/schema-props.txt \
    /tmp/platform-db-YOUR.c

# Install it
cp /tmp/platform-db-YOUR.c ../config-parser/property-database-platform-YOUR.c

Note: Each platform tracks only its own implementation. Properties showing as "Unknown" in base repo may be "CONFIGURED" in your platform.

Schema Update Procedures

Included Schema File

This repository includes a default schema file in config-samples/:

ucentral.schema.pretty.json - uCentral JSON schema (human-readable format)

This file allows immediate use of the testing framework without fetching external repositories.

When to Update Schema

Update the schema when:

New version of ols-ucentral-schema is released
New configuration properties are added to the schema
Property definitions change (type, constraints, etc.)
Schema validation errors appear for valid configurations
Preparing for new feature development

Schema Update Process

Step 1: Identify Current Schema Version

cd /path/to/ols-ucentral-client

# Check current schema version
head -20 config-samples/ucentral.schema.pretty.json | grep -i version

# Note the current version for rollback if needed

Step 2: Obtain New Schema

Repository Location:

GitHub: https://github.com/Telecominfraproject/ols-ucentral-schema
Schema files are in the schema/ directory (YAML format)
Built/converted JSON schemas may be in releases or build artifacts

Option A: Using fetch-schema.sh Helper (Recommended)

cd tests/tools

# Fetch main branch
./fetch-schema.sh

# OR fetch specific branch
./fetch-schema.sh release-1.0

# View help
./fetch-schema.sh --help

The script clones the schema repository to tests/tools/ols-ucentral-schema/.

Option B: Manual Git Clone

cd tests/tools

# Clone the schema repository
git clone https://github.com/Telecominfraproject/ols-ucentral-schema.git

# Checkout specific version (recommended for stability)
cd ols-ucentral-schema
git tag -l                    # List available versions
git checkout v4.2.0           # Checkout specific version

# OR use specific branch
git checkout release-1.0

Option C: Download Specific File

If you only need the JSON schema file:

# Download directly from GitHub
cd config-samples
wget https://raw.githubusercontent.com/Telecominfraproject/ols-ucentral-schema/main/schema/ucentral.schema.json \
    -O ucentral.schema.pretty.json

Step 3: Validate Schema File

cd /path/to/ols-ucentral-client/tests/config-parser

# Verify schema is valid JSON
python3 -c "import json; json.load(open('../../config-samples/ucentral.schema.pretty.json'))"

# Test that validator can load it
python3 validate-schema.py --schema ../../config-samples/ucentral.schema.pretty.json \
    ../../config-samples/cfg0.json

Step 4: Test Against Existing Configurations

# Run schema validation on all test configs
make validate-schema

# Review results for new validation errors

Expected outcomes:

All valid: Schema is backward compatible
Some failures: Schema may have added requirements or changed definitions
Many failures: Schema may be incompatible - review changes carefully

Step 5: Address Validation Failures

If existing valid configs now fail validation:

A. Investigate Schema Changes

# Compare old and new schema
cd config-samples
diff <(jq . ols.ucentral.schema.json.old) <(jq . ols.ucentral.schema.json) > schema-changes.diff

# Look for:
# - New required fields
# - Changed property types
# - New constraints (min/max, enums)
# - Removed properties

B. Update Affected Configurations

# For each failing config:
# 1. Review the validation error
./validate-schema.py ../../config-samples/failing-config.json

# 2. Fix the configuration to meet new requirements
vi ../../config-samples/failing-config.json

# 3. Revalidate
./validate-schema.py ../../config-samples/failing-config.json

C. Document Breaking Changes

Create or update SCHEMA_CHANGES.md:

## Schema Update: v4.1.0 → v4.2.0

### Breaking Changes
- `unit.timezone` now required (was optional)
- `interfaces.ethernet.speed` changed from string to enum
- New required field: `switch.system-name`

### Configurations Updated
- cfg0.json - Added unit.timezone
- ECS4150-TM.json - Changed speed format
- All configs - Added switch.system-name with default value

### Migration Guide
For existing configurations:
1. Add `"timezone": "UTC"` to unit section
2. Change speed: `"1000"` → `"1G"`
3. Add `"system-name": "switch"` to switch section

Step 6: Update Schema Reference in Code

If schema location or format changed:

# Update validate-schema.py search paths if needed
vi tests/schema/validate-schema.py

# Update _find_default_schema() method:
#   script_dir / "../../config-samples/ucentral.schema.pretty.json",
#   script_dir / "../../config-samples/ols.ucentral.schema.json",

Step 7: Commit Schema Update

cd /path/to/ols-ucentral-client

# Add updated schema
git add config-samples/ols.ucentral.schema*.json

# Add any fixed configurations
git add config-samples/*.json

# Add documentation
git add src/ucentral-client/SCHEMA_CHANGES.md

# Commit with clear message
git commit -m "Update uCentral schema to v4.2.0

- Updated schema from ols-ucentral-schema v4.2.0
- Fixed 5 test configurations for new requirements
- Added timezone, updated speed enums, added system-name
- See SCHEMA_CHANGES.md for migration guide"

Schema Rollback Procedure

If new schema causes major issues:

# Revert to previous schema
git checkout HEAD~1 -- config-samples/ols.ucentral.schema*.json

# Or restore from backup
cp config-samples/ucentral.schema.pretty.json.backup \
   config-samples/ucentral.schema.pretty.json

# Verify old schema works
make validate-schema

Property Database Update Procedures

When to Update Property Database

Update the property database when:

New parser functions added to proto.c
Existing parser functions modified (name change, scope change)
Properties removed from parser implementation
Parser refactoring changes function organization
Adding support for new configuration features

Property Database Update Process

Step 1: Identify Parser Changes

A. New Feature Development

If you're actively developing:

# You know what functions you added
# Example: Added cfg_port_mirroring_parse()
grep -n "cfg_port_mirroring_parse" src/ucentral-client/proto.c

B. Code Update/Merge

If updating from upstream or merging branches:

cd /path/to/ols-ucentral-client/tests/config-parser

# Compare parser functions between versions
git diff HEAD~1 proto.c | grep "^+.*cfg_.*_parse"

# List all current parser functions
grep -n "^static.*cfg_.*_parse\|^cfg_.*_parse" proto.c | awk '{print $3}' | sort

C. Comprehensive Audit

Periodically audit all functions:

# Extract all cfg_*_parse functions from proto.c
grep -o "cfg_[a-z_]*_parse" proto.c | sort -u > current-functions.txt

# Extract all parser_function references from property database
grep "parser_function" test-config-parser.c | \
    sed 's/.*"\(cfg_[^"]*\)".*/\1/' | sort -u > database-functions.txt

# Find functions in proto.c but NOT in database (missing entries)
comm -23 current-functions.txt database-functions.txt > missing-from-database.txt

# Find functions in database but NOT in proto.c (invalid entries)
comm -13 current-functions.txt database-functions.txt > invalid-in-database.txt

# Review both files
cat missing-from-database.txt
cat invalid-in-database.txt

Step 2: Remove Invalid Property Entries

For functions that no longer exist:

A. Identify Properties to Remove

# For each invalid function, find its properties
INVALID_FUNC="cfg_old_feature_parse"

grep -n "\"$INVALID_FUNC\"" test-config-parser.c

# This shows line numbers of all property entries using this function

B. Remove Property Entries

#!/usr/bin/env python3
# remove-properties.py - Helper script to remove property entries

import sys
import re

if len(sys.argv) < 2:
    print("Usage: ./remove-properties.py <function_name>")
    sys.exit(1)

function_name = sys.argv[1]

with open('test-config-parser.c', 'r') as f:
    lines = f.readlines()

# Find and remove entries for this function
output_lines = []
skip_entry = False
brace_count = 0

for line in lines:
    # Check if this line starts a property entry with our function
    if f'parser_function = "{function_name}"' in line:
        # Find the start of this struct (previous { )
        # Mark to skip this entire entry
        skip_entry = True
        # Walk back to find the opening brace
        idx = len(output_lines) - 1
        while idx >= 0:
            if '{' in output_lines[idx]:
                output_lines = output_lines[:idx]
                break
            idx -= 1
        continue

    if skip_entry:
        if '},' in line:
            skip_entry = False
        continue

    output_lines.append(line)

with open('test-config-parser.c', 'w') as f:
    f.writelines(output_lines)

print(f"Removed entries for {function_name}")

Usage:

cd /path/to/ols-ucentral-client/tests/config-parser

# Remove entries for obsolete function
python3 remove-properties.py cfg_old_feature_parse

# Verify compilation still works
make clean
make test-config-parser

Manual Removal Alternative:

# Edit test-config-parser.c
vi test-config-parser.c

# Search for the function name: /cfg_old_feature_parse
# Delete the entire property entry (from opening { to closing },)

# Example - DELETE THIS ENTIRE BLOCK:
# {
#     .path = "some.old.property",
#     .parser_function = "cfg_old_feature_parse()",
#     .status = PROP_CONFIGURED,
#     .notes = "Old feature"
# },

Step 3: Add New Property Entries

For new parser functions:

A. Determine What Properties the Function Handles

# Read the function to understand what it parses
vi proto.c
# Search for: /cfg_new_feature_parse

# Look for cJSON_GetObjectItem calls to find property names
grep -A 50 "cfg_new_feature_parse" proto.c | grep "cJSON_GetObjectItem"

# Example output:
# cJSON_GetObjectItem(obj, "enabled")
# cJSON_GetObjectItem(obj, "mode")
# cJSON_GetObjectItem(obj, "timeout")

B. Determine Property Paths

Property paths follow JSON structure:

services.new-feature.enabled     → "services.new-feature.enabled"
interfaces.ethernet.speed        → "interfaces.ethernet.speed"
switch.spanning-tree.enabled     → "switch.spanning-tree.enabled"

For array items:

interfaces.ethernet[].name       → "interfaces.ethernet.name"
vlans[].id                       → "vlans.id"

C. Add Property Entries to Database

vi test-config-parser.c

# Find the properties[] array definition
# Add new entries in logical grouping with related properties

# Template:
# {
#     .path = "full.property.path",
#     .parser_function = "cfg_function_name()",
#     .status = PROP_CONFIGURED,
#     .notes = "Description of what this property does"
# },

Example Addition:

static struct property_info properties[] = {
    // ... existing entries ...

    // Port Mirroring Configuration (NEW)
    {
        .path = "services.port-mirroring.enabled",
        .parser_function = "cfg_port_mirroring_parse()",
        .status = PROP_CONFIGURED,
        .notes = "Enable/disable port mirroring service"
    },
    {
        .path = "services.port-mirroring.sessions",
        .parser_function = "cfg_port_mirroring_parse()",
        .status = PROP_CONFIGURED,
        .notes = "Array of mirroring session configurations"
    },
    {
        .path = "services.port-mirroring.sessions.id",
        .parser_function = "cfg_port_mirroring_parse()",
        .status = PROP_CONFIGURED,
        .notes = "Session identifier (1-4)"
    },
    {
        .path = "services.port-mirroring.sessions.source-ports",
        .parser_function = "cfg_port_mirroring_parse()",
        .status = PROP_CONFIGURED,
        .notes = "Array of source port names to mirror"
    },
    {
        .path = "services.port-mirroring.sessions.destination-port",
        .parser_function = "cfg_port_mirroring_parse()",
        .status = PROP_CONFIGURED,
        .notes = "Destination port name for mirrored traffic"
    },
    {
        .path = "services.port-mirroring.sessions.direction",
        .parser_function = "cfg_port_mirroring_parse()",
        .status = PROP_CONFIGURED,
        .notes = "Mirror direction: rx, tx, or both"
    },

    // ... rest of entries ...
};

Guidelines for Property Entries:

Grouping: Keep related properties together with a comment header
Ordering: Follow JSON structure hierarchy (parent before children)
Naming: Use exact JSON property names (hyphens, not underscores)
Status: Use PROP_CONFIGURED for actively parsed properties
Notes: Provide clear, concise description including valid values/ranges
Arrays: Use singular form without [] in path (e.g., "sessions.id" not "sessions[].id")

Step 4: Verify Property Database Accuracy

cd /path/to/ols-ucentral-client/tests/config-parser

# Rebuild test suite
make clean
make test-config-parser

# Run tests to see property usage report
make test-config

# Review the [PROPERTY USAGE REPORT] section
# Check for:
# - "Unknown (not in property database)" - missing entries
# - Properties with correct parser_function references
# - Properties marked as CONFIGURED that should be

Step 5: Test with Configurations Using New Properties

A. Create Test Configuration

cd config-samples

# Create test config demonstrating new feature
cat > test-new-feature.json <<'EOF'
{
  "uuid": 1,
  "unit": {
    "name": "test-new-feature",
    "timezone": "UTC"
  },
  "interfaces": {
    "ethernet": [
      {"name": "Ethernet0", "enabled": true}
    ]
  },
  "services": {
    "port-mirroring": {
      "enabled": true,
      "sessions": [
        {
          "id": 1,
          "source-ports": ["Ethernet0", "Ethernet1"],
          "destination-port": "Ethernet10",
          "direction": "both"
        }
      ]
    }
  }
}
EOF

B. Validate Configuration

cd /path/to/ols-ucentral-client/tests/config-parser

# Schema validation
./validate-schema.py ../../config-samples/test-new-feature.json

# Parser test
./test-config-parser ../../config-samples/test-new-feature.json

# Check property report shows properties as CONFIGURED
make test-config | grep -A 5 "port-mirroring"

Step 6: Document Property Database Changes

Create or update PROPERTY_DATABASE_CHANGES.md:

## Property Database Update: 2025-12-12

### Added Properties
- `services.port-mirroring.*` (6 properties)
  - Parser: cfg_port_mirroring_parse()
  - Feature: Port mirroring/SPAN configuration
  - Test config: test-new-feature.json

### Removed Properties
- `services.legacy-feature.*` (4 properties)
  - Reason: cfg_legacy_feature_parse() removed in commit abc123
  - Migration: Feature deprecated, no replacement

### Modified Properties
- `switch.spanning-tree.mode`
  - Changed parser: cfg_stp_parse() → cfg_spanning_tree_parse()
  - Reason: Parser function renamed for consistency

Step 7: Commit Property Database Changes

cd /path/to/ols-ucentral-client

# Add modified test file
git add tests/config-parser/test-config-parser.c

# Add test configuration if created
git add config-samples/test-new-feature.json

# Add documentation if created
git add tests/PROPERTY_DATABASE_CHANGES.md

# Commit
git commit -m "Update property database for port mirroring feature

- Added 6 property entries for services.port-mirroring
- Properties handled by cfg_port_mirroring_parse()
- Added test-new-feature.json demonstrating configuration
- All tests passing"

Property Database Maintenance Best Practices

Update Immediately: When adding new parser functions, add property entries immediately
Remove Promptly: When removing parser functions, clean up property entries in same commit
Test Always: Run full test suite after any property database changes
Document Changes: Maintain changelog of database modifications
Review Periodically: Audit database accuracy quarterly or after major updates
Platform Sync: If porting to platform repos, document platform-specific additions

Version Synchronization

Keeping Schema and Property Database in Sync

The schema and property database serve different but complementary purposes:

Schema - Defines what's structurally valid Property Database - Tracks what's actually implemented

Version Compatibility Matrix

Maintain a compatibility matrix:

## Version Compatibility

| Client Version | Schema Version | Property Count | Notes |
|----------------|----------------|----------------|-------|
| 1.0.0          | v4.0.0         | 420            | Initial release |
| 1.1.0          | v4.1.0         | 450            | Added STP, IGMP |
| 1.2.0          | v4.1.0         | 465            | Added PoE, 802.1X |
| 2.0.0          | v4.2.0         | 510            | Major feature update |

Update Coordination

When updating both schema and property database:

Schema First: Update schema, verify existing configs
Implement Features: Add parser functions for new schema properties
Update Database: Add property entries for new implementations
Test Thoroughly: Run complete test suite
Document Together: Update documentation explaining the changes

Tracking Implementation Status

Use property reports to track implementation progress:

# Generate property usage report
make test-config > report.txt

# Count properties by status
grep "Status: CONFIGURED" report.txt | wc -l
grep "Status: Unknown" report.txt | wc -l

# Identify unimplemented schema properties
grep "Unknown (not in property database)" report.txt

Testing After Updates

Complete Test Sequence

After updating schema or property database:

cd /path/to/ols-ucentral-client

# 1. Clean build
cd tests/config-parser
make clean

# 2. Rebuild test tools
make test-config-parser

# 3. Validate schema file
python3 -c "import json; json.load(open('../../config-samples/ucentral.schema.pretty.json'))"

# 4. Run schema validation
make validate-schema

# 5. Run parser tests
make test-config

# 6. Run full test suite
make test-config-full

# 7. Review property usage report
make test-config | grep -A 100 "PROPERTY USAGE REPORT"

Validation Checklist

Schema file is valid JSON
Schema validator loads successfully
All positive test configs pass schema validation
All negative test configs fail schema validation (expected)
All positive test configs pass parser tests
Property database has no references to non-existent functions
New properties appear in usage report with correct status
No unexpected "Unknown" properties for implemented features
Test results match expectations (pass/fail counts)
No memory leaks (valgrind if available)

Regression Testing

Keep baseline test results:

# Save baseline before changes
make test-config-full > test-results-baseline.txt

# After changes, compare
make test-config-full > test-results-new.txt
diff test-results-baseline.txt test-results-new.txt

# Expected differences:
# - New properties in usage report
# - Updated parser function references
# - New test configs results

# Unexpected differences:
# - Previously passing tests now fail
# - Properties changing status unexpectedly
# - Parser errors on existing configs

Troubleshooting

Schema Update Issues

Issue: Schema validation fails for all configs

Possible causes:

Schema file is corrupted or invalid JSON
Schema path incorrect in validate-schema.py
Schema format changed (Draft-7 vs Draft-4)

Resolution:

# Verify schema is valid JSON
python3 -m json.tool config-samples/ucentral.schema.pretty.json > /dev/null

# Check schema path detection
python3 validate-schema.py --schema ../../config-samples/ucentral.schema.pretty.json \
    ../../config-samples/cfg0.json

# Compare schema $schema property
grep '$schema' config-samples/ucentral.schema.pretty.json

Issue: New schema rejects previously valid configs

Possible causes:

Schema added new required fields
Schema changed property types
Schema added constraints (min/max, enums)

Resolution:

# Get detailed error
./validate-schema.py ../../config-samples/failing-config.json

# Compare schemas to find changes
diff old-schema.json new-schema.json

# Update configs to meet new requirements

Property Database Issues

Issue: Compilation fails after property database update

Possible causes:

Syntax error in property entry (missing comma, quote)
Invalid struct member
Property array not properly terminated

Resolution:

# Check compilation error message
make test-config-parser 2>&1 | head -20

# Common fixes:
# - Add missing comma after previous entry
# - Ensure notes string has closing quote
# - Check .path, .parser_function, .status, .notes are all present

Issue: Tests fail after property database update

Possible causes:

Removed properties still referenced elsewhere
Added properties with wrong parser function
Property paths don't match JSON structure

Resolution:

# Run specific config test
./test-config-parser ../../config-samples/failing-config.json

# Check property usage report
make test-config | grep -A 10 "property-name"

# Verify parser function exists
grep "function_name" proto.c

Issue: Properties showing as "Unknown" after adding to database

Possible causes:

Property path doesn't match JSON exactly
Parser function name has typo
Property entry not in properties[] array

Resolution:

# Check property path in JSON
cat config-samples/test-config.json | jq '.path.to.property'

# Verify parser function name exactly
grep -n "cfg_function_name" proto.c

# Ensure property entry is within properties[] array bounds
# (Check that it's before the closing }; )

General Maintenance Issues

Issue: Test results inconsistent between runs

Possible causes:

Configs modified between runs
Schema file changed
Test order dependency (should not happen)

Resolution:

# Check for uncommitted changes
git status

# Verify schema hasn't changed
git diff config-samples/ucentral.schema.pretty.json

# Run tests multiple times
for i in {1..3}; do make test-config; done

Issue: Docker environment tests fail but local tests pass

Possible causes:

Different schema version in container
Path differences
Missing dependencies

Resolution:

# Check schema in container
docker exec ucentral_client_build_env bash -c \
    "cat /root/ols-nos/config-samples/ucentral.schema.pretty.json" | head -20

# Verify paths in container
docker exec ucentral_client_build_env bash -c \
    "cd /root/ols-nos/tests/config-parser && ls -la ../../config-samples/*.json"

# Rebuild container if needed
make clean
make build-host-env

Quick Reference

Schema Update Commands

# Check current version
head -20 config-samples/ucentral.schema.pretty.json | grep version

# Update schema
cp /path/to/new/schema.json config-samples/ucentral.schema.pretty.json

# Validate schema
python3 -m json.tool config-samples/ucentral.schema.pretty.json > /dev/null

# Test with configs
make validate-schema

Property Database Commands

# List all parser functions
grep -n "^static.*cfg_.*_parse\|^cfg_.*_parse" proto.c | awk '{print $3}' | sort

# Find function in database
grep -n "function_name" test-config-parser.c

# Rebuild and test
make clean && make test-config

# View property report
make test-config | grep -A 100 "PROPERTY USAGE REPORT"

Testing Commands

# Full test suite
make test-config-full

# Schema only
make validate-schema

# Parser only
make test-config

# Single config
./test-config-parser ../../config-samples/specific-config.json

Legacy Approach

Note: The config-based approach described below has been superseded by the schema-based approach (see above). Legacy tools are archived in tests/tools/legacy/.

Old Config-Based Method (Pre-December 2024)

The original approach extracted properties FROM configuration files:

# OLD METHOD - No longer recommended
cd tests/tools/legacy
python3 generate-property-database.py ../../config-samples/*.json > /tmp/props.txt
python3 find-property-line-numbers.py ../../src/ucentral-client/proto.c /tmp/props.txt

Limitations of config-based approach:

Only tracked properties that appeared in test configs (628 properties)
Missed properties defined in schema but not in configs
No way to identify unimplemented schema properties
Inconsistent property paths (array indices varied)
Required manual curation

Migration to schema-based:

December 2024: Switched to schema as source of truth
Now tracks all 398 schema properties consistently
Shows implementation status for each property
Fully automated regeneration

For historical reference, see tests/tools/legacy/README.md.

32 KiB Raw Permalink Blame History

Configuration Testing Framework Maintenance Guide

Overview

Key Components

Schema-Based Workflow

Table of Contents

Quick Start: Schema-Based Regeneration

Property Database Regeneration

When to Regenerate

Why Schema-Based?

Base Database Generation

Platform Database Generation

Adding New Parser Functions

Platform-Specific Updates

For Platform Developers

Schema Update Procedures

Included Schema File

When to Update Schema

Schema Update Process

Step 1: Identify Current Schema Version

Step 2: Obtain New Schema

Step 3: Validate Schema File

Step 4: Test Against Existing Configurations

Step 5: Address Validation Failures

Step 6: Update Schema Reference in Code

Step 7: Commit Schema Update

Schema Rollback Procedure

Property Database Update Procedures

When to Update Property Database

Property Database Update Process

Step 1: Identify Parser Changes

Step 2: Remove Invalid Property Entries

Step 3: Add New Property Entries

Step 4: Verify Property Database Accuracy

Step 5: Test with Configurations Using New Properties

Step 6: Document Property Database Changes

Step 7: Commit Property Database Changes

Property Database Maintenance Best Practices

Version Synchronization

Keeping Schema and Property Database in Sync

Version Compatibility Matrix

Update Coordination

Tracking Implementation Status

Testing After Updates

Complete Test Sequence

Validation Checklist

Regression Testing

Troubleshooting

Schema Update Issues

Property Database Issues

General Maintenance Issues

Quick Reference

Schema Update Commands

Property Database Commands

Testing Commands

See Also

Legacy Approach

Old Config-Based Method (Pre-December 2024)

32 KiB

Raw Permalink Blame History