Skip to content

Conversation

@felipestanzani
Copy link
Collaborator

Conformance Test Suite and Decoder Improvements

Overview

This PR introduces a comprehensive conformance test suite for both encoding and decoding operations, along with significant improvements to the ValueDecoder parsing logic, error handling, and code maintainability.

Changes Summary

  • 47 files changed: 4,500 insertions(+), 282 deletions(-)

Key Features

🧪 Comprehensive Conformance Test Suite

  • Decoding conformance tests covering:

    • Arrays (nested, primitive, tabular)
    • Objects and nested structures
    • Primitives and root-form values
    • Delimiters (comma, tab, pipe)
    • Whitespace handling
    • Blank lines
    • Indentation validation and error cases
    • Path expansion
    • Number parsing
    • Validation errors
  • Encoding conformance tests covering:

    • Arrays (nested, primitive, objects)
    • Objects and nested structures
    • Primitives
    • Delimiters
    • Whitespace handling
    • Custom options
  • Test infrastructure:

    • Refactored ConformanceTest to utilize JUnit 5 DynamicContainer for better test organization
    • Dynamic test generation from JSON fixture files
    • Improved error handling and test structure

🔧 Decoder Improvements

ValueDecoder Refactoring

  • Streamlined parsing logic: Simplified validation of multiple primitives at root level by reducing redundant checks
  • Enhanced tabular array processing: Introduced a new method for processing lines in tabular arrays, enhancing readability and maintainability
  • Improved whitespace handling: Enhanced whitespace tolerance and error handling in decoding processes
  • Better indentation validation: Refactored indentation validation logic for improved accuracy
  • Enhanced error handling: Improved parsing logic with better error messages and validation
  • Path expansion support: Added support for path expansion in DecodeOptions and ValueDecoder

Code Quality Improvements

  • Enhanced comments and documentation throughout ValueDecoder
  • Method simplifications for better maintainability
  • Improved clarity and performance in parsing logic

🛠️ Additional Changes

  • StringEscaper utility: Added StringEscaper class with unescaping functionality for handling escaped strings
  • PathExpansion support: Enhanced DecodeOptions with path expansion capabilities
  • Test structure refactoring: Reorganized test project structure for better maintainability
  • Dependabot configuration: Updated Dependabot schedule from weekly to monthly

Testing

All conformance tests are passing ✅

The test suite validates:

  • Round-trip encoding/decoding
  • Edge cases and error conditions
  • Various delimiter formats
  • Whitespace and indentation handling
  • Path expansion functionality

Impact

This PR significantly improves:

  • Test coverage: Comprehensive conformance tests ensure TOON format compliance
  • Code maintainability: Refactored decoder logic is more readable and maintainable
  • Error handling: Better error messages and validation improve developer experience
  • Robustness: Enhanced whitespace and indentation handling make the decoder more tolerant of various input formats

Files Changed

Core Changes

  • src/main/java/com/felipestanzani/jtoon/decoder/ValueDecoder.java - Major refactoring (1,228+ lines)
  • src/main/java/com/felipestanzani/jtoon/decoder/PrimitiveDecoder.java - Improvements
  • src/main/java/com/felipestanzani/jtoon/DecodeOptions.java - Path expansion support
  • src/main/java/com/felipestanzani/jtoon/PathExpansion.java - New utility
  • src/main/java/com/felipestanzani/jtoon/util/StringEscaper.java - New utility

Test Infrastructure

  • src/test/java/com/felipestanzani/jtoon/conformance/ConformanceTest.java - Comprehensive test suite
  • src/test/java/com/felipestanzani/jtoon/conformance/model/ - Test fixture models
  • src/test/resources/conformance/decode/ - 13 decode test fixtures
  • src/test/resources/conformance/encode/ - 8 encode test fixtures

Breaking Changes

None - this is a backward-compatible enhancement.

Invokedzz and others added 20 commits November 9, 2025 20:00
feature: create test case for json encoding
…ptions handling

- Refactored the encoding conformance tests to use dynamic tests via JUnit's TestFactory.
- Introduced a new JsonTestOptions model to encapsulate encoding options.
- Updated JsonTestCase to use JsonTestOptions instead of a String for options.
- Improved test fixture loading and parsing for better maintainability.
- Removed unnecessary ObjectMapper instantiation from JToon.
- Simplified the decodeToJson method in JToon by delegating to ValueDecoder.
- Enhanced PrimitiveDecoder with a switch statement for better readability.
- Updated ValueDecoder to streamline parsing logic and improve error handling.
…structures

- Introduced EncodeTestFixture and DecodeTestFixture to encapsulate test data.
- Added JsonEncodeTestCase and JsonDecodeTestCase for encoding and decoding test cases.
- Refactored ConformanceTest to support dynamic test generation for both encoding and decoding.
- Updated parsing methods to handle new test case structures and options.
- Removed obsolete JsonTestCase and TestFixture classes for improved clarity.
…or model structures

- Added error handling for decoding test cases, allowing tests to expect exceptions.
- Updated JsonDecodeTestCase to include a shouldError flag for better test case management.
- Refactored parsing methods to remove unnecessary ObjectMapper instantiation.
- Removed ExpectedDecodeResult model as part of the cleanup process.
- Introduced PathExpansion enum to manage dotted key expansion modes.
- Updated DecodeOptions to include expandPaths parameter with default set to OFF.
- Refactored ValueDecoder to handle path expansion for keys, allowing nested object structures.
- Enhanced parsing methods to validate and expand keys based on the specified path expansion mode.
- Improved error handling for blank lines and delimiter mismatches in strict mode.
… parsing logic

- Changed project SDK language level from JDK_25 to JDK_21.
- Added checks in PrimitiveDecoder to return whole numbers as Long if within range.
- Enhanced ValueDecoder to handle special cases for null input and improved error handling for blank lines.
- Introduced new methods for parsing arrays and handling path expansion conflicts in strict mode.
    - Enhanced parsing logic in ValueDecoder to streamline handling of keyed arrays and key-value pairs.
    - Introduced methods for processing blank lines and direct child lines to improve readability and maintainability.
    - Updated error handling for blank lines and indentation validation in strict mode.
    - Removed redundant checks and improved comments for clarity.
…ut handling

- Modified shouldTerminateListArray method to include line content checks for improved array parsing.
- Enhanced parsing logic to handle empty values with nested content, ensuring correct object creation.
- Updated JToonDecodeTest and RoundTripTest to assert empty input returns empty map instead of null, improving consistency in decoding behavior.
…validation

- Enhanced parsing logic to avoid trimming leading whitespace, ensuring proper indentation validation.
- Updated handling of trailing whitespace to prevent issues with empty lines.
- Improved error handling for tab characters in strict mode, providing clearer feedback on indentation errors.
…ation

- Updated testJSONFile methods to return DynamicNode instead of Stream<DynamicTest>, enhancing test structure.
- Introduced createTestContainer method to group tests under their respective file names for better readability.
- Simplified dynamic test creation by removing redundant parameters in createDynamicTest methods.
… to monthly; refactor ValueDecoder for improved parsing logic and error handling, including method simplifications and enhanced comments.
…andling

- Simplified the validation of multiple primitives at root level by reducing redundant checks.
- Introduced a new method for processing lines in tabular arrays, enhancing readability and maintainability.
- Improved handling of blank lines and termination conditions in both tabular and list arrays.
- Updated comments for clarity and better understanding of the parsing flow.
This was linked to issues Nov 12, 2025
@felipestanzani felipestanzani merged commit 28d8358 into main Nov 12, 2025
1 check passed
@felipestanzani felipestanzani deleted the conformance branch November 12, 2025 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Conformance Tests Decode function

3 participants