Improve YAML validation when using check endpoint

[AhmedFatthy1040]

This PR fixes an issue with Configuration as Code (CasC) YAML validation by improving error handling for incorrect indentation and invalid structures. The fix ensures that malformed YAML is correctly detected and provides detailed error messages.

Your checklist for this pull request

🚨 Please review the guidelines for contributing to this repository.

• Make sure you are requesting to pull a topic/feature/bugfix branch (right side) and not your master branch!

• Ensure that the pull request title represents the desired changelog entry

• Please describe what you did

  • Improved YAML validation to handle syntax errors more effectively.
  • Enhanced error reporting by providing line numbers and detailed messages.

• Link to relevant issues in GitHub or in Jenkins JIRA

  • Fixes Make /manage/configuration-as-code/check respond with a JSON list of issues #2628

• Link to relevant pull requests, esp. upstream and downstream changes

• Did you provide a test-case? That demonstrates feature works or fixes the issue.

  • Yes, I tested the fix using the Jenkins Script Console with the following Groovy script:

   // Test script for Configuration as Code YAML validation
import io.jenkins.plugins.casc.ConfigurationAsCode
import io.jenkins.plugins.casc.yaml.YamlSource
import org.yaml.snakeyaml.Yaml
import net.sf.json.JSONArray
import net.sf.json.JSONObject
import java.io.StringReader
import java.nio.file.Files
import java.nio.file.Path

// Valid YAML test case
def validYaml = """
jenkins:
  systemMessage: "Hello from Groovy script"
  numExecutors: 2
"""

// Invalid YAML test case (bad indentation)
def invalidYaml = """
jenkins:
systemMessage: "This YAML is invalid"
  numExecutors: {
"""

// Mock response class to capture results
class MockResponse {
    int status = 200
    String contentType
    StringWriter writer = new StringWriter()
    
    void setContentType(String type) {
        this.contentType = type
    }
    
    void setStatus(int status) {
        this.status = status
    }
    
    PrintWriter getWriter() {
        return new PrintWriter(writer)
    }
    
    String getContent() {
        return writer.toString()
    }
}

// Mock request class
class MockRequest {
    private String yamlContent
    
    MockRequest(String yamlContent) {
        this.yamlContent = yamlContent
    }
}

// Function to simulate API call
def simulateApiCall(String yaml, String testName) {
    println "\n==== Testing ${testName} ===="
    println "YAML Content:"
    println yaml
    
    try {
        // Create a temporary file with the YAML content
        def tempFile = Files.createTempFile("config", ".yaml")
        tempFile.toFile().text = yaml
        
        // Create a YamlSource from the temp file
        def source = YamlSource.of(tempFile)
        
        // Get the ConfigurationAsCode singleton
        def casc = ConfigurationAsCode.get()
        
        // Validate the YAML using checkWith method (this tests the internal logic)
        def result = casc.checkWith(source)
        
        // First show the standard validation result
        println "\nStandard Validation Result:"
        if (result.isEmpty()) {
            println "✅ YAML is valid! No issues found."
        } else {
            println "⚠️ Validation messages:"
            result.each { sourceObj, message ->
                println "- Line ${sourceObj?.line ?: 'unknown'}: ${message}"
            }
        }
        
        // Now create a MockResponse to simulate API behavior
        def mockResponse = new MockResponse()
        
        // Simulate API call by creating JSON response
        JSONArray response = new JSONArray()
        result.entrySet().stream()
                .forEach { entry ->
                    response.add(new JSONObject()
                        .accumulate("line", entry.getKey()?.line ?: -1)
                        .accumulate("message", entry.getValue()))
                }
                
        mockResponse.setContentType("application/json")
        response.write(mockResponse.getWriter())
        mockResponse.getWriter().flush()
        
        println "\nSimulated API Response:"
        println "Status Code: ${mockResponse.status}"
        println "Content Type: ${mockResponse.contentType}"
        println "Response Body: ${mockResponse.getContent()}"
        
        // Parse the response back to verify it's valid JSON
        def parsedJson = new groovy.json.JsonSlurper().parseText(mockResponse.getContent())
        println "Parsed JSON contains ${parsedJson.size()} issue(s)"
        
        // Cleanup
        Files.delete(tempFile)
        
    } catch (Exception e) {
        println "\n❌ Error during validation:"
        println e.toString()
        e.printStackTrace()
        if (e.message?.contains("<html")) {
            println "FAILED: Got HTML error instead of proper response"
        }
    }
}

// Test with syntax error YAML
def syntaxErrorYaml = """
jenkins:
  - this: is not valid
  - because: 
    - systemMessage is supposed to be a string
    not: an array
"""

// Test with unknown property
def unknownPropertyYaml = """
jenkins:
  nonExistentProperty: "This property doesn't exist in Jenkins config"
  systemMessage: "But this one is valid"
"""

// Run the tests
println "Starting Configuration as Code YAML validation tests..."
simulateApiCall(validYaml, "Valid YAML configuration")
simulateApiCall(invalidYaml, "Invalid YAML syntax")
simulateApiCall(syntaxErrorYaml, "YAML with syntax errors")
simulateApiCall(unknownPropertyYaml, "YAML with unknown properties")

println "\n==== Test Summary ===="
println "Tests completed! Check that all scenarios return proper JSON responses without HTML errors."
println "Valid YAML should return status 200 with empty array."
println "Invalid YAML should return status 400 with error messages in JSON format."

// Return 'null' to prevent Jenkins from trying to serialize the last expression
return null

• Results:
  • ✅ Valid YAML: Passed without errors.
  • ❌ Invalid YAML: Detected syntax issues with appropriate error messages.

Starting Configuration as Code YAML validation tests...

==== Testing Valid YAML configuration ====
YAML Content:

jenkins:
  systemMessage: "Hello from Groovy script"
  numExecutors: 2


Standard Validation Result:
✅ YAML is valid! No issues found.

Simulated API Response:
Status Code: 200
Content Type: application/json
Response Body: []
Parsed JSON contains 0 issue(s)

==== Testing Invalid YAML syntax ====
YAML Content:

jenkins:
systemMessage: "This YAML is invalid"
  numExecutors: {


❌ Error during validation:
while parsing a block mapping
 in C:\Users\Ahmed\AppData\Local\Temp\config451284278673233401.yaml, line 2, column 1:
    jenkins:
    ^
expected <block end>, but found '<block mapping start>'
 in C:\Users\Ahmed\AppData\Local\Temp\config451284278673233401.yaml, line 4, column 3:
      numExecutors: {
      ^


==== Testing YAML with syntax errors ====
YAML Content:

jenkins:
  - this: is not valid
  - because: 
    - systemMessage is supposed to be a string
    not: an array


❌ Error during validation:
io.jenkins.plugins.casc.ConfiguratorException: Item isn't a Mapping

==== Testing YAML with unknown properties ====
YAML Content:

jenkins:
  nonExistentProperty: "This property doesn't exist in Jenkins config"
  systemMessage: "But this one is valid"


❌ Error during validation:
io.jenkins.plugins.casc.UnknownAttributesException: jenkins: Invalid configuration elements for type: class jenkins.model.Jenkins : nonExistentProperty.
Available attributes : authorizationStrategy, clouds, crumbIssuer, disableRememberMe, disabledAdministrativeMonitors, globalNodeProperties, labelAtoms, labelString, log, markupFormatter, mode, myViewsTabBar, noUsageStatistics, nodeMonitors, nodeName, nodeProperties, nodes, numExecutors, primaryView, projectNamingStrategy, proxy, quietPeriod, remotingSecurity, scmCheckoutRetryCount, securityRealm, slaveAgentPort, systemMessage, updateCenter, views, viewsTabBar

==== Test Summary ====
Tests completed! Check that all scenarios return proper JSON responses without HTML errors.
Valid YAML should return status 200 with empty array.
Invalid YAML should return status 400 with error messages in JSON format.

[timja]

Hey this looks good, instead of testing it via the script console could you provide an integration test please?

There should be plenty of examples in e.g https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/test-harness/src/test/java/io/jenkins/plugins/casc/ConfigurationAsCodeTest.java

[AhmedFatthy1040]

Thanks, I've added the integration test as requested. @timja