[Bug][azuredevops_go] YAML Syntax Error breaks Pipeline

[techdecline]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

The azuredevops_go plugin fails the entire DevLake pipeline when it encounters Azure DevOps builds created by YAML syntax errors. These builds are valid Azure DevOps build records but do not have a usable Timeline, causing timeline parsing to fail and abort the pipeline.

This behavior blocks data ingestion even though the underlying build failure is expected and should be handled gracefully.

What do you expect to happen

The Azure DevOps Go plugin should gracefully handle missing or invalid timelines, for example:

• Skip builds whose timeline response cannot be parsed
• Log a warning instead of failing the pipeline
• Mark the build as failed/incomplete and continue
• Treat timeline collection as best effort, not mandatory

At minimum, timeline parsing errors should be non‑fatal for the overall pipeline.

How to reproduce

Add an Azure DevOps Project to the connection's data scope that contains a pipeline with a YAML error that has been triggered at least once within the selected timeframe. Run the data collection. The data collection job will fail at ado:<scope name>.

This an error description based on the original logs, but refined using AI to strip confidential information and increase readability.

Error running DevLake pipeline

Root cause:
subtask collectApiTimelineRecords ended unexpectedly

Stack (compressed):
- server/services.runPipeline
- core/runner.RunPipeline
- core/runner.RunPluginTask
- core/runner.RunPluginSubTasks
- plugins/azuredevops_go/tasks.CollectRecords
- helpers/pluginhelper/api.(*ApiCollector).Execute
- helpers/pluginhelper/api.(*WorkerScheduler).WaitAsync

Combined errors:
{
  error parsing response from <ORG>/<PROJECT>/_apis/build/builds/<BUILD_ID_1>/Timeline?api-version=7.1
  =====================
  error parsing response from <ORG>/<PROJECT>/_apis/build/builds/<BUILD_ID_2>/Timeline?api-version=7.1
  =====================
  error parsing response from <ORG>/<PROJECT>/_apis/build/builds/<BUILD_ID_3>/Timeline?api-version=7.1
}

Notes:
- All affected builds failed due to YAML syntax errors in pipeline definitions
- Timeline API responses are empty, malformed, or non-JSON
- Failure of a single timeline causes the entire pipeline to abort

Anything else

No response

Version

v1.0.3-beta10@99376a8

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[dosubot]

Hi @techdecline! I'm Dosu and I’m helping the incubator-devlake team.

Thanks for the detailed report. I dug into the code and can confirm the root cause and suggest where a fix should go.

Root cause: The ParseRawMessageFromRecords response parser calls UnmarshalResponse, which returns a hard error on both empty responses (ErrEmptyResponse) and malformed/non-JSON bodies. These errors propagate up through the ApiCollector and abort the entire subtask.

The plugin already has a pattern for gracefully skipping problematic builds — ignoreDeletedBuilds uses api.ErrIgnoreAndContinue in the AfterResponse handler to skip 404s. However, when a YAML-broken build returns a 200 with an empty or non-JSON timeline body, the AfterResponse check passes (it's not a 404), and the error only surfaces later during response parsing where there's no ignore/skip mechanism.

Suggested fix approach: There are a couple of options:

1. Expand AfterResponse — Check for empty or non-JSON response bodies in ignoreDeletedBuilds (or a new wrapper) and return api.ErrIgnoreAndContinue for those cases too. This is the minimal change.

2. Make ParseRawMessageFromRecords tolerant — Return an empty []json.RawMessage instead of an error when the response is empty or cannot be parsed as JSON, with a warning log. This treats missing timelines as "no records" rather than a fatal error.

Also worth noting: the collector query on line 52 already filters out result != "failed" builds, but builds broken by YAML errors may still have a non-failed result (e.g., none) depending on how Azure DevOps categorizes them, so they slip through.

Since you've indicated you're willing to submit a PR — option 1 is probably the cleanest and most consistent with the existing codebase patterns. You could also consider adding a filter for builds with result = "none" at the query level if those are consistently the problematic ones.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}
   

[techdecline]

Hey @dosu, Just created a Pull Request based on your and my own analysis.

[dosubot]

Nice, thanks for jumping on that so quickly! If you link the PR here, I'd be happy to take a look at the implementation and provide feedback.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}
   

[techdecline]

There you go @dosu #8839

[dosubot]

Thanks for sharing! I took a look at the PR — the approach looks solid. You implemented both fixes we discussed:

1. ignoreInvalidTimelineResponse handler — replaces ignoreDeletedBuilds and now also skips empty/non-JSON bodies via api.ErrIgnoreAndContinue, with the body peek-and-restore via io.NopCloser so valid responses still flow through to the parser. Clean solution.
2. Query-level filter — excluding result = 'none' builds in addition to 'failed', which avoids even hitting the Timeline API for YAML-broken builds in the first place.

Good to see unit tests covering the key scenarios (404, empty body, non-JSON body, valid JSON passthrough, and valid empty JSON object). The defense-in-depth approach here is nice — the query filter prevents most of the bad requests, and the AfterResponse handler catches anything that slips through.

Looks good to me overall. Hope it gets picked up by the maintainers soon!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}