Description of the bug
Summary
This PR addresses two failure modes observed in production runs that can terminate otherwise valid trait-unit executions:
Incorrect file-type detection in pgscatalog-intersect can route non-BIM input through BIM parsing.
Versions collation can fail when malformed YAML reaches DUMPSOFTWAREVERSIONS.
Both issues caused hard pipeline stops and reduced resume reliability.
Versioning
Workflow: pgscatalog/pgsc_calc 2.2.0
Nextflow: 25.10.4
Containers seen in failing runs:
ghcr.io/pgscatalog/pygscatalog:pgscatalog-utils-2.0.0
ghcr.io/pgscatalog/pyyaml:6.0
Affected versions
Confirmed on pgsc_calc 2.2.0 (and corresponding runtime containers above).
Fixed in
Local integration fix released in PGS_autorun v2.2.
Upstream pgsc_calc fix version: TBD by maintainers (to be assigned in release notes once merged).
Problem 1: pgscatalog-intersect misclassifies inputs
read_var_general() uses substring matching on filename ("bim" in name) instead of extension-aware detection.
If a non-BIM temporary/work filename contains "bim", parsing is misrouted and fails with:
KeyError: 'CHR:POS:A0:A1'
Expected behavior
Only true .bim inputs should take the BIM parser path.
Actual behavior
Any filename containing "bim" may be treated as BIM.
Impact
Trait-unit failure in ancestry/intersection stage despite valid data.
Proposed upstream fix
Use strict extension detection (name.endswith(".bim")) or equivalent normalized suffix check.
Problem 2: DUMPSOFTWAREVERSIONS YAML collation failure
DUMPSOFTWAREVERSIONS reads collated_versions.yml; malformed YAML causes:
yaml.scanner.ScannerError: while scanning a simple key ... could not find expected ':'
Observed failure is consistent with heredoc/content leakage producing invalid YAML structure.
Expected behavior
Software-version collation should tolerate/avoid malformed intermediate YAML and complete report generation.
Actual behavior
Trait unit fails at final collation/report step.
Impact
Run terminates late even when core compute stages have completed.
Proposed upstream fix
Ensure version files are emitted with strict YAML-safe writing paths (no heredoc leakage into content).
Add defensive validation before merge/collation with clearer diagnostics (and optional graceful fallback when feasible).
Reproduction notes (high-level)
Run a trait unit where non-BIM paths can include the substring "bim" in work/temp naming.
Observe intersect-stage failure with KeyError: 'CHR:POS:A0:A1'.
In affected runs, inspect collated_versions.yml near the reported line and observe malformed YAML.
Observe DUMPSOFTWAREVERSIONS failure with yaml.scanner.ScannerError.
Command used and terminal output
Relevant files
No response
System information
No response
Description of the bug
Summary
This PR addresses two failure modes observed in production runs that can terminate otherwise valid trait-unit executions:
Incorrect file-type detection in pgscatalog-intersect can route non-BIM input through BIM parsing.
Versions collation can fail when malformed YAML reaches DUMPSOFTWAREVERSIONS.
Both issues caused hard pipeline stops and reduced resume reliability.
Versioning
Workflow: pgscatalog/pgsc_calc 2.2.0
Nextflow: 25.10.4
Containers seen in failing runs:
ghcr.io/pgscatalog/pygscatalog:pgscatalog-utils-2.0.0
ghcr.io/pgscatalog/pyyaml:6.0
Affected versions
Confirmed on pgsc_calc 2.2.0 (and corresponding runtime containers above).
Fixed in
Local integration fix released in PGS_autorun v2.2.
Upstream pgsc_calc fix version: TBD by maintainers (to be assigned in release notes once merged).
Problem 1: pgscatalog-intersect misclassifies inputs
read_var_general() uses substring matching on filename ("bim" in name) instead of extension-aware detection.
If a non-BIM temporary/work filename contains "bim", parsing is misrouted and fails with:
KeyError: 'CHR:POS:A0:A1'
Expected behavior
Only true .bim inputs should take the BIM parser path.
Actual behavior
Any filename containing "bim" may be treated as BIM.
Impact
Trait-unit failure in ancestry/intersection stage despite valid data.
Proposed upstream fix
Use strict extension detection (name.endswith(".bim")) or equivalent normalized suffix check.
Problem 2: DUMPSOFTWAREVERSIONS YAML collation failure
DUMPSOFTWAREVERSIONS reads collated_versions.yml; malformed YAML causes:
yaml.scanner.ScannerError: while scanning a simple key ... could not find expected ':'
Observed failure is consistent with heredoc/content leakage producing invalid YAML structure.
Expected behavior
Software-version collation should tolerate/avoid malformed intermediate YAML and complete report generation.
Actual behavior
Trait unit fails at final collation/report step.
Impact
Run terminates late even when core compute stages have completed.
Proposed upstream fix
Ensure version files are emitted with strict YAML-safe writing paths (no heredoc leakage into content).
Add defensive validation before merge/collation with clearer diagnostics (and optional graceful fallback when feasible).
Reproduction notes (high-level)
Run a trait unit where non-BIM paths can include the substring "bim" in work/temp naming.
Observe intersect-stage failure with KeyError: 'CHR:POS:A0:A1'.
In affected runs, inspect collated_versions.yml near the reported line and observe malformed YAML.
Observe DUMPSOFTWAREVERSIONS failure with yaml.scanner.ScannerError.
Command used and terminal output
Relevant files
No response
System information
No response