get_record_info(fields=[...], info=[...]) silently drops the requested INFO column

[d-laub]

Summary

VCF.get_record_info(fields=[...], info=[...]) (and by extension VCF._write_gvi_index(info=[...])) silently drops the requested INFO column when BOTH a non-empty fields= list and a non-empty info= list are passed together. No error is raised -- the resulting DataFrame/LazyFrame schema simply omits the INFO column entirely (confirmed via both .collect_schema() immediately after the call and .collect()).

Repro

from genoray import VCF

v = VCF("some.vcf.gz")  # header declares INFO/AF (Number=1, Type=Float)

lf = v.get_record_info(fields=["CHROM", "POS", "REF", "ALT"], info=["AF"], lazy=True)
print(lf.collect_schema())
# Schema([('CHROM', Enum(...)), ('POS', Int32), ('REF', String), ('ALT', List(String))])
# -- no AF, no nested "info"/"INFO" struct column at all.

df = v.get_record_info(fields=["CHROM", "POS", "REF", "ALT"], info=["AF"], lazy=False)
print(df.schema)
# same -- AF is missing even when eagerly collected.

By contrast, VCF._fetch_info_cols(["AF"]) (genoray's own internal helper, used only for the SVLEN/END/IMPRECISE special case in _write_gvi_index) returns the column correctly:

v._fetch_info_cols(["AF"]).collect()
# shape: (3, 2)  columns: AF, POS -- correct values.

Suspected root cause

get_record_info chains .rename(lambda c: c.upper()) onto the freshly-constructed oxbow-registered io-source LazyFrame before it's ever collected:

df = (
    cast(pl.LazyFrame, reader(self.path, samples=[], fields=fields, info_fields=info, regions=region).pl(lazy=True))
    .rename(lambda c: c.upper())
    .with_columns(pl.col("CHROM").cast(pl.Enum(self.contigs)))
)

_fetch_info_cols does not rename -- it reads the raw lowercase-named io-source frame directly and does an explicit .unnest("info").

The io-source's own column-projection callback (VariantFile._batchreader_builder's inner builder(columns, batch_size)) does a case-sensitive check:

if columns is not None:
    ...
    if "info" not in columns:
        scan_kwargs["info_fields"] = []
    ...

If polars' projection pushdown, when passing through the .rename() node, hands the builder a columns list that doesn't contain the lowercase "info" sentinel (e.g. because it uses the renamed/uppercase names, or otherwise fails to round-trip through the rename), this callback force-resets info_fields=[], and the INFO data never gets fetched from the underlying VCF/BCF at all -- explaining why the column is missing entirely rather than merely un-flattened.

This exactly matches why _write_gvi_index's SV-field handling (SVLEN/END/IMPRECISE) deliberately routes through _fetch_info_cols instead of get_record_info's info= parameter -- that workaround sidesteps this bug for the SV-specific case, but the general user-supplied info= parameter of _write_gvi_index still uses the broken get_record_info path and is therefore broken for any caller.

Impact

Any caller of VCF.get_record_info(fields=[a non-empty list], info=[...]) or VCF._write_gvi_index(info=[...]) silently loses the requested INFO column(s) with no error/warning, producing a schema and on-disk index that's missing data the caller explicitly asked for.

Suggested fix

Either:

1. Have get_record_info route its info= fetch through the same _fetch_info_cols-style (no-rename, explicit-unnest, eager-collect) path already used for SV fields, or
2. Investigate/fix the projection-pushdown-through-rename interaction directly so .rename() doesn't corrupt the io-source builder's column-projection callback.

Workaround (downstream)

genvarloader (_write.py) works around this by calling _write_gvi_index() with no info= at all, then separately fetching the requested INFO column(s) via _fetch_info_cols(...) and horizontally attaching them onto the written .gvi index itself (with a POS-alignment cross-check), mirroring genoray's own SV-field workaround.