Hi @ajnebro and @rorro6787,
I would like to include a feature in SAES. That is, allow loading Parquet files as well as CSV as input for SAES.
This would allow users to load files that, when stored as CSV, occupied significantly more disk space.
I have already spotted where the changes need to be made, and it wouldn't break any functionality of the library:
|
data = pd.read_csv(data, delimiter=",") if isinstance(data, str) else data |
|
|
|
# Load the metrics DataFrame, either from a CSV file or as an existing DataFrame |
|
metrics = pd.read_csv(metrics, delimiter=",") if isinstance(metrics, str) else metrics |
The Parquet files should have the same schema as the expected CSV file, and once loaded, the resulting pd.Dataframe can be used as always.
I would like to contribute to SAES by making this change, if you allow me.
Hi @ajnebro and @rorro6787,
I would like to include a feature in SAES. That is, allow loading Parquet files as well as CSV as input for SAES.
This would allow users to load files that, when stored as CSV, occupied significantly more disk space.
I have already spotted where the changes need to be made, and it wouldn't break any functionality of the library:
SAES/SAES/utils/dataframe_processor.py
Lines 94 to 97 in 97e152b
The Parquet files should have the same schema as the expected CSV file, and once loaded, the resulting pd.Dataframe can be used as always.
I would like to contribute to SAES by making this change, if you allow me.