You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I am having problems using target-postgres because I could not find a way to do a fresh load of the data, i.e. truncate / delete whatever is already in the DB and insert incoming data.
I want to be able to only load the rows that are coming from the extractors and overwrite whatever was in the tables from a previous load.
Am I missing something? Note: I am using target-postgres in Meltano
Describe alternatives you've considered
The only alternative I can actually use (given I am working with multiple types extractors) is to write my own code that truncates a table if the table is available.
Additional context
I understand the point of having incremental loads and merging strategies , but there should also be the option to start fresh.
Perhaps the feature is there and I don't know how to use it?
As a simple example of working with a CSV file with 10 rows:
run extract and load for the first time on the CSV file with 10 rows => 10 rows in the DB table , all good
delete 3 rows from the CSV file and run extract and load again => 10 rows in the DB table... there needs to be the option to end up with 7 rows because the file has changed.
Is your feature request related to a problem? Please describe.
I am having problems using
target-postgresbecause I could not find a way to do a fresh load of the data, i.e. truncate / delete whatever is already in the DB and insert incoming data.I want to be able to only load the rows that are coming from the extractors and overwrite whatever was in the tables from a previous load.
Am I missing something?
Note: I am using target-postgres in Meltano
Describe the solution you'd like
I would like there to be a flag similar to https://hub.meltano.com/loaders/target-bigquery/#replication_method-setting
in which one can specify
replication_method: truncateand have the tables truncated before loading in the incoming dataDescribe alternatives you've considered
The only alternative I can actually use (given I am working with multiple types extractors) is to write my own code that truncates a table if the table is available.
Additional context
I understand the point of having incremental loads and merging strategies , but there should also be the option to start fresh.
Perhaps the feature is there and I don't know how to use it?
As a simple example of working with a CSV file with 10 rows: