Skip to content

fix(log_object_drops): correct commit LSN, DROP DATABASE path, TEMP skip, and PITR docs (v5 review fixes) #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
NikolayS wants to merge 1 commit into
base: master
Choose a base branch
from
Open

fix(log_object_drops): correct commit LSN, DROP DATABASE path, TEMP skip, and PITR docs (v5 review fixes) #55

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions contrib/postgres_fdw/connection.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ static void do_sql_command_end(PGconn *conn, const char *sql,
static void begin_remote_xact(ConnCacheEntry *entry);
static void pgfdw_report_internal(int elevel, PGresult *res, PGconn *conn,
const char *sql);
static void pgfdw_xact_callback(XactEvent event, void *arg);
static void pgfdw_xact_callback(XactEvent event, void *arg, XLogRecPtr lsn);
static void pgfdw_subxact_callback(SubXactEvent event,
SubTransactionId mySubid,
SubTransactionId parentSubid,
Expand Down Expand Up @@ -1170,7 +1170,7 @@ pgfdw_report_internal(int elevel, PGresult *res, PGconn *conn,
* COMMIT TRANSACTION may run deferred triggers.)
*/
static void
pgfdw_xact_callback(XactEvent event, void *arg)
pgfdw_xact_callback(XactEvent event, void *arg, XLogRecPtr lsn)
{
HASH_SEQ_STATUS scan;
ConnCacheEntry *entry;
Expand Down
2 changes: 1 addition & 1 deletion contrib/sepgsql/label.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ sepgsql_set_client_label(const char *new_label)
* changes in the client_label_pending list.
*/
static void
sepgsql_xact_callback(XactEvent event, void *arg)
sepgsql_xact_callback(XactEvent event, void *arg, XLogRecPtr lsn)
{
if (event == XACT_EVENT_COMMIT)
{
Expand Down
69 changes: 69 additions & 0 deletions doc/src/sgml/config.sgml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8055,6 +8055,75 @@ local0.* /var/log/postgresql
</listitem>
</varlistentry>

<varlistentry id="guc-log_object_drops" xreflabel="log_object_drops">
<term><varname>log_object_drops</varname> (<type>boolean</type>)
<indexterm>
<primary><varname>log_object_drops</varname> configuration parameter</primary>
</indexterm>
</term>
<listitem>
<para>
Causes <command>DROP TABLE</command> and <command>DROP DATABASE</command>
operations to be logged with their commit LSN (Log Sequence Number).
The commit LSN represents the point in the write-ahead log where the
drop operation becomes durable and visible to other transactions.
</para>

<para>
Each logged entry includes the schema name, table name, OID, and the
commit LSN. For example:
<programlisting>
LOG: DROP TABLE: relation "public.employees" (OID 16384), commit LSN: 0/015D4A48
LOG: DROP DATABASE: database "testdb" (OID 16385), commit LSN: 0/015D4B20
</programlisting>
</para>

<para>
Logged operations include: direct <command>DROP TABLE</command> statements,
tables dropped via <command>DROP SCHEMA CASCADE</command>,
partitioned tables and their partitions, tables in inheritance hierarchies,
and <command>DROP DATABASE</command> commands.
Operations that are rolled back (via <command>ROLLBACK</command> or
<command>ROLLBACK TO SAVEPOINT</command>) are not logged.
Temporary tables are not logged, since their contents cannot be
recovered by point-in-time recovery.
</para>

<para>
This parameter is useful for tracking and auditing destructive operations,
and for coordinating with external systems that monitor the write-ahead log.
Note that enabling this option may generate substantial log volume
when dropping schemas or databases containing many tables.
</para>

<para>
The default is <literal>off</literal>. Only superusers and users with
the appropriate <literal>SET</literal> privilege can change this setting.
</para>

<para>
<emphasis>Using the logged LSN for point-in-time recovery.</emphasis>
The logged value is the LSN at which the <emphasis>commit</emphasis>
record of the transaction that performed the drop begins, which is the
value <xref linkend="guc-recovery-target-lsn"/> compares against. To
recover the dropped object, recovery must stop
<emphasis>before</emphasis> that record is replayed.
Because <xref linkend="guc-recovery-target-inclusive"/> defaults to
<literal>on</literal>, which stops recovery <emphasis>after</emphasis>
the target LSN, setting <varname>recovery_target_lsn</varname> to the
logged LSN alone would replay the drop and lose the object again.
You must also set <varname>recovery_target_inclusive</varname> to
<literal>off</literal>:
<programlisting>
recovery_target_lsn = '0/015D4A48' # the commit LSN from the log message
recovery_target_inclusive = off # required: stop BEFORE the drop commits
</programlisting>
With these settings, recovery stops just short of the drop, and the
object is present in the recovered cluster. Note that any transaction
that committed after this LSN is also not recovered.
</para>
</listitem>
</varlistentry>

<varlistentry id="guc-log-duration" xreflabel="log_duration">
<term><varname>log_duration</varname> (<type>boolean</type>)
Expand Down
54 changes: 36 additions & 18 deletions src/backend/access/transam/xact.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ static void AtCommit_Memory(void);
static void AtStart_Cache(void);
static void AtStart_Memory(void);
static void AtStart_ResourceOwner(void);
static void CallXactCallbacks(XactEvent event);
static void CallXactCallbacks(XactEvent event, XLogRecPtr lsn);
static void CallSubXactCallbacks(SubXactEvent event,
SubTransactionId mySubid,
SubTransactionId parentSubid);
Expand Down Expand Up @@ -1342,9 +1342,10 @@ AtSubStart_ResourceOwner(void)
* If you change this function, see RecordTransactionCommitPrepared also.
*/
static TransactionId
RecordTransactionCommit(void)
RecordTransactionCommit(XLogRecPtr *commit_lsn_out)
{
TransactionId xid = GetTopTransactionIdIfAny();
XLogRecPtr commit_lsn = InvalidXLogRecPtr;
bool markXidCommitted = TransactionIdIsValid(xid);
TransactionId latestXid = InvalidTransactionId;
int nrels;
Expand Down Expand Up @@ -1481,13 +1482,24 @@ RecordTransactionCommit(void)
/*
* Insert the commit XLOG record.
*/
XactLogCommitRecord(GetCurrentTransactionStopTimestamp(),
nchildren, children, nrels, rels,
ndroppedstats, droppedstats,
nmsgs, invalMessages,
RelcacheInitFileInval,
MyXactFlags,
InvalidTransactionId, NULL /* plain commit */ );
(void) XactLogCommitRecord(GetCurrentTransactionStopTimestamp(),
nchildren, children, nrels, rels,
ndroppedstats, droppedstats,
nmsgs, invalMessages,
RelcacheInitFileInval,
MyXactFlags,
InvalidTransactionId, NULL /* plain commit */ );

/*
* Report the *start* of the commit record to XactCallbacks, not its
* end. Recovery compares recovery_target_lsn against the start LSN of
* each record (see recoveryStopsBefore()), so the start LSN is the
* value a DBA can feed back to recovery_target_lsn, together with
* recovery_target_inclusive = off, to stop just before this
* transaction commits. The end LSN would be the start of the *next*
* record and would therefore replay this commit.
*/
commit_lsn = ProcLastRecPtr;

if (replorigin)
/* Move LSNs forward for this replication origin */
Expand Down Expand Up @@ -1610,6 +1622,9 @@ RecordTransactionCommit(void)
if (ndroppedstats)
pfree(droppedstats);

if (commit_lsn_out)
*commit_lsn_out = commit_lsn;

return latestXid;
}

Expand Down Expand Up @@ -2272,6 +2287,7 @@ CommitTransaction(void)
TransactionState s = CurrentTransactionState;
TransactionId latestXid;
bool is_parallel_worker;
XLogRecPtr commit_lsn = InvalidXLogRecPtr;

is_parallel_worker = (s->blockState == TBLOCK_PARALLEL_INPROGRESS);

Expand Down Expand Up @@ -2320,7 +2336,8 @@ CommitTransaction(void)
*/

CallXactCallbacks(is_parallel_worker ? XACT_EVENT_PARALLEL_PRE_COMMIT
: XACT_EVENT_PRE_COMMIT);
: XACT_EVENT_PRE_COMMIT,
InvalidXLogRecPtr);

/*
* If this xact has started any unfinished parallel operation, clean up
Expand Down Expand Up @@ -2404,7 +2421,7 @@ CommitTransaction(void)
* We need to mark our XIDs as committed in pg_xact. This is where we
* durably commit.
*/
latestXid = RecordTransactionCommit();
latestXid = RecordTransactionCommit(&commit_lsn);
}
else
{
Expand Down Expand Up @@ -2447,7 +2464,8 @@ CommitTransaction(void)
*/

CallXactCallbacks(is_parallel_worker ? XACT_EVENT_PARALLEL_COMMIT
: XACT_EVENT_COMMIT);
: XACT_EVENT_COMMIT,
commit_lsn);

CurrentResourceOwner = NULL;
ResourceOwnerRelease(TopTransactionResourceOwner,
Expand Down Expand Up @@ -2597,7 +2615,7 @@ PrepareTransaction(void)
break;
}

CallXactCallbacks(XACT_EVENT_PRE_PREPARE);
CallXactCallbacks(XACT_EVENT_PRE_PREPARE, InvalidXLogRecPtr);

/*
* The remaining actions cannot call any user-defined code, so it's safe
Expand Down Expand Up @@ -2757,7 +2775,7 @@ PrepareTransaction(void)
* that cure could be worse than the disease.
*/

CallXactCallbacks(XACT_EVENT_PREPARE);
CallXactCallbacks(XACT_EVENT_PREPARE, InvalidXLogRecPtr);

ResourceOwnerRelease(TopTransactionResourceOwner,
RESOURCE_RELEASE_BEFORE_LOCKS,
Expand Down Expand Up @@ -3011,9 +3029,9 @@ AbortTransaction(void)
if (TopTransactionResourceOwner != NULL)
{
if (is_parallel_worker)
CallXactCallbacks(XACT_EVENT_PARALLEL_ABORT);
CallXactCallbacks(XACT_EVENT_PARALLEL_ABORT, InvalidXLogRecPtr);
else
CallXactCallbacks(XACT_EVENT_ABORT);
CallXactCallbacks(XACT_EVENT_ABORT, InvalidXLogRecPtr);

ResourceOwnerRelease(TopTransactionResourceOwner,
RESOURCE_RELEASE_BEFORE_LOCKS,
Expand Down Expand Up @@ -3889,7 +3907,7 @@ UnregisterXactCallback(XactCallback callback, void *arg)
}

static void
CallXactCallbacks(XactEvent event)
CallXactCallbacks(XactEvent event, XLogRecPtr lsn)
{
XactCallbackItem *item;
XactCallbackItem *next;
Expand All @@ -3898,7 +3916,7 @@ CallXactCallbacks(XactEvent event)
{
/* allow callbacks to unregister themselves when called */
next = item->next;
item->callback(event, item->arg);
item->callback(event, item->arg, lsn);
}
}

Expand Down
Loading
Loading