Oracle-DB: Cleanup Unified Audit Trail with dynamic number of rows and oldest timestamp, but limited storage size
For housekeeping of Unified Audit Trail in a recent project there were some objectives:
This would require to estimate a maximum age of held audit records that ensures also in worst cases that the storage size limit will not be exceeded. In result, outside the worst case of audit volume you will purge the audit records far too early.
You could have stored them much longer because there's enough space below the storage limit.
The following SQL script accepts all named objectives by ensuring:
The really allocated storage will be higher, e.g. due to PctFree, LOB indexes and the possibly nearly empty oldest partition.
- A hard rule that audit records must be kept in all cases for at least x days
- A hard rule that the stored audit records should not exceed a limited storage size, except the the above age limit requires this
- A soft objective thet audit records should be stored as long as possible if the storage size limit is not reached
This would require to estimate a maximum age of held audit records that ensures also in worst cases that the storage size limit will not be exceeded. In result, outside the worst case of audit volume you will purge the audit records far too early.
You could have stored them much longer because there's enough space below the storage limit.
The following SQL script accepts all named objectives by ensuring:
- Audit records younger than x will never be purged
- Old audit records are purged if necessary to reach the defined size limit if they do not violate the minimum age limit
- Audit records are stored as long as the storage size limit is not exceeded
-- purge the unified audit trail with follwing rules:
-- Audit records are purged if size/amount of stored audit data exceeds a threshold
-- Audit records younger than minimum age are never purged
-- 2025-01-27, Peter Ramm, OSP Dresden
/*
If the period to purge is larger than the interval of one partition in audsys.AUD$Unified,
then DBMS_AUDIT_MGMT.CLEAN_AUDIT_TRAIL will use DROP PARTITION instead of DELETE if all records of a partition should be purged.
For efficient purge it is suggested to set the partioning interval smaller than the call cycle of this script.
Use DBMS_AUDIT_MGMT.ALTER_PARTITION_INTERVAL to define the used interval.
Oracle's default of 1 month would be too coarse in most cases.
*/
SET SERVEROUTPUT ON LINESIZE 300;
DECLARE
MIN_DAYS_KEEP CONSTANT NUMBER := 14; -- Minimum age of audit records before considered for purge, adjust to your needs
MAX_DATA_SIZE_MB CONSTANT NUMBER := 500; -- Net size limit of stored audit records, adjust to your needs
v_Avg_Row_Len NUMBER;
v_Min_Timestamp TIMESTAMP; -- timestamp of oldest record
v_Max_Timestamp TIMESTAMP;
v_Max_Rows_to_Keep NUMBER;
v_Current_Rows_Count NUMBER;
v_Rows_to_purge NUMBER;
PROCEDURE Log(p_Msg IN VARCHAR2) IS
BEGIN
DBMS_OUTPUT.PUT_LINE(TO_CHAR(SYSDATE, 'YYYY-MM-DD HH24:MI:SS')||' '||p_Msg);
END Log;
BEGIN
Log('===== Start purge Unified Audit Trail =====');
-- Use audsys.AUD$Unified to calculate the avg. net size of an audit entry
SELECT Avg_Row_Len INTO v_Avg_Row_Len FROM DBA_Tables WHERE Owner = 'AUDSYS' AND Table_Name = 'AUD$UNIFIED';
IF v_Avg_Row_Len IS NULL THEN
RAISE_APPLICATION_ERROR(-20999, 'audsys.AUD$Unified must be analyzed to know the avg. row len');
END IF;
v_Max_Rows_to_Keep := ROUND(MAX_DATA_SIZE_MB * 1024*1024 / v_Avg_Row_Len);
Log('Minimum days to keep : '||MIN_DAYS_KEEP);
Log('Maximum remaining data size after purge : '||MAX_DATA_SIZE_MB||' MB');
Log('Maximum number of rows accepted : '||v_Max_Rows_to_Keep);
-- Use view Unified_Audit_Trai to count all audit entries
-- table rows from audsys.AUD$Unified as well as files in filesystem for r/o instances (GV$Unified_Audit_Trail)
SELECT /*+ PARALLEL(2) */ COUNT(*), MIN(Event_Timestamp) INTO v_Current_Rows_Count, v_Min_Timestamp FROM Unified_Audit_Trail;
Log('Current number of rows in Unified_Audit_Trail : '||v_Current_Rows_Count);
Log('Current oldest stored record is from : '||TO_CHAR(v_Min_Timestamp, 'YYYY-MM-DD HH24:MI_SS'));
if v_Current_Rows_Count < v_Max_Rows_to_Keep THEN
Log('Nothing to purge because size limit is not reached');
Log('Calculated net data size is : '||ROUND(v_Current_Rows_Count * v_Avg_Row_Len / (1024*1024), 2)||' MB');
ELSE
IF v_Min_Timestamp > SYSDATE - MIN_DAYS_KEEP THEN
Log('Size limit is exceeded, but all audit records are younger than age threshold. Nothing to purge.');
Log('Calculated net data size is : '||ROUND(v_Current_Rows_Count * v_Avg_Row_Len / (1024*1024), 2)||' MB');
ELSE
Log('Calculated net data size is : '||ROUND(v_Current_Rows_Count * v_Avg_Row_Len / (1024*1024), 2)||' MB');
-- Use view Unified_Audit_Trai to count all audit entries incl. file system
SELECT /*+ PARALLEL(2) */ MAX(Event_Timestamp), COUNT(*) INTO v_Max_Timestamp, v_Rows_To_Purge
FROM (SELECT RowNum Row_Num, Event_Timestamp
FROM (SELECT Event_Timestamp
FROM Unified_Audit_Trail
WHERE Event_Timestamp < SYSDATE - MIN_DAYS_KEEP
ORDER BY Event_Timestamp DESC
)
)
WHERE Row_num > v_Max_Rows_to_Keep
;
Log('Approximate number of rows to purge : '||v_Rows_To_Purge);
Log('Purge audit records older than : '||TO_CHAR(v_Max_Timestamp, 'YYYY-MM-DD HH24:MI_SS'));
DBMS_AUDIT_MGMT.SET_LAST_ARCHIVE_TIMESTAMP(audit_trail_type => DBMS_AUDIT_MGMT.AUDIT_TRAIL_UNIFIED,
last_archive_time => v_Max_Timestamp
);
DBMS_AUDIT_MGMT.CLEAN_AUDIT_TRAIL(audit_trail_type => DBMS_AUDIT_MGMT.AUDIT_TRAIL_UNIFIED,
use_last_arch_timestamp => TRUE
);
SELECT /*+ PARALLEL(2) */ COUNT(*), MIN(Event_Timestamp) INTO v_Current_Rows_Count, v_Min_Timestamp FROM Unified_Audit_Trail;
Log('No. of rows after purge in Unified_Audit_Trail: '||v_Current_Rows_Count);
Log('Oldest stored record after purge is from : '||TO_CHAR(v_Min_Timestamp, 'YYYY-MM-DD HH24:MI_SS'));
END IF;
END IF;
Log('===== Finished purge Unified Audit Trail =====');
END;
/
Please keep in mind that the storage size limit is treated as net data size (avg. row length * number of rows).The really allocated storage will be higher, e.g. due to PctFree, LOB indexes and the possibly nearly empty oldest partition.
Comments
Post a Comment