Introduction
Maintaining global indexes were always a expensive task in Oracle, particularly in the context of partitioned tables, where dropping or truncating a table partition could cause a global index to become UNUSABLE/INVALID unless specified to update the indexes during the drop/truncate operation. However, updating the index entries during partition drop/truncate can eventually slow down the actual drop/truncate operation.
With Oracle 12c, this is now a story of past. In Oracle 12c, Oracle has improvised/optimized a lot of existing database functionalities. One of these improvisation is the maintenance of global indexes while dropping or truncating a table partition. With Oracle 12c, a drop or truncate table partition (with update indexes clause) is optimized by deferring the maintenance of associated global indexes, while still leaving the indexes in VALID state.
Prior to 12c, a drop/truncate table partition (with update indexes clause) would cause a synchronous maintenance for the associated global indexes and thereby would delay the actual drop/truncate operation. However, with 12c this global index maintenance operation is performed asynchronously which optimizes the drop or truncate partition operation.
In today’s article, we are going to explore this new enhancement introduced with Oracle database 12c.
How it works?
Before, starting with the exploration and demonstration; let me provide a brief idea about how this enhancement works within a Oracle 12c database.
Starting with 12c, Oracle defers (postpones) the deletion of global index entries associated with a dropped or truncated table partition. Oracle internally maintains the list of global index entries whose associated table partitions has been dropped and mark those index entries as ORPHAN entries. These OPRPHAN index entries are cleaned by default on a regular basis through a scheduler job named SYS.PMO_DEFERRED_GIDX_MAINT_JOB. The OPRPHAN index entries can also be cleaned up manually on demand either by manually running the scheduler job or by executing DBMS_PART.CLEANUP_GIDX procedure or by rebuilding global index/index partitions or by coalescing the index/index partitions. Additionally, when we query data from a table; Oracle scans through the orphaned index entries and ignores any record which points to a orphan entry.
Since, Oracle need not update the global index entries synchronously during the drop or truncate table partition operation, it optimizes and speeds up the actual drop or truncate table partition operation by a much higher magnitude.
Demonstration
Lets go through a quick demonstration, to understand this functionality. In the following example, I am creating a partitioned table for our demonstration. I would be demonstrating the functionality with truncate operation. You can expect a similar result for drop operations too.
---//
---// create a partitioned table for demo //---
---//
SQL> create table T_PART_AYSNC
2 (
3 id number,
4 name varchar(15),
5 join_date date
6 )
7 partition by range (join_date)
8 (
9 PARTITION P1 values less than (TO_DATE('01-FEB-2016','DD-MON-YYYY')) tablespace MYAPP_TS,
10 PARTITION P2 values less than (TO_DATE('01-MAR-2016','DD-MON-YYYY')) tablespace MYAPP_TS,
11 PARTITION P3 values less than (TO_DATE('01-APR-2016','DD-MON-YYYY')) tablespace MYAPP_TS
12 );
Table created.
Now, let’s populate the partitioned table with some data.
---//
---// populating partitioned table //---
---//
SQL> insert /*+ APPEND */ into T_PART_AYSNC
2 select rownum, rpad('X',15,'X'), '13-JAN-2016'
3 from dual connect by rownum <= 1e5;
100000 rows created.
SQL> commit;
Commit complete.
SQL> insert /*+ APPEND */ into T_PART_AYSNC
2 select rownum+1e5, rpad('X',15,'X'), '13-FEB-2016'
3 from dual connect by rownum<= 1e5;
100000 rows created.
SQL> commit;
Commit complete.
SQL> insert /*+ APPEND */ into T_PART_AYSNC
2 select rownum+2e5, rpad('X',15,'X'), '13-MAR-2016'
3 from dual connect by rownum <= 1e5;
100000 rows created.
SQL> commit;
Commit complete.
Now, lets create a couple of global indexes on the partitioned table
---// ---// create global partitioned index on the partitioned table //--- ---// SQL> create unique index T_PART_AYSNC_PK on T_PART_AYSNC (id) global 2 partition by range (id) 3 ( 4 partition id_p1 values less than (100001) tablespace MYAPP_TS, 5 partition id_p2 values less than (200001) tablespace MYAPP_TS, 6 partition id_p3 values less than (maxvalue) tablespace MYAPP_TS 7 ); Index created. ---// ---// create global index on the partitioned table //--- ---// SQL> create index idx_T_PART_AYSNC_G on T_PART_AYSNC (join_date); Index created.
At this point, we have a partitioned table with three partitions having 100000 records in each partition. We also have two global indexes on this partitioned table, one of which a partitioned global index and the other is a normal global index.
Let’s validate the status of the global indexes, before we actually drop a table partition
---// ---// global index status //--- ---// SQL> select i.index_name,null partition_name,i.num_rows,s.blocks, 2 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries 3 from dba_indexes i, dba_segments s 4 where i.index_name=s.segment_name and i.index_name='IDX_T_PART_AYSNC_G' 5 union 6 select i.index_name,i.partition_name,i.num_rows,s.blocks, 7 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries 8 from dba_ind_partitions i, dba_segments s 9 where i.partition_name=s.partition_name and i.index_name='T_PART_AYSNC_PK' 10 ; INDEX_NAME PARTITION_ NUM_ROWS BLOCKS LEAF_BLOCKS SIZE_MB STATUS ORP -------------------- ---------- ---------- ---------- ----------- ---------- -------- --- IDX_T_PART_AYSNC_G 300000 1024 962 8 VALID NO T_PART_AYSNC_PK ID_P1 100000 1024 264 8 USABLE NO T_PART_AYSNC_PK ID_P2 100000 1024 265 8 USABLE NO T_PART_AYSNC_PK ID_P3 100000 1024 265 8 USABLE NO
There is a new column ORPHANED_ENTRIES in this output. This column indicates, whether an index has any ORPHAN entries associated with it as a result of a drop/truncate partition operation.
Now, let us examine; how expensive would be the drop table partition operation with index maintenance. To measure the cost, we will take a note of the “db block gets” and “redo size” for the current database session before and after the truncate table partition operation and will compare the cost with truncate table partition operation performed without index maintenance.
Let’s first analyze the cost involved in truncate table partition operation without index maintenance.
---//
---// noting db block gets and redo size for current session (before truncate partition) //---
---//
SQL> select sn.name,ss.value from v$statname sn, v$sesstat ss
2 where sn.statistic#=ss.statistic#
3 and sn.name in ('redo size','db block gets')
4 and ss.sid = (select sys_context('userenv','sid') from dual)
5 ;
NAME VALUE
-------------------- ----------
db block gets 3
redo size 992
---//
---// truncate table partition without index maintenance //---
---//
SQL> alter table T_PART_AYSNC truncate partition P1;
Table truncated.
---//
---// noting db block gets and redo size for current session (after truncate partition) //---
---//
SQL> select sn.name,ss.value from v$statname sn, v$sesstat ss
2 where sn.statistic#=ss.statistic#
3 and sn.name in ('redo size','db block gets')
4 and ss.sid = (select sys_context('userenv','sid') from dual)
5 ;
NAME VALUE
-------------------- ----------
db block gets 106
redo size 22112
The truncate table partition (without index maintenance) operation took 103 block gets and 21120 bytes of redo to complete the truncate operation. However, it left the global indexes in INVALID/UNUSABLE state as found below.
---// ---// Index status after drop partition without index maintenance //--- ---// SQL> select i.index_name,null partition_name,i.num_rows,s.blocks, 2 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries 3 from dba_indexes i, dba_segments s 4 where i.index_name=s.segment_name and i.index_name='IDX_T_PART_AYSNC_G' 5 union 6 select i.index_name,i.partition_name,i.num_rows,s.blocks, 7 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries 8 from dba_ind_partitions i, dba_segments s 9 where i.partition_name=s.partition_name and i.index_name='T_PART_AYSNC_PK' 10 ; INDEX_NAME PARTITION_ NUM_ROWS BLOCKS LEAF_BLOCKS SIZE_MB STATUS ORP -------------------- ---------- ---------- ---------- ----------- ---------- -------- --- IDX_T_PART_AYSNC_G 300000 1024 962 8 UNUSABLE NO T_PART_AYSNC_PK ID_P1 100000 1024 264 8 UNUSABLE NO T_PART_AYSNC_PK ID_P2 100000 1024 265 8 UNUSABLE NO T_PART_AYSNC_PK ID_P3 100000 1024 265 8 UNUSABLE NO
Lets rebuild the indexes and evaluate the cost of truncating the table partition with index maintenance in place.
---//
---// noting db block gets and redo size for current session (before truncate partition) //---
---//
SQL> select sn.name,ss.value from v$statname sn, v$sesstat ss
2 where sn.statistic#=ss.statistic#
3 and sn.name in ('redo size','db block gets')
4 and ss.sid = (select sys_context('userenv','sid') from dual)
5 ;
NAME VALUE
-------------------- ----------
db block gets 3
redo size 992
---//
---// truncate table partition with index maintenance //---
---//
SQL> alter table T_PART_AYSNC truncate partition P2 update global indexes;
Table truncated.
---//
---// noting db block gets and redo size for current session (after truncate partition) //---
---//
SQL> select sn.name,ss.value from v$statname sn, v$sesstat ss
2 where sn.statistic#=ss.statistic#
3 and sn.name in ('redo size','db block gets')
4 and ss.sid = (select sys_context('userenv','sid') from dual)
5 ;
NAME VALUE
-------------------- ----------
db block gets 122
redo size 25284
As we can observe, it took just 119 block gets and 24292 bytes of redo to complete the truncate partition operation with index maintenance. This is almost similar to the cost of truncating the table partition without updating the global indexes.
Now, let’s find out the status of the global indexes after this truncate operation.
---//
---// global index status after truncate partition with index maintenance //---
---//
SQL> exec dbms_stats.gather_table_stats('MYAPP','T_PART_AYSNC');
PL/SQL procedure successfully completed.
SQL> select i.index_name,null partition_name,i.num_rows,s.blocks,
2 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries
3 from dba_indexes i, dba_segments s
4 where i.index_name=s.segment_name and i.index_name='IDX_T_PART_AYSNC_G'
5 union
6 select i.index_name,i.partition_name,i.num_rows,s.blocks,
7 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries
8 from dba_ind_partitions i, dba_segments s
9 where i.partition_name=s.partition_name and i.index_name='T_PART_AYSNC_PK'
10 ;
INDEX_NAME PARTITION_ NUM_ROWS BLOCKS LEAF_BLOCKS SIZE_MB STATUS ORP
-------------------- ---------- ---------- ---------- ----------- ---------- -------- ---
IDX_T_PART_AYSNC_G 100000 768 322 6 VALID YES
T_PART_AYSNC_PK ID_P1 0 1024 0 8 USABLE YES
T_PART_AYSNC_PK ID_P2 0 1024 0 8 USABLE YES
T_PART_AYSNC_PK ID_P3 100000 1024 265 8 USABLE YES
As we can observe, the global indexes are still in VALID/USABLE state. However, this time the new column ORPHANED_ENTRIES is showing a value YES. This indicates that the associated global indexes are now having orphan index entries as a result of the truncate table partition operation.
Now the question comes, how Oracle ensures that these indexes will return correct result as they have orphaned entries associated with them? Well, as I mentioned earlier; Oracle internally maintains the list of all orphaned entries associated with a global index. We can view the orphaned entries by querying SYS.INDEX_ORPHANED_ENTRY$ or SYS.INDEX_ORPHANED_ENTRY_V$ views as shown below.
---//
---// viewing orphaned index entries //---
---//
SQL> select * from sys.index_orphaned_entry$ order by 1;
INDEXOBJ# TABPARTDOBJ# H
---------- ------------ -
86586 86583 O
86587 86583 O
86588 86583 O
86589 86583 O
SQL> select * from SYS.INDEX_ORPHANED_ENTRY_V$;
INDEX_OWNER INDEX_NAME INDEX_SUBNAME INDEX_OBJECT_ID TABLE_OWNER TABLE_NAME TABLE_SUBNAME TABLE_OBJECT_ID T
--------------- -------------------- ------------- --------------- --------------- --------------- --------------- --------------- -
MYAPP T_PART_AYSNC_PK ID_P1 86586 MYAPP T_PART_AYSNC 86581 O
MYAPP T_PART_AYSNC_PK ID_P2 86587 MYAPP T_PART_AYSNC 86581 O
MYAPP T_PART_AYSNC_PK ID_P3 86588 MYAPP T_PART_AYSNC 86581 O
MYAPP IDX_T_PART_AYSNC_G 86589 MYAPP T_PART_AYSNC 86581 O
Entries from this view point to the list of indexes (INDEXOBJ#) which are having orphaned index entries and the table partition (TABPARTDOBJ#) to which all these orphaned entries belong to.
Here, we can see all the orphaned index entries are linked to table partition object 86583, which is the partition P2 that we have truncated earlier.
When we query data (using index) from a truncated/dropped table partition, Oracle scans through this list of orphaned index entries to avoid/ignore querying data from the respective table partitions (which is/are truncated or dropped).
---//
---// query data from table with dropped/truncated partition //---
---//
SQL> select /*+ gather_plan_statistics */ * from T_PART_AYSNC where id>10 and id<100;
no rows selected
SQL> select * from table(dbms_xplan.display_cursor(null,null,'allstats last'));
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------
SQL_ID 6md0spb139jqj, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ * from T_PART_AYSNC where id>10
and id<100
Plan hash value: 1060913503
-------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |
-------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 0 |00:00:00.01 | 1 |
| 1 | PARTITION RANGE SINGLE | | 1 | 1 | 0 |00:00:00.01 | 1 |
| 2 | TABLE ACCESS BY GLOBAL INDEX ROWID BATCHED| T_PART_AYSNC | 1 | 1 | 0 |00:00:00.01 | 1 |
|* 3 | INDEX RANGE SCAN | T_PART_AYSNC_PK | 1 | 1 | 0 |00:00:00.01 | 1 |
-------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("ID">10 AND "ID"<100)
filter(TBL$OR$IDX$PART$NUM(>,0,8,0,"T_PART_AYSNC".ROWID)=1)
22 rows selected.
Here is another example, where we query data from both dropped and existing partitions.
---//
---// query data from table with dropped/truncated partition //---
---//
SQL> select /*+ gather_plan_statistics */ * from T_PART_AYSNC where id>199999 and id<200002;
ID NAME JOIN_DATE
---------- -------------------- ---------
200001 XXXXXXXXXXXXXXX 13-MAR-16
SQL> select * from table(dbms_xplan.display_cursor(null,null,'allstats last'));
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------
SQL_ID 7ph2mc3j5p58w, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ * from T_PART_AYSNC where
id>199999 and id<200002
Plan hash value: 1377415075
-------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |
-------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.01 | 6 |
| 1 | PARTITION RANGE ITERATOR | | 1 | 1 | 1 |00:00:00.01 | 6 |
| 2 | TABLE ACCESS BY GLOBAL INDEX ROWID BATCHED| T_PART_AYSNC | 2 | 1 | 1 |00:00:00.01 | 6 |
|* 3 | INDEX RANGE SCAN | T_PART_AYSNC_PK | 2 | 1 | 1 |00:00:00.01 | 5 |
-------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("ID">199999 AND "ID"<200002)
filter(TBL$OR$IDX$PART$NUM(>,0,8,0,"T_PART_AYSNC".ROWID)=1)
22 rows selected.
As we can see, in all the cases, Oracle is able to return the correct result from the existing table partitions even though the index has orphaned entries pointing to the dropped/truncated partition, by simply ignoring the index entries (table partitions) which are marked as orphan by the drop or truncate table partition operation. Oracle utilizes the undocumented function TBL$OR$IDX$PART$NUM for partition pruning and thereby avoids reading the truncated or dropped table partition.
Oracle will periodically clean up the orphan index entries by means of a scheduler job SYS.PMO_DEFERRED_GIDX_MAINT_JOB which is scheduled to run at 2:00 AM every day as shown below.
SQL> select owner,job_name,program_name,next_run_date,state,enabled from dba_scheduler_jobs where job_name='PMO_DEFERRED_GIDX_MAINT_JOB'; OWNER JOB_NAME PROGRAM_NAME NEXT_RUN_DATE STATE ENABL ---------- --------------------------- ------------------------- ------------------------------------------ --------------- ----- SYS PMO_DEFERRED_GIDX_MAINT_JOB PMO_DEFERRED_GIDX_MAINT 08-MAY-16 02.00.00.823524 AM ASIA/CALCUTTA SCHEDULED TRUE
We can also clean up the orphan index entries on demand either by running the scheduler job SYS.PMO_DEFERRED_GIDX_MAINT_JOB or by calling the procedure DBMS_PART.CLEANUP_GIDX as shown below.
---//
---// clean up orphan index entries by running maintenance job //---
---//
SQL> exec dbms_scheduler.run_job('PMO_DEFERRED_GIDX_MAINT_JOB');
PL/SQL procedure successfully completed.
---//
---// clean up orphan index entries by calling cleanup_gidx procedure //---
---//
SQL> exec dbms_part.cleanup_gidx('MYAPP','T_PART_AYSNC');
PL/SQL procedure successfully completed.
We can also rebuild/coalesce individual index or index partitions to get rid of the orphaned index entries using following syntax.
---// ---// rebuild index to cleanup orphan entries //--- ---// ALTER INDEX REBUILD [PARTITION]; ---// ---// coalesce index to cleanup orphan entries //--- ---// ALTER INDEX [PARTITION] COALESCE CLEANUP;
Once, we perform the clean up, the respective index entries would be removed the orphaned_entries list as found below.
---// ---// no orphaned index entries after clean up //--- ---// SQL> select * from sys.index_orphaned_entry$; no rows selected ---// ---// orphaned_entries flag is cleared after clean up //--- ---// SQL> select i.index_name,null partition_name,i.num_rows,s.blocks, 2 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries 3 from dba_indexes i, dba_segments s 4 where i.index_name=s.segment_name and i.index_name='IDX_T_PART_AYSNC_G' 5 union 6 select i.index_name,i.partition_name,i.num_rows,s.blocks, 7 i.leaf_blocks,s.bytes/1024/1024 Size_MB,i.status,i.orphaned_entries 8 from dba_ind_partitions i, dba_segments s 9 where i.partition_name=s.partition_name and i.index_name='T_PART_AYSNC_PK' 10 ; INDEX_NAME PARTITION_ NUM_ROWS BLOCKS LEAF_BLOCKS SIZE_MB STATUS ORP -------------------- ---------- ---------- ---------- ----------- ---------- -------- --- IDX_T_PART_AYSNC_G 100000 768 322 6 VALID NO T_PART_AYSNC_PK ID_P1 0 1024 0 8 USABLE NO T_PART_AYSNC_PK ID_P2 0 1024 0 8 USABLE NO T_PART_AYSNC_PK ID_P3 100000 1024 265 8 USABLE NO
We can also see that, the queries are now not applying additional filters for pruning the orphan records
---//
---// no filter applied after cleanup of orphan index entries //---
---//
SQL> select /*+ gather_plan_statistics */ * from T_PART_AYSNC where id=13;
no rows selected
Elapsed: 00:00:00.00
SQL> select * from table(dbms_xplan.display_cursor(null,null,'allstats last'));
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------------------
SQL_ID 7wtduvyjzbxwt, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ * from T_PART_AYSNC where id=13
Plan hash value: 2879828060
-----------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |
-----------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 0 |00:00:00.01 | 1 |
| 1 | PARTITION RANGE SINGLE | | 1 | 1 | 0 |00:00:00.01 | 1 |
| 2 | TABLE ACCESS BY GLOBAL INDEX ROWID| T_PART_AYSNC | 1 | 1 | 0 |00:00:00.01 | 1 |
|* 3 | INDEX UNIQUE SCAN | T_PART_AYSNC_PK | 1 | 1 | 0 |00:00:00.01 | 1 |
-----------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("ID"=13)
20 rows selected.
Conclusion
Asynchronous global index maintenance is a great enhancement to the existing index maintenance functionalities. It enables us to efficiently drop/truncate table partitions without disturbing the associated global indexes. We also have a choice to perform the global index maintenance during the default maintenance window (2:00 AM every day) or on demand based on our convenience, which gives a great flexibility in terms of deciding the maintenance window. Even though the index maintenance is asynchronous, it doesn’t impact the accuracy and efficiency of the index search as the index remains USABLE as well as Oracle internally handles the orphan index entries during query processing.