Let’s discuss having multiple jobs unloading the same table at the same time. NGT’s support for this is unique and robust. I’ll break this down into two topics, using multiple concurrent jobs to have:
- Each job unloads partition ranges, with all parts to a common point of consistency.
- Each job unloads the same table, possibly the whole table, at the same time with a point of consistency.
Review of unloading to a Point of Consistency (POC)
This is done by having an asynchronous task, for NGT that’s the NGT Subsystem, capture before Images of any page that changes during the unload, then any rows unloaded from one of these pages would come from that before image. However, once the NGT Subsystem is collecting these before images for a POC, and possibly a subset of partitions, the NGT Subsystem cannot simultaneously manage a second POC of the same object for another job or add partitions to an existing object’s POC. However, with NGT there is a way.
NGT puts unload parallelism on steroids!
First, are multiple submitted unloads even needed with NGT? It is common to want to unload a very large partitioned table space quickly, and to create a clean point-of-consistency (POC). This is accomplished by unloading the partitions in parallel. Any unload product can multi-task and process some number of partitions in parallel. NGT with its server jobs supercharge this by having its server jobs multi-task several partitions just like the master job you submitted is doing. The use of NGT servers may eliminate the need for multiple user-submitted job to increase parallelism. However, what if you have thousands of partitions and you want to submit multiple jobs that each can submit multiple server jobs to go nuclear with parallelism. NGT can do this too.
To run multiple jobs which each unload a subset of partitions of the same table space you specify the +CONNECTALL(YES) global parameter in your set of jobs. This tells the NGT Subsystem to connect to and collect before images for all parts of the table space. This way whichever of your jobs runs first, it will have the NGT subsystem monitor all parts and all the subsequent jobs will share this existing connection, share the point of consistency. You get a clean POC unload of many parts using many jobs running concurrently.
The following JCL could be one of many jobs that you run concurrently; this one unloads the first 100 parts.
//ULDPARMS DD *
//SYSIN DD *
OUTPUT SYSREC DSNAME hlq.UNLOAD.P&PART..T&TIME
SHRLEVEL CHANGE CONSISTENT YES
UNLOADDN SYSREC UNLOADDNPFX
SELECT * FROM creator.table_name
Multiple unrelated unload jobs processing the same table at the same time
NGT Unload has always allowed multiple unload jobs to process the same table even when a POC is requested, SHRLEVEL CHANGE CONSISTENT YES. If a CONSISTENT unload is running and another CONSISTENT unload is submitted for that same table, NGT will let this and any subsequent overlapping unloads share the existing POC.
If you don’t want this behavior, the sharing of the existing POC, you can specify +POC_SHARE(NO) and subsequent jobs will fail rather than sharing the existing POC. Specify +POC_SHARE(NO) when it is important that the data is unloaded is consistent as of the time the job is submitted and cannot be from a few minutes prior.
Note: +POC_SHARE(YES/NO) was added with PTF BQU2543 in August 2019. The prior behavior and the new default is YES.