The above is correct, you need to manage understand the entire transaction path and manage any overlaps. the above is a good exmaple of this but you also need to consider MBO level overlaps and locking. A great example of this can be loading location hierarchies where hierarchy records are created my multiple transactions simoultaniously, possibly ending is disaster. Of course, in locations this can be managed by understand the hierarchy structure, but you can see how this can apply to other situations.
We have in the past done exactly as Prashant advices, with the additional element of slicing the datasets carefully to account for my point above
Recommendation, look for natural data segmentation like sites or sets
------------------------------
Rene Grozdanovski
Certus Digital
------------------------------
Original Message:
Sent: 10-06-2023 10:51
From: Prashant Sharma
Subject: Multi Thread IFACETABLECONSUMER Data load?
If you are looking for true multi threading with IFACETABLECONSUMER then -
- As IFACETABLECONSUMER will require data to be inserted into queue table mx_inter_trans then you should need to create separate QUEUETABLE for each instances.
- Create separate custom queue and increase the limit of JMS Queue on SQOUT and Custom One so record can be processed fast.
- SIBs are not a problem.
In theory there should not be any issue but time it will take to load millions of records is something which you should consider specially when you have multiple objects that will get the data defined in your OS. Generally use of queue which obviously will have some delays in loading is not ideal way to load this big data volume.
------------------------------
Prashant Sharma
Sedin Technologies
Connect with me @ https://www.linkedin.com/in/psharmamaximo/
#IBM Champion 2022
#IBM Champion 2023
Original Message:
Sent: 10-05-2023 15:22
From: David Miller
Subject: Multi Thread IFACETABLECONSUMER Data load?
Has anyone attempted to make the IFACETABLECONSUMER CRON TASK work multi-threaded with Cron Task Instances set to run on specific JVMs so that I could load a large data set split up on multiple JVMs.
I am planning to create additional Cron Task Instances in the IFACETABLECONSUMER Cron Task and the use mxe.crontask.dorun and mxe.crontask.donotrun to make different instances run on different JVMs so that I get true multi-threading in my load and hopefully take advantage of multiple processors at same time.
Not sure if I go through all this effort, I may end up at a bottleneck of the SIB tables or WAS Queues or some other spot and not really accomplish anything?
Any others insight would be greatly appreciated. I have 10s of millions or records to load and looking to speed it up any way I can.
Thanks,
------------------------------
#Integrations
------------------------------
David Miller
Enterprise Integration Partners
------------------------------