Maximo Open Forum

 View Only

Related Content

  • 1.  Multi Thread IFACETABLECONSUMER Data load?

    Posted 10-05-2023 15:22

    Has anyone attempted to make the IFACETABLECONSUMER CRON TASK work multi-threaded with Cron Task Instances set to run on specific JVMs so that I could load a large data set split up on multiple JVMs.

    I am planning to create additional Cron Task Instances in the IFACETABLECONSUMER Cron Task and the use mxe.crontask.dorun and mxe.crontask.donotrun to make different instances run on different JVMs so that I get true multi-threading in my load and hopefully take advantage of multiple processors at same time.

    Not sure if I go through all this effort, I may end up at a bottleneck of the SIB tables or WAS Queues or some other spot and not really accomplish anything?

    Any others insight would be greatly appreciated. I have 10s of millions or records to load and looking to speed it up any way I can.

    Thanks,

    ------------------------------


    #Integrations

    ------------------------------
    David Miller
    Enterprise Integration Partners
    ------------------------------


  • 2.  RE: Multi Thread IFACETABLECONSUMER Data load?

    Posted 10-06-2023 10:51

    If you are looking for true multi threading with IFACETABLECONSUMER then -

    1. As IFACETABLECONSUMER will require data to be inserted into queue table mx_inter_trans then you should need to create separate QUEUETABLE for each instances.
    2.  Create separate custom queue and increase the limit of JMS Queue on SQOUT and Custom One so record can be processed fast.
    3. SIBs are not a problem.

    In theory there should not be any issue but time it will take to load millions of records is something which you should consider specially when you have multiple objects that will get the data defined in your OS. Generally use of queue which obviously will have some delays in loading is not ideal way to load this big data volume.



    ------------------------------
    Prashant Sharma
    Sedin Technologies
    Connect with me @ https://www.linkedin.com/in/psharmamaximo/

    #IBM Champion 2022
    #IBM Champion 2023
    ------------------------------



  • 3.  RE: Multi Thread IFACETABLECONSUMER Data load?

    Posted 10-07-2023 10:03

    The above is correct, you need to manage understand the entire transaction path and manage any overlaps. the above is a good exmaple of this but you also need to consider MBO level overlaps and locking. A great example of this can be loading location hierarchies where hierarchy records are created my multiple transactions simoultaniously, possibly ending is disaster. Of course, in locations this can be managed by understand the hierarchy structure, but you can see how this can apply to other situations.

    We have in the past done exactly as Prashant advices, with the additional element of slicing the datasets carefully to account for my point above

    Recommendation, look for natural data segmentation like sites or sets



    ------------------------------
    Rene Grozdanovski
    Certus Digital
    ------------------------------

    Original Message