Maximo Open Forum

 View Only

  • 1.  Issue with outofMemory leak and integrations are not working

    Posted 26 days ago

    Hello All,

    Greetings!

    We came across an issue with outofMemory java heap space issue.

    Also, integrations are not working.

    Error: java.net.SocketTimeoutException: Read timed out

    please can you help us to fix the issue.

    Thanks & Regards,

    Kiran


    #Integrations

    ------------------------------
    Komindala Kiran Kumar
    CTS
    ------------------------------


  • 2.  RE: Issue with outofMemory leak and integrations are not working

    Posted 9 days ago

    Hi Kiran,

    We resolved the issue by:

    1. Increasing JVM heap size for the Maximo/MIF server

    2. Cleaning up integrations (Integration → Message Reprocessing)

    3. Restarting Maximo JVMs (UI + MIF) after the changes.

    4. Adjusting timeouts

      • Increased integration/web service timeouts to avoid SocketTimeoutException.

    After these changes the OutOfMemory and java.net.SocketTimeoutException: Read timed out errors stopped and integrations started working normally.



    ------------------------------
    Pavithra Nallamothu
    Vaaridha Technologies
    ------------------------------

    Original Message


  • 3.  RE: Issue with outofMemory leak and integrations are not working

    Posted 8 days ago

    @Komindala Kiran Kumar this is very vague description of the problem and you basically need to start using common troubleshooting techniques. There are different ways to approach this and although @Pavithra Nallamothu gave some suggestions on how they resolved this problem most likely in the past this may not be your problem. Personaly I would start with the following:
    1. Ask the question(s) when did this start to happen?

    2. What changes where made recently?

    3. Look at the systemOut.log or SystemErr.log  (There may be more that one if you have a cluster setup) for a Java stack trace. Analyzed the stack trace as it will likely point you to where the failure is occurring (ie Cron Task, Auto Script, maybe a piece of code in the <application> class)

    4. Check Websphere Messaging Engine is it started (I mention this as you say that all integrations are not working).

    5. Are your using Sequential Queues? If so check Message Tracking and Message Reprocessing as if there is an error(s)  then messages will be on hold until the error is cleared.

    Depending on the above it would then help to start to dig deeper into specific areas. Hope this helps. 

    Mike



    ------------------------------
    Michael Marsonet
    MRM-EAM Consulting Inc.
    ------------------------------

    Original Message


  • 4.  RE: Issue with outofMemory leak and integrations are not working

    Posted 8 days ago

    Hi, @Komindala Kiran Kumar

    Your problem statement is vague so any advice will be generic.

    Check the memory usage entries in the JVM logs to see when the free memory dropped to low levels e.g. below 10% free memory. - BMXAA7019I - The total memory is yyyy and the memory available is zzzz

    I have seen memory problems start hours before the users actually report the problem.

    Check the mbo counts - [29/11/21 17:00:22:237 GMT] 0000015e SystemOut    O 29 Nov 2021 17:00:22:237 [INFO] [UI1] [] WORKORDER: mbosets (1), mbos (1) - I discuss these log entries here - https://www.linkedin.com/pulse/seeing-which-mbosetsmbos-using-memory-mark-robbins. Counts over 20,000 often require investigation to understand why that many MBOs are in memory. if you follow the link then you will find that the article is part of a whole series on MBOs and how they use memory

    There are various other things that you can check to see what is using the memory and where the memory leak may be.

    If you aren't confident investigating this then consider using consultancy. I have written the Cohesive Log Analyser and is a specialist tool designed to investigate performance problems exactly like this.

    The analysis includes graphs / information / likely suspects / known leaks spotted on your system.

    Memory related problems can occur for a number of reasons including:

    • Code that isn't releasing memory when it finishes
    • users / interfaces submitting requests that retrieve too much data
    • JVMs not configured with the most appropriate settings e.g. to disable port sharing on the latest WAS releases
    • Missing patches - IBM have released fix patches with corrections for memory leaks - I have helped identify a number of those leaks

    I'm happy to discuss this offline if you want to reply privately to me,

    @Pavithra Nallamothu - the actions that you describe probably won't resolve the underlying memory leak that caused the problem.

    Restarting the JVM often hides the problem because it restarts the JVM before the free memory reaches a critical low point

    It is critical to understand where the memory is being used and how to resolve it.

    If you graph the available memory entries mentioned above then I would expect to see that the free memory drops over the longer term.



    ------------------------------
    mark robbins
    Cohesive
    IBM Champion 2017-2024 Inclusive
    See my blog on Maximo support related topics here:
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------

    Original Message


Related Content