Maximo Open Forum

 View Only
  • 1.  Maximo Mobile Attchment Heapdump issue

    Posted 20 days ago

    Hi Everyone ,

    We are using MAS 9.0 and MS sql DB. We are  facing heap dump issue which is bringing ui pod down while accessing the attachments from mobile or role based technician app . We are using oob relationship between work order and doclink due to business requirements . 
    Did anyone faced similar issue , any leads please 

     


    #MaximoApplicationSuite

    ------------------------------
    Balaji Reddy
    infy
    ------------------------------


  • 2.  RE: Maximo Mobile Attchment Heapdump issue

    Posted 18 days ago

    Hi Balaji,

    I believe this is a known issue from IBM around the OOB WORKORDER -> DOCLINKS relationship. The OOB relationship can bring back attachments from related records such as tasks, assets, locations, job plans, PMs, safety plans, tickets, etc., which can cause a very large MXAPIWODETAIL payload and memory pressure on the UI pod. I would work towards reducing this and seeing if it helps.

    You can see a note here.

    I am always hesitant to change anything directly OOB but that is up to you. I typically try to create duplicate with a prefix to know that I changed it. This way my changes do not get overwritten when you upgrade to new versions of Maximo/Manage.

    Thank you,



    ------------------------------
    Allan Henle
    Naviam
    ------------------------------



  • 3.  RE: Maximo Mobile Attchment Heapdump issue

    Posted 17 days ago

    Hi Balaji,

    You mentioned about using the OOB relationship due to business requirement. May I know whether it is due to a technical challenge to update the relationship? or any other fucntionality related. Based on that, I can recommend a way to customize.

    Thanks! 



    ------------------------------
    Sankar Ganesh V S
    Naviam
    ------------------------------



  • 4.  RE: Maximo Mobile Attchment Heapdump issue

    Posted 15 days ago

    Hi Sankar ,

    we changed the relationship to have only work order related attachments without any union with other objects , still issue persists .

    While analysing the heap , we found select * from work order triggering  from somewhere , as our system is having around 30 million records , it might be the reason for jvm crash . Please advise if you have any leads . 



    ------------------------------
    Balaji Reddy
    infosys
    ------------------------------



  • 5.  RE: Maximo Mobile Attchment Heapdump issue

    Posted 15 days ago

    Balaj,

    The select * from workorder with 30 million records is likely to be the cause. I am not confident where this is coming from. It could be a user running the query. Do you have the full select * from workorder query? Or is that all the information you have.

    I am not sure if mobile would do that when adding an attachment.

    Thank you,



    ------------------------------
    Allan Henle
    Naviam
    ------------------------------



  • 6.  RE: Maximo Mobile Attchment Heapdump issue

    Posted 12 days ago

    Hi Balaj,

    I would look at the BIRT reports that were being run.

    If this is a problem with an integration / user action then Maximo's fetch stop limit (see link below) should have stopped the query long before it returns a lot of rows. Maximo's fetch stop limit is not applied to queries executed via BIRT. The core dump should be able to give you more clues about how the data is being used and thus what generated it.

    Have you configured the SQL logger to give you information about WORKORDER related queries?

    Have you looked in the http_access logs to see what HTTP requests were being submitted e.g. which request asked for lots of records ?

    best regards,

    Mark

    https://www.ibm.com/docs/en/masv-and-l/maximo-manage/cd?topic=properties-database-in-maximo-manage - mxe.db.fetchResultStopLimit



    ------------------------------
    mark robbins
    Cohesive
    IBM Champion 2017-2024 Inclusive
    See my blog on Maximo support related topics here:
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 7.  RE: Maximo Mobile Attchment Heapdump issue

    Posted 13 days ago

    @Balaji Reddy

    I suggest taking a look at the Query Definition, such as ASSIGNEDWOLIST, on the MXAPIWODETAIL Object Structure, which is used by the Mobile Technician app. This should help identify why it is running a query that returns approximately 30 million records.

    The attachments may simply be the symptom or the breaking point, rather than the root cause. These queries can be modified to reduce the number of records the technician sees, but make sure you back up the current query definition before making any changes.

    You can also take the query and run it directly against the database to validate its performance and confirm how many records it returns. 



    ------------------------------
    Michael Marsonet
    MRM-EAM Consulting Inc.
    ------------------------------



  • 8.  RE: Maximo Mobile Attchment Heapdump issue

    Posted 9 hours ago
      |   view attached
    Hi Balaji,

    The challenge you are facing is complex. I used AI tools to provide concise explanations and practical tools to help your team identify the root cause of the issue you're facing. I did not alter the AI's word choices to keep things simple. 
    To stop the pod crashes and pinpoint the exact source of this issue immediately, your database and integration teams must account for three structural variables that the out-of-the-box (OOB) architecture guide overlooks:

    1. Core Architectural Vulnerabilities

    A. The Relational Trap: WONUM vs. WORKORDERID

    If you look closely at your modified DOCLINKS relationship, check how the tables are bound.
    • The Failure Mode: WONUM is not a unique value in Maximo. It duplicates across different sites (siteid), organizations (orgid), and historical records.
    • The Performance Impact: If your optimized mobile query maps attachments via a clause like ownertable='WORKORDER' and ownerid=:workorderid, the reality is OOB, MXAPIWODETAIL has a significant number of relationships making it difficult to pinpoint the root cause of the 'select *' root cause. The source might be an Object Structure Relation, OSLC Query, or an Application's saved query to name a few.
    • The Fix: Refine the combinations and permutations driving the where clauses in addition to the change your team made,(i.e., ownertable='WORKORDER' and ownerid=:workorderid). Consider altering the MobileDB Crontask instance's where clause parameter if possible. 

    B. Polymorphism and the copylinktowo Flag

    The physical WORKORDER table is highly polymorphic, every parent records (woclass = 'WORKORDER') and children records (woclass = 'ACTIVITY' or 'TASK') have a unique :workorderid where both parent and child records can hold independent attachments.
    When a child task inherits or references a parent file, Maximo logs a tracking row where doc.copylinktowo = 1. See the highlighted logic reference query below: 

    SELECT wo2.workorderid
    ,di.document
    ,di.description
    ,di.urlname
    ,doc.docinfoid
    ,doc.doclinksid
    ,doc.ownertable
    ,doc.copylinktowo
    ,doc.ownerid
    FROM docinfo di
    JOIN doclinks doc ON doc.docinfoid = di.docinfoid
    JOIN workorder wo2 ON wo2.workorderid = doc.ownerid
    WHERE doc.ownertable = 'WORKORDER'

    C. The Legacy SQL Server Sequencing Gap

    In older Maximo environments that migrated up to MAS 9 on Microsoft SQL Server, primary keys did not natively use SQL Server's database-level auto-sequences. This changed in MAS, Maximo now manages sequencing via the MAXSEQUENCE table framework as described in A3J Article - SQL Server Now Using Sequences in MAS 

    2. The Diagnostic Query (Isolate the Root Cause)

    By wrapping the DOCLINKS tracking inside an aggregated subquery mapped over the unique workorderid index, this query will tell you exactly where your attachment data volume resides without triggering a table scan or multiplying row counts:
    --=================================================================================================================================================== 
    -- COHESIVE SYSTEM MONITORING MATRIX: PARENT VS CHILD ATTACHMENT DIFFERENTIATION (SQL SERVER) 
    --=================================================================================================================================================== 
    SELECT 
         @@SERVERNAME AS [Maximo_Environment]
        ,p.siteid AS [Site_ID]
        ,p.worktype AS [Work_Type]
        ,wt.wtypedesc AS [Work_Type_Description]
        ,COUNT(p.workorderid) AS [Total_Qualifying_WOs]
        
        -- 1. Differentiate Base Table Records by Core Class 
        ,SUM(CASE WHEN p.woclass = 'WORKORDER' AND p.istask = 0 THEN 1 ELSE 0 END) AS [Parent_WOs]
        ,SUM(CASE WHEN p.woclass = 'ACTIVITY' OR p.istask = 1 THEN 1 ELSE 0 END) AS [Child_Activities_and_Tasks]
        
        -- 2. Cleanly Count Parent vs Child Attachments via Unique ID Subquery 
        ,ISNULL(SUM(CASE WHEN p.woclass = 'WORKORDER' AND p.istask = 0 THEN att.Attachment_Count ELSE 0 END), 0) AS [Parent_WO_Attachments]
        ,ISNULL(SUM(CASE WHEN p.woclass = 'ACTIVITY' OR p.istask = 1 THEN att.Attachment_Count ELSE 0 END), 0) AS [Child_Activity_Attachments]
        ,ISNULL(SUM(att.Attachment_Count), 0) AS [Combined_Total_Attachments]
        
        -- 3. Operational Queue Metrics 
        ,SUM(CASE WHEN p.wopriority <= 2 AND p.wopriority > 0 THEN 1 ELSE 0 END) AS [High_Priority_WOs]
        ,SUM(CASE WHEN p.wopriority > 2 AND p.wopriority <= 5 THEN 1 ELSE 0 END) AS [Medium_Priority_WOs]
        ,SUM(CASE WHEN p.wopriority > 5 OR p.wopriority = 0 THEN 1 ELSE 0 END) AS [Low_or_Unassigned_Priority]
        ,SUM(CASE WHEN p.amcrew IS NOT NULL OR p.crewworkgroup IS NOT NULL THEN 1 ELSE 0 END) AS [Crew_Assigned_WOs]
        ,SUM(CASE WHEN p.parentchgsstatus = 1 THEN 1 ELSE 0 END) AS [WOs_With_Inherit_Status_Enabled]
        ,COUNT(DISTINCT p.reportedby) AS [Unique_Reporters_In_Dataset]
        
        -- 4. Active Queue Breakdown 
        ,SUM(CASE WHEN sd.maxvalue = 'APPR' THEN 1 ELSE 0 END) AS [Status_Approved]
        ,SUM(CASE WHEN sd.maxvalue = 'INPRG' THEN 1 ELSE 0 END) AS [Status_In_Progress]
        ,SUM(CASE WHEN sd.maxvalue = 'WMATL' THEN 1 ELSE 0 END) AS [Status_Waiting_on_Material]
        ,SUM(CASE WHEN sd.maxvalue = 'COMP' THEN 1 ELSE 0 END) AS [Status_Completed]
        ,MIN(p.reportdate) AS [Oldest_Report_Date_In_Queue]
        ,MAX(p.reportdate) AS [Newest_Report_Date_In_Queue] 
    FROM workorder p 
    LEFT JOIN synonymdomain sd 
        ON sd.domainid = 'WOSTATUS' 
       AND sd.value = p.STATUS 
       AND sd.defaults = 1 
    LEFT JOIN worktype wt 
        ON wt.worktype = p.worktype 
       AND wt.orgid = p.orgid 
       
    -- THE STRATEGIC FIX: Left Join to an aggregated subquery targeting the unique physical index 
    LEFT OUTER JOIN ( 
        SELECT 
            doc.ownerid AS target_workorderid, 
            COUNT(doc.doclinksid) AS Attachment_Count 
        FROM doclinks doc 
        INNER JOIN docinfo di 
            ON doc.docinfoid = di.docinfoid 
        WHERE doc.ownertable = 'WORKORDER' 
          AND doc.ownerid IS NOT NULL 
        GROUP BY doc.ownerid 
    ) att ON att.target_workorderid = p.workorderid 
    WHERE p.historyflag = 0 
      AND sd.maxvalue IN ('APPR', 'WSCH', 'WMATL', 'WAPPR', 'INPRG', 'COMP') 
    GROUP BY 
         p.siteid 
        ,p.worktype 
        ,wt.wtypedesc 
    ORDER BY 
         p.siteid 
        ,[Total_Qualifying_WOs] DESC;

    3. Use the attached IBM Techdoc to finetune your Role Based App: Attached "Maximo Mobile - How to setup preloaded database to improve lookup download performance?"

    Follow the practical steps to set up scenarios that can help your team identify the issue's root cause.

    High-Level Steps

    The MXAPIWODETAIL Object Structure used by the mobile app contains over 14 nested child objects out-of-the-box (OOB). Follow the steps below MobileDB troubleshooting steps below. 

    Step 1: Extract Maximo Mobile's SQLite MobileDB maximoMobile.db

    1. Extract the active, completed preloaded database identifier from your backend: In SQL Server you can correlate performance to query execution times by selecting all columns.
      SELECT mobiledbid, persongroup, siteid, changedate FROM mobiledb WHERE mobiledbid={YOUR_MOBILE_DB_ID}
    2. Establish an active browser session cookie by logging into the Maximo desktop GUI as an administrator, then open a clean tab to stream the raw binary database payload directly to your desktop:
      http://{hostname}:{port}/maximo/oslc/graphite/mobile/db?mobileDbId={YOUR_MOBILE_DB_ID}
      
    3. Open the downloaded file using DB Browser for SQLite and run this audit in the Execute SQL tab to pinpoint which tables are harboring the mega-byte bloat:
      SELECT name AS [Table Name], SUM(length(json)) / 1024.0 / 1024.0 AS [Size (MB)] 
      FROM sqlite_master CROSS JOIN json_each(name) WHERE type = 'table' AND name LIKE '%_lookup'
      GROUP BY name ORDER BY [Size (MB)] DESC;
      

    Step 2: Establish a Sandboxed MobileDB Cron Task Control Group

    1. Go to the Person Groups application in the Maximo UI and create a test group (e.g., YOUR_UNIQUE_MOBILE_TEST). Populate it with exactly one testing person/labor record.
    2. In the Cron Task application, create a testing crontask instance in the MobileDbGeneration Cron Task.
    3. Create a brand-new Cron Task Instance (e.g., YOUR_UNIQUE_MOBILE_INSTANCE).
    4. Set the PersonGroup parameter for this test instance strictly to YOUR_MOBILE_PERSONGROUP and set a group default to receive runtime error notification emails.
    5. Assign a tightly restricted, controlled WHERE clause parameter to this instance to fetch a test worklist.
    6. Activate only this test instance. This allows you to generate, stress-test, and debug mobiledb causing your pod issue.

    Step 3: Verify both Global and Mobile System Properties' Recommended settings 

    System Properties application in Maximo Manage
    • mxe.db.fetchResultStopLimit 
    • mxe.db.fetchStopLimitEnabled
    • mobile.% prefix system properties


    ------------------------------
    [Fredrick] [Ndwaru]
    [Perpetual Ignition]
    ------------------------------



Newest Episode
Ep. 2 | Adding a QR Code to a BIRT Report

Watch Steven Shull walk through how to add a QR code to a BIRT report and make Maximo reporting more scannable, connected, and useful in the field.

MORE by Naviam Episode 2 cover
Watch Episode 2
Also available: Ep. 1 | IBM Maximo Application Suite 9.2