I'll start with the second question first. For Windows specifically (not iOS/Android yet, though this will be changing in 9.1) we have two methods to disable Windows hello. If you are using a MDM like Intune, you should follow this: https://www.ibm.com/docs/en/masv-and-l/maximo-manage/cd?topic=mmiw-configuring-maximo-mobile-application-settings-windows-mobile-device-manager
If you are not using a MDM, you can follow the steps here and set the enableWindowsHello to false https://www.ibm.com/docs/en/masv-and-l/maximo-manage/cd?topic=mmiw-configuring-maximo-mobile-application-settings-windows-from-command-line
Regarding 1, the Maximo Mobile configuration is identical to configuring SAML for Maximo. Some customers have configured a filter to ignore the /maximo/oslc route historically to enable authentication with LDAP credential for integrations that would not work for Maximo Mobile. However, in your example it sounds like that piece is configured correctly since WebSphere is triggering the redirect to the identity provider. It sounds like your issue might be with the target URL in WebSphere or that you haven't mapped the trust all authenticated realms to the application.