Using A Tivoli Business Service Management Solution To Monitor

Transcript

Using a Tivoli Business Service Management Solution to Monitor PeopleSoft Environments This paper is a step-by-step solution guide to managing and monitoring PeopleSoft environments with IBM® Tivoli® solutions. The suite of Tivoli software works together to show a picture of the overall health of the PeopleSoft environment. This picture helps you stay ahead of your clients by knowing critical availability and status information, and being able to proactively act on that information. Tivoli software products integrate with each other to discover the resources in the PeopleSoft environment, collect related elements into business services, and display notifications when events occur that the environment. From high-level visual overviews of the PeopleSoft system, you can drill down to more detailed information about the infrastructure and events within that infrastructure. The solution described in this paper does not include an IBM Tivoli Application Dependency Discover Manager (TADDM) sensor or a Discovery Library Adapter for detecting PeopleSoft applications. A sensor provides details about the PeopleSoft application. But the testing of the solution described in this paper has contributed to development of a TADDM sensor. This solution uses alerts from IBM Tivoli Monitoring agents and resources like TADDM custom server templates for PeopleSoft processes. Solution environment overview Six key products are installed for this solution: • • • • • • • IBM Tivoli Business Service Manager (TBSM) IBM Tivoli Netcool®/OMNIbus IBM Tivoli Monitoring IBM Tivoli Application Dependency Discovery Manager (TADDM) IBM Tivoli Composite Application Manager Agent (ITCAM) for DB2 IBM Tivoli Composite Application Manager Agent (ITCAM) for PeopleSoft Enterprise Process Scheduler IBM Tivoli Composite Application Manager Agent (ITCAM) for PeopleSoft Enterprise Application Domain Weblogic, DB2, OS sensors and custom server templates TADDM Discovers PeopleSoft components TBSM Services Portlets DLT pulls from TADDM OMNIBus Object Server Events Forwarded Discovery Library Toolkit ITM event rules Launch TIP TBSM EIF Probe Situlation Events Forwarded Tivoli Monitoring OS agent, PeopleSoft agents ITM HUB Tivoli Enterprise Monitoring Server ITM HUB Tivoli Enterprise Portal Server Agent Data to TEMS Weblogic tier machine OS agent, PeopleSoft agents PeopleSoft Application tier machine OS agent, DB2 agent Database tier (DB2) machine Figure 1. Data flow of business service management for PeopleSoft OMNIBus Active Event List Installing and configuring the products In this section, the installation and configuration for each of the products is described. Tivoli Composite Application Manager Agent for PeopleSoft Enterprise Application Domain This agent monitors the availability, health, and performance of key PeopleSoft Application Domain resources: application server, Tuxedo, Process Schedulers, Analytic Servers, and Domains. The Tivoli Monitoring for Applications: PeopleSoft domain agent, Version 6.2.1 is installed in an existing IBM Tivoli Monitoring V6.2.1 infrastructure that includes the hub Tivoli Enterprise Monitoring Server (TEMS or monitoring server) and the Tivoli Enterprise Portal Server (TEPS or portal server). After installing the agent, you configure the instance by configuring the monitoring domain agent for PeopleSoft component by using the Manage Tivoli Enterprise Monitoring Services (MTEMS) utility. Figure 2. MTEMS utility showing agent instances The following example includes the domain agent configuration panel: Figure 3. Agent configuration for the PeopleSoft domain agent For installing, configuring, and troubleshooting the PeopleSoft application domain agent, see the user’s guide at the following web address: http://publib.boulder.ibm.com/infocenter/tivihelp/v24r1/topic/com.ibm.itcama.doc_6.2.2/PSD/om_net_agen t_template.htm Tivoli Composite Application Manager Agent for PeopleSoft Enterprise Process Scheduler This agent monitors the availability, health, and performance of key PeopleSoft Process Scheduler resources such as batch processes, the scheduler, and Tuxedo logs. The Tivoli Monitoring for Applications: PeopleSoft Process Scheduler agent, Version 6.2.1 is also installed in an existing IBM Tivoli Monitoring V6.2.1 infrastructure that includes the hub Tivoli Enterprise Monitoring Server (TEMS or monitoring server), and the Tivoli Enterprise Portal Server (TEPS or portal server). In most cases, the agent is installed alongside the domain agent on the same computer. After installing the agent, you configure the instance by configuring the monitoring domain agent for PeopleSoft component by using the Manage Tivoli Enterprise Monitoring Services (MTEMS) utility. The following example shows the process scheduler agent configuration panel: Figure 4. Agent configuration for the PeopleSoft process scheduler agent For installing, configuring, and troubleshooting the PeopleSoft application domain agent, see the user's guide at the following web address: http://publib.boulder.ibm.com/infocenter/tivihelp/v24r1/topic/com.ibm.itcama.doc_6.2.2/PSS/om_net_agen t_template.htm Tivoli Monitoring for DB2 This agent offers a central point of management for the PeopleSoft DB2 environment. If another database like Oracle is used, you can use the Tivoli Composite Application Manager Extended Agent for Oracle Database. The DB2 agent is installed on the computer where the DB2 server is running. Tivoli Business Service Manager, OMNIBus ObjectServer As a nonroot user (for example, bsmadmin), use the Tivoli Business Service Manager, Version 4.2.1 installer to install the TBSM data server, TBSM dashboard server (Netcool/WebTop), and OMNIbus ObjectServer. Update the TBSM environment with TBSM Fix Pack 2. In a later section, see how to configure TBSM to enable status and availability of the PeopleSoft components in the service tree. Tivoli EIF probe Install an EIF probe from the IBM Tivoli Business Service Manager (TBSM) V4.2.1 installation media on the same computer where the ObjectServer is located. Also install the itm_event.rules file because you are integrating the OMNIbus ObjectServer with the Tivoli Monitoring software. The EIF probe is also stopped and started by using the bsmadmin nonroot user. Tivoli Application Dependency Discovery Manager TADDM, Version 7.2.1 storage and discovery servers are installed with a DB2 backend database on one computer. In a later section, you configure the sensors for discovering the WebLogic and other servers, and creating the business applications for the PeopleSoft environment. TBSM Discovery Library Toolkit Install the Discovery Library Toolkit (DLT) that is included with the TBSM installation media. Configure the DLT to point to the TADDM server and database. You must also apply the update from TBSM V4.2.1 Fix Pack 2 that can be done after installing the DLT. Lastly, copy the latest taddm-api-client.jar and platform-model.jar jar files from the TADDM server into the XMLtoolkit. Sending Tivoli Monitoring events to the OMNIbus ObjectServer In this section, you complete the procedures to enable the data flow shown in Figure 1 that goes from the agent to Tivoli Monitoring and from Tivoli Monitoring to the OMNIbus ObjectServer. Validating workspaces in Tivoli Monitoring When the PeopleSoft Tivoli Monitoring domain agent and process scheduler instances are configured and started, the workspaces are displayed in the Tivoli Monitoring portal server console: Figure 5. PeopleSoft agents and Windows OS agent navigator trees Ensure that all agents (such as the PeopleSoft agents, OS agents, and DB2 agents) are running, and each workspace contains active data. For troubleshooting PeopleSoft agent issues, see the following documentation: • Troubleshooting section in the IBM Tivoli Composite Application Manager Agent for PeopleSoft Enterprise Application Domain Forwarding situation alerts from Tivoli Monitoring to the EIF probe Given a Tivoli Monitoring environment consisting of a monitoring server, a portal server, and the PeopleSoft agent, configure the monitoring server to forward situation events to the EIF probe. Complete this setup by configuring the monitoring server through the Manage Tivoli Enterprise Monitoring Services utility and reconfiguring the monitoring server as shown in the following figure: Figure 6. Configuration settings for event forwarding The default situations for all agents should be started and sending events to the EIF. However, you can validate and manage the situations by selecting an agent subnode and right-clicking Manage Situations, which launches the Manage Situation panel. In following example screen capture, the KP8 situations belong to the PeopleSoft domain agent: Figure 7. Tivoli Monitoring Manage Situations Figure 7 shows the default situations that can be stopped, started, or edited from this view. To immediately send situation events to OMNIbus, click Select All, and click the Start Situation icon at the top-left side of the panel. Validating events in the OMNIbus Active Events List To ensure that events are flowing from Tivoli Monitoring to the OMNIbus ObjectServer, log in to the TBSM dashboard server (or the OMNIbus Event List application), and validate that you are receiving events from the Tivoli Monitoring agent instances. If you have many events, filter on the Node field that corresponds to the node in the Tivoli Monitoring navigator tree in the Tivoli Enterprise Portal (portal) console. The following figure shows events from a PeopleSoft application and web tier (filtered by Node) that are in very bad shape, all showing critical alerts: Figure 8. Tivoli Monitoring Manage Situations These events come from the PeopleSoft domain and process scheduler, OS, and DB2 agents. Drill down on an event and examine the event fields. For example, take a look at the BSM identity. This field will need to match the BSM identity of a service node in TBSM in order to receive status from OMNIbus. Figure 9. Alert field information The summary field provides the Key Performance Indicator (KPI) that triggered the event; the KPI should contain the condition that was violated. When you verify that an event flow is coming from your PeopleSoft servers through Tivoli Monitoring, you must build a service tree in TBSM. Building PeopleSoft groups by using TADDM In this section, you use TADDM to discover and build the TADDM group containing PeopleSoft components such as the Jolt, Workstation, and Tuxedo components that ultimately become the business service tree in TBSM. -Configure the scope In general, a PeopleSoft environment consists of a database, a web browser, and an application tier. In the scopeset, create scope entries for the DB2 server host, WebLogic server host, and the PeopleSoft application host. Configure the access list Create access list entries for each user that will securely access the operating system for each computer in the PeopleSoft tier. This access is required so that TADDM can discover the PeopleSoft components when logged in to a secure session. The following figure shows an access list entry for a WebLogic user, a UNIX computer system user for the DB2 server, and Windows system users for the application and web tier computers: Figure 10. Access list Configure anchors and gateways Configure anchors for private networks (for example, a DB2 server in a private network) and Windows gateways. For instance, in tests of this solution, the DB2 server was running on a computer with a Linux OS and the application and web servers were running on Windows systems. Configure the WebLogic sensor: If you are using WebLogic, configure the WebLogic SSH sensor which is a more comprehensive sensor compared to the basic WebLogic sensor. For details on configuring the sensor, see the WebLogic section of the TADDM sensor documentation at the following location: http://publib.boulder.ibm.com/infocenter/tivihelp/v10r1/index.jsp?topic=/com.ibm.taddmsensors.doc_72/w elcome_page/welcome.html Copy the weblogic.jar, webservices.jar, and wljmxclient.jar jar files from the WebLogic server to the TADDM discovery server (for example, $COLLATION_HOME/lib/weblogic/9.0). On the TADDM discovery server, edit the plugin.xml file for the WebLogic launcher sensor. The following example of the domain tag in the plugin.xml file (under $TADDM_HOME/osgi/plugins/com.ibm.cdb.discover.app.j2ee.weblogic.sensor.weblog iclaunchersensor_1.2.1) that shows the configuration details for a single domain listening on port 80: C:\apps\psoft\hrms9\webserv\peoplesoft\config config.xml 9.48.189.35 80 After editing and saving this file, restart the TADDM discovery server. Run an initial discovery Run an initial discovery to monitor the session sensors to ensure TADDM is able to discover the components. In addition, the initial discovery can be used to collect the unknown servers that might need to be created as custom server templates, which are required by TBSM. Ensure that the WebLogic components are being discovered as shown in the following figure: Figure 11. WebLogic sensor discovery status in TADDM Create custom server templates for the PeopleSoft components Use the BIRT (Business Intelligence and Reporting Tool) report that lists the unknown servers for details. The following is a sample table of custom servers created for testing purposes: Custom server template name WebLogic Service PS JSH PS WSH PS JSL PS WSL Tuxedo Service Program name contents beasvc.exe JSH WSH JSL WSL tuxipc.exe Comments WebLogic BEA service Jolt station handler Workstation handler Jolt station listener Workstation listener Tuxedo service Restart the discovery to create the new custom server templates. Create PeopleSoft business application by using the TADDM 7.2.1 Grouping Composer Use the Grouping Composer in the Discovery Management console to create a business application containing the components that will end up in the service component repository (SCR) in TBSM. In testing, within the Grouping Composer, the Inventory Summary tool was used that included the components such as the BEA Service, DB2 database components, PeopleSoft processes, and WebLogic components as shown in the following example: Figure 12. Grouping Composer showing PeopleSoft components After running another discovery, verify the physical topology of your business application. The following example topology is a very simple business application with a small selection of components: Figure 13. Physical topology of a small PeopleSoft environment Loading the IDML data In this section, you enable data flow of configuration items (service nodes) from TADDM to TBSM by running the discovery library adapters (DLAs), loading the resulting IDML books into TADDM, and using the TBSM Discovery Library Toolkit to pull the TADDM information into TBSM. Running the Tivoli Monitoring DLA Ensure all agents are running, and on the portal server computer, run the tmsdla.bat batch file or the tmsdla.sh script (on UNIX and Linux platforms, use the itmcmd execute command) to create the Tivoli Monitoring IDML book. Transfer the book file to the TADDM discovery server. Load the IDML data into TADDM On the TADDM discovery server, use the TADDM bulkload tool to load both IDML books: $TADDM_HOME/bin/loadidml.sh –f You should see the following output: Bulk Load Program succeeded. Return code is: 0 In the data management portal under Analytics, use the Inventory Summary report to verify that the IDML data was reconciled with the computer systems using the MSS info: Figure 14. MSS info for computer system Business Service Management In this section, you focus on the event flow between the OMNIbus ObjectServer and the TBSM service tree shown in Figure 1 at the beginning of this document. You will use the toolkit to pull configuration items from TADDM into the TBSM component repository. You can see how to launch from the TBSM service to the ObjectServer for drilling down to events and to TADDM for drilling down to configuration items. Running and verifying the Discovery Library Toolkit discovery Start the DLT service and start the discovery as the nonroot user (for example, bsmadmin). You can optionally use the -e option to rebuild all of the explicit relationships, for example: /data/IBM/Tivoli/tbsm/XMLtoolkit/bin/cmdbdiscovery –e –b –r In this example, you are also running an immediate bulk discovery that pulls the PeopleSoft business applications into TBSM service component repository. Verify the TBSM service tree After a successful discovery, log in to the TBSM dashboard and navigate to Availability > Service Availability. Expand the Imported Business Services node. The service node is created by TBSM with the name of the TADDM business application: Figure 15. Business service node “PeopleSoftMini” As seen in the example in Figure 16 all the components contained in the business application container are created in TADDM. Verify each node by expanding the node and ensure each subnode is shown. Notice that the state and event icons are affected by Tivoli Monitoring events. When clicking on a service node in the tree, you can see the hierarchy of services and their status in the service viewer portal, as shown in the following figure: Figure 16. Service viewer showing service node hierarchy and status The service availability page also provides portlets that show a summary of urgent services and events as shown in the following figure (note that the service availability page shows all services affecting the selected service node): Figure 17. Service event summaries The service availability portlet shows the states of the PeopleSoft services. In the next section, you can see how to drill down on the event to obtain more details on the events from the PeopleSoft services and configuration details of the service in TADDM and Tivoli Monitoring (through launch-in-context). Drill down on events, TADDM details, and Tivoli Monitoring workspaces To show the affecting event in the OMNIbus Active Event List (AEL), right-click an event in the service tree. In the context menu, you can view the affecting events in the Active Event List or table, rule status, and root cause events: Figure 18. Context menu showing how to drill down on the OMNIbus Active Event List Figure 19. Events in the OMNIbus Active Event List launched from the TBSM service tree Selecting affecting events launches to the OMNIbus event portals. You can select events if you want to act on the event, such as acknowledging or deleting the event. Showing the root cause event surfaces the event in the service details portal in the same page. From the service details portal, you can click the event to pull up the alert status details: Figure 20. Viewing the alert status details of a root cause event In this example, details are shown of a critical event for the AlertGroup ITM_KP8_AGENT_DATA_PROVIDER_LOG event. The summary field can contain the details and possible root cause. You can also drill down on a service node and launch in context to TADDM configuration details or change history. Right-click a service node and click Launch to > Show Configuration Details (TADDM): Figure 21. TADDM details for a TBSM service node from a PeopleSoft WebLogic domain The TADDM UI is started. If you configured single-signon, you cannot see a login screen. If you see a login screen, type in the TADDM user name, password, and click OK. In addition, you can also drill down on the Tivoli Monitoring managed system details by selecting and then right-clicking the service. In the context menu, you can choose Show Managed System (TEP) to launch to the workspace view: Figure 22. Tivoli portal context menu item The Tivoli portal console is started and the correct workspace view is automatically displayed. The following figure shows the availability workspace view that includes some critical availability issues with the some of the process scheduler monitors: Figure 23. Tivoli Monitoring workspaces Common problems in TBSM In this section, a description of some of the possible problems in TBSM are included. Missing service components under TBSM service tree The following problems might cause service nodes to not be displayed under “Imported Business Services” after a discovery library toolkit data pull: • • • • The TBSM DLT pull failed. In this case, check the logs under $TBSM_HOME/XMLtoolkit/log. Issues with the TBSM data server. The discovery library toolkit might be working but the data might be blocked from going into the service component repository. In this case, check the logs under $TIP_HOME/profiles/TBSMProfile/logs/server1. The TADDM business application does not exist in TADDM. A business application container is required to be automatically populated under Imported Business Services in the TBSM service tree. Use the Grouping Composer or create application descriptors to automate the process. You find missing components in an existing TADDM business application. Use the Grouping Composer to include the missing data. State and event icons are not being affected by incoming events • • The most common problem is that IDML data is not loaded into TADDM. Ensure the IDML book is being bulk loaded into TADDM and verify the MSS info of the computer system in question. Events not flowing to the OMNIbus ObjectServer. In this case, check the status of the EIF probe and OMNIbus ObjectServer. Single sign-on is not working If you receive a login prompt when launching to Tivoli Monitoring or TADDM, check the following requirements are met: • • Users have the correct roles in Tivoli Monitoring and TADDM. For instance, you must enable the user in TADDM through the User Administration tool for the LDAP user to access the TADDM UI. Check security settings under the IBM WebSphere® Application Server administration: - The realm name is common between the products - The domain name is common between the products - The LDAP base entries are common between the products - A common LTPA key is shared between all the products (TADDM, Tivoli Monitoring, and Netcool/Webtop) Conclusion In this paper, you learned how to set up the Tivoli Monitoring PeopleSoft agents to provide monitoring data to Tivoli Monitoring, which then forwards the events to the Tivoli Netcool/OMNIbus ObjectServer. You also learned how to create business application containers for discovered components in the PeopleSoft environment such as Tuxedo and WebLogic that are pulled in by TBSM to populate a service tree. Finally, you learned about affecting and root cause events for a service node and drilling down to TADDM for configuration details. Appendix A. Architecture Computer Tivoli Monitoring environment (hub monitoring server and portal server) OS Product name and version number W2K3 R2 Tivoli Monitoring V6.2.2 Fix Pack 2 IBM Tivoli Composite Application Manager Agent for PeopleSoft Enterprise Process Scheduler V6.2.1 PeopleSoft Application and WebLogic tier IBM Tivoli Composite Application Manager Agent for PeopleSoft Enterprise Application Domain V6.2.1 W2K3 R2 PeopleSoft DB tier Tivoli Netcool/OMNIbus ObjectServer Tivoli EIF Probe Tivoli Application Dependency Discovery Manager Tivoli Netcool/Webtop RedHat 5.2 RedHat 5.2 RedHat 5.2 RedHat 5.2 RedHat 5.2 IBM Tivoli Composite Application Manager Agent for DB2 V6.2.1 OMNIbus V7.3 TBSM V4.2.1 Fix Pack 2 TADDM V7.2.1 (pre-GA) Tivoli Netcool/Webtop V2.2.x (TBSM V4.2.1 Fix Pack 2) Appendix B. Information centers Products TBSM data server, TBSM discovery library toolkit, Tivoli EIF Probe, Tivoli Netcool/Webtop (TBSM dashboard) IBM Tivoli Composite Application Manager Agents (PeopleSoft, DB2) Tivoli Application Dependency Discovery Manager Tivoli Netcool/OMNIbus ObjectServer IBM Tivoli Monitoring Information center IBM Tivoli Business Service Manager IBM Tivoli Composite Application Manager Agents IBM Tivoli Dependency Discovery Manager IBM Tivoli Netcool/OMNIbus Information Center IBM Tivoli Monitoring Appendix C. Acronyms Acronym DLT ITCAM EIF TADDM TBSM TEP TEPS TEMS AEL IDML Definition Discovery Library Toolkit IBM Tivoli Composite Application Monitoring Event Integration Facility Tivoli Application Dependency Discovery Manager Tivoli Business Service Manager Tivoli Enterprise Portal Tivoli Enterprise Portal Server Tivoli Enterprise Management Server Active Event List International Development Markup Language