Configuring HP Storage Management Pack for SNMP devices in SCOM

In my previous posts I’ve covered Configuring HP ProLiant Server Management Pack and Configuring HP BladeSystem Management Pack and now it is time of another MP by HP. In this post I will cover how to configure HP Storage Management Pack for SNMP Devices. This MP supports two more methods (WEBES and WMI) for monitoring bedsides SNMP and depending on the method various HP Storage systems are monitored. The HP devices supported by SNMP method are:

  • P9000
  • P2000/MSA
  • Tape Storage
  • P4000
  • B-Series Switches
  • HP 3PAR Storage System
  • HP D2D Backup System

Note: I will cover configuration that only needs to be done on SCOM side. Configuration on the different storage systems for SNMP is available in the user’s guide of the management pack. Make sure you have the latest firmware versions of these devices

It is common practice management packs such HP Storage to be installed on the RMS server but depending on your environment you can use any other server with SCOM agent on it. Follow these general steps to install and configure HP Storage MP for SNMP devices:

1. Install Windows SNMP Service feature on your RMS server.

2. Download the HP Storage Management Pack from HP site.

3. One zip file is download which when extracted contains the following files:
•    HP Storage Management Pack for System Center.zip
•    HP Storage WEBESListener for System Center.zip
•    HPStorageManagementPackforSystemCenterUserGuide-2.x pdf (User Guide)

4. If you will not use WEBES method you can only unzip HP Storage Management Pack for System Center.zip.

5. Copy the zip files to your RMS server.

6. Unzip the HP Storage Management Pack for System Center.zip file.

7. Navigate to the unzipped folder, and run setup.exe.

8. Click Next to continue.

image

9. Click Next to confirm the installation.

image

10. Click Close after installation is finished.

Note: All files including the HPEnterpriseStorageMP.mp are installed in ~:\Program Files\HP\HP Enterprise Storage directory for a 32-bit operating system, and ~:\Program Files(x86)\HP\HP Enterprise Storage directory for a 64-bit operating system.

11. The management pack will be automatically imported in SCOM if not you can locate the mp file and import it manually.

12. After the MP is installed SNMP Service have to be configured.

12. Start Services.msc. Locate SNMP Service and double click on it to open Properties.

13. Click on Security tab.

14. Check Send authentication trap.

15. Add Community name with READ rights on Accepted community names. This is the community name that you will use for Storage devices when configure them.

16. Select Accept SNMP packets from these hosts.

17. Add the IP addresses of every storage device you will monitor trough HP Storage MP.

18. Click OK to save settings.

19. After SNMP Service is configured properly SNMP Trap Catcher Rule in HP Storage MP have to be enabled.

20. Open SCOM console.

21. Click on Authoring pane.

22. Click on Rules.

23. In the Rules window, search for HP Storage Trap Catcher rule using the Look for: search bar.

24. Right-click the HP Storage Trap Catcher rule and select Overrides > Override the Rule. Select For a specific object of the type: HP Storage SNMP Trap Catcher discovery and in the search filed find the FQDN of your RMS server. Select it and click OK.

25. Override Properties window will appear.

image

26. Check the box under Override column. Change the value under Override Value from False to True. Select the management pack that you save overrides and click OK.

Note: Management Server Action Account has to be a member of the Operations Manager Administrators user role in SCOM. If this not set you may have trouble with some PowerShell scripts not being able to finish successfully.

Another additional configuration that I recommend is that if you do not have P4000/E5000/X5000 systems in your environment is to disable HPEnterpriseStorage.SNMP_CommandExecutor.Rule and HPEnterpriseStorage.WMI_CommandExecutor.Rule. These rules are executed every minute and could cause some performance issues on your RMS server also they are improperly targeted to Group. Follow these steps to disable them:

1. Open SCOM console.

2. Click on Authoring pane.

3. Click on Rules.

4. In the Rules window, search for SNMP_Command using the Look for: search bar.

5. In the listed rules search for the Type :  Group.

image

image

6. Right Click on the HPEnterpriseStorage.SNMP_CommandExecutor.Rule -> Overrides -> Override the Rule -> For all the Objects of the class :Group.

7. Select the check in front of Enabled. Set Override value from True to False. For Management pack select  your MP for overrides and click OK.

image

image

8. Repeat steps from 4 till 7 searching by WMI_Command for HPEnterpriseStorage.WMI_CommandExecutor.Rule rule.

After these steps are executed you are left with only configuring the storage devices to send SNMP packets to your RMS server.

Routing Alerts from SCOM in SCSM by using Custom Field Criteria Type

Recently I faced the task to route alerts from SCOM in SCSM to different Support Groups. It seemed like an easy task because in most cases routing is based in Management Pack Name criteria. For example alerts that come from Management Pack that contains “SQL” in its name are assigned to SQL Support Group, alerts that come from Management Pack that contains “BizTalk” in its name are assigned to BizTalk Support Group and etc. You get the idea you can create such routing rule for every Management Pack. Besides this routing rule you can also route alerts based on SCSM groups membership of computer, Custom Fields and  Monitoring Classes.

image

When I started configuring routing based on Management Pack Name I didn’t had any issues everything was working as it was suppose to work you just have to be careful not put make any conflicts with rules by overlapping them. But when I tried to configure routing based on Custom Field I faced issues. In the next lines I will describe how stumbled on that issue and how I fixed it. I couldn’t find any such issue over Internet so I’ve decided to share it with the community.

Lets say that we have two Support Groups – Backup and Storage. Those two Support Groups are using one management pack in SCOM to monitor their devices. So in SCSM we need to configure: alerts that are coming from devices supported by Backup Support Group to be assigned to Backup team and alerts that are coming from devices supported by Storage to be assigned to Storage team. Most of you will probably suggest that we can put these devices in groups in SCSM and route alerts based on that or even easier we can route them based on Monitoring Class. But these two options are also not available because all these devices are monitored by SNMP so they they do not have CI record in SCSM and all alerts come from the server where the management pack is installed in our case this is the RMS server. Such management pack is HP Storage Management Pack. This management pack monitors various storage devices manufactured by HP and all is put in one MP file. Lets say we want to monitor 3PAR Storage, SAN Switches, D2D Devices and Tape Libraries with this management pack. All of these device are monitored by SNMP and we want 3PAR and SAN switches alerts to go to Storage Support Group and D2D device and Tape Libraries alerts to go Backup Support Group. When alerts for these devices are created in SCOM the first 6 custom fields are filled with values:

  • Custom Field 1 – Source of the Event
  • Custom Field 2 – Logging Computer name
  • Custom Field 3 – Device Id
  • Custom Field 4 – Device Name
  • Custom Field 5 – Source Computer Name – the computer that generated the event
  • Custom Field 6 – Source Computer Domain Name – the domain of the computer that generated the event

So custom fields for alert could look like this:

image

Or like this:

image

From the examples above it is clearly that the best option is to route alerts based on Custom Field 1. Before creating the route rule I will show you the steps for creating the templates that will be used by these rules.

If we go in SCSM console –> Library –> Lists and open the properties of Incident Tier Queue list we can see that we have 3 Support Groups – Storage, Backup and Windows:

image

So we need to configure 2 Templates in SCSM – one for Storage and one for Backup Support Group. We go to Templates and from Action Menu we choose Create Template and new window appears:

image

We can name the templates “SCOM Incidents Storage”, for class to choose Incident and for management pack you we can select a custom management pack where we store such settings. When we click OK an incident form will open. This is our template and here we have to fill the fields that will be changed when alert meets certain routing rule criteria. In our case we can populate Classification category, Source and Support group:

image

You can choose to populate different fields but Support group is the field that is actually used for assignment. When We click OK the template will be saved. Another template have to be created the same way for Backup:

image

Now we are ready with the templates and we can configure the routing rules in SCOM Alert Connector. I will not show how this connector is configured because it is pretty simple operation and there are a lot of articles over Internet about that.

When we open the SCOM Alert Connector Properties there is Alert Routing Rules tab and on that tab routing rules are added:

image

You can see even the option that if alert doesn’t meet any of the specified routing rules Operations Manager Incident Template will be used for them. This is the default SCSM Template. When we click on Add button a new window appears. In this window I gave distinguishing name for the routing rule, which template to use and the criteria for the alerts:

image

So I was ready with my first routing rule so I’ve clicked OK on the rule and OK on Connector’s window. Before creating more rules I’ve decided to test if my routing was right. You can create test alert from your device or you can take any alert that is with status new and it is not forwarded to your SCSM server and modify the custom fields like those for your device. After you modify them you can forward that alert to SCSM to see if it will be routed correctly just like this by selecting Forward to –> Alert Sync: SCOM alerts:

image

After the alert was forwarded I’ve open the SCSM console and found the alert created as a incident:

image

As you can see from the screenshot the Storage template I’ve created wasn’t applied to this incident because Support Group field was empty which meant that the default Operations Manager Incident Template was applied and the alert didn’t matched my routing criteria. At this point I understood that I have to make some troubleshooting in order to solve this.

The first thing I wanted to see if Custom Fields properties arrived in SCSM from SCOM properly. This can be seen in the Extensions tab of the incident:

image

As we can see from the screenshot all properties are the same as they appear in SCOM. I couldn’t find any reason why this solution is not work so I’ve started to modify the routing rule by different methods like using Custom Field 3 for rule instead of 1.

image

But this didn’t work also so I’ve switched back to Custom Field 1 and realized that the value of “3PAR” that I’ve put for that field was still there. I thought when I select Custom Field 3 the value for Custom Field 1 will be automatically reset but this was not the case. This lead me to the thought that all used Custom Fields have to be defined in the routing rule in order to work so I’ve created the routing rule for Storage to use all Custom Fields:

image

image

image

image

image

image

I’ve also created the routing rule for Backup to see if they will work in parallel:

image

image

image

image

image

image

After creating the tow rules they looked like this in the SCOM Alert Connector:

image

As you can see the routing rules are different only for the definition of Custom Field 1.

After saving the SCOM Alert Connector configuration I’ve modified the custom fields of two alerts in SCOM and forward them to SCSM:

image

image

When the alerts were forwarded successful I’ve checked the SCSM console to see how both alerts look:

image

image

As you can see both alerts are routed correctly and assigned to the right Support Group.

In order to use routing of alerts for custom fields all used fields have to be configured in the routing rule.

The behavior of the connector for routing alerts using Custom Fields criteria is the same for SCOM 2007 R2 and 2012.

Configuring HP ProLiant Server Management Pack for SCOM

In my previous post I’ve covered the configuration of the management pack for HP Enclosures. But this MP only monitors health of the enclosures and not the blade servers inside via SCOM. In order to monitor blade servers or any other standalone HP server you have to install and configure HP ProLiant Server Management Pack. Actually if you want to use HP BaldeSystem Management pack you must import HP ProLiant Server Management Pack also.

HP ProLiant Server Management Pack can monitor HP servers trough two different methods:

  • SNMP method – For this method you must install HP Insight Management Agents on every server you will monitor besides the SCOM agent. This method uses SNMP to send trap messages.
  • WMI method – For this method you must install HP Insight Management WBEM Providers on every server you will monitor besides the SCOM agent. This method relays on WMI service.

As I am a Windows Engineer I prefer WMI method and on this article I will cover only this way of implementation.

The guide that is provided with this management pack covers both methods and I recommend reading it. This article will walk you trough the basic actions you need to do to install and configure HP ProLiant Management Pack for SCOM:

1. First you need to download the management pack itself from here. The MP is part of HP Insight Control for Microsoft System Center solution.

2. You will download one zip file that contains – 32-bit installation file, 64-bit installation file, License agreement file, Release notes file, User Guide and Troubleshooting Guide.

3. Copy 64-bit installation (if your SCOM server is 32-bit use 32-bit installation) to your RMS server and install it. It is pretty simple installation you can leave the default options. During the installation you will be present to choose if you want to install SNMP based method, WMI based method or both. If you choose only one of the methods the installation will only import the management pack that corresponds to the chosen method. It is best to choose both because you can always switch between methods or for example you can monitor one group of servers with one method and another group with the other one.

image

Note In the documentation of the MP it is not mentioned that the MP requires to be installed on the RMS server. In fact the MP can be installed on any Management Server or any server that have SCOM agent installed.

4. After the installation is finished the HP ProLiant Management Pack should be imported in SCOM and HP Custom Data Manager application should be installed. This application will allow you to enter additional information (fore example end of hardware support date) for individual servers that will be populated in SCOM Database.

5. After the MPs are imported discovery of HP servers will start. Discoveries are enabled by default so there is no additional configuration there. At this step it is best to check if all your HP monitored servers have HP Insight Management WBEM Providers installed. This can be achieved by opening SCOM console and navigating to Monitoring Pane –> HP Systems –> Windows Server State. Here you can see the discovered HP Servers in your environment. Modify this view to show HP Management Version column. This will show you if SCOM agent found HP Insight Management WBEM Providers.

image

6. Next and final step is to configure the MP to use WMI based method. By default HP ProLiant Server MP uses SNMP based method and this can be switched to WMI by overriding a rule. To do that you have to navigate to Authoring Pane –> Management Pack Object –> Object Discoveries. There you have to look for HP ProLiant Server Discovery Rule. When you find that rule select it and right click on it. Select Overrides –> Override the Object Discovery ->For all objects of type: Windows Computer. A new dialog will appear. Select all check boxes under Override column, change the value for SNMPWMIPrefernece from SNMP to WMI and save the override to your HP ProLiant Server Management  Pack for overrides.

image

Note If you want to switch to WMI based method only for particular servers create SCOM group and populate the group with the servers you want to monitor via that method. When you make the override in step 6 override for specific group and choose the custom created group.

7. This will switch to WMI based method for all Windows computers in your environment. After you made this setting you can again navigate to Monitoring Pane –> HP Systems –> Windows Server State.Edit the view to show Monitoring Source column. You will see your HP servers slowly switching from HP Insight Management Agents for monitoring source to HP Insight Management WBEM Providers.

image

After step 7 no further overrides are need in order to make HP ProLiant Server Management pack to work via WMI based method.

Configuring HP BladeSystem Management Pack for SCOM

Most of the large organizations today do not buy individual servers (exceptions are present of course) instead they buy Enclosures. In one such enclosure you can put up to 16 server and/or storage blades. HP as one of the biggest hardware vendors offers two such solutions – HP BladeSystem c3000 and HP BladeSystem c7000. If you bought such solution you receive for free HP BladeSystem Management Pack from HP for SCOM to monitor HP Enclosures. To start monitoring you HP Enclosures of course you have to first install and configure that MP on your SCOM environment. HP is providing installation and configuration guide for the management pack but I will walk you trough the steps you need to perform if you stumble on some difficulties. These steps are applicable for SCOM 2007 and 2012 versions:

1. First you need to download HP BladeSystem MP. You can find it here. The MP is part of Insight Control for Microsoft System Center solution.

2. After the MP is downloaded it is best to install SNMP Services feature on your RMS server (RMS Emulator). As in 2012 you have RMS emulator and you can move that role to other Management Server it is good idea to install SNMP Services on another Management server if you want to failover in case your RMS Emulator server goes down. SNMP Services installation does not require restart.

image

3. The HP BladeSystem MP when it is downloaded is archived in a zip file in which you can find several files – 64-bit installation file, 32-bit installation file, License agreement file, Release notes file, User guide and Troubleshoot guide. When you unzip it copy the 64-bit installation file (these days there are not many 32-bit SCOM servers Smile ) to your RMS server.

4. Start the 64-bit installation. Keep in mind that this management pack have certain requirements – HP ProLiant Server Management Pack imported (I will describe installation and configuration of that MP in another article), Windows PowerShell 1.0, .NET 2.0 Framework, SNMP Trap Service (We installed it in step 2) and of course at least SCOM 2007. Finish the installation with the default settings. It is pretty simple installation. If you have SCOM 2012 you perform the installation on the your failover Management Server also.

image

5. After the installation is finished you will see two new service and one application installed:

image

image

6. Next step is to configure SNMP. Open Services console. Find SNMP service and double-click on it. A dialog box will appear on which you have to click on Security tab:

image

7. You have to check Send authentication trap option. Add community name that you will use for communication between the RMS server and the enclosures. You can use READ ONLY rights.

8. Select Accept SNMP packets from these hosts and add all your IP’s of your Onboard Administrators (Primary and Secondary). Onboard Administrator is the control interface from Enclosures. Click OK when you are done. Now the RMS server is configured to receive SNMP packets from the Enclosures you will monitor. If you are using SCOM 2012 you can make the same settings on your Management Server that you will use for failover of your RMS Emulator role.

9. Next step is to add your HP Enclosures to HP Device Monitor Console. You can find that console in Start Menu -> All Programs -> HP SCOM Management Packs.

10. When you start it you need to make a connection to localhost, FQDN (of your RMS server)or NetBIOS name (of your RMS Server). The console actually connects to that HP Monitor Device Service that is installed on the RMS.

11. After you are connect to that service trough the console on the right of the console you will find Actions menu from where you can select Add Onboard Administrator option.

image

12. Click on Add Onboard Administrator.

13. A new dialog box will appear where you have to add IP address or FQDN of your Onboard Administrator. You also have to provide your administrator username and password to your Onboard Administrator interface.

image

14. Add all your Onboard Administrators to the console using steps 12 and 13. Adding OAs in the console will discover all components of your Enclosures and will draw SCOM diagrams for them. If you have SCOM 2012 you can connect the HP Device Monitor Console to your failover Management Server and add the same OAs there.

15. The final configuration step is to configure OAs to send SNMP traps to the RMS server. This can be achieved by opening OA web interface. When you open the interface you have to expand Enclosure Information than expand Enclosure Settings and finally click on SNMP Settings. Check Enable SNMP option, fill the other fields and click Apply. For Read and Write community name use the one entered you’ve used in step 7.

image

16. Now that SNMP is enabled we have to configure OA to send traps to specific IP address. Right beneath Enable SNMP settings there is SNMP Alert Destination setting. For host you have to enter the IP address for your RMS server and for community name enter the one you’ve used in step 7. Click Add. There is Send Test Alert button which will send trap to your RMS server and will appear as warning alert in SCOM console. This is very handful option to verify that your configuration was successful.

image

17. Repeat steps 15 and 16 for all your OAs.

When you complete your configuration you can use Send Test Alert option to see if all enclosures are configured correctly and are receiving alerts. You also should be able to use all your enclosures by opening the SCOM console, navigating to Monitoring pane –> HP Systems –> HP BladeSystem Enclosures –> Enclosure State:

image

Note: HP BladeSystem Management Pack should be imported during the installation. If for some reason this is not done you can import the MPs manually by finding them on the RMS server in the installation folder of the MP. This also refers to HP ProLiant Server Management  Pack.

If you have SCOM 2012 and your RMS Emulator server fails you can follow two steps to recover your monitoring for HP BladeSystem Enclosures:

1. Transfer RMS Emulator role to your failover Management Server

2. Change the IP address of your failover Management Server to the IP address of your failed RMS Emulator server because SNMP traps are send to specific IP address. If you do not want to change the IP address on your Management server you have to reconfigure all OAs to sent SNMP traps to your new RMS Emulator.

Note: HP BladeSystem Management Pack does not require to be installed on RMS server (RMS Emulator). It can be installed on any SCOM Management Server or Server that have agent installed but is common practice such monitoring solutions to be installed on RMS. Also the documentation of the management pack doesn’t mention that the MP relays on the RMS (RMS Emulator) server to function properly.

HP ProLiant Servers not monitored by WBEM Providers method in SCOM

HP have Management Pack for monitoring ProLiant Servers in SCOM. This management pack can monitor Windows Server by using 2 different methods:

  • SNMP via HP Insight Management Agents
  • WBEM via HP Insight Management WBEM Providers

In order to monitor HP servers with SCOM you have to have at least one of these two tools installed on your HP Servers.

When you import HP ProLiant Server Management Pack in SCOM the default method for monitoring is trough SNMP. In order to change WBEM Providers method you have to make override.

Recently I’ve stumbled on issue where after I’ve enabled the WBEM Providers method there were still several servers left monitored by SNMO method:

wbem

As you can see in the screenshot I have two servers that have monitoring source HP Insight Management Agents (SNMP) and because of that they do not have Health checkmark for Networks ProLiant WMI (WBEM method). In the detail view below it can be seen that HP Insight Management WBEM Providers are not detected at all.

When I logon to these two servers I can see that they both have HP Insight Management WBEM Providers software installed but for some reason it is not working properly.

The solution is very simple:

  1. Uninstall the HP Insight Management WBEM Providers.
  2. Install them again. If you do not have the installation files you can download them by going to HP download page.
  3. Wait a while until Discovery finds the HP WBEM Providers software and all components are discovered and healthy.
  4. Note: The uninstallation and the installation doesn’t require any restart of the server. Everything happens on the fly and doesn’t affect any service.