New version 6.3.173.1 for SQL Server Management Pack is out

A new version of SQL Server Management Pack for SCOM is released but this is just a minor release and contains only one bug fix:

  • Fixed DB File Group Free Space bug

Download the MP from here.

SCOM Configuration Item Connector in SCSM is stuck at 100 % and events with ID 34081 are logged in Operations Manager Log

Not so while ago I was challenged with this issue:

SYMPTOMS

  • Events logged in Operations Manager log on System Center Service Manager server:

Log Name:      Operations Manager
Source:        Operations Manager Connector
Date:         
<DATE>
Event ID:      34081
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      scsmserver.contoso.com
Description:
Connector Name=SCOM CI Connector, Id=<GUID>
Encountered unexpected exception of type Object reference not set to an instance of an object. during synchronization. The synchronization will resume on the next scheduled time.
Message: %3.
Event Xml:
<Event xmlns=”http://schemas.microsoft.com/win/2004/08/events/event”>
<System>
<Provider Name=”Operations Manager Connector” />
<EventID Qualifiers=”49152″>34081</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime=”
<DATE>” />
<EventRecordID>1845675</EventRecordID>
    <Channel>Operations Manager</Channel>
    <Computer>scsmserver.contoso.com</Computer>
<Security />
  </System>
  <EventData>
    <Data>Name=SCOM CI Connector, Id=<GUID></Data>
<Data>Object reference not set to an instance of an object.</Data>
  </EventData>
</Event>

———————–

Log Name:      Operations Manager
Source:        Operations Manager Connector
Date:         
<DATE>
Event ID:      34081
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:     scsmserver.contoso.com
Description:
Connector Name=SCOM CI Connector, Id=<GUID>
Encountered unexpected exception of type NullReferenceException during synchronization. The synchronization will resume on the next scheduled time.
Message: Object reference not set to an instance of an object..
Event Xml:
<Event xmlns=”http://schemas.microsoft.com/win/2004/08/events/event”>
<System>
<Provider Name=”Operations Manager Connector” />
<EventID Qualifiers=”49152″>34081</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime=”
<DATE>” />
<EventRecordID>1845677</EventRecordID>
    <Channel>Operations Manager</Channel>
   <Computer>scsmserver.contoso.com</Computer>
<Security />
  </System>
  <EventData>
<Data>Name=SCOM CI Connector, Id=<GUID></Data>
<Data>NullReferenceException</Data>
    <Data>Object reference not set to an instance of an object.</Data>
  </EventData>
</Event>
———————–

Log Name:      Operations Manager
Source:        Operations Manager Connector
Date:         
<DATE>
Event ID:      34081
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      scsmserver.contoso.com
Description:
Connector <SCSM Management Group Name>
Encountered unexpected exception of type OMConnector.<GUID>_SyncRule during synchronization. The synchronization will resume on the next scheduled time.
Message: System Center Operations Manager Synchronization Workflow (internal).
Event Xml:
<Event xmlns=”http://schemas.microsoft.com/win/2004/08/events/event”>
<System>
<Provider Name=”Operations Manager Connector” />
<EventID Qualifiers=”49152″>34081</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime=”
<DATE>” />
<EventRecordID>1845678</EventRecordID>
    <Channel>Operations Manager</Channel>
<Computer>scsmserver.contoso.com</Computer>
<Security />
  </System>
  <EventData>
<Data><SCSM Management Group Name></Data>
<Data>OMConnector.<GUID>_SyncRule</Data>
<Data>System Center Operations Manager Synchronization Workflow (internal)</Data>
<Data><GUID></Data>
<Data>CI Sync with connector BME Id <GUID> to server scsmserver.contoso.com, Source Domain\User = <SCOM CI Connector Account></Data>
<Data>Object reference not set to an instance of an object.</Data>
</EventData>
</Event>

Note: The text has been modified to hide custom data.

  • SCOM CI Connector is stuck at 100% without finish date and finishes after next schedule synchronization date is start. When next scheduled synchronization date starts the SCOM CI Connector finishes with error.

CAUSE

  • You change SCOM CI Connector configuration by refreshing the synchronized management packs and one of the synced management pack is a custom unsealed management pack. That custom management pack was modified in SCOM but its version number was never changed and the modified version was not imported in SCSM. As result SCOM and SCSM have the same version number of this custom unsealed management pack but the management packs have different code in them.

RESOLUTION

  • Disable the unsealed custom management pack from syncing and save the SCOM CI Connector configuration. Synchronize the connector to test.
  • Delete the unsealed management pack from SCSM. Export the unsealed custom management pack from SCOM and import it in SCSM. Refresh the management packs in the SCOM CI Connector, select for synchronization the unsealed custom management pack and save the configuration. Synchronize the connector to test.

Routing Alerts from SCOM in SCSM by using Custom Field Criteria Type

Recently I faced the task to route alerts from SCOM in SCSM to different Support Groups. It seemed like an easy task because in most cases routing is based in Management Pack Name criteria. For example alerts that come from Management Pack that contains “SQL” in its name are assigned to SQL Support Group, alerts that come from Management Pack that contains “BizTalk” in its name are assigned to BizTalk Support Group and etc. You get the idea you can create such routing rule for every Management Pack. Besides this routing rule you can also route alerts based on SCSM groups membership of computer, Custom Fields and  Monitoring Classes.

image

When I started configuring routing based on Management Pack Name I didn’t had any issues everything was working as it was suppose to work you just have to be careful not put make any conflicts with rules by overlapping them. But when I tried to configure routing based on Custom Field I faced issues. In the next lines I will describe how stumbled on that issue and how I fixed it. I couldn’t find any such issue over Internet so I’ve decided to share it with the community.

Lets say that we have two Support Groups – Backup and Storage. Those two Support Groups are using one management pack in SCOM to monitor their devices. So in SCSM we need to configure: alerts that are coming from devices supported by Backup Support Group to be assigned to Backup team and alerts that are coming from devices supported by Storage to be assigned to Storage team. Most of you will probably suggest that we can put these devices in groups in SCSM and route alerts based on that or even easier we can route them based on Monitoring Class. But these two options are also not available because all these devices are monitored by SNMP so they they do not have CI record in SCSM and all alerts come from the server where the management pack is installed in our case this is the RMS server. Such management pack is HP Storage Management Pack. This management pack monitors various storage devices manufactured by HP and all is put in one MP file. Lets say we want to monitor 3PAR Storage, SAN Switches, D2D Devices and Tape Libraries with this management pack. All of these device are monitored by SNMP and we want 3PAR and SAN switches alerts to go to Storage Support Group and D2D device and Tape Libraries alerts to go Backup Support Group. When alerts for these devices are created in SCOM the first 6 custom fields are filled with values:

  • Custom Field 1 – Source of the Event
  • Custom Field 2 – Logging Computer name
  • Custom Field 3 – Device Id
  • Custom Field 4 – Device Name
  • Custom Field 5 – Source Computer Name – the computer that generated the event
  • Custom Field 6 – Source Computer Domain Name – the domain of the computer that generated the event

So custom fields for alert could look like this:

image

Or like this:

image

From the examples above it is clearly that the best option is to route alerts based on Custom Field 1. Before creating the route rule I will show you the steps for creating the templates that will be used by these rules.

If we go in SCSM console –> Library –> Lists and open the properties of Incident Tier Queue list we can see that we have 3 Support Groups – Storage, Backup and Windows:

image

So we need to configure 2 Templates in SCSM – one for Storage and one for Backup Support Group. We go to Templates and from Action Menu we choose Create Template and new window appears:

image

We can name the templates “SCOM Incidents Storage”, for class to choose Incident and for management pack you we can select a custom management pack where we store such settings. When we click OK an incident form will open. This is our template and here we have to fill the fields that will be changed when alert meets certain routing rule criteria. In our case we can populate Classification category, Source and Support group:

image

You can choose to populate different fields but Support group is the field that is actually used for assignment. When We click OK the template will be saved. Another template have to be created the same way for Backup:

image

Now we are ready with the templates and we can configure the routing rules in SCOM Alert Connector. I will not show how this connector is configured because it is pretty simple operation and there are a lot of articles over Internet about that.

When we open the SCOM Alert Connector Properties there is Alert Routing Rules tab and on that tab routing rules are added:

image

You can see even the option that if alert doesn’t meet any of the specified routing rules Operations Manager Incident Template will be used for them. This is the default SCSM Template. When we click on Add button a new window appears. In this window I gave distinguishing name for the routing rule, which template to use and the criteria for the alerts:

image

So I was ready with my first routing rule so I’ve clicked OK on the rule and OK on Connector’s window. Before creating more rules I’ve decided to test if my routing was right. You can create test alert from your device or you can take any alert that is with status new and it is not forwarded to your SCSM server and modify the custom fields like those for your device. After you modify them you can forward that alert to SCSM to see if it will be routed correctly just like this by selecting Forward to –> Alert Sync: SCOM alerts:

image

After the alert was forwarded I’ve open the SCSM console and found the alert created as a incident:

image

As you can see from the screenshot the Storage template I’ve created wasn’t applied to this incident because Support Group field was empty which meant that the default Operations Manager Incident Template was applied and the alert didn’t matched my routing criteria. At this point I understood that I have to make some troubleshooting in order to solve this.

The first thing I wanted to see if Custom Fields properties arrived in SCSM from SCOM properly. This can be seen in the Extensions tab of the incident:

image

As we can see from the screenshot all properties are the same as they appear in SCOM. I couldn’t find any reason why this solution is not work so I’ve started to modify the routing rule by different methods like using Custom Field 3 for rule instead of 1.

image

But this didn’t work also so I’ve switched back to Custom Field 1 and realized that the value of “3PAR” that I’ve put for that field was still there. I thought when I select Custom Field 3 the value for Custom Field 1 will be automatically reset but this was not the case. This lead me to the thought that all used Custom Fields have to be defined in the routing rule in order to work so I’ve created the routing rule for Storage to use all Custom Fields:

image

image

image

image

image

image

I’ve also created the routing rule for Backup to see if they will work in parallel:

image

image

image

image

image

image

After creating the tow rules they looked like this in the SCOM Alert Connector:

image

As you can see the routing rules are different only for the definition of Custom Field 1.

After saving the SCOM Alert Connector configuration I’ve modified the custom fields of two alerts in SCOM and forward them to SCSM:

image

image

When the alerts were forwarded successful I’ve checked the SCSM console to see how both alerts look:

image

image

As you can see both alerts are routed correctly and assigned to the right Support Group.

In order to use routing of alerts for custom fields all used fields have to be configured in the routing rule.

The behavior of the connector for routing alerts using Custom Fields criteria is the same for SCOM 2007 R2 and 2012.

Cluster disk monitoring for the latest Windows OS MP

Daniele Grandini a well known MVP has made a management pack that extends the monitoring of Cluster Disks for the latest Windows OS management pack. As you know the latest Windows OS MP contains a lot of changes of how logical disks are monitored in SCOM. You can read about all changes in Kevin’s blog. As these changes were introduced there were missing some performance counters for Cluster Disks and CSV. Daniele took the challenge to add these counters for Cluster Disk by creating an management pack. You can read about this solution and download the management pack on his blog.

Thank you Daniele for sharing this solution with the community.

Configuring HP ProLiant Server Management Pack for SCOM

In my previous post I’ve covered the configuration of the management pack for HP Enclosures. But this MP only monitors health of the enclosures and not the blade servers inside via SCOM. In order to monitor blade servers or any other standalone HP server you have to install and configure HP ProLiant Server Management Pack. Actually if you want to use HP BaldeSystem Management pack you must import HP ProLiant Server Management Pack also.

HP ProLiant Server Management Pack can monitor HP servers trough two different methods:

  • SNMP method – For this method you must install HP Insight Management Agents on every server you will monitor besides the SCOM agent. This method uses SNMP to send trap messages.
  • WMI method – For this method you must install HP Insight Management WBEM Providers on every server you will monitor besides the SCOM agent. This method relays on WMI service.

As I am a Windows Engineer I prefer WMI method and on this article I will cover only this way of implementation.

The guide that is provided with this management pack covers both methods and I recommend reading it. This article will walk you trough the basic actions you need to do to install and configure HP ProLiant Management Pack for SCOM:

1. First you need to download the management pack itself from here. The MP is part of HP Insight Control for Microsoft System Center solution.

2. You will download one zip file that contains – 32-bit installation file, 64-bit installation file, License agreement file, Release notes file, User Guide and Troubleshooting Guide.

3. Copy 64-bit installation (if your SCOM server is 32-bit use 32-bit installation) to your RMS server and install it. It is pretty simple installation you can leave the default options. During the installation you will be present to choose if you want to install SNMP based method, WMI based method or both. If you choose only one of the methods the installation will only import the management pack that corresponds to the chosen method. It is best to choose both because you can always switch between methods or for example you can monitor one group of servers with one method and another group with the other one.

image

Note In the documentation of the MP it is not mentioned that the MP requires to be installed on the RMS server. In fact the MP can be installed on any Management Server or any server that have SCOM agent installed.

4. After the installation is finished the HP ProLiant Management Pack should be imported in SCOM and HP Custom Data Manager application should be installed. This application will allow you to enter additional information (fore example end of hardware support date) for individual servers that will be populated in SCOM Database.

5. After the MPs are imported discovery of HP servers will start. Discoveries are enabled by default so there is no additional configuration there. At this step it is best to check if all your HP monitored servers have HP Insight Management WBEM Providers installed. This can be achieved by opening SCOM console and navigating to Monitoring Pane –> HP Systems –> Windows Server State. Here you can see the discovered HP Servers in your environment. Modify this view to show HP Management Version column. This will show you if SCOM agent found HP Insight Management WBEM Providers.

image

6. Next and final step is to configure the MP to use WMI based method. By default HP ProLiant Server MP uses SNMP based method and this can be switched to WMI by overriding a rule. To do that you have to navigate to Authoring Pane –> Management Pack Object –> Object Discoveries. There you have to look for HP ProLiant Server Discovery Rule. When you find that rule select it and right click on it. Select Overrides –> Override the Object Discovery ->For all objects of type: Windows Computer. A new dialog will appear. Select all check boxes under Override column, change the value for SNMPWMIPrefernece from SNMP to WMI and save the override to your HP ProLiant Server Management  Pack for overrides.

image

Note If you want to switch to WMI based method only for particular servers create SCOM group and populate the group with the servers you want to monitor via that method. When you make the override in step 6 override for specific group and choose the custom created group.

7. This will switch to WMI based method for all Windows computers in your environment. After you made this setting you can again navigate to Monitoring Pane –> HP Systems –> Windows Server State.Edit the view to show Monitoring Source column. You will see your HP servers slowly switching from HP Insight Management Agents for monitoring source to HP Insight Management WBEM Providers.

image

After step 7 no further overrides are need in order to make HP ProLiant Server Management pack to work via WMI based method.