How can we tell which server is the active resource pool member? | Quisitive

Have you ever tried to debug a situation where notifications seem to be working but only part of the time in Operations Manager? I ran into this situation recently and found a couple of items which assist with this specifically for Operations Manager 2012. This blog post will cover:

What is a resource pool?

Resource Pools were added in Operations Manager 2012. As a quick definition for what they are: “A resource pool is a collection of management servers used to distribute work amongst themselves and take over work from a failed member.” – subset from http://technet.microsoft.com/en-us/library/hh230706.aspx. I would update that slightly to the following definition based upon this discussion that a gateway can also be a member of a resource pool:  http://www.systemcentercentral.com/BlogDetails/tabid/143/IndexID/94138/Default.aspx . “A resource pool is a collection of management servers and/or gateway servers used to distribute work amongst themselves and take over work from a failed member.” 

For additional readings on resource pools I recommend:

General recommendations to debug notifications send through a resource pool:

The following is a summary of insights and best practices I’ve been combined based on feedback from most of the OpsMgr alpha-geeks on the planet. (Thank you Kevin, Scott, Flemming, Tao, Dieter and Kevin).

How to determine what server is active in a resource pool

Update: A member of the System Center Central community (Alex) provided an updated query which covers all application pools and has results which are more consistent than what I put together. Alex, thank you for writing this and for sharing it!

select
BaseManagedEntity.DisplayName
,cs.agent.AGentGuid
,cs.WorkFlowExecutionLocationAgent.AgentRowId
,cs.workflowexecutionlocation.WorkflowExecutionLocationRowId
,cs.workflowexecutionlocation.DisplayName
from cs.WorkFlowExecutionLocationAgent
inner join cs.workflowexecutionlocation
ON cs.WorkFlowExecutionLocationAgent.WorkFlowExecutionLocationAgentRowId = cs.workflowexecutionlocation.WorkflowExecutionLocationRowId
inner join CS.agent
ON CS.agent.AgentRowId=cs.WorkFlowExecutionLocationAgent.AgentRowId
inner join BaseManagedEntity
ON BaseManagedEntity.BaseManagedEntityId = CS.agent.AGentGuid
where cs.workflowexecutionlocation.DisplayName like '%Pool%'

Sample output results from my lab (single management server) and another lab are shown below: (truncated to only the two relevant fields – name of the management server and name of the resource pool)

OM01.cloud.pvtAD Assignment Resource Pool
OM01.cloud.pvtAll Management Servers Resource Pool
OM01.cloud.pvtNotifications Resource Pool
OM1.CAT.DEMOGSM Pool
OM2.CAT.DEMONetwork Device Pool
OM1.CAT.DEMOAD Assignment Resource Pool
OM2.CAT.DEMOAll Management Servers Resource Pool
OM1.CAT.DEMONotifications Resource Pool

Summary: I hope that this blog post provided some interesting insights into resource pools, how to debug them and how to determine who the active resource pool member is!

Update: Alexey Zhuravlev put together a sample pack on how to debug the active pool member which is available at: http://www.systemcentercentral.com/pack-catalog/demo-pool-owner/

One of the best parts of TechEd was getting to talk with customers who are using Operations Manager and had a specific question that they were looking for a resolution to. One of these conversations that I had was around how to monitor certificates and provide a notification of when they will expire beyond the three week default timeframe. I spent a little time digging into this and found that the community has a management pack available which could assist with this question.

It’s available from SystemCenterCentral at: http://www.systemcentercentral.com/pack-catalog/pki-certificate-verification-mp/

The default certificate expiration is set to 21 days. However, this can be changed via an override as shown below. This is set on the Certificate lifespan monitor by changing the Lifetime threshold as shown below.

A sample alert is shown below:

The certificate has expired on 31.15.2002 09:00. Certificate Name: Microsoft Windows Hardware Compatibility Serial number: 198b11d13f9a8ffe69a0 Certificate store: Intermediate Certification Authorities

Lifetime threshold (days)

Default: 21 days

Calculates how many days are left until the certificate expires by evaluating the ‘Valid to’ property of a certificate

Summary: If you are looking for a way to get an alert more than three weeks before your certificate expire, you may want to check out the SystemCenterCentral pack available at http://www.systemcentercentral.com/pack-catalog/pki-certificate-verification-mp/. Once it is added you can change the default behavior by setting an override on the Lifetime threshold field.

During a recent discussion on monitoring for devices which can be monitored with Operations Manager (but without installing an agent) one of my co-workers mentioned that we can use the new Operations Manager 2012 network monitoring for non-SNMP devices by using ICMP only mode. This approach lets you monitor devices which can only be monitored via ping type monitors (similarly to the discussion on the free OpsLogix ping management pack). When I think of network monitoring in Operations Manager my brain automatically goes to SNMP – I had forgotten completely that Operations Manager can do ICMP access mode which is a pretty useful tool to have available.

This blog post will explain how we discover network devices using only ICMP, what we get from using network monitoring with ICMP only mode in Operations Manager, and what can be configured for this functionality.

How to discover network devices via ICMP only monitoring

You can create a new network discovery or add an entry for the new items to an existing network discovery. Use the Access Mode of ICMP to specify that it will be ICMP only monitored device.

What do we get from using Network Monitoring running in ICMP only mode in OpsMgr 2012?

Where do we see the device if we are monitoring it only via ICMP? It appears in the Network Monitoring folder in the Hosts view as shown below.

Since it is a network device, it also appears on the Network Summary Dashboard.

Since it’s part of the Operations Manager framework, we can use Health Explorer to see it’s health and details on what the interval, number of retries and timeout information as shown below.

Health explorer:


The existing dashboards also work for this type of a device such as the vicinity view and average availability.

What performance counter(s) are collected?

The ICM ping response time counter is gathered for the ICMP only monitored device as shown below in the ICMP Ping Response Time performance view.

Configuration of the ICMP only monitored device:

You can also configure different ways to use this functionality through overrides as shown below where the network device can be configured to create an alert, or you can change the interval, number of retries, timeout, packet size and more.


Summary: Using the built-in network ICMP only network monitoring capabilities for Operations Manager makes it easy to add devices which can only be monitored via ping. This approach integrates with the pre-built network monitoring dashboards in Operations Manager and can be customized through the use of overrides.

Thank you to Brian Pavnick for bringing this question to my attention!

This article identifies changes seen in Operations Manager 2012 R2 preview release and details the steps required to install Operations Manager 2012 R2 preview release running on Windows Server 2012 R2 preview release. For the to be an all-in-one System Center Operations Manager 2012 R2 environment, it will need to be domain controller, SQL server and Operations Manager server.

The following were the steps taken to install the all-in-one environment (documented in this blog post):

Installing the server as a domain controller

The process was not noticeably different than promoting a domain controller in Windows Server 2012.

image
image
image
image
image
image
image

I rebooted after the domain controller promotion process had completed.

Pre-requisites for Operations Manager 2012 R2 all-in-one environment:

Report viewer controls:

I downloaded and installed report viewer per the link in the Operations Manager prerequisite checker.

image

IIS requirements: I added roles requested by the OpsMgr 2012 R2 setup:

image

Windows Identify Foundation

I added Windows Identify Foundation as a warning was raised indicating that this was required for System center Advisor during the Operations Manager prerequisite checker..

image

Installed the WIF feature:

image

Installation of SQL Server 2012:

This was a standard SQL server 2012 installation, with no notable configuration changes required for Windows Server 2012 R2.

image
image

Operations Manager 2012 R2 Preview release installation:

There were a few minor changes during the Operations Manager 2012 R2 prerelease installation steps.

(Note the checkbox to download the latest updates to the setup program – does this remind anyone else of the configuration manager installer?)

image

Since we’re installing this as an all-in-one, all features were chosen. For the remainder of the setup process defaults were chosen.

image
image

For my lab environment, I ended up running it on four processors and four gigabytes of memory to have a successful install unlike the warning condition shown below.

image
image
image
image
image
image
image
image
image
image
image
image

Addition of the advisor management pack during the installation is shown below:

image

On my first attempt installation failed on the data warehouse configuration step.

image

After the failure I rebooted and re-tried the installation after moving the VM’s configuration up to four processors and four gigabytes of memory resulted in a successful installation.

What are the changes in the Operations Manager 2012 R2?

Based upon Microsoft’s statement at: http://www.microsoft.com/en-us/server-cloud/system-center/system-center-2012-r2.aspx, this section appears to map to the System Center 2012 R2 Operations Manager enhancements:

System Center 2012 R2 provides deep insight into your applications and workloads, enabling you to deliver a predictable application SLA to your business. It provides application performance monitoring for Java applications, including line-of-code traceability. In addition, the System Center Advisor connector allows you to consume workload configuration best practices in your familiar Operations Manager console.

We expect that there are changes to the Java monitoring capabilities as well as the System Center Advisor connector. Some of these were apparent after installation of the 2012 R2 preview release.

During my touring of the new Operations Manager 2012 R2 preview release what did I find which was different when compared with Operations Manager 2012 SP1?

Updated launch screen:

image

New version screen showing the 2012 R2 update:

image

Monitoring pane changes:

New Advisor views:

image

And the Windows client management pack is added by default:

Administration pane:

New Advisor settings in the Administration pane:

image
image

And new version number for the Operations Manager management server:

image

Authoring pane:

And in the Authoring pane there is a pre-built “Application Health SLA” defined within Service Level Tracking:

image

Summary: The System Center 2012 R2 preview release installed on Windows Server 2012 R2 preview release in an all-in-one configuration with no notable issues and several changes in this version are easy to identify in the Operations Manager console.

Bonus screenshot!

Check out this good screenshot taken of the “Windows Reboot Fish” during a reboot of my Windows Server 2012 R2 preview release 🙂

I was recently working with a dynamic group membership situation where we needed to include all of the sub-OU’s within the group. Our approach was to create a dynamic membership which matched based on a wildcard value that would exist because the sub-OU naming includes the top level OU naming. The result was just what we needed so we didn’t need to specify each sub-OU’s membership in the group. I was going to write up the details on this, but I found that someone else had done so (it’s great being a member of a community that shares information like this – way to go!). The following is a subset of his article available a http://00shep.blogspot.com/2012/03/scom-groups-dyanmic-members-ou.html

“Note the highest level OU for which you want to capture all sub-systems

  1. Go to one of the systems in SCOM and view the properties in “Monitoring”. One of the values will be “Organizational Unit” > Copy it
  2. Create your Dynamic Members inclusion rule
  3. Select “Windows Computer” > Add

e.g.
*OU=XenApp-65,OU=Servers,DC=MYDOMAIN,DC=com”

Summary: Operator role and higher (operator, advanced operator, and administrator) should be able to put systems into maintenance mode.

My testing: For due diligence I tested this in my lab environment as well for the Administrator role, Operator Role and Read-Only Operator Role and the results are shown below.

Administrator: (Maintenance mode is available)

Operator: (Maintenance mode is available)

Read-Only Operator: (Maintenance mode is NOT available)

For additional details, Microsoft has provided a set of what privileges are associated with each role in OpsMgr which is included in this post for reference purposes (The following content of what roles have what privileges is re-arranged from: http://technet.microsoft.com/en-us/library/hh872885.aspx)

1.1 Administrator

The Administrator profile includes full privileges to Operations Manager. No scoping of the Administrator profile is supported. The Administrator profile contains all of the privileges found in the Author, Advanced Operator, Operator, and Read-Only Operator profiles in addition to those listed below.

1.2 Author

The Author profile includes a set of privileges designed for authoring of monitoring configuration. A role based on the Authors profile grants members the ability to create, edit, and delete monitoring configuration (tasks, rules, monitors, and views) within the configured scope. For convenience, Authors can also be configured to have Advanced Operator privileges scoped by group. The Author profile contains all of the privileges found in the Advanced Operator, Operator, and Read-Only Operator profiles in addition to those listed below.

1.3 Advanced Operator

The Advanced Operator profile includes a set of privileges designed for users who need access to limited tweaking of monitoring configuration in addition to the Operators privileges. A role based on the Advanced Operators profile grants members the ability to override the configuration of rules and monitors for specific targets or groups of targets within the configured scope. The Advanced Operator profile contains all of the privileges found in the Operator and Read-Only Operator profiles in addition to those listed below.

1.4 Operator

The Operator profile includes a set of privileges designed for users who need access to alerts, views, and tasks. A role based on the Operators profile grants members the ability to interact with alerts, run tasks, and access views according to their configured scope. The Operator profile contains all of the privileges found in the Read-Only Operator profile in addition to those listed below.

1.5 Read-Only Operator

The Read-Only Operator profile includes a set of privileges designed for users who need read-only access to alerts and views. A role based on the Read-Only Operators profile grants members the ability to view alerts and access views according to their configured scope.

1.6 Report Operator

The Report Operator profile includes a set of privileges designed for users who need access to Reports. A role based on the Report Operator profile grants members the ability to view reports according to their configured scope.

1.7 Report Security Administrator

Thank you to Paul Johnson for his assistance putting this together!

For an upcoming demonstration I was tasked with showing how End-User Recovery works in Data Protection Manager 2012. For most of this process I worked from Robert’s article at: http://robertanddpm.blogspot.com/2011/02/eur.html. The following were the steps that I ended up doing to achieve the same task in DPM 2012:

1) The first step was to deploy the DPM agent to my Windows 7 system (see https://www.catapultsystems.com/cfuller/archive/2012/03/02/windows-server-8-beta%E2%80%93managing-it-with-configmgr-2012-rc-sccm-and-backing-it-up-with-dpm-2012-rc-scdpm.aspx for examples on how to manually install a DPM 2012 agent and point it to the DPM server)

2) Next I enabled the schema extension for DPM / End User Recovery by running the DPMADSchemaExtension (run as administrator) with a schema admin account. The default file location is shown below from my DPM server (c:\program files\Microsoft DPM\DPM\End User Recovery).

The following were the prompts and steps required to configure this for my domain (cloud.pvt) and my DPM server (cloud-dp01).

3) Next I enabled Self Service recovery from the DPM console in the Protection section of the UI.

In DPM enabled End-User Recovery from the Protection tab by checking the box shown below:

4) On my Windows 7 system I created a new file (testfile2.txt) and stored it on the desktop of the Windows 7 system.

5) Next I created a new protection group to backup specific folders on my Windows 7 system (including the desktop). On the DPM server I validated the backup was working as shown below where I was able to see the file that I had recently created on the Windows 7 system.

6) To enable the user’s self service recovery I installed this option from the DPM 2012 media on the Windows 7 client:

7) I changed the DPM icon so it would be visible as an icon on the taskbar and left-clicked on it to see options for the DPM client as shown below:

8) After a successful synchronization I was able to see recent backups as shown below:

9) To test this functional I deleted my new file that I had backed up and successfully used the DPM client to restore it on my Windows 7 client.

10) Finally to remove existing recovery options I disabled them rom the UI as shown below (idea from http://robertanddpm.blogspot.com/2011/10/complements-for-eur.html). The original view available included the Previous Versions tab shown below:

Once these changes were made the Previous Versions tab was removed from the UI:

Summary: Based upon my tests it appears that End User Recovery is functional even in the RC version of the product based upon the tests I was performing in my lab environment.

Good additional links for reading on this topic include:

http://robertanddpm.blogspot.com/2011/02/eur.html: Getting started on End User Recovery in DPM

http://robertanddpm.blogspot.com/2011/10/complements-for-eur.html: Disable the existing recovery options via the previous versions functionality

OpsMgr 2007 introduced the concept of a Root Management Server and Secondary Management Servers. In OpsMgr2007 the RMS comprised 3 services providing the following functionality:

1) SDK Service

2)Health Service

3) Configuration Service

The RMS was both a single point of failure and a potential bottleneck.

In SCOM 2012 the Root Management Server was removed but an RMS Emulator role (RMSE) has been retained. These seem to be logical next-step questions to ask based upon the new RMSE:

Questions/answers below to these are below:

What do we know about the RMS Emulator?

The RMS Emulator Role is assigned to the first Management Server installed in the management group. This role can be moved to another Management Server using PowerShell:

§ Get-SCOMManagementServer –Name <New RMS Emulator> | Set-SCOMRMSEmulator

From – http://blogs.technet.com/b/momteam/archive/2011/08/22/topology-changes-in-system-center-2012-operations-manager-overview.aspx – In order for us to not to break backwards compatibility and provide support for legacy management packs we decided to leave the Root Management Server instance and add a special role to one of the management servers in the Management group called the RMS Emulator. This RMS Emulator is only for backwards compatibility to legacy management packs and is in no way required for the management group to function correctly.” An example of a management pack which contains a workflow which would require the RMS Emulator is shown below:

The RMS Emulator Role is only necessary to trick those workflows that were targeted to the RMS into thinking that the RMS still exists.

Subset from http://blogs.technet.com/b/momteam/archive/2011/08/22/topology-changes-in-system-center-2012-operations-manager-overview.aspx “At this point you may be wondering about workflows targeted to the RMS that are outside of the OpsMgr product groups control (other management packs from different Microsoft teams or third party vendors). They have all been moved to target Resource Pools OR singleton objects (that also live in the “all management servers” pool). Other applications management packs or custom management packs might still want to find that object, and that is the only reason why this object exists. The OpsMgr management packs don’t target the RMS object anymore which brings us onto Resource Pools.

Does the RMS Emulator do all of the same items that the RMS originally provided?

The majority of the services previously provided by the RMS can now be performed by any management server and as a result inherently provide redundancy when multiple management servers exist in the appropriate management server resource pool.

The services which were previously managed by the RMS are listed below with their new homes:

· Console access: Role based access control is done by ANY SDK Service when you connect to it. Any management server although you can control which management server (as with some other SDK activity) via Load Balancing. To use load balancing provide users with the virtual name of the Management Server load-balancing pool and just add management servers into and out of the resource pool on the load balancer.

· Role based access control: Role based access control is done by ANY SDK Service when you connect to it

· Distribution of configurations to agents: Distribution of configurations to agents is done by any MS for its agents (“federated config service”)

· Connectors to other mgmt systems: Connectors to other mgmt systems can “talk to” any MS they like – like consoles

· Alert notifications: Alert notifications is a workflow, runs in a pool [Notifications Resource Pool]

· Group Calculations: Group Calculations is a workflow, runs in pools and or distributed across MS’s depending on which MS owns which group/container [All Management Servers Resource Pool]

· Availability: Availability – runs distributed as usual – pieces on the agent, pieces on the MS’s, depending on which MS owns which group/container/singleton object [All Management Servers Resource Pool]

· Dependency Monitor: Dependency Monitor – runs where the corresponding objects live… on one of the servers in the “all management servers” pool

· Health aggregation: Health aggregation is a workflow, runs in pools and or distributed across MS’s depending on which MS owns which container [All Management Servers Resource Pool]

· DB Grooming: DB Grooming – runs in a pool [All Management Servers Resource Pool]

· Enables model based mgmt: While the RMS handed some of the model based management

· Resource pools only cover Health Service functionality.

· Resource pools do not cover Config service – There is now a distributed configuration with a Configuration Service on every management server. Failover and assignment works as it did in OpsMgr 2007 R2.

<side-bar of extra cool OpsMgr goodness

The config service was rewritten almost from scratch. It has a new service name – the original was OMCFG and the new is CSHOST. The data is now federated among management servers, all contributing to the configuration. A new set of tables were created in the OpsDB (CS.*) to persist the instance space (as opposed to having it all in memory like the old config service did). The “new set of tables” is called the Configuration Store (hence the CS.* schema), and was initially meant to be a separate database, but was then collapsed into OpsDB after it was understood that it really would not impact performance to keep the two together – as it is one less DB to maintain and backup, so easier for the user.

</side-bar of extra cool OpsMgr goodness

· Resource Pools don’t cover SDK functionality which you demo by connecting to a management server or reporting through a console. If you stop the sdk service on the management server you are connected to then the console or reporting won’t failover to another management server. But you can leverage load balancing to achieve this.

What classes should be used to target instead of using the RootManagementServer class?

This will vary depending upon the requirements for the management pack which is being developed but primarily the Microsoft.SystemCenter.AllManagementServersPool class from the Microsoft.SystemCenter.Library management pack would be used for a ‘general’ server-based workflows. Another option is to create your own singleton class that “makes sense” for your application, and target that one. Many other workflows will need a different pool as a target (like a “user pools” or a pools for network device monitoring or a pool for Linux/Unix monitoring).

Summary: The RMS Emulator only provides backward compatibility to management pack using things which target the ‘Root Management Server’ class instance (Target=”SC!Microsoft.SystemCenter.RootManagementServer). The RMS emulator is hosted on a management server and as such it does a lot of activities that aren’t specific to the RMSE role. Everything else the RMS used to be responsible for is managed by the “All Management Servers Resource Pool”, the “Notifications Resource Pool” and the “AD Assignment Resource Pool”.

I owe a huge thanks to the entire OpsMgr MVP community for their assistance in dumbing this down to my level! Specific thanks to: (alphabetically since so many folks helped with this!)

The following is another dashboard component for the Service Manager Dashboard customized for OpsMgr (#SCOM). This queries for from the OperationsManagerDW database to gather the most recent counters for Processor Utilization for all servers in the environment. This query has been added to the SQL queries provided on SystemCenterCentral (and remember I am not a SQL guy so if the query is ugly don’t blame me!)

All Server processor utilization most recent counter: (displayed as a bar chart and data grid)

DECLARE @HOURsOffset NUMERIC
DECLARE @ObjectName varchar(200)      
DECLARE @CounterName varchar(200)             
DECLARE @InstanceName varchar(200)             
          
set @ObjectName=’Processor‘             
set @CounterName=’% Processor Time‘             
set @InstanceName=’_Total‘      
SET @HOURsOffset = (select datediff(hour,getdate(),getutcdate()) )    
 

DECLARE @TABLE TABLE ([NAME] VARCHAR(255),[SampleValue] NUMERIC(9,2),[TimeAdded] DATETIME, [Rank] INT) 

INSERT INTO @TABLE   

SELECT  left(Path,CHARINDEX(‘.’, Path)-1) as Path, SampleValue, dateadd(HOUR,-@HOURsOffset,DateTime) AS  TimeAdded,
rank() OVER (PARTITION BY Path ORDER BY DateTime DESC) AS Rank 

FROM Perf.vPerfRaw p

INNER JOIN vManagedEntity me ON me.ManagedEntityRowID = p.ManagedEntityRowID
INNER JOIN vPerformanceRuleInstance pri ON pri.PerformanceRuleInstanceRowId = p.PerformanceRuleInstanceRowId
INNER JOIN vPerformanceRule pr ON pr.RuleRowId = pri.RuleRowId
  where objectname = @ObjectName and instancename = @InstanceName and countername = @CounterName and Path != ‘NULL’
  and p.DateTime > DATEADD(minute, -60, getutcdate())


SELECT DISTINCT (NAME)as PATH,SampleValue as ProcessorUtil, TimeAdded FROM @TABLE
WHERE rank = 1      
  order by [Path], [TimeAdded]

Note: By changing the bolded items above, this SQL query could be used to to provide any other performance counter which is gathered by Operations Manager.

Additional Note: Performance counters which use optimized data can cause challenges as they often will not have reported the value within the timeframe identified above (-60 in the query indicates the number of minutes to look back for data).

Summary: Looking for a dashboard to show the most recent value for a performance counter in OperationsManager using the OperationsManagerDW? Check this one out!