We ran into an interesting issue when we attempted to fail over our OperationsManager database from one node to the other node in an Always On SQL 2012 configuration. Operations Manager was functioning correctly when the database was on the primary node, but when it was moved to the other node we started to receive a variety of critical alerts. These included:
- Performance data collection process unable to write to the Data Warehouse
- Object Health State data collection process unable to write to the Data Warehouse
- Event data collection process unable to write to the Data Warehouse
- Event data collection process unable to write to the Data Warehouse in a timely manner
- Data Warehouse relationship synchronization process failed to read state
- Data Warehouse object health state data dedicated maintenance process failed to perform
- Data Warehouse monitor initial state data synchronization process failed to read state
- Data Warehouse managed object type synchronization process failed to read state
- Data Warehouse managed object synchronization process failed to read state
- Data Warehouse maintenance mode synchronization process failed to read state
- Data Warehouse health service availability data synchronization failed to read state
- Data Warehouse failed to request a list of management packs that have all of their dependencies
- Data Warehouse failed to request a list of management packs installed for their management
- Data Warehouse failed to enumerate database components to be deployed
- Data Warehouse failed to deploy database component
- Data Warehouse data set enumeration process failed to obtain list of data sets
- Data Warehouse configuration synchronization process failed to read state
- Data Warehouse alert data dedicated maintenance process failed to perform maintenance
Needless to say, it was pretty obvious that SOMETHING was wrong. When reverted the database back the original node, the alerts closed so this was a repeatable problem.
Resolution: When configuring SQL always on, it is important that the logins are configured consistently on each node of the cluster. Otherwise the above alerts may occur if the appropriate account cannot log into the database. In our case, the primary node had the logins defined correctly but the secondary node was missing one of the required accounts. Once we fixed this the alerts auto-closed and the issue has not recurred.