cisco onprem uc · published

Troubleshooting Unified CM database replication

Verified 2026-09-30 · 57 sources · tier 2

Also known as CUCM database replication, Unified CM Database Status report, utils dbreplication runtimestate.

Unified CM database replication operates in a fully meshed topology in which the publisher and every subscriber establish logical connections to all other servers in the cluster 7. Verifying and troubleshooting this replication topology relies on Command Line Interface (CLI) diagnostic utilities, system reports, and service checks 56362.

Replication States and Verification

Replication health is verified by running utils dbreplication runtimestate on the first node (publisher), where each node should report a Connected replication status and a setup value of (2) Setup Completed 56. In the Replication Setup (RTMT) & Details column of the utils dbreplication runtimestate output, a value of 2 indicates replication is good 43. The 12.5(1)SU1 Administration Guide states that any Replication Setup value other than 2 indicates a problem with database replication 14. WarmTransfer's reading of the sources is that the meanings of setup values 0 through 4 appear unchanged between Release 12.5(1)SU1 and Release 15 55.

WarmTransfer's reading of the sources is that the documented replication setup values indicate specific states: 55

Value	Documented meaning	Detailed source description
0	Replication not started 40	The Release 15 Administration Guide attributes this to no subscribers existing or the Database Layer Monitor service not running 40; the TAC technote describes it as an initialization state where staying over an hour indicates a possible setup failure 41.
1	Replicates created but count incorrect 42	Replicates have been created, but their count is incorrect 42.
2	Replication good / Setup Completed 4356	Connected status with Setup Completed across all nodes 56.
3	Replication bad in cluster / Mismatched tables 4445	The Administration Guide defines this as replication being bad in the cluster 44; the TAC technote describes it as logical connections being established while table matches remain uncertain 45.
4	Setup failed / dropped 4647	The Administration Guide defines this as setup not succeeding 46; the TAC technote describes it as a server lacking an active logical connection to receive tables 47.

Running utils dbreplication runtimestate requires command privilege level 0, is allowed during upgrade, and reports status from the perspective of a specific node when provided that node's name 35. In a healthy utils dbreplication runtimestate output, status checks show complete with no errors or mismatches, DB/RPC/DbMon reports Y/Y/Y for each server, the replication queue is typically 0, and setup displays Setup Completed for all servers 9. When reviewing utils dbreplication runtimestate, checking the timestamp prevents reading the Cluster Replication State section from outdated synchronization information 39.

To trigger a background check of replication, run utils dbreplication status on the first node (publisher), and review its progress and results via utils dbreplication runtimestate 4849. The Release 15 CLI reference states that utils dbreplication status should run only on the publisher and is not allowed during an upgrade 49.

Prerequisites and Environmental Diagnostics

WarmTransfer's reading of the sources is that non-destructive checks across services, network connectivity, DNS, NTP, cluster authentication, and configuration files precede repair actions, while reset procedures remain a final resort 18.

Four specific services must display a STARTED state in the output of utils service list: Cluster Manager, A Cisco DB, A Cisco DB Replicator, and Cisco Database Layer Monitor 36.

Network and environment verification requires passing checks:

Diagnostic tests: When running utils diagnose test, validate_network, ntp_reachability, and ntp_stratum must all report Passed 3.
NTP configuration: The NTP stratum must be less than 5 to avoid being deemed unreliable 16. In utils ntp status, the publisher synchronizes to an external NTP server, and subscribers report the publisher as their synchronization source at 1 stratum level higher 17. The address-change troubleshooting chapter gives utils ntp status, utils ntp restart, and utils ntp server list as the NTP checks on both publisher and subscriber nodes 15.
DNS records: If DNS is configured on any Unified CM server, both forward and reverse records (such as A and PTR) are required 4. A failed reverse DNS lookup causes replication-related network diagnostic errors, which are checked using utils network eth0 all and utils network host 34.
Cluster authentication: If show network cluster reports nodes as unauthenticated, verify network connectivity and ensure the security password matches across all nodes 1. If a subscriber displays incorrect publisher details after an address change, update it using set network cluster publisher 37.
Network ports and latency: Intracluster database traffic requires 1500/TCP for the primary database connection, 1501/TCP for the secondary connection, and 8500 TCP/UDP for IPSec Cluster Manager intracluster system data replication 1923. Port 1515/TCP is used for database replication during installation and must be permitted across the network to allow Database Layer RPC (DBL RPC) hello communication 2021. Port 1516/TCP handles database replication during upgrade 22. Nodes spread over a WAN that show Out of Sync or Not Requested statuses require network latency well under 80 ms 57.
Hosts file consistency: The hosts, rhosts, and sqlhosts files must match across all nodes in the cluster, as reflected in the Unified CM Database Status report generated under Cisco Unified Reporting > System Reports > Unified CM Database Status 102. When this report flags a mismatch in those files, restart Cluster Manager and then A Cisco DB from the publisher CLI, generate a new report, and evaluate if the mismatch clears 11.

Remediation and Operational Procedures

When utils dbreplication runtimestate flags table mismatches or errors, running utils dbreplication repair all attempts resynchronization before a full reset is considered 29. The utils dbreplication repair command brings subscriber data into sync with the publisher and is permitted during an upgrade, whereas repairtable and repairreplicate do not repair replication setup 28. For clusters containing more than 8 nodes, Cisco TAC should be consulted before executing repair and reset steps (Steps 7 and 8) 53.

If a reset becomes necessary, utils dbreplication stop must first run on all subscriber nodes before executing utils dbreplication reset 50. In scenarios where cluster nodes are switched back to an older software version, replication must be manually reset by executing utils dbreplication reset all from the publisher CLI 51.

Additional CLI tuning and recovery options include:

utils dbreplication setprocess: Configures a process setting between 1 and 40 (default 1) to accelerate setup performance at the expense of higher system resource consumption 38.
utils dbreplication setrepltimeout: Configures replication timeout between 300 and 3600 seconds (default 300) to batch more subscriber servers on larger clusters 30. Sizing guidelines calculate timeout as 1 minute per server for nodes 1 through 5, 2 minutes per server for nodes 6 through 10, and 3 minutes per server beyond 10, totaling 21 minutes for a 12-server cluster 31.
utils dbreplication forcedatasyncsub: Replaces a subscriber's database with the publisher's data with a default recovery timeout of 40 minutes, but must not be executed if only dynamic tables are out of sync 6.
utils dbreplication rebuild: Can affect cluster performance across other nodes and is recommended to run within a maintenance window 27.

Prior to initiating an IP address or hostname change, verified health checks require confirming the absence of active ServerDown alerts, verifying a replication status of 2 for all nodes in utils dbreplication runtimestate, and executing utils diagnose module validate_network on the first node 24. Pre-upgrade manual workflows require running utils dbreplication status followed by utils dbreplication runtimestate, resolving any node not showing a replication setup value of 2 before upgrading 25. The Upgrade Readiness COP file checks authentication and replication status before and after a Release 15 upgrade 26.

When opening a TAC case regarding replication issues, Cisco asks for utils dbreplication runtimestate from the publisher, the Unified CM Database Status report, as well as show network cluster, show status, utils diagnose test, utils service list, utils network connectivity, show open ports all, and show open ports regexp 8500 gathered from the publisher and each affected subscriber 522.

Applicability

Applies to: Cisco Unified Communications Manager. Deployments: on-premises. Sources checked 2026-09-30. The 12.5(1)SU1 Administration Guide addresses database replication verification rules 14. The Release 15 Administration Guide and CLI reference cover replication commands, port definitions, and setup states 403520.

What remains uncertain

The field-by-field meaning of runtimestate output columns (Repl Queue, Replication Group ID g_N, DbMon, and PING) is not covered by the sources below. Known defects that break replication after an upgrade, certificate change, or IP change are not covered by the sources below. Resolving the dropadmindb conflict between a troubleshooting technote and the Release 15 CLI reference is not covered by the sources below. The exact node scope wording for reset and stop in the Release 15 CLI reference is not covered by the sources below. Reconciling the intracluster round-trip delay limit from the design guide against the 80 ms figure in the technote is not covered by the sources below. Procedures to rebuild a CUCM publisher from a subscriber are not covered by the sources below. The recovery procedure when the cluster security password does not match across nodes is not covered by the sources below. IM and Presence Service database replication troubleshooting is not covered by the sources below.