Introduction

ActiveMQ uses JDBC master-slave replication where one broker holds a database lock. If the lock renewal fails or the lease expires, the broker loses its master status, potentially causing split-brain or failover issues.

Symptoms

Lock expired:

```bash $ tail -f /var/log/activemq/activemq.log

[ERROR] Failed to renew database lock: Lock has been held by another broker [WARN] Master broker lost lock, becoming slave [ERROR] org.apache.activemq.store.jdbc.JDBCPersistenceAdapter - Lock keepalive failed ```

Database connection error:

bash
[ERROR] Cannot acquire database lock: Connection refused to database
[ERROR] java.sql.SQLException: Connection is closed

Split-brain warning:

bash
[WARN] Multiple brokers claiming master status detected
[ERROR] Network partition detected: both brokers claiming lock ownership

Common Causes

  1. 1.Lock keepalive timeout too short - Broker can't renew lock fast enough
  2. 2.Database connection issues - Connection dropped before renewal
  3. 3.Database overloaded - Slow response prevents lock renewal
  4. 4.Network latency - High latency causes renewal delays
  5. 5.Lock acquired by another broker - Another broker stole the lock
  6. 6.JDBC driver issues - Driver connection pooling problems

Step-by-Step Fix

Step 1: Check Lock Status

```bash # Check if broker is master curl -u admin:admin http://localhost:8161/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=localhost/MasterBroker

# Check lock table in database # MySQL: mysql -u activemq -p -e "SELECT * FROM activemq.ACTIVEMQ_LOCK;"

# PostgreSQL: psql -U activemq -d activemq -c "SELECT * FROM activemq_lock;"

# Oracle: sqlplus activemq@activemq "SELECT * FROM ACTIVEMQ_LOCK;"

# Check lock acquisition time # In activemq.log: grep -i "acquired lock|lost lock" /var/log/activemq/activemq.log | tail -20 ```

Step 2: Configure Lock Keepalive

```xml <!-- In activemq.xml, adjust JDBC persistence adapter --> <persistenceAdapter> <jdbcPersistenceAdapter dataSource="#mysql-ds" lockKeepAlivePeriod="5000" lockAcquireSleepInterval="10000" createTablesOnStartup="false"> <locker> <databaseLocker lockAcquireSleepInterval="10000"/> </locker> </jdbcPersistenceAdapter> </persistenceAdapter>

<!-- Key parameters --> <!-- lockKeepAlivePeriod: How often to renew lock (default 30000ms) --> <!-- lockAcquireSleepInterval: How long to wait before retrying lock acquisition -->

<!-- For high-latency environments, increase keepalive period --> <jdbcPersistenceAdapter dataSource="#mysql-ds" lockKeepAlivePeriod="10000"/> ```

Step 3: Configure Database Connection Pool

```xml <!-- In activemq.xml, use connection pool --> <bean id="mysql-ds" class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close"> <property name="driverClassName" value="com.mysql.jdbc.Driver"/> <property name="url" value="jdbc:mysql://localhost:3306/activemq?autoReconnect=true"/> <property name="username" value="activemq"/> <property name="password" value="password"/> <property name="maxTotal" value="20"/> <property name="maxIdle" value="10"/> <property name="minIdle" value="5"/> <property name="validationQuery" value="SELECT 1"/> <property name="testOnBorrow" value="true"/> <property name="testWhileIdle" value="true"/> <property name="timeBetweenEvictionRunsMillis" value="30000"/> <property name="minEvictableIdleTimeMillis" value="60000"/> </bean>

<!-- Enable auto-reconnect --> <property name="url" value="jdbc:mysql://localhost:3306/activemq?autoReconnect=true&amp;connectTimeout=5000&amp;socketTimeout=30000"/> ```

Step 4: Check Database Connectivity

```bash # Test database connection mysql -h db-host -u activemq -p activemq -e "SELECT 1;"

# Check database response time mysql -h db-host -u activemq -p -e "SELECT BENCHMARK(1000000, SHA1('test'));"

# Check database status mysqladmin -h db-host status

# Check for connection limits mysql -h db-host -u activemq -p -e "SHOW VARIABLES LIKE 'max_connections';" mysql -h db-host -u activemq -p -e "SHOW STATUS LIKE 'Threads_connected';"

# Check network latency to database ping db-host traceroute db-host ```

Step 5: Handle Lock Acquisition Failures

```bash # If lock was stolen by another broker # Check which broker has the lock curl -u admin:admin http://localhost:8161/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=*/MasterBroker

# Stop broker that shouldn't be master systemctl stop activemq

# Clear lock table manually (emergency only) mysql -u activemq -p -e "DELETE FROM activemq.ACTIVEMQ_LOCK;"

# Restart broker that should be master systemctl start activemq

# Watch lock acquisition tail -f /var/log/activemq/activemq.log | grep -i "lock" ```

Step 6: Use Lease Database Locker

```xml <!-- Use lease-based locker for better timeout handling --> <persistenceAdapter> <jdbcPersistenceAdapter dataSource="#mysql-ds"> <locker> <lease-databaseLocker lockAcquireSleepInterval="10000" leaseHolderId="broker-1" leaseKeepAlivePeriod="5000" leaseExpirationTime="60000"/> </locker> </jdbcPersistenceAdapter> </persistenceAdapter>

<!-- Lease-based locker benefits --> <!-- - Explicit expiration time --> <!-- - Better timeout handling --> <!-- - Avoids split-brain scenarios --> ```

Step 7: Configure Network Timeout

```xml <!-- In JDBC URL, set timeouts --> <bean id="mysql-ds" class="org.apache.commons.dbcp2.BasicDataSource"> <property name="url" value="jdbc:mysql://db-host:3306/activemq?connectTimeout=10000&amp;socketTimeout=60000&amp;autoReconnect=true"/> </bean>

<!-- For PostgreSQL --> <property name="url" value="jdbc:postgresql://db-host:5432/activemq?connectTimeout=10&amp;socketTimeout=60&amp;loginTimeout=10"/>

<!-- For Oracle --> <property name="url" value="jdbc:oracle:thin:@db-host:1521:activemq?connectTimeout=10000"/> ```

Step 8: Monitor Lock Health

```bash # Check lock renewal in logs grep -i "renewed lock|keepalive" /var/log/activemq/activemq.log | tail -50

# Enable lock logging # In logging.properties: log4j.logger.org.apache.activemq.store.jdbc=DEBUG log4j.logger.org.apache.activemq.store.jdbc.locker=DEBUG

# Monitor via JMX curl -u admin:admin http://localhost:8161/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=localhost/PersistenceAdapter/lockTime

# Set up alerting for lock failures grep -i "lost lock|failed to renew" /var/log/activemq/activemq.log ```

Step 9: Handle Split-Brain Scenario

```bash # If multiple brokers claiming master status # Stop all brokers systemctl stop activemq

# Check lock table mysql -u activemq -p -e "SELECT * FROM activemq.ACTIVEMQ_LOCK;"

# Clear lock table mysql -u activemq -p -e "DELETE FROM activemq.ACTIVEMQ_LOCK;"

# Start designated master first # On master broker: systemctl start activemq

# Wait for lock acquisition tail -f /var/log/activemq/activemq.log | grep -i "acquired lock"

# Start slave brokers # On slave brokers: systemctl start activemq

# Verify only one master curl -u admin:admin http://localhost:8161/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=*/MasterBroker ```

Step 10: Configure Failover URL

```java // Configure clients with failover URL ConnectionFactory factory = new ActiveMQConnectionFactory( "failover:(tcp://master:61616,tcp://slave:61616)?randomize=false&amp;priorityBackup=true&amp;timeout=5000" );

// Client detects master change and reconnects // Failover parameters: // randomize=false: Always try master first // priorityBackup=true: Prefer master // timeout: Max wait time for connection ```

JDBC Lock Settings

SettingDefaultRecommended
lockKeepAlivePeriod30000ms5000-10000ms
lockAcquireSleepInterval10000ms5000-10000ms
connectTimeout0 (infinite)10000ms
socketTimeout0 (infinite)60000ms

Verification

```bash # After configuration changes # Restart ActiveMQ systemctl restart activemq

# Check lock acquisition tail -f /var/log/activemq/activemq.log | grep -i "lock"

# Should see: "Acquired lock" and regular "Renewed lock"

# Verify master status curl -u admin:admin http://localhost:8161/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=localhost/MasterBroker

# Should return true for master broker

# Check database lock table mysql -u activemq -p -e "SELECT * FROM activemq.ACTIVEMQ_LOCK;"

# Monitor for lock renewals grep -c "renewed lock" /var/log/activemq/activemq.log ```

Prevention

To prevent ActiveMQ JDBC lock expired issues from recurring, implement these proactive measures:

1. Monitor Lock Status

yaml
groups:
- name: activemq-ha
  rules:
  - alert: ActiveMQJDBCLockExpired
    expr: |
      activemq_jdbc_lock_expired > 0
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "ActiveMQ JDBC lock expired"

2. Configure Proper Lock Settings

```xml <!-- activemq.xml --> <persistenceAdapter> <jdbcPersistenceAdapter dataDirectory="${activemq.data}" dataSource="#mysql-ds" lockKeepAlivePeriod="5000" lockAcquireSleepInterval="5000"/> </persistenceAdapter>

<bean id="mysql-ds" class="org.apache.commons.dbcp2.BasicDataSource"> <property name="url" value="jdbc:mysql://mysql-host:3306/activemq"/> <property name="username" value="activemq"/> <property name="password" value="password"/> <property name="maxTotal" value="20"/> <property name="maxIdle" value="10"/> <property name="connectionProperties" value="connectTimeout=10000;socketTimeout=60000"/> </bean> ```

3. Monitor Database Connectivity

```bash # Regular connectivity test cat << 'EOF' > /usr/local/bin/check_activemq_db.sh #!/bin/bash mysql -u activemq -p$ACTIVEMQ_DB_PASS -e "SELECT 1" activemq if [ $? -ne 0 ]; then echo "Database connectivity issue" systemctl restart activemq fi EOF

chmod +x /usr/local/bin/check_activemq_db.sh echo "*/5 * * * * root /usr/local/bin/check_activemq_db.sh" > /etc/cron.d/activemq-db-check ```

Best Practices Checklist

  • [ ] Monitor lock status with alerts
  • [ ] Configure proper lock keep-alive
  • [ ] Monitor database connectivity
  • [ ] Set appropriate timeouts
  • [ ] Use connection pooling
  • [ ] Test failover regularly
  • [Fix ActiveMQ Master Slave Failover](/articles/fix-activemq-master-slave-failover)
  • [Fix ActiveMQ Destination Full](/articles/fix-activemq-destination-full)
  • [Fix MySQL Connection Refused](/articles/fix-mysql-connection-refused)
  • [Fix Fix Activemq Broker Down Issue in Messaging](fix-activemq-broker-down)
  • [Fix ActiveMQ Destination Full](fix-activemq-destination-full)
  • [Fix ActiveMQ Kaha PIndex Corrupted](fix-activemq-kaha-pindex-corrupted)
  • [Fix ActiveMQ Master Slave Failover](fix-activemq-master-slave-failover)
  • [Fix ActiveMQ Slow Consumer](fix-activemq-slow-consumer)

<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "TechArticle", "headline": "Fix ActiveMQ JDBC Lock Expired", "description": "Troubleshoot ActiveMQ JDBC lock expired errors. Configure lock keepalive, database connections, and failover handling.", "url": "https://www.fixwikihub.com/fix-activemq-jdbc-lock-expired", "publisher": { "@type": "Organization", "name": "FixWikiHub", "url": "https://www.fixwikihub.com" }, "author": { "@type": "Person", "name": "FixWikiHub Editorial Team" }, "datePublished": "2026-04-03T21:44:40.140Z", "dateModified": "2026-04-03T21:44:40.140Z" } </script>