A little background

In April I wrote about our BPM Server installation, that is actually a single host cluster on dev and test. But installed like it was a dual-node  server, so we had a cloned domain.

A BPM installation has a component that is called the Process Asset Manager. Under the hood it uses a replicated SubVersion server. Each node has one, so they had to synchronize. But since we're on the same host, we needed to differentiate in port numbers. Although we used virtual host names for each server node. In an article in april 'BPM BAC Subversion Server refusing connections' I wrote about that.

It turns out: it did not work appropriately. After some investigation, apparently the bac_node1/subversion server calls the subversion server on bac_node2, with it's own address, but with the port of the other:

...
<Jul 24, 2017, 2:37:29,543 PM CEST> <Debug> <oracle.bpm.bac.svnserver.replication> <darlin01> <bpm_server1> <Active Sync Thread [/54680efc-9328-478e-953c-834bbb250725/]> <<anonymous>> <> <20a54a15-d8d8-4e58-a4f6-11a90e992967-00000008> <1500899849543> <[severity-value: 128] [rid: 0:490:13:19] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000000> <[oracle.bpm.log.Logger:debug] About to synchronize against node Member(Id=2, Timestamp=2017-07-12 08:01:13.309, Address=10.0.0.38:37055, MachineId=9002, Location=site:tst.darwin-it.local,machine:bpm_machine1,process:24566,member:bpm_server2, Role=bpm_cluster), url svn://syncuser@t-bpm-1-bpm-1-vhn.tst.darwin-it.local:8424/54680efc-9328-478e-953c-834bbb250725>
...

Of course there's no listen address so unsurprisingly we get errors like:

...
[2017-05-30T12:55:51.704+02:00] [bpm_server1] [ERROR] [] [oracle.bpm.bac.svnserver.replication] [tid: Active Sync Thread [/b91abb78-6b3d-4448-af6a-e82125f261f0/]] [userId: ] [ecid: ed0bc6ce-fefb-4608-8dbe-46f7206a1573-0000000a,0:527] [APP: OracleBPMBACServerApp] [partition-name: DOMAIN] [tenant-name: GLOBAL] org.tmatesoft.svn.core.SVNException: svn: E210003: connection refused by the server[[
oracle.bpm.bac.subversion.server.repository.exceptions.RepositoryException: org.tmatesoft.svn.core.SVNException: svn: E210003: connection refused by the server
at oracle.bpm.bac.subversion.server.repository.exceptions.RepositoryException.wrap(RepositoryException.java:56)
at oracle.bpm.bac.subversion.server.repository.SVNKitRepositorySession.getRepositoryUUID(SVNKitRepositorySession.java:98)
at oracle.bpm.bac.subversion.server.repository.RepositorySVNSync.sync(RepositorySVNSync.java:74)
at oracle.bpm.bac.subversion.server.repository.RepositorySVNSync.sync(RepositorySVNSync.java:59)
at oracle.bpm.bac.subversion.server.repository.ha.aa.ActiveAARepository$Synchronizer.runImpl(ActiveAARepository.java:341)
at oracle.bpm.bac.subversion.server.repository.ha.aa.ActiveAARepository$Synchronizer.run(ActiveAARepository.java:304)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.tmatesoft.svn.core.SVNException: svn: E210003: connection refused by the server
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:85)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:69)
at org.tmatesoft.svn.core.internal.io.svn.SVNPlainConnector.open(SVNPlainConnector.java:62)
at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.open(SVNConnection.java:77)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.openConnection(SVNRepositoryImpl.java:1252)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.testConnection(SVNRepositoryImpl.java:95)
at org.tmatesoft.svn.core.io.SVNRepository.getRepositoryUUID(SVNRepository.java:280)
at oracle.bpm.bac.subversion.server.repository.SVNKitRepositorySession.getRepositoryUUID(SVNKitRepositorySession.java:95)
... 5 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:112)
at org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:68)
at org.tmatesoft.svn.core.internal.io.svn.SVNPlainConnector.open(SVNPlainConnector.java:53)
... 10 more

]] 

(to get this page searchable).

Solution

But now, since this week, Oracle Support presents us.... (drum rolls)  Patch 26775572 for version 12.2.1.2.0, the official fix for this issue, is finally ready and available for download.
I hope to be able to install it next monday. But feel free to try with me: 'Patch 26775572: BAC node using the wrong port when attempts synchronization with Virtual IP '

By the way, for about 2 months we have diagnostic patch running that solves this. But named patch is the official delivery of our diagnostic variant.

By the way, since 12.2.1.3.0 is out in the field for quite some time already, I expect that this did not landed in that version yet. If you're running in the same issue for 12.2.1.3.0 then create a SR to request for a port of this patch.