svnsync fails to replicate large revision

[Colin Foster]

Hello,

I'm not sure if this is a "bug", a "configuration issue", or simply a network issue. I'm hoping it is just a configuration issue.

I'm creating an off-site replica of our SVN server. Our current server is running the latest CollabNet SVNEdge server. Subversion version 1.8.19-4419.51

There was a large commit that fails after an indeterminate amount of time - sometimes 45 minutes, sometime 4.5 hours. I've been tinkering with Timout and KeepAlive settings to no avail.

The error I continue to see when the failure happens is:
[Fri Jul 13 11:22:33.773429 2018] [dav:error] [pid 1906] [client XX:38654] Provider encountered an error while streaming a REPORT response. [500, #0]
[Fri Jul 13 11:22:33.773445 2018] [dav:error] [pid 1906] [client XX:38654] Problem replaying revision [500, #104]
[Fri Jul 13 11:22:33.773448 2018] [dav:error] [pid 1906] [client XX:38654] Error writing base64 data: Connection reset by peer [500, #104]


The initial requests are seen here:


[13/Jul/2018:09:45:46 -0700] gregc IA rev-proplist r18116 9

With the failure showing up as such (note the timestamp of the two messages):

[13/Jul/2018:11:20:01 -0700] jenkins IA get-dir / /trunk r44281 props 0

[13/Jul/2018:09:45:54 -0700] gregc IA replay / r18116 5799


Relevant settings:
Bulk updates allowed. I've tried preferring but that didn't seem to fix the issue
Timeout: I've set it between 12,000 and 120,000. Issue not fixed
KeepAlive: Yes (always)
MaxKeelAliveRequests: I've set it between 10,000 and 100,000
KeepAliveTimeout: I've set it between 15 and 900
Compression set to 5
Server cache: 64 MB, enabled for Full Texts, Deltas, and RevProps

I have had to remove "mod_deflate" from loading - I believe this was due to crashes when transferring files > 4GB.


The server seems to be syncing at about 500KBps (4Mbps). The file "db/revs/18/18116" is 441 MB on disk: FSFS V6, Repo format 5, Not packed.

This is supposedly the message the client is exiting with (garbled with the output of "time"?):
57:43.26
.svnsync: E175002: REPORT request on '/svn/IA/!svn/rev/18116' failed
Command exited with non-zero status 1
1158.67user 425.49system 57:43.26elapsed 45%CPU (0avgtext+0avgdata 35588maxresident)k
216336inputs+1877472outputs (19major+808715minor)pagefaults 0swaps



That is all the relevant information I can think of in this instance. Seemingly the client is terminating the "svnsync synchronize" request. Or is this something in the server settings that is causing this issue? Or could this be something else?


Thank you very much!

[143421_logo_final_25]
Colin Foster
Software Engineer
Tel [phone]
***@in-advantage.com<mailto:***@in-advantage.com>
www.in-advantage.com

[Bo Berglund]

On Fri, 13 Jul 2018 20:01:13 +0000, Colin Foster

Post by Colin Foster
I'm not sure if this is a "bug", a "configuration issue", or simply a network issue.

FWIW:
Back in March I set up a svnsync replication of our main SVN
repository (which is on Windows Server 16 with SVN 1.9.7) across the
Internet.

I also got hit by a failure to initialize the synced repository
properly, it stopped in the middle at a revision that contained a
*lot* of stuff. It seemed to be caused by the slow network speed and a
timeout in the svnsync transfer.

So I switched approach and created a dump of the source repository,
then transferred the dump file (rather big at several GB even in
compressed form) over the Internet to the target server, which is
running Ubuntu Server 16.04-3 LTS) and SVN 1.9.3.

After loading the dump file into the target I could start svnsync with
command switches that recognized the target server as already
populated from the dump so no data for existing revisions needed to be
transfered.

I don't remember the exact commands I used but searching the svn users
list for my discussion of this dated 2018-03-24 to 29 with subject:
"Subversion 1.9.7 server on Windows, advice on setting up svnsync?"
might help you too. I now have a working backup system using svnsync.