Apache SVN commit fail

[黃靖翔Bill Huang]

Dear Sir

I have a serious problem when I commit data to SVN server ,
It always in *Committing transaction* have a unknow problem .
When commit run to *Committing transaction *, it always need to wait 10 min
and reply below error message

Committing transaction...
svn: E175012: Commit failed (details follow):
svn: E175012: Connection timed out
svn: E200042: Additional errors:
svn: E175002: Unexpected server error 500 'Internal Server Error' on
'/svn/0_IT_test/Bill/!svn/txn/0-0'

May I know under what circumstances will this error result ?

I have tried many ways to exclude this error , but still has not been
solved this problem

can please help us trubleshooting this problem ?

SVN server is ubuntu 16.04
SVN version is 1.9.3


Thanks
--
[image: unizyx-logo_color_ch_eg]

www.unizyx.com.tw

*Bill Huang* *黃靖翔*

Unizyx Holding Corporation

Information Service Section 1

[phone] <+886%203%20578%203942> ext. 81659

***@unizyx.com.tw <***@unizyx.com.tw>

All The Best,

[Johan Corveleyn]

Post by 黃靖翔Bill Huang
I have a serious problem when I commit data to SVN server ,
It always in Committing transaction have a unknow problem .
When commit run to Committing transaction , it always need to wait 10 min and reply below error message：
Committing transaction...
svn: E175012: Connection timed out
svn: E175002: Unexpected server error 500 'Internal Server Error' on '/svn/0_IT_test/Bill/!svn/txn/0-0'
May I know under what circumstances will this error result ?
I have tried many ways to exclude this error , but still has not been solved this problem
can please help us trubleshooting this problem ?
SVN server is ubuntu 16.04
SVN version is 1.9.3

You'll have to take a look at the error log on the server (or ask a
sysadmin to do that for you). There should be more information about
the reason for the 500 Internal Server Error.

[黃靖翔Bill Huang]

Dear Sir

Attachment is my SVN server log , as described earlier ,
when my user commit data and finally will wait for 10 minutes after the
failure
I check SVN Log not show any detail message about this error.
and during that time keep monitor SVN server performance is normal

The only relevant is committing transaction will wait for 10 min and reply
error.
In access log have show execute MERGE at 18:48:53 and after 10 minutes
(18:58:54) show delete
error log shows Could not MERGE resource at 18:59:47

I don't have any idea for this error message can do something

I try to adjust ./subversion http-timeout and apache2.conf ,
but did not solve it successfully.

May I know have any advice can provide?

Thank you for your help

Post by 黃靖翔Bill Huang
I have a serious problem when I commit data to SVN server ,
It always in Committing transaction have a unknow problem .
When commit run to Committing transaction , it always need to wait 10

min and reply below error message

Post by 黃靖翔Bill Huang
Committing transaction...
svn: E175012: Connection timed out
svn: E175002: Unexpected server error 500 'Internal Server Error' on

'/svn/0_IT_test/Bill/!svn/txn/0-0'

Post by 黃靖翔Bill Huang
May I know under what circumstances will this error result ?
I have tried many ways to exclude this error , but still has not been

solved this problem

Post by 黃靖翔Bill Huang
can please help us trubleshooting this problem ?
SVN server is ubuntu 16.04
SVN version is 1.9.3

You'll have to take a look at the error log on the server (or ask a
sysadmin to do that for you). There should be more information about
the reason for the 500 Internal Server Error.
--
Johan

--
[image: unizyx-logo_color_ch_eg]

www.unizyx.com.tw

*Bill Huang* *黃靖翔*

Unizyx Holding Corporation

Information Service Section 1

[phone] <+886%203%20578%203942> ext. 81659

***@unizyx.com.tw <***@unizyx.com.tw>

All The Best,

[Johan Corveleyn]

[ Hi, please don't top-post on this list, i.e. put your reply at the
bottom or inline. And preferably use plain-text emails. I've
re-arranged your reply into a bottom-post. More below. ]

Post by Johan Corveleyn

Post by 黃靖翔Bill Huang
I have a serious problem when I commit data to SVN server ,
It always in Committing transaction have a unknow problem .
When commit run to Committing transaction , it always need to wait 10 min and reply below error message：
Committing transaction...
svn: E175012: Connection timed out
svn: E175002: Unexpected server error 500 'Internal Server Error' on '/svn/0_IT_test/Bill/!svn/txn/0-0'
May I know under what circumstances will this error result ?
I have tried many ways to exclude this error , but still has not been solved this problem
can please help us trubleshooting this problem ?
SVN server is ubuntu 16.04
SVN version is 1.9.3

You'll have to take a look at the error log on the server (or ask a
sysadmin to do that for you). There should be more information about
the reason for the 500 Internal Server Error.
--
Johan

Dear Sir：
Attachment is my SVN server log , as described earlier ,
when my user commit data and finally will wait for 10 minutes after the failure
I check SVN Log not show any detail message about this error.
and during that time keep monitor SVN server performance is normal
The only relevant is committing transaction will wait for 10 min and reply error.
In access log have show execute MERGE at 18:48:53 and after 10 minutes (18:58:54) show delete
error log shows Could not MERGE resource at 18:59:47
I don't have any idea for this error message can do something
I try to adjust ./subversion http-timeout and apache2.conf ,
but did not solve it successfully.
May I know have any advice can provide?
Thank you for your help

You should focus on the first series of errors in the error log. Which is this:

[Thu Jan 18 18:59:47.799933 2018] Could not MERGE resource
"/svn/0_IT_test/Bill/!svn/txn/4-e" into "/svn/0_IT_test/Bill". [500,
#0]
[Thu Jan 18 18:59:47.799948 2018] An error occurred while committing
the transaction. [500, #160014]
[Thu Jan 18 18:59:47.799950 2018] Reference to non-existent node
'_2j3j.0.t4-e' in filesystem '/var/lib/svn/0_IT_test/Bill/db' [500,
#160014]

I think the essential part of the error message is this:

Reference to non-existent node '_2j3j.0.t4-e' in filesystem
'/var/lib/svn/0_IT_test/Bill/db'

I'm not sure what that means, but I'd check for any corruption of the
repository. I googled that "Reference to non-existent node" message,
and found one potentially relevant post (and some very old ones, all
about repository corruption):

https://svn.haxx.se/users/archive-2006-08/0013.shtml

So I'd try:
* svnadmin verify
* svnadmin dump (sometimes dump find problems that verify misses)
* svnadmin recover (this apparently helped the user in the post I
referred to above)

And check for anything unusual on a storage level, verify the filesystem etc ...

[Johan Corveleyn]

Hi Bill,

Please use "reply all" to keep the mailinglist in cc, so others can
help as well.
And as I said before, we prefer "bottom-posting" on this list, or
inline replying (if at all possible). Anyway, some more info below ...

On Thu, Jan 25, 2018 at 3:10 AM, Bill Huang-黃靖翔

Dear Johan
Thanks for your relpy !
I create a new SVN server and create new repository
But commit file still have a problem
May I know SVN commit file have any limit ?
Such as commit file size total can't exceed 10GB ?
Because when I commit less file (total size 1G) it can normal.
But when commit a lot of file (total size about 50GB) it will fail
Error message as I submitted to you before

For context, I'm pasting the error messages from the previous mail:

[Thu Jan 18 18:59:47.799933 2018] Could not MERGE resource
"/svn/0_IT_test/Bill/!svn/txn/4-e" into "/svn/0_IT_test/Bill". [500,
#0]
[Thu Jan 18 18:59:47.799948 2018] An error occurred while
committing the transaction. [500, #160014]
[Thu Jan 18 18:59:47.799950 2018] Reference to non-existent node
'_2j3j.0.t4-e' in filesystem '/var/lib/svn/0_IT_test/Bill/db' [500,
#160014]

Post by 黃靖翔Bill Huang
SVN server is ubuntu 16.04
SVN version is 1.9.3

Good thing that you reproduced it on a new, clean repository. This new
repository is on a similar environment, with the same SVN server
version?

Since you mentioned that the error only occurs if the total size of
the commit gets large, the first suspect is the back-end filesystem. I
think SVN internally has no (practical) limit on the total size of the
commit. But in the end a commit becomes a single "rev" file on the
back-end. The size of that rev file depends on the incoming files, and
compression and deltification that the server applies.

In the error message above that back-end filesystem is on
'/var/lib/svn/0_IT_test/Bill/db'. If you take a look there, you'll see
directories 'revs', 'transactions', 'txn-protorevs', ... Is there a
maximum file size limit on the level of the filesystem there? Is there
enough diskspace? Is there anything special with the way you set
things up? Are these local filesystems, or networked filesystems
connected via NFS or SMB? Did you perhaps mount 'revs' and
'transactions' from different filesystems? Any special write-caching
options that could be playing a role here? ... Those are all potential
areas for further investigation.

[Bill Huang-黃靖翔]

Dear Sir

Thanks for your reply

1.
Good thing that you reproduced it on a new, clean repository. This new repository is on a similar environment, with the same SVN server version?

Yes , same SVN server version .


2.
Since you mentioned that the error only occurs if the total size of the commit gets large, the first suspect is the back-end filesystem. I think SVN internally has no (practical) limit on the total size of the commit. But in the end a commit becomes a single "rev" file on the back-end. The size of that rev file depends on the incoming files, and compression and deltification that the server applies.

In the error message above that back-end filesystem is on '/var/lib/svn/0_IT_test/Bill/db'. If you take a look there, you'll see directories 'revs', 'transactions', 'txn-protorevs', ... Is there a maximum file size limit on the level of the filesystem there? Is there enough diskspace? Is there anything special with the way you set things up? Are these local filesystems, or networked filesystems connected via NFS or SMB? Did you perhaps mount 'revs' and 'transactions' from different filesystems? Any special write-caching options that could be playing a role here? ... Those are all potential areas for further investigation.


May I know how do I setting it ? which file can I adjust ?
I think my SVN server is default setting, because I do not have adjust any setting.
And my server is physical machine Dell-R230 and free space is 1.4T , so I think diskspace is normal

Attachments for my settings and server status
And I just setting attachments file 02 and 03
May I know have any more setting about SVN how can I try to setting it



Thanks





-----Original Message-----
From: Johan Corveleyn [mailto:***@gmail.com]
Sent: Thursday, January 25, 2018 5:23 PM
To: Bill Huang-黃靖翔 <***@unizyx.com.tw>
Cc: 黃靖翔 <***@yahoo.com.tw>; Subversion <***@subversion.apache.org>
Subject: Re: Apache SVN commit fail

Hi Bill,

Please use "reply all" to keep the mailinglist in cc, so others can help as well.
And as I said before, we prefer "bottom-posting" on this list, or inline replying (if at all possible). Anyway, some more info below ...

On Thu, Jan 25, 2018 at 3:10 AM, Bill Huang-黃靖翔

Dear Johan
Thanks for your relpy !
I create a new SVN server and create new repository But commit file
still have a problem
May I know SVN commit file have any limit ?
Such as commit file size total can't exceed 10GB ?
Because when I commit less file (total size 1G) it can normal.
But when commit a lot of file (total size about 50GB) it will fail
Error message as I submitted to you before

For context, I'm pasting the error messages from the previous mail:

[Thu Jan 18 18:59:47.799933 2018] Could not MERGE resource "/svn/0_IT_test/Bill/!svn/txn/4-e" into "/svn/0_IT_test/Bill". [500, #0]
[Thu Jan 18 18:59:47.799948 2018] An error occurred while committing the transaction. [500, #160014]
[Thu Jan 18 18:59:47.799950 2018] Reference to non-existent node '_2j3j.0.t4-e' in filesystem '/var/lib/svn/0_IT_test/Bill/db' [500, #160014]

Post by 黃靖翔Bill Huang
SVN server is ubuntu 16.04
SVN version is 1.9.3

Good thing that you reproduced it on a new, clean repository. This new repository is on a similar environment, with the same SVN server version?

Since you mentioned that the error only occurs if the total size of the commit gets large, the first suspect is the back-end filesystem. I think SVN internally has no (practical) limit on the total size of the commit. But in the end a commit becomes a single "rev" file on the back-end. The size of that rev file depends on the incoming files, and compression and deltification that the server applies.

In the error message above that back-end filesystem is on '/var/lib/svn/0_IT_test/Bill/db'. If you take a look there, you'll see directories 'revs', 'transactions', 'txn-protorevs', ... Is there a maximum file size limit on the level of the filesystem there? Is there enough diskspace? Is there anything special with the way you set things up? Are these local filesystems, or networked filesystems connected via NFS or SMB? Did you perhaps mount 'revs' and 'transactions' from different filesystems? Any special write-caching options that could be playing a role here? ... Those are all potential areas for further investigation.

[Johan Corveleyn]

On Thu, Jan 25, 2018 at 11:14 AM, Bill Huang-黃靖翔

Post by Bill Huang-黃靖翔

Post by Bill Huang-黃靖翔
Since you mentioned that the error only occurs if the total size of the commit gets large, the first suspect is the back-end filesystem. I think SVN internally has no (practical) limit on the total size of the commit. But in the end a commit becomes a single "rev" file on the back-end. The size of that rev file depends on the incoming files, and compression and deltification that the server applies.
In the error message above that back-end filesystem is on '/var/lib/svn/0_IT_test/Bill/db'. If you take a look there, you'll see directories 'revs', 'transactions', 'txn-protorevs', ... Is there a maximum file size limit on the level of the filesystem there? Is there enough diskspace? Is there anything special with the way you set things up? Are these local filesystems, or networked filesystems connected via NFS or SMB? Did you perhaps mount 'revs' and 'transactions' from different filesystems? Any special write-caching options that could be playing a role here? ... Those are all potential areas for further investigation.

May I know how do I setting it ? which file can I adjust ?
I think my SVN server is default setting, because I do not have adjust any setting.
And my server is physical machine Dell-R230 and free space is 1.4T , so I think diskspace is normal
Attachments for my settings and server status
And I just setting attachments file 02 and 03
May I know have any more setting about SVN how can I try to setting it

I'm not talking about a setting in SVN. I'm talking about the
filesystem on which the svn backend (/var/lib/svn/0_IT_test/Bill/db)
is stored. That's /dev/sda1 if I read your screenshots correctly. Is
that filesystem capable of storing such large files?

Okay, so it has enough diskspace, and it's a local disk. What is the
type of the filesystem? I think 'mount' should show you that. Maybe
you can perform a couple of tests outside of SVN, to just test the
disk? For instance, try to create a file of 10 GB (with 'dd' or
something), then 20 GB, 50 GB (don't forget to clean them up again
:-)).

I'm also wondering whether "write-caching" could play a role here.
Perhaps you can take a look and see if disabling it fixes the problem.
See for example these articles:
https://linuxconfig.org/improve-hard-drive-write-speed-with-write-back-caching
http://ubuntuhandbook.org/index.php/2014/01/disable-disk-caching-prevent-data-loss/
(I'm just guessing here -- not sure if it's related and if it's the
way to go -- but maybe you can eliminate this by trying it out)

That said, I'm not very proficient in Ubuntu or Linux in general.
Maybe someone else on this list can help more.

P.S.: one of your screenshots contained a visible password. I strongly
suggest you change that, since this is a public and archived
mailinglist.