SVN Commit Failed For Data larger Than 2 GB [How To Resolve]

[Mohsin Abbas]

Good Day Team,

I am using Subversion 1.8.9 server on linux OS and tortoise SVN client
at windows. When I try to commit data larger then 2+ GB my commit
failed. I tried to google on different websites they provided below
solution :

1 : Set LimitRequestBody to 0 in the server side httpd.conf file.

But Apache Web server ( Apache 2.2 ) allows 2 GB only max limit for
data in HTTP request. Now tell me how can I commit files which having
size more than 2 GB ? How can I increase data limit in apache
webserver more than 2 GB ? OR I assume we can not commit data more
than 2 GB because svn not allows us for this ?
Please shed some light on this issue this is urgent.


Regards
Mohsin Abbas

[Nico Kadel-Garcia]

Post by Mohsin Abbas
Good Day Team,
I am using Subversion 1.8.9 server on linux OS and tortoise SVN client
at windows. When I try to commit data larger then 2+ GB my commit
failed. I tried to google on different websites they provided below
1 : Set LimitRequestBody to 0 in the server side httpd.conf file.
But Apache Web server ( Apache 2.2 ) allows 2 GB only max limit for
data in HTTP request. Now tell me how can I commit files which having
size more than 2 GB ? How can I increase data limit in apache
webserver more than 2 GB ? OR I assume we can not commit data more
than 2 GB because svn not allows us for this ?
Please shed some light on this issue this is urgent.

For pete's sake, it's like asking how to put a pool ball in your
mouth. Just Don't Do That.......

If you have to use files that large, switch to svn+ssh:// or svn://.

[Philip Martin]

Post by Mohsin Abbas
I am using Subversion 1.8.9 server on linux OS and tortoise SVN client
at windows. When I try to commit data larger then 2+ GB my commit
failed. I tried to google on different websites they provided below
1 : Set LimitRequestBody to 0 in the server side httpd.conf file.
But Apache Web server ( Apache 2.2 ) allows 2 GB only max limit for
data in HTTP request. Now tell me how can I commit files which having
size more than 2 GB ? How can I increase data limit in apache
webserver more than 2 GB ? OR I assume we can not commit data more
than 2 GB because svn not allows us for this ?
Please shed some light on this issue this is urgent.

Subversion supports commits greater than 2GB over HTTP. There are many
things that could cause it to fail so you need to provide more
information.

Which HTTP request is failing?

The commit will use temporary disk space, do you have enough on the
client and the server?

What error message does the client give?

Look at the server logs: what error message does the server give?

If you are using a proxy then look at the the proxy logs: what error
does the proxy give?

[Mohsin Abbas]

Thanks Philip Martin for your reply.

I'll share error.log portion sortly meanwhile I have question I am
using Apache 2.2.24 (DAV module via http) with svn V1.8.9 and apache
web server have limit of request body 2 GB then how can you say more
than 2 GB data can be commited in svn ? This thing is confusing for me
please explain.

Regards
Mohsin

Post by Philip Martin

Post by Mohsin Abbas
I am using Subversion 1.8.9 server on linux OS and tortoise SVN client
at windows. When I try to commit data larger then 2+ GB my commit
failed. I tried to google on different websites they provided below
1 : Set LimitRequestBody to 0 in the server side httpd.conf file.
But Apache Web server ( Apache 2.2 ) allows 2 GB only max limit for
data in HTTP request. Now tell me how can I commit files which having
size more than 2 GB ? How can I increase data limit in apache
webserver more than 2 GB ? OR I assume we can not commit data more
than 2 GB because svn not allows us for this ?
Please shed some light on this issue this is urgent.

Subversion supports commits greater than 2GB over HTTP. There are many
things that could cause it to fail so you need to provide more
information.
Which HTTP request is failing?
The commit will use temporary disk space, do you have enough on the
client and the server?
What error message does the client give?
Look at the server logs: what error message does the server give?
If you are using a proxy then look at the the proxy logs: what error
does the proxy give?
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*

[Mohsin Abbas]

HI Philip Martin,

Here is Tortoise SVN client and Apache error log portion :

*Tortoise SVN Client Error :*

Error: Commit failed (details follow):
Error: PUT of
Error:
'/svn/test/!svn/wrk/6f036c59-c3a8-5947-91dd-bd0f1eeccd84/SomeFolder/brands.zip':

Error: Could not send request body: An existing connection was forcibly
closed by the
Error: remote host.
Error: (http://x.x.com)

*Apache error.log portion :*

[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] mod_dav_svn
close_stream: error closing write stream [500, #185004]
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] Unexpected end of
svndiff input [500, #185004]

Please let me know how can i figure out this issue ?

FYI
My folder size is 4.73 GB which i am committing .


Regards
Mohsin

Post by Philip Martin

Post by Mohsin Abbas
I am using Subversion 1.8.9 server on linux OS and tortoise SVN client
at windows. When I try to commit data larger then 2+ GB my commit
failed. I tried to google on different websites they provided below
1 : Set LimitRequestBody to 0 in the server side httpd.conf file.
But Apache Web server ( Apache 2.2 ) allows 2 GB only max limit for
data in HTTP request. Now tell me how can I commit files which having
size more than 2 GB ? How can I increase data limit in apache
webserver more than 2 GB ? OR I assume we can not commit data more
than 2 GB because svn not allows us for this ?
Please shed some light on this issue this is urgent.

Subversion supports commits greater than 2GB over HTTP. There are many
things that could cause it to fail so you need to provide more
information.
Which HTTP request is failing?
The commit will use temporary disk space, do you have enough on the
client and the server?
What error message does the client give?
Look at the server logs: what error message does the server give?
If you are using a proxy then look at the the proxy logs: what error
does the proxy give?
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*

[Branko Čibej]

Post by Mohsin Abbas
HI Philip Martin,
*Tortoise SVN Client Error :*
Error: PUT of
Error: Could not send request body: An existing connection was
forcibly closed by the
Error: remote host.
Error: (http://x.x.com)
*Apache error.log portion :*
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] mod_dav_svn
close_stream: error closing write stream [500, #185004]
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] Unexpected end of
svndiff input [500, #185004]
From the above, I'm almost certain that this is not Subversion's fault.

Look at the error messages: the client thinks that the server closed the
connection, but the server thinks that the client closed the connection;
so the only reasonable explanation is that something *between* the
client and server is in fact closing the connection.

This could be a proxy, or a load balancer, or even a packet
inspector/firewall (see http://markmail.org/message/tazxq3jexaopcdum for
example) that's forcibly closing the connection.

You'll have to talk to the server/site administrators to determine
what's happening. There's not much anyone else can do without knowing
how not just the server, but the whole data-centre are configured.

-- Brane

[Mohsin Abbas]

Thanks Branko Cibej for your feedback.

I have checked we are not using any proxy or load balancer for our svn but
firewall in involved in our scenario. In case of firewall there is not
limit for data transfer for specific your over the network.You said there
is in between client and server which is Apache may be due to Apache
request time out this is happening ? I have searched on internet regarding
Apache request body parameter whose max limit is 2 GB means Apache can
carry max 2 GB request please confirm me if I am wrong may be due to this
limit svn commit is failing before 2 GB ? Should i consider Apache request
timeout parameter in httpd.conf ? Please suggest


Regards
Mohsin

Post by Mohsin Abbas
HI Philip Martin,
*Tortoise SVN Client Error :*
Error: PUT of
Error: Could not send request body: An existing connection was forcibly
closed by the
Error: remote host.
Error: (http://x.x.com)
*Apache error.log portion :*
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] mod_dav_svn
close_stream: error closing write stream [500, #185004]
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] Unexpected end of
svndiff input [500, #185004]
From the above, I'm almost certain that this is not Subversion's fault.
Look at the error messages: the client thinks that the server closed the
connection, but the server thinks that the client closed the connection; so
the only reasonable explanation is that something *between* the client
and server is in fact closing the connection.
This could be a proxy, or a load balancer, or even a packet
inspector/firewall (see http://markmail.org/message/tazxq3jexaopcdum for
example) that's forcibly closing the connection.
You'll have to talk to the server/site administrators to determine what's
happening. There's not much anyone else can do without knowing how not just
the server, but the whole data-centre are configured.
-- Brane

[Philip Martin]

Post by Mohsin Abbas

Post by Mohsin Abbas
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] mod_dav_svn
close_stream: error closing write stream [500, #185004]
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] Unexpected end of
svndiff input [500, #185004]

From the above, I'm almost certain that this is not Subversion's fault.
Look at the error messages: the client thinks that the server closed the
connection, but the server thinks that the client closed the connection;
so the only reasonable explanation is that something *between* the
client and server is in fact closing the connection.

Error 185004 is SVN_ERR_SVNDIFF_UNEXPECTED_END and that could be caused
by some proxy/router closing the connection. But it could also be
caused by a Subversion bug that causes the client to write invalid
svndiff data. It's not trivial to distinguish one from the other, we
would need access to all the commit data to decode the svndiff. Another
way would be to attempt the commit from the server itself over HTTP to
localhost, if this works the problem is most likely the network.

[Mohsin Abbas]

HI,

I have committed folder with size 4.7 GB successfully in SVN repository by
using svn CLI (svn commit command). But when I try to commit folder by
using Tortoise SVN client my commit was failed. It means we have limitation
in tortoise svn client for commit. Can any one confirm this ? There is no
issue in svn at all but may be in svn client.

Regards
Mohsin

Post by Philip Martin

Post by Mohsin Abbas

Post by Mohsin Abbas
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] mod_dav_svn
close_stream: error closing write stream [500, #185004]
[Sat Sep 27 11:21:44 2014] [error] [client x.x.x.x] Unexpected end of
svndiff input [500, #185004]

From the above, I'm almost certain that this is not Subversion's fault.
Look at the error messages: the client thinks that the server closed the
connection, but the server thinks that the client closed the connection;
so the only reasonable explanation is that something *between* the
client and server is in fact closing the connection.

Error 185004 is SVN_ERR_SVNDIFF_UNEXPECTED_END and that could be caused
by some proxy/router closing the connection. But it could also be
caused by a Subversion bug that causes the client to write invalid
svndiff data. It's not trivial to distinguish one from the other, we
would need access to all the commit data to decode the svndiff. Another
way would be to attempt the commit from the server itself over HTTP to
localhost, if this works the problem is most likely the network.
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*

[Branko Čibej]

Post by Mohsin Abbas
HI,
I have committed folder with size 4.7 GB successfully in SVN
repository by using svn CLI (svn commit command). But when I try to
commit folder by using Tortoise SVN client my commit was failed. It
means we have limitation in tortoise svn client for commit. Can any
one confirm this ? There is no issue in svn at all but may be in svn
client.

The last sencence is both confusing and important: "the svn client" is
the command-line tool you get when installing a stock Subversion
package, and that's what this project supports. TortoiseSVN is *not*
supported by us, it's a separate project; I don't think more than a
couple Subversion committers ever saw the TSVN source code.

So it turns out we've been going back and forth about this issue for 4
days, because we did not know a very important piece of information: you
did not use Subversion to perform the commit, you used TortoiseSVN ...

Given that we do know now that the problem is most likely in TSVN, I
suggest you report it there; I suggest you start by reading

http://tortoisesvn.net/reportbug.html

which describes the steps you need to take to create a bug report that's
most likely to help resolve the issue.

-- Brane

[Mohsin Abbas]

Brane,

Just for your information we have SVN installed on Linux machine and I
committed folder of size 4.7 GB on Linux server (I am administrator of
SVN). I have checked out that particular repository on Linux server and
committed that folder which was successful. But all our users have windows
machine so they use Tortoise SVN (client) for svn check out or svn commit
or svn update. I am using windows machine and Tortoise SVN too for svn
operations like commit , update n etc. Point is I can do commit on Linux
server as a administrator. But for windows users how they commit on their
machines with Tortoise SVN client which is not possible for normal user to
perform heavy data commit so we should consider some other tool which
provide this facility . Can we have some others svn clients for windows
which support your SVN?

Surely, I will report bug for this issue.


Regards
Mohsin

Post by Mohsin Abbas
HI,
I have committed folder with size 4.7 GB successfully in SVN repository by
using svn CLI (svn commit command). But when I try to commit folder by
using Tortoise SVN client my commit was failed. It means we have limitation
in tortoise svn client for commit. Can any one confirm this ? There is no
issue in svn at all but may be in svn client.
The last sencence is both confusing and important: "the svn client" is the
command-line tool you get when installing a stock Subversion package, and
that's what this project supports. TortoiseSVN is *not* supported by us,
it's a separate project; I don't think more than a couple Subversion
committers ever saw the TSVN source code.
So it turns out we've been going back and forth about this issue for 4
days, because we did not know a very important piece of information: you
did not use Subversion to perform the commit, you used TortoiseSVN ...
Given that we do know now that the problem is most likely in TSVN, I
suggest you report it there; I suggest you start by reading
http://tortoisesvn.net/reportbug.html
which describes the steps you need to take to create a bug report that's
most likely to help resolve the issue.
-- Brane

[Andreas Stieger]

Hi,

Post by Mohsin Abbas
Just for your information we have SVN installed on Linux machine and I
committed folder of size 4.7 GB on Linux server (I am administrator of
SVN). I have checked out that particular repository on Linux server and
committed that folder which was successful.

Which URL scheme were you using locally on GNU/Linux? If it was file I
will once again point to the serf issues below and httpd/network config.
https://code.google.com/p/serf/issues/detail?id=152
https://code.google.com/p/serf/source/detail?r=2419

Andreas

[Mohsin]

Hi,

For accessing repo in browser we are using HTTP protocol while on linux
server i think (not sure) we are using file scheme . Here is import command
which we had used on linux server for importing data in newly created
repository.

FYI

svn import -m "First Import" /tmp/Test/ file:///X/Y/
Adding /tmp/Test/Trunk
Adding /tmp/Test/Branches
Adding /tmp/Test/Tags

Is this file scheme which we are using ? Please suggest


Regards
Mohsin



--
View this message in context: http://subversion.1072662.n5.nabble.com/SVN-Commit-Failed-For-Data-larger-Than-2-GB-How-To-Resolve-tp190417p190483.html
Sent from the Subversion Users mailing list archive at Nabble.com.

[Andreas Stieger]

Post by Mohsin
For accessing repo in browser we are using HTTP protocol while on linux
server i think (not sure) we are using file scheme .

If you are not sure then go check. I gave interpretations on what the difference means. If if fails on http but works on file, you will know where to look.

Check your httpd for timeout related settings and bugs, as previously indicated by others.
Note that the 2G serf bug fix linked to twice is not available in a public TortioseSVN release.

Andreas

[Mohsin]

Hi,

How can i check which scheme we are using on linux server ? can you share
command so that i can tell you ?


Mohsin



--
View this message in context: http://subversion.1072662.n5.nabble.com/SVN-Commit-Failed-For-Data-larger-Than-2-GB-How-To-Resolve-tp190417p190486.html
Sent from the Subversion Users mailing list archive at Nabble.com.

[Markus Schaber]

Hi, Mohsin,

How can i check which scheme we are using on linux server ? can you share command so that i can tell you ?

Check the start of the URLs you're using to access the repository:

http://server/some/path -> http access, apache configuration may be the problem.
https://server/some/path -> http access with ssl protection, apache configuration may be the problem.

svn://server/some/path... -> svn protocol, not affected.
file:///some/path/... -> direct file access, not affected.
svn+ssh://server/some/path -> svn protocol tunneled through ssh, not affected.

svn+XXX://server/some/path -> svn protocol tunneled through some custom proxy, unlikely to be affected unless your tunnel configuration imposes a size limit on the total amount of tunneled data.


Best regards

Markus Schaber

CODESYS(r) a trademark of 3S-Smart Software Solutions GmbH

Inspiring Automation Solutions

3S-Smart Software Solutions GmbH
Dipl.-Inf. Markus Schaber | Product Development Core Technology
Memminger Str. 151 | 87439 Kempten | Germany
Tel. [phone] | Fax [phone]

E-Mail: ***@codesys.com | Web: http://www.codesys.com | CODESYS store: http://store.codesys.com
CODESYS forum: http://forum.codesys.com

Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received
this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure
or distribution of the material in this e-mail is strictly forbidden.

[Mohsin]

Hi,

we are using HTTP protocol for repository access in browser e.g

http://server/some/path


Mohsin



--
View this message in context: http://subversion.1072662.n5.nabble.com/SVN-Commit-Failed-For-Data-larger-Than-2-GB-How-To-Resolve-tp190417p190491.html
Sent from the Subversion Users mailing list archive at Nabble.com.

[Les Mikesell]

Post by Mohsin
Hi,
we are using HTTP protocol for repository access in browser e.g
http://server/some/path

I'm not sure it is clear from this thread whether you have succeeded
in committing >2Gb with a command line svn client using http protocol
(your import example showed file://). If the command line client
still shows the problem over http, then the issue may be with apache
on the server side. However if the command line svn works with an
http url but tortoise fails, then the issue is obviously with the
tortoise libraries on the client side.

[Mohsin]

Dear,


That's what i am saying from last 2 days I was successful in committing data
by using svn command line on Linux server but I faced issue with tortoise
svn client on my window machine . That clearly depicts issue is with
tortoise svn i know that but most important thing which i am eagerly to
listen from you people how to resolve this issue ? What should I do to
figure out this issue with tortoise svn client ?


~Mohsin



--
View this message in context: http://subversion.1072662.n5.nabble.com/SVN-Commit-Failed-For-Data-larger-Than-2-GB-How-To-Resolve-tp190417p190498.html
Sent from the Subversion Users mailing list archive at Nabble.com.

[Les Mikesell]

Post by Mohsin
Dear,
That's what i am saying from last 2 days I was successful in committing data
by using svn command line on Linux server but I faced issue with tortoise
svn client on my window machine .

Thanks - it would have been more clear if you had shown that svn
command line instead of the one that used the file:// protocol.

Post by Mohsin
That clearly depicts issue is with
tortoise svn i know that but most important thing which i am eagerly to
listen from you people how to resolve this issue ? What should I do to
figure out this issue with tortoise svn client ?

First note that the svn command line client is available for windows
too, and it might be worth verifying that it can succeed in exactly
the same circumstances where tortoise fails. As others have noted,
this mail list doesn't have much to do with the tortoise client, so
you probably won't get the best advice about this problem here. But,
make sure you are using the latest tortoise - if the issue is really
in the neon libraries, it looks like neon has been dropped in the 1.8
release:

http://subversion.apache.org/docs/release-notes/1.8.html#neon-deleted

[Mohsin]

Thanks Dear,

Post by Les Mikesell
Thanks - it would have been more clear if you had shown that svn
command line instead of the one that used the file:// protocol.

Ignore my file:// protocol post that was in some other context . I am using
HTTP protocol for repository access.

Post by Les Mikesell
First note that the svn command line client is available for windows
too, and it might be worth verifying that it can succeed in exactly
the same circumstances where tortoise fails. As others have noted,
this mail list doesn't have much to do with the tortoise client, so
you probably won't get the best advice about this problem here. But,
make sure you are using the latest tortoise - if the issue is really
in the neon libraries, it looks like neon has been dropped in the 1.8

release:

How can I use svn command line for windows ? Can you tell me in this regard
?

I am using Tortoise svn client older version (1.6 or 1.7) may be due to this
older version Tortoise svn client was not able to commit larger data. I'll
upgrade my Tortoise svn client version to latest and try to commit data may
be with latest version I will be able to commit data on windows machine too.
I'll update you regarding this shortly.


Mohsin



--
View this message in context: http://subversion.1072662.n5.nabble.com/SVN-Commit-Failed-For-Data-larger-Than-2-GB-How-To-Resolve-tp190417p190504.html
Sent from the Subversion Users mailing list archive at Nabble.com.

[Les Mikesell]

Post by Mohsin
Thanks Dear,

Post by Les Mikesell
Thanks - it would have been more clear if you had shown that svn
command line instead of the one that used the file:// protocol.

Ignore my file:// protocol post that was in some other context . I am using
HTTP protocol for repository access.

Post by Les Mikesell
First note that the svn command line client is available for windows
too, and it might be worth verifying that it can succeed in exactly
the same circumstances where tortoise fails. As others have noted,
this mail list doesn't have much to do with the tortoise client, so
you probably won't get the best advice about this problem here. But,
make sure you are using the latest tortoise - if the issue is really
in the neon libraries, it looks like neon has been dropped in the 1.8

How can I use svn command line for windows ? Can you tell me in this regard
?

The tortoise installer should offer to include a command line client
too, but it will probably be built with the same library as the GUI.
There are several other builds linked from:
https://subversion.apache.org/packages.html#windows
plus one from the Cygwin environment (which, being a more linux-like
environment may confuse your line endings if you aren't careful).

Post by Mohsin
I am using Tortoise svn client older version (1.6 or 1.7) may be due to this
older version Tortoise svn client was not able to commit larger data. I'll
upgrade my Tortoise svn client version to latest and try to commit data may
be with latest version I will be able to commit data on windows machine too.
I'll update you regarding this shortly.

As a general recommendation: I usually try to update free software
before fighting bugs that might already be fixed.

[Mohsin]

Hi ,

After upgrading my Tortoise SVN client version (1.8.7) on my windows machine
I was able to commit folder of size 4.7 GB : ) There is no issue in tortoise
svn client nor in SVN server. Thanks all of you for your support .


Regards
Mohsin Abbas



--
View this message in context: http://subversion.1072662.n5.nabble.com/SVN-Commit-Failed-For-Data-larger-Than-2-GB-How-To-Resolve-tp190417p190510.html
Sent from the Subversion Users mailing list archive at Nabble.com.