Not so cheap moves

[Ignacio González (Eliop)]

Client svn 1.8.13
Server svn 1.8.13 on Centos 7 64 bit
-----
I have a repository with 400 000 revisions.
Proyect X in that repository has the usual trunk / branches / tags
structure.
Tags are structured in a dozen of sub-directories:
Project X
+- trunk
+- tags
+-- setA
+-- setB
+-- setC

The number of tags under 'setB' is over 3000. All of them are copies of the
trunk at several points in the past.
I want to rename 'setB' to 'setF'.
Easy, I thought: svn mv ....setB .....setF.
No way: after 30 minutes, the server gives up.
No problem, I thought: instead of 1 big rename, do 3000 small moves.
So I created tags/setF and wrote a client-side script with 3000 svn mv
.....setB/label<n> .....setF/label<n>
The script is running fine, but I do not know why it takes around 30
seconds to do each move (3000 x 30 s > 20 hours). Is not each move just one
cheap copy plus a delete? Each new revision occupies about 200 000 bytes
(fsfs); revisions are packed every 1000 revisions.
No real problem, just wondering what is going behind the stage.
Thanks.

[Branko Čibej]

Post by Ignacio González (Eliop)
Client svn 1.8.13
Server svn 1.8.13 on Centos 7 64 bit
-----
I have a repository with 400 000 revisions.
Proyect X in that repository has the usual trunk / branches / tags
structure.
Project X
 +- trunk
 +- tags
    +-- setA
    +-- setB
    +-- setC
The number of tags under 'setB' is over 3000. All of them are copies
of the trunk at several points in the past.
I want to rename 'setB' to 'setF'.
Easy, I thought: svn mv ....setB .....setF.

Did you use server-side move or client-side move + commit? Please show
us the *exact* command you used, not some summary that you think is
sufficient, but really is not.

-- Brane

[Ignacio González (Eliop)]

Post by Branko Čibej

Post by Ignacio González (Eliop)
Client svn 1.8.13
Server svn 1.8.13 on Centos 7 64 bit
-----
I have a repository with 400 000 revisions.
Proyect X in that repository has the usual trunk / branches / tags
structure.
Project X
+- trunk
+- tags
+-- setA
+-- setB
+-- setC
The number of tags under 'setB' is over 3000. All of them are copies
of the trunk at several points in the past.
I want to rename 'setB' to 'setF'.
Easy, I thought: svn mv ....setB .....setF.

Did you use server-side move or client-side move + commit? Please show
us the *exact* command you used, not some summary that you think is
sufficient, but really is not.

It was a server-side move launched from my Windows 10 PC client.
The original command, with some names changed, was:

svn mv http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setB/ http://
myserver.mydomain.com/svn/myrepo/ProjectX/tags/setF/ -m"Renombrando
etiquetas setB a setF."

This command bailed out after 30 minutes aprox.

The commands included in the script that is (still) running are like this
one:

svn mv http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setB/
MY_FIRST_TAG/ http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setF/
MY_FIRTS_TAG/ -m"Moviendo etiquetas de setB a setF."

These commands take from 15 to 60 s and create 200 000 bytes files in the
fsfs repo.

[Johan Corveleyn]

On Tue, Feb 20, 2018 at 10:35 AM, Ignacio González (Eliop)

Post by Ignacio González (Eliop)

Post by Branko Čibej

Post by Ignacio González (Eliop)
Client svn 1.8.13
Server svn 1.8.13 on Centos 7 64 bit
-----
I have a repository with 400 000 revisions.
Proyect X in that repository has the usual trunk / branches / tags
structure.
Project X
+- trunk
+- tags
+-- setA
+-- setB
+-- setC
The number of tags under 'setB' is over 3000. All of them are copies
of the trunk at several points in the past.
I want to rename 'setB' to 'setF'.
Easy, I thought: svn mv ....setB .....setF.

Did you use server-side move or client-side move + commit? Please show
us the *exact* command you used, not some summary that you think is
sufficient, but really is not.

It was a server-side move launched from my Windows 10 PC client.
svn mv http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setB/
http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setF/ -m"Renombrando
etiquetas setB a setF."
This command bailed out after 30 minutes aprox.
The commands included in the script that is (still) running are like this
svn mv
http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setB/MY_FIRST_TAG/
http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setF/MY_FIRTS_TAG/
-m"Moviendo etiquetas de setB a setF."
These commands take from 15 to 60 s and create 200 000 bytes files in the
fsfs repo.

I think there are two things:

- Slowness of the server-side move: I think that's because of this
issue: https://issues.apache.org/jira/browse/SVN-4531 (server-side
copy (over dav) is slow and uses too much memory). This was fixed in
1.8.14 (server-side), and you're using 1.8.13 unfortunately. There is
also a workaround documented in the issue, by using certain Apache
directives.

- Large revision files when editing large directories: that's because
in your version of the server / FSFS backend there is not yet any
"deltification" of directories, I believe (or not by default anyway,
because a 1.8 server cannot handle that performantly). That means that
directory listings are stored entirely in full for every revision that
changes them (and not merely the diff is stored). As of 1.9 the
default for new repositories is to enable directory deltification. See
http://subversion.apache.org/docs/release-notes/1.9.html#fsfs-defaults.

[Ignacio González (Eliop)]

On Tue, Feb 20, 2018 at 10:35 AM, Ignacio González (Eliop)

Post by Ignacio González (Eliop)

Post by Branko Čibej

Post by Ignacio González (Eliop)
Client svn 1.8.13
Server svn 1.8.13 on Centos 7 64 bit
-----
I have a repository with 400 000 revisions.
Proyect X in that repository has the usual trunk / branches / tags
structure.
Project X
+- trunk
+- tags
+-- setA
+-- setB
+-- setC
The number of tags under 'setB' is over 3000. All of them are copies
of the trunk at several points in the past.
I want to rename 'setB' to 'setF'.
Easy, I thought: svn mv ....setB .....setF.

Did you use server-side move or client-side move + commit? Please show
us the *exact* command you used, not some summary that you think is
sufficient, but really is not.

It was a server-side move launched from my Windows 10 PC client.
svn mv http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setB/
http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setF/

-m"Renombrando

Post by Ignacio González (Eliop)
etiquetas setB a setF."
This command bailed out after 30 minutes aprox.
The commands included in the script that is (still) running are like

this

Post by Ignacio González (Eliop)
svn mv
http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setB/MY_FIRST_TAG/
http://myserver.mydomain.com/svn/myrepo/ProjectX/tags/setF/MY_FIRTS_TAG/
-m"Moviendo etiquetas de setB a setF."
These commands take from 15 to 60 s and create 200 000 bytes files in

the

Post by Ignacio González (Eliop)
fsfs repo.

- Slowness of the server-side move: I think that's because of this
issue: https://issues.apache.org/jira/browse/SVN-4531 (server-side
copy (over dav) is slow and uses too much memory). This was fixed in
1.8.14 (server-side), and you're using 1.8.13 unfortunately. There is
also a workaround documented in the issue, by using certain Apache
directives.
- Large revision files when editing large directories: that's because
in your version of the server / FSFS backend there is not yet any
"deltification" of directories, I believe (or not by default anyway,
because a 1.8 server cannot handle that performantly). That means that
directory listings are stored entirely in full for every revision that
changes them (and not merely the diff is stored). As of 1.9 the
default for new repositories is to enable directory deltification. See
http://subversion.apache.org/docs/release-notes/1.9.html#fsfs-defaults.
--
Johan

Thanks, Johan, very instructive.