Checksum mismatch still with 1.8.19

[Grant Drake]

Our current Subversion server is 1.8.5. We are trying to setup a replacement server, on which I have installed 1.8.19.

The svnadmin load command on the 1.8.19 server for one of the repository dump files only comes up repeatedly with this error about 80% of the way through the revisions:

svnadmin: E160000: SHA1 of reps '-1 133 284 793 a21e1fc00eb3e762b9b269b65b16a7bc ac7b8b00ada08b3e6bba37a0be206ad5faab70c1 6476-4zw/_a' and '-1 641 284 793 a21e1fc00eb3e762b9b269b65b16a7bc ac7b8b00ada08b3e6bba37a0be206ad5faab70c1 6476-4zw/_a' matches (ac7b8b00ada08b3e6bba37a0be206ad5faab70c1) but contents differ
svnadmin: E160004: Filesystem is corrupt
svnadmin: E200014: Checksum mismatch while reading representation:
expected: a21e1fc00eb3e762b9b269b65b16a7bc
actual: ac1cc0244040d0134191ec8cec175e1f

I have run svnadmin verify on the original server repo, it shows no indication of corruption.

What can we do? Must we upgrade the original source server to 1.8.19 in order to produce a new dump file that can be loaded?



________________________________

This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page

Please visit http://www.markit.com/en/about/contact/contact-us.page for contact information on our offices worldwide.

[Eric Johnson]

Does an svnadmin load, with the same dump file, work without errors, when
you load back into version 1.8.5?

How did you create the dump file? Did you use the incremental form with
deltas?

Eric.

Post by Grant Drake
Our current Subversion server is 1.8.5. We are trying to setup a
replacement server, on which I have installed 1.8.19.
The svnadmin load command on the 1.8.19 server for one of the repository
dump files only comes up repeatedly with this error about 80% of the way
svnadmin: E160000: SHA1 of reps '-1 133 284 793
a21e1fc00eb3e762b9b269b65b16a7bc ac7b8b00ada08b3e6bba37a0be206ad5faab70c1
6476-4zw/_a' and '-1 641 284 793 a21e1fc00eb3e762b9b269b65b16a7bc
ac7b8b00ada08b3e6bba37a0be206ad5faab70c1 6476-4zw/_a' matches (
ac7b8b00ada08b3e6bba37a0be206ad5faab70c1) but contents differ
svnadmin: E160004: Filesystem is corrupt
expected: a21e1fc00eb3e762b9b269b65b16a7bc
actual: ac1cc0244040d0134191ec8cec175e1f
I have run svnadmin verify on the original server repo, it shows no
indication of corruption.
What can we do? Must we upgrade the original source server to 1.8.19 in
order to produce a new dump file that can be loaded?
------------------------------
This e-mail, including accompanying communications and attachments, is
strictly confidential and only for the intended recipient. Any retention,
use or disclosure not expressly authorised by Markit is prohibited. This
http://www.markit.com/en/about/legal/email-disclaimer.page
Please visit http://www.markit.com/en/about/contact/contact-us.page for
contact information on our offices worldwide.

[Daniel Shahaf]

Post by Grant Drake
Our current Subversion server is 1.8.5. We are trying to setup a replacement server, on which I have installed 1.8.19.
svnadmin: E160000: SHA1 of reps '-1 133 284 793 a21e1fc00eb3e762b9b269b65b16a7bc ac7b8b00ada08b3e6bba37a0be206ad5faab70c1 6476-4zw/_a' and '-1 641 284 793 a21e1fc00eb3e762b9b269b65b16a7bc ac7b8b00ada08b3e6bba37a0be206ad5faab70c1 6476-4zw/_a' matches (ac7b8b00ada08b3e6bba37a0be206ad5faab70c1) but contents differ

Does it load successfully if you disable rep-sharing in fsfs.conf on the
1.8.19 repository?

The two -1's imply that both sides of the collision are part of the same
revision --- the revision that 'load' fails to commit. Inspect that
revision on the 1.8.5 server. Does it, by any chance, really add two
different files with the same sha1?

Post by Grant Drake
svnadmin: E160004: Filesystem is corrupt
expected: a21e1fc00eb3e762b9b269b65b16a7bc
actual: ac1cc0244040d0134191ec8cec175e1f
I have run svnadmin verify on the original server repo, it shows no indication of corruption.
What can we do? Must we upgrade the original source server to 1.8.19 in order to produce a new dump file that can be loaded?

I don't expect that would help. This doesn't seem like a bug in the dump code.

[Grant Drake]

Thanks all for the responses.

Post by Daniel Shahaf
Does it load successfully if you disable rep-sharing in fsfs.conf on the
1.8.19 repository?

I do not have rep-sharing enabled (on either repository).

Post by Daniel Shahaf
The two -1's imply that both sides of the collision are part of the same
revision --- the revision that 'load' fails to commit. Inspect that revision on
the 1.8.5 server. Does it, by any chance, really add two different files with
the same sha1?

I inspected the revision. It does indeed have a number of files with the same MD-5 / SHA1 hash. The contents seem to be identical for them (they are NuGet packages.config files for different projects in the solution, all with the same package references, so their xml files are identical). But this is surely not an unusual situation?

Post by Daniel Shahaf
Just quickly skimming through this mail, this sounds like the problem
http://mail-archives.apache.org/mod_mbox/subversion-users/201707.mbox/%3C5aae07a9-bb6d-6340-206a-dafcf91e35e6%40apache.org%3E
Note that the issue reported in that mail was not fixed in 1.8.19 (it's
currently expected to get into 1.8.20).
If you are in fact running into that described problem, try svnadmin
load -M 0 as a workaround. If that won't work, you are likely running
into a different issue here. As danielsh already suggested, try to
verify that indeed you don't have two different files with the same
sha-1 hash (I doubt that atm, though).

I I had indeed read that thread before I posted here, however the last message on that thread said the fix would be in 1.8.19. Which is why I posted - that the fix is still pending is news to me :-)

I have tried the svnadmin load -M 0 trick, and that does seem to work to allow the dump to be loaded. Exactly why I have no clue though of course, but thanks! Is there any downside to this?

________________________________

This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page

Please visit http://www.markit.com/en/about/contact/contact-us.page for contact informati

[Daniel Shahaf]

Post by Grant Drake

Post by Daniel Shahaf
The two -1's imply that both sides of the collision are part of the same
revision --- the revision that 'load' fails to commit. Inspect that revision on
the 1.8.5 server. Does it, by any chance, really add two different files with
the same sha1?

I inspected the revision. It does indeed have a number of files with
the same MD-5 / SHA1 hash. The contents seem to be identical for them
(they are NuGet packages.config files for different projects in the
solution, all with the same package references, so their xml files are
identical). But this is surely not an unusual situation?

I meant, files that have the same sha1 and different contents, as in
http://shattered.io/. Multiple identical files are of course supported.

[Stefan Hett]

Post by Grant Drake
[...]

Post by Stefan
Just quickly skimming through this mail, this sounds like the problem
http://mail-archives.apache.org/mod_mbox/subversion-users/201707.mbox/%3C5aae07a9-bb6d-6340-206a-dafcf91e35e6%40apache.org%3E
Note that the issue reported in that mail was not fixed in 1.8.19 (it's
currently expected to get into 1.8.20).
If you are in fact running into that described problem, try svnadmin
load -M 0 as a workaround. If that won't work, you are likely running
into a different issue here. As danielsh already suggested, try to
verify that indeed you don't have two different files with the same
sha-1 hash (I doubt that atm, though).

I I had indeed read that thread before I posted here, however the last message on that thread said the fix would be in 1.8.19. Which is why I posted - that the fix is still pending is news to me :-)
I have tried the svnadmin load -M 0 trick, and that does seem to work to allow the dump to be loaded. Exactly why I have no clue though of course, but thanks! Is there any downside to this?
[...]

The only downside I'm aware of is that svnadmin load performs a bit less
performant (most notably with large repositories on slower storage
systems (i.e. HDD or NAS)). This however only applies to the performance
of running the load command. The repository's performance is not
impacted to my knowledge.

P.S. your mail signature is not quite suitable for posting on mailing
lists. ;-)

[Stefan]

Post by Grant Drake
Our current Subversion server is 1.8.5. We are trying to setup a
replacement server, on which I have installed 1.8.19.
 
The svnadmin load command on the 1.8.19 server for one of the
repository dump files only comes up repeatedly with this error about
 
svnadmin: E160000: SHA1 of reps '-1 133 284 793
a21e1fc00eb3e762b9b269b65b16a7bc
ac7b8b00ada08b3e6bba37a0be206ad5faab70c1 6476-4zw/_a' and '-1 641 284
793 a21e1fc00eb3e762b9b269b65b16a7bc
ac7b8b00ada08b3e6bba37a0be206ad5faab70c1 6476-4zw/_a' matches
(ac7b8b00ada08b3e6bba37a0be206ad5faab70c1) but contents differ
svnadmin: E160004: Filesystem is corrupt
   expected:  a21e1fc00eb3e762b9b269b65b16a7bc
     actual:  ac1cc0244040d0134191ec8cec175e1f
 
I have run svnadmin verify on the original server repo, it shows no
indication of corruption.
 
What can we do? Must we upgrade the original source server to 1.8.19
in order to produce a new dump file that can be loaded?
 
 
------------------------------------------------------------------------
This e-mail, including accompanying communications and attachments, is
strictly confidential and only for the intended recipient. Any
retention, use or disclosure not expressly authorised by Markit is
prohibited. This email is subject to all waivers and other terms at
http://www.markit.com/en/about/legal/email-disclaimer.page
Please visit http://www.markit.com/en/about/contact/contact-us.page
for contact information on our offices worldwide.

Just quickly skimming through this mail, this sounds like the problem
described and investigated in late July:
http://mail-archives.apache.org/mod_mbox/subversion-users/201707.mbox/%3C5aae07a9-bb6d-6340-206a-dafcf91e35e6%40apache.org%3E

Note that the issue reported in that mail was not fixed in 1.8.19 (it's
currently expected to get into 1.8.20).

If you are in fact running into that described problem, try svnadmin
load -M 0 as a workaround. If that won't work, you are likely running
into a different issue here. As danielsh already suggested, try to
verify that indeed you don't have two different files with the same
sha-1 hash (I doubt that atm, though).

Regards,
Stefan