inconsistent revision numbers

[Norbert Preining]

Dear all,

(please Cc, I am not subscribed)

I have blogged about this [1] and both Branko Čibej and Stefan Sperling
asked me to move the discussion here.
[1] https://www.preining.info/blog/2017/10/inconsistent-version-numbers-in-subversion/

I describe my problem

* Assume a subversion server and two checkouts
* both checkout are completely clean, no out-of-vcs files, no
ignored files, no mixed revisions, dead plain svn checkouts.
* Furthermore, assume the following set of commits:

revision A: file xxx is changed
revision A+n: file xxx is changed again
revision A+n+m: file xxx is changed and takes the same content
as back in revision A

Assume two users:
User A did 'svn up' between revision A+n and A+n+m
User B did 'svn up' in [A, A+n), but NOT in [A+n, A+n+m), and
again after A+n+m


In this case, the "last comitted revision" (according to 'svn status -v')
differ:
User A will get A+n+m
User B will get A

In practice, in our TeX Live main repository, when I do 'svn status -v'
in my own checkout, I see:
45627 39121 karl Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
and when I do it on our master checkout (the one that does the daily
updates for TeX Live) I see:
45643 44192 karl Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
(empty initial columns deleted).

The second number here is -- according to 'svn status -h' --
The last committed revision and last committed author (with -v)
^^^^^^^^^^^^^^^^^^^^^^^

My assumption *was* that this is *consistent* across checkout, but it is
not.

Is this an intended behavior?

All the best

Norbert


--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[Branko Čibej]

Post by Norbert Preining
Dear all,
(please Cc, I am not subscribed)
I have blogged about this [1] and both Branko Čibej and Stefan Sperling
asked me to move the discussion here.
[1] https://www.preining.info/blog/2017/10/inconsistent-version-numbers-in-subversion/
I describe my problem
* Assume a subversion server and two checkouts
* both checkout are completely clean, no out-of-vcs files, no
ignored files, no mixed revisions, dead plain svn checkouts.

I'm pretty sure they're not, but see below.

Post by Norbert Preining
revision A: file xxx is changed
revision A+n: file xxx is changed again
revision A+n+m: file xxx is changed and takes the same content
as back in revision A
User A did 'svn up' between revision A+n and A+n+m
User B did 'svn up' in [A, A+n), but NOT in [A+n, A+n+m), and
again after A+n+m
In this case, the "last comitted revision" (according to 'svn status -v')
User A will get A+n+m
User B will get A
In practice, in our TeX Live main repository, when I do 'svn status -v'
45627 39121 karl Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
and when I do it on our master checkout (the one that does the daily
45643 44192 karl Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
(empty initial columns deleted).
The second number here is -- according to 'svn status -h' --
The last committed revision and last committed author (with -v)
^^^^^^^^^^^^^^^^^^^^^^^
My assumption *was* that this is *consistent* across checkout, but it is
not.

It should be.

I took a look at the TeXlive repository, specifically at that file. In
both cases, wheather the working copy revision (svn update -r X) is
45627 or 45643, that file's last changed revision should be 44192.

Here's the proof:

$ svn info --show-item=last-changed-revision svn://tug.org/texlive/trunk/Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
44192

$ svn info --show-item=last-changed-revision svn://tug.org/texlive/trunk/Master/tlpkg/tlpsrc/collection-***@45627
44192

Post by Norbert Preining
Is this an intended behavior?

That depends on what you did. :)

For example, what does `svnversion` say about your working copy?
Compared to the master?

Unless this is a strange artefact of git-svn, I'd recommend an "svn
update" in your working copy ("your checkout," as you call it) and let's
see what happens.

-- Brane

[Norbert Preining]

Hi Branko,

Post by Branko Čibej

Post by Norbert Preining
* both checkout are completely clean, no out-of-vcs files, no
ignored files, no mixed revisions, dead plain svn checkouts.

I'm pretty sure they're not, but see below.

Disprove me, please!

Post by Branko Čibej

Post by Norbert Preining
My assumption *was* that this is *consistent* across checkout, but it is
not.

It should be.

That is good to hear.

Post by Branko Čibej
$ svn info --show-item=last-changed-revision svn://tug.org/texlive/trunk/Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
44192

Indeed, I get the same.

Post by Branko Čibej

Post by Norbert Preining
Is this an intended behavior?

That depends on what you did. :)

Nothing but very sporadic svn up

Post by Branko Čibej
For example, what does `svnversion` say about your working copy?
Compared to the master?

mine: 45643
master: 45643

Post by Branko Čibej
Unless this is a strange artefact of git-svn, I'd recommend an "svn

It is NOT about git-svn, please stop claiming that. I clearly stated
that it is about a standard default svn checkout.

I have *two* checkouts, one over svn and one over git-svn. The reason
why the plain svn is only updated once in a while is that I practically
only use git-svn.



Now for the supposed help of "svn up", even after that I still have
on my local machine:
$ svn status -v | grep collection-langgreek.tlpsrc
45643 39121 karl Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
$

This is BTW not the only file. We have this pattern (due to some users
doing remove/copy/add of the same file sometimes) at about 40 files.

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[Stefan Sperling]

Post by Branko Čibej

Post by Norbert Preining
Dear all,
(please Cc, I am not subscribed)
I have blogged about this [1] and both Branko Čibej and Stefan Sperling
asked me to move the discussion here.
[1] https://www.preining.info/blog/2017/10/inconsistent-version-numbers-in-subversion/
I describe my problem
* Assume a subversion server and two checkouts
* both checkout are completely clean, no out-of-vcs files, no
ignored files, no mixed revisions, dead plain svn checkouts.

I'm pretty sure they're not, but see below.

Post by Norbert Preining
revision A: file xxx is changed
revision A+n: file xxx is changed again
revision A+n+m: file xxx is changed and takes the same content
as back in revision A
User A did 'svn up' between revision A+n and A+n+m
User B did 'svn up' in [A, A+n), but NOT in [A+n, A+n+m), and
again after A+n+m
In this case, the "last comitted revision" (according to 'svn status -v')
User A will get A+n+m
User B will get A
In practice, in our TeX Live main repository, when I do 'svn status -v'
45627 39121 karl Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
and when I do it on our master checkout (the one that does the daily
45643 44192 karl Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
(empty initial columns deleted).
The second number here is -- according to 'svn status -h' --
The last committed revision and last committed author (with -v)
^^^^^^^^^^^^^^^^^^^^^^^
My assumption *was* that this is *consistent* across checkout, but it is
not.

It should be.
I took a look at the TeXlive repository, specifically at that file. In
both cases, wheather the working copy revision (svn update -r X) is
45627 or 45643, that file's last changed revision should be 44192.
$ svn info --show-item=last-changed-revision svn://tug.org/texlive/trunk/Master/tlpkg/tlpsrc/collection-langgreek.tlpsrc
44192
44192

Post by Norbert Preining
Is this an intended behavior?

That depends on what you did. :)
For example, what does `svnversion` say about your working copy?
Compared to the master?
Unless this is a strange artefact of git-svn, I'd recommend an "svn
update" in your working copy ("your checkout," as you call it) and let's
see what happens.
-- Brane

It looks like a bug. The last-changed revision of the file is not
bumped even though it should be. It seems to be related to the
fact that the server does not send a delta for the file during
client B's update because the file's content is the same.

I can reproduce the problem with the attached script.
Below is the full output.

Note that the file 'alpha' is last-changed in revision 4 but has
an incorrect last-changed revision of r2 instead of r4:

4 2 stsp last-changed-rev/trunkB/alpha

The output changes if this line in the script:

echo change1 > $trunk/alpha # change back to content of r2

is changed to:

echo change3 > $trunk/alpha # change to other content

which results in a non-null delta on user B's update.

4 4 stsp last-changed-rev/trunkB/alpha



+ set -e
+ pwd
+ cwd=/tmp
+ basename last-changed-rev.sh .sh
+ scratch_area=last-changed-rev
+ repos=last-changed-rev/repos
+ trunk=last-changed-rev/trunk
+ branch=last-changed-rev/branch
+ trunk_url=file:////tmp/last-changed-rev/repos/trunk
+ branch_url=file:////tmp/last-changed-rev/repos/branch
+ set -x
+ rm -rf last-changed-rev
+ mkdir -p last-changed-rev
+ mkdir -p last-changed-rev/trunk
+ echo alpha
+ > last-changed-rev/trunk/alpha
+ svnadmin create /tmp/last-changed-rev/repos
+ svn import last-changed-rev/trunk file:////tmp/last-changed-rev/repos/trunk -m importing project tree
Adding last-changed-rev/trunk/alpha
Committing transaction...
Committed revision 1.
+ rm -rf last-changed-rev/trunk
+ svn checkout file:////tmp/last-changed-rev/repos/trunk last-changed-rev/trunk
A last-changed-rev/trunk/alpha
Checked out revision 1.
+ svn checkout file:////tmp/last-changed-rev/repos/trunk last-changed-rev/trunkA
A last-changed-rev/trunkA/alpha
Checked out revision 1.
+ svn checkout file:////tmp/last-changed-rev/repos/trunk last-changed-rev/trunkB
A last-changed-rev/trunkB/alpha
Checked out revision 1.
+ echo change1
+ > last-changed-rev/trunk/alpha
+ svn ci last-changed-rev/trunk -m revision A: file xxx is changed
Sending last-changed-rev/trunk/alpha
Transmitting file data .done
Committing transaction...
Committed revision 2.
+ svn up last-changed-rev/trunkB
Updating 'last-changed-rev/trunkB':
U last-changed-rev/trunkB/alpha
Updated to revision 2.
+ echo change2
+ > last-changed-rev/trunk/alpha
+ svn ci last-changed-rev/trunk -m revision A+n: file xxx is changed again
Sending last-changed-rev/trunk/alpha
Transmitting file data .done
Committing transaction...
Committed revision 3.
+ svn up last-changed-rev/trunkA
Updating 'last-changed-rev/trunkA':
U last-changed-rev/trunkA/alpha
Updated to revision 3.
+ echo change1
+ > last-changed-rev/trunk/alpha
+ svn ci last-changed-rev/trunk -m revision A+n+m: file xxx is changed and takes the same content as back in revision A
Sending last-changed-rev/trunk/alpha
Transmitting file data .done
Committing transaction...
Committed revision 4.
+ svn up last-changed-rev/trunkB
Updating 'last-changed-rev/trunkB':
At revision 4.
+ svn status -v last-changed-rev/trunkA
3 3 stsp last-changed-rev/trunkA
3 3 stsp last-changed-rev/trunkA/alpha
+ svn status -v last-changed-rev/trunkB
4 4 stsp last-changed-rev/trunkB
4 2 stsp last-changed-rev/trunkB/alpha

[Daniel Shahaf]

Post by Stefan Sperling
It looks like a bug. The last-changed revision of the file is not
bumped even though it should be. It seems to be related to the
fact that the server does not send a delta for the file during
client B's update because the file's content is the same.
I can reproduce the problem with the attached script.

I've added a regression test in https://svn.apache.org/r1813771.

Daniel

[Branko Čibej]

Post by Daniel Shahaf

Post by Stefan Sperling
It looks like a bug. The last-changed revision of the file is not
bumped even though it should be. It seems to be related to the
fact that the server does not send a delta for the file during
client B's update because the file's content is the same.
I can reproduce the problem with the attached script.

I've added a regression test in https://svn.apache.org/r1813771.

Well I'll be unprintabled. I can't imagine how or when such a bug could
have slipped through; and it's not RA-specific, either.

As mentioned elsewhere: this may be a new bug. Otherwise I'm sure it'd
have been caught in the 1.0 testing, 15+ years ago ...


Norbert, congratulations, you found a bug that we'd all have expected to
be impossible. Regarding the "what were we thinking" rant in your blog;
clearly, we were thinking that this bug could not possibly have crept
through all our testing. And we were mistaken.

Duh.

-- Brane

[Stefan Sperling]

Post by Daniel Shahaf

Post by Stefan Sperling
It looks like a bug. The last-changed revision of the file is not
bumped even though it should be. It seems to be related to the
fact that the server does not send a delta for the file during
client B's update because the file's content is the same.
I can reproduce the problem with the attached script.

I've added a regression test in https://svn.apache.org/r1813771.
Daniel

Cheers!

[Daniel Shahaf]

Post by Daniel Shahaf

Post by Stefan Sperling
It looks like a bug. The last-changed revision of the file is not
bumped even though it should be. It seems to be related to the
fact that the server does not send a delta for the file during
client B's update because the file's content is the same.
I can reproduce the problem with the attached script.

I've added a regression test in https://svn.apache.org/r1813771.

Fixed in https://svn.apache.org/r1813794 and nominated for backport
to 1.9.8. Pizzas to stsp :-)

Daniel

[Norbert Preining]

Post by Daniel Shahaf
Fixed in https://svn.apache.org/r1813794 and nominated for backport
to 1.9.8. Pizzas to stsp :-)

Thanks a lot.

With a fixed/updated subversion client, will I get the updates
automatically, or do I need also an updated server, and also do some
special svn up invocations?

Best

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[Daniel Shahaf]

Post by Norbert Preining

Post by Daniel Shahaf
Fixed in https://svn.apache.org/r1813794 and nominated for backport
to 1.9.8. Pizzas to stsp :-)

Thanks a lot.
With a fixed/updated subversion client, will I get the updates
automatically, or do I need also an updated server, and also do some
special svn up invocations?

You need to upgrade the server (only; the client needn't be upgraded).

As to "special svn up invocations", let me first say that the easiest
way would be to checkout a new working copy.

Having said that, the following should work in an existing working copy:
For each affected file, run an 'svn update' (to either a newer or an
older revision) that crosses a revision that modified that file.

The easiest way to determine the affected files would be to diff the
outputs of «svn info -R ./» and «svn info -R $(svn info --show-item=url)»,
specifically, the "Last Changed Rev" entries in the output. This
assumes the working copy has been updated to HEAD; if that's not the
case, the second 'info' invocation should be passed the working copy's
BASE revision for a peg revision.

Yet another approach would be to run 'svn up -r N && svn up', where N =
HEAD - 1000 or so; note, this would only be 95% correct.

Cheers,

Daniel

[Norbert Preining]

Hi everyone

thanks for the incredible quick check and ACK.

Post by Branko Čibej
Norbert, congratulations, you found a bug that we'd all have expected to
be impossible. Regarding the "what were we thinking" rant in your blog;
clearly, we were thinking that this bug could not possibly have crept
through all our testing. And we were mistaken.

Sorry for the "rant" - take it as what it is - I was soo dead surprised
to find something like this in svn that I thought it must be on purpose
- I couldn't find any other explanation.

Good night from Japan

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13