Tests failing post-merge with main Daikon tree (C# format)

[mernst]

Originally reported on Google Code with ID 43

Project Member Reported by [email protected], Feb 21, 2014

After merging with the main Daikon tree and running the tests inside /tests/daikon-tests,
there are 87 failures. 27 of these failures are because  csharp test goals have not
been updated. 4 of these failures were already present. This leaves 56 new failures.
I've attached the results of 'make summary' after running the tests.

    test_summary.txt 
19.1 KB   View   Download   


Feb 21, 2014 Delete comment #1 [email protected]
I'm getting 95 failures (see attached). What is your time frame for updating the C#
contract?


Feb 21, 2014 Delete comment Project Member #2 [email protected]
I am getting 87 failures when I run only /tests/daikon-test. When I run all the tests
I get 95 failures like you, where the additional 8 are from /tests/chicory-tests.

Do you mean when I plan on updating the csharp goal files?
Feb 21, 2014 Delete comment #3 [email protected]
At a glance, it looks like many of the errors might be due to changes in how the orig
works for derived variables. For example:

-size(this.elementData[]) >= orig(size(this.elementData[]))
-size(this.elementData[])-1 >= orig(size(this.elementData[]))-1
+size(this.elementData[]) >= size(orig(this.elementData[]))
+size(this.elementData[])-1 >= size(orig(this.elementData[]))-1

This is mostly likely due to the change you made in a2086720bd03 which we needed to
fix the linking of orig variables to parent PPTs first introduced in my revision 45d28d97c397.

Feb 22, 2014 Delete comment Project Member #4 [email protected]
The majority of the differences do appear to be related to how orig works for derived
variables, but there are also changes I'm seeing in some other files that do not seem
related to this.

For example, here is TinySet.txt-jml.diff:

--- TinySet.txt-jml.goal    2014-02-20 16:07:31.808256874 -0800
+++ TinySet.txt-jml 2014-02-21 11:39:55.937329203 -0800
@@ -29,6 +29,16 @@
 (\result == true)  ==>  (\old(this) != null)
 (\result == true)  ==>  (this.bits > \old(n))
 ===========================================================================
+six170.TinySet.contains(int):::EXIT;condition="return == true"
+    Variables: this this.bits n return orig(this) orig(this.bits) orig(n)
+\result == true
+\old(this) != null
+this.bits > \old(n)
+===========================================================================
+six170.TinySet.contains(int):::EXIT;condition="not(return == true)"
+    Variables: this this.bits n return orig(this) orig(this.bits) orig(n)
+\result == false
+===========================================================================
 six170.TinySet.intersect(six170.TinySet):::ENTER
     Variables: this this.bits other other.bits
 other != null

This is the addition of two new program points on the default splitter for boolean
returns.

Here is a portion of the diff from Suppress02.txt-esc.diff:

-      Unmodified variables: this this.theArray this.theArray[] n size(this.theArray[])
this.theArray[n] this.theArray[n..] this.theArray[n+1..] this.theArray[0..n] this.theArray[0..n-1]
+      Unmodified variables: this this.theArray this.theArray[] n this.theArray[n]
this.theArray[n..] this.theArray[n+1..] this.theArray[0..n] this.theArray[0..n-1]

See my next post for a summary of the errors I am seeing.


Feb 22, 2014 Delete comment Project Member #5 [email protected]
There a few classes of differences I'm seeing in the test diffs:

1) orig(size(..)) versus size(orig(..)). This shows up both in invariants and in the
variable lists for program points
2) new program points on the default splitter for boolean returns. Another example
of this is RatPoly.txt-simplify.diff
3) changes in the unmodified variables lists. In particular, size(..) derived variables
have been removed from every unmodified variable list

From what I've seen, these differences represent the cause of every test failure (aside
from the non-updated csharp goals).

Like you said, we will probably have to have the targets updated for issue (1), but
I'm not sure what is going on with issues (2) or (3). 

Do you have any ideas?


Feb 25, 2014 Delete comment Project Member #6 Todd.Schiller
For #2, did you ever make changes to how splitter PPTs are handled? I don't recall
making changes, so this would be a side effect. From the example you give, our output
looks like it might be correct but that there's some configuration issue going on (where
the splitter PPTs should not be printed out for that particular test).

For #3, can you double-check the test case to see what the correct output should be?
After that you're best bet is probably use the Eclipse debugger to figure out why the
size(this.theArray[]) is not being printed. I could imagine it being an orig(...) issue
where the pre-state and post-state variables are not being matched properly.

Reported by [email protected] on 2015-05-04 15:29:18

---

- _Attachment: [test_summary.txt](https://storage.googleapis.com/google-code-attachments/daikon/issue-43/comment-0/test_summary.txt)_

[mernst]

Reported by [email protected] on 2015-05-04 15:34:57