Update test references

[lczech]

As mentioned in #324, the tests do not seem to be up to date with what Binder generates any more. This PR attempts to fix this. There are some major changes to some of the test cases though - I'd appreciate your feedback @lyskov on whether this is expected and in order. If not, let me know how to fix this :-)

[lyskov]

Thank you for PR @lczech ! This is a quite a big difference, - let me run the ~same thing locally on current master and see if i can reproduce this.

[lyskov]

Hm… something does not seems right, - for instance there is diff for T42.stl.names.multi.cpp but it should not be compilable (string include was missing in original .hpp file)… - i just pushed my version to master, could you please merge it (or maybe just re-run tests on your system using the latest master if that easier?).

Maybe this is compiler difference, - was your diff set produced with Clang or GCC?

Thanks,

[lczech]

Interesting, hm yes, it could be different clang versions then? I am using --llvm-version 14.0.5. What do you mean though by

was your diff set produced with Clang or GCC?

that? Binder is using LLVM, so where would GCC come into play when creating the test references?

[lyskov]

that? Binder is using LLVM, so where would GCC come into play when creating the test references?

-- The code will be always produced by Binder and so with LLVM, however you can (try) compile it with any compiler including GCC (and this is where the error that I mentioned will manifest).

Just to double check: you used the default LLVM version when running build-and-run-tests.py script?

[lczech]

The code will be always produced by Binder and so with LLVM, however you can (try) compile it with any compiler including GCC (and this is where the error that I mentioned will manifest).

Ah I see what you mean. Yes, I am also compiling on GCC, and AppleClang, see, e.g., here - all of that works fine except for the int sizes issue #258

Just to double check: you used the default LLVM version when running build-and-run-tests.py script?

I'm using this:

python3 build.py -j 8 --llvm-version 14.0.5 --update-binder
python3 build-and-run-tests.py -j 8 --llvm-version 14.0.5

which includes my build tool improvement from #327.

[lyskov]

Aha, now that's makes sense, - thank you for clarifying. I am not sure we want to use custom LLVM version for reference results, I would rather stick with what is coded in scripts so users could re-run this locally without knowing the extra options.

[lczech]

Right, that makes sense! I guess #337 does this anyway already, so this PR can be closed then?!

[lczech]

Also, would it make sense to upgrade the default LLVM? I think the current one is 8? Seems a bit old already.

Edit: Nope, sorry, just checked here, it's 6... But I see in the comments there that you want to keep this for backwards compatibility?

[lyskov]

Right, that makes sense! I guess #337 does this anyway already, so this PR can be closed then?!

probably yes

[lyskov]

Also, would it make sense to upgrade the default LLVM? I think the current one is 8? Seems a bit old already

Current default version is v6 which is the last LLVM version that could be compiled on legacy systems like CentOS-7 (I know, long deprecated) so I would rather keep it that way unless there is a compelling reason to do so.