[Simh] clang (was Re: XCode and LTO)

[Craig A. Berry]

On Apr 18, 2012, at 11:36 AM, Mark Pizzolato - Info Comm wrote:

> On Tuesday, April 17, 2012, at 8:53 PM, Craig A. Berry wrote:
>> 
>> Hmm.  Putting GCC=clang in the environment gets a successful compile,
>> though with lots of warnings.  However, starting the simulator hangs after
>> "Eth: opened OS device en0" which it didn't do with other compilers, so there
>> would be more work to support that compiler or it may not like something
>> about the libpcap library.
> 
> The current makefile (or at least it's attempts at optimizing) presumes strongly that gcc is the compiler engine (as it expects that make will be gnu-make).  Please let me know if you come up with any way which will provide better results using the most common compilers which an OSX environment might have.  
> 

Finally got back to this and debugged the hang upon start-up after building with clang (the native front-end to llvm), which reports itself like so:

% clang -v
Apple clang version 3.1 (tags/Apple/clang-318.0.58) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin11.3.0
Thread model: posix

Clang claims to be highly gcc-compatible and it looks like that's true, but the makefile doesn't recognize it so it doesn't select any non-default compiler options.  The following command line with explicit options is what I landed on after experimentation; it suppresses some of the more voluminous warnings as well as enabling some optimizations that clang seems to recognize:

% make GCC=clang CFLAGS_O='-O4 -fno-strict-overflow -flto -fwhole-program -Wno-parentheses -Wno-comment' vax

That builds, but it doesn't run, due to the hang I mentioned.  Sampling the process in the Apple Activity Monitor showed the following:

Call graph:
    2621 Thread_3712539   DispatchQueue_1: com.apple.main-thread  (serial)
    + 328 start  (in vax) + 52  [0x1094727e4]
    + ! 328 main  (in vax) + 1193  [0x1094a7cf9]
    + !   328 do_cmd  (in vax) + 1070  [0x1094a86de]
    + !     328 run_cmd  (in vax) + 1276  [0x1094af8ec]
    + !       328 sim_instr  (in vax) + 2466  [0x109473f82]
    + !         328 get_istr  (in vax) + 312  [0x10947ef98]
    + !           262 rom_rd  (in vax) + 171  [0x10948a6fb]
    + !           : 212 gettimeofday  (in libsystem_c.dylib) + 43,179,...  [0x7fff9384d1f0,0x7fff9384d278,...]
    + !           : 27 gettimeofday  (in libsystem_c.dylib) + 43  [0x7fff9384d1f0]
    + !           : | 27 __commpage_gettimeofday  (in libsystem_c.dylib) + 0,97  [0x7fff9384d290,0x7fff9384d2f1]
    + !           : 23 ???  (in <unknown binary>)  [0x7fff6906e618]
    + !           :   23 __commpage_gettimeofday  (in libsystem_c.dylib) + 7  [0x7fff9384d297]
    + !           66 rom_rd  (in vax) + 160,186,...  [0x10948a6f0,0x10948a70a,...]
    + 19 DYLD-STUB$$gettimeofday  (in vax) + 0  [0x1094c0a48]
    + 13 __commpage_gettimeofday  (in libsystem_c.dylib) + 92,96  [0x7fff9384d2ec,0x7fff9384d2f0]
    + 2 ???  (in <unknown binary>)  [0x7fff00096b0f]

[and a gazillion of these last 2 lines repeated]

So I located rom_rd in VAX/vax_sysdev.c and read its comments about trying to appear busy without being caught doing so by optimizing compilers.  I didn't understand what all it was doing, but it calls a tiny function named rom_swapb().  The comments in the code say, "To avoid smart compilers, the loopval variable is referenced in the function arguments so that the function expression is not loop invariant," which led me to the lucky guess that the compiler was inlining the function and then optimizing away the inlined code.  So I made the following (non-portable) change:

% git diff
diff --git a/VAX/vax_sysdev.c b/VAX/vax_sysdev.c
index 885e64b..2588d4f 100644
--- a/VAX/vax_sysdev.c
+++ b/VAX/vax_sysdev.c
@@ -475,7 +475,7 @@ DEVICE sysd_dev = {
    issues with the embedded timing loops.  
 */
 
-int32 rom_swapb(int32 val)
+int32 __attribute__((noinline)) rom_swapb(int32 val)
 {
 return ((val << 24) & 0xff000000) | (( val << 8) & 0xff0000) |
     ((val >> 8) & 0xff00) | ((val >> 24) & 0xff);
[end]

and that did the trick.  The simulator now starts and VMS boots fine.  If anyone knows of a more portable way to prevent inlining of rom_swapb(), it seems like it would be a  good idea to do so.  

I added one more line at the end of my performance comparisons.  Clang did a good job but still not quite as good as gcc:

VUPS	SIMH version, Compiler
_____	___________________________
14.0	v3.8-1, gcc (4.0?)
18.5	v3.8-2-rc2-190, llvm-gcc 4.2.1, NO_LTO=1
22.5	v3.8-2-rc2-190, gcc v4.7.0
15.4	v3.8-2-rc2-190, llvm-gcc 4.2.1, NO_LTO=1 CFLAGS_O=-O4
19.8	v3.8-2-rc2-190, clang 3.1, flags above

________________________________________
Craig A. Berry
mailto:craigberry at mac.com

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser