[Simh] SIMH test suite / target CPU speed control

[Vince Mulhollon]

On Fri, Sep 03, 2010 at 08:20:49AM -0400, Armistead, Jason wrote:
> One potential problem would be accounting for any variation in the execution behavior, e.g. if the time-of-day clock was read from the host or if network interactions occur (assuming networking is enabled).

Yes sir, we can tell here whom spends most of their time in the oldies 
(paper tape, etc) and whom spends most of their time in the moderns that
had those fancy real time clock things...

There are two ways to work around this with "modern" OS.  One is to check
for results on the line printer etc.  So, my expect script installs
fortran from papertape and then enters, compiles, and runs a test fortran 
program that verifies if 2+2 really does equal 2*2, etc.  This is a 
good test of overall machine operation and can also handle some pretty
complicated tests.  Perhaps you can even skip verification of the 
executable output, in that merely being able to install a compiler
and compile "anything" without error messages is strong indication
that all is well.

Another way to do it is just tough it out the hard way without simple
md5sum of the whole works.  So, if I run it ten times on my "golden"
machine and compare the results and list the differences, those ten 
lists of differences should be vaguely similar in their differences.

More specifically, feed the disk images into a dump program, feed 
the "ascii" dumps into diff against a similar dump of the golden test 
result, the resulting diif/patch file against any date/time and network 
access should be "about 150K and 12345 lines".  Within 5% of normal
patch file size is probably correct, but a patch thats fifty times bigger 
than ever seen before is almost certainly indicating a problem.

> If you wanted a "near enough is good enough" approximation and had an instruction which took between, say, 6 and 10 cycles to execute, you might split the difference and say that 8 cycles is an OK average.

Getting within an order of magnitude would probably be accurate enough at 
using "enough of the host CPU that you're not wasting time" yet also not 
hogging the CPU.

I'd be happy even just seeing it with I/O... If my "110 baud console" 
is outputting multiple pages of text per second, theres something wrong.  
At least on the PDP8 it should be pretty easy to do some kind of 
leaky token bucket that returns the busy flag if you've printed more 
than ten characters per second or whatever.  Of simh could emulate
disk drive seek times by simply sleeping for an appropriate amount
of time.

Although its tremendously host dependent, it would be fun to do some
control system theory to make a self controlled system that constantly
corrects the # of emulated instructions per second  equal to 50% of 
CPU load or some other configurable percentage.  My laptop gets 
uncomfortably hot in some simh experiments..