[Simh] How do we test the emulators?

[Peter Lund]

While mucking around with the build script(s) it annoyed me that I
basically didn't have any tests I could run on the resulting binaries.

Jason Stevens seems to have had the same problem when he recently built
lots and lots of binaries for various platforms.  If it had been dead
easy to test an emulator I don't think he would have written:
"Again these binaries are largely compiled and untested.."

So how could we test them?  I'm not really thinking about huge, in-depth
tests.  More like small and easy smoke tests.

Most of us probably only really care about a small number of emulators
that emulate interesting machines or machines one used to work on for
years, a long time ago.  So we have in-depth knowledge of a few of the
emulators and practically none about the others.

Even the ones we know well require active work to test: find this OS
image, attach it to the right emulated peripheral device, boot it, run
this command, then that command, pretend to send something to an output
device, etc...

Are there a bunch of smallish programs we can gather somewhere on a web
site (or in a tarball, sorry, zipfile) and script so anybody can run the
smoketests automatically without having to know anything about all the
emulators he or she doesn't care about?

I think we should start small and then (slowly) build upwards.  Simple
things like OS images we can boot test automatically or machine code
sorting and searching programs for the really old machines.

We can add coverage testing later on and gradually improve the tests
until we know all instructions have been tested, all addressing modes,
all external hardware, etc.

Anyone?

-Peter