[Simh] EXT : [simh] testing simulated CPUs

[Hittner, David T (IS)]

An interesting read. I might delve into this further and read the articles cited by this article.

I'm somewhat disappointed that the authors only discussed user-mode defect identification, and failed to discuss the behavioral differences by the various physical implementations of the CPU architectures themselves. However the defect identification techniques described seem reasonable.

They assume a single de facto behavioral standard by "the physical CPU". However, as we all remember from the notorious Pentium bug, different physical CPUs within the same "class" (IA32, VAX, etc..) can and do behave differently.

If you read Bob Supnik's articles about some of Digital's CPUs deviating from the expected behavioral "standard", you'll get a better idea of the how fuzzy this behavior matching and the original CPU specifications really are.

Perfect emulation probably won't happen until a CPU specification is designed with the express purpose of perfect emulation in mind.

Dave Hittner

-----Original Message-----
From: simh-bounces at trailing-edge.com [mailto:simh-bounces at trailing-edge.com] On Behalf Of Nelson H. F. Beebe
Sent: Wednesday, October 30, 2013 2:27 PM
To: simh at trailing-edge.com
Cc: beebe at math.utah.edu
Subject: EXT :[Simh] [simh] testing simulated CPUs

This new journal article may be of interest to some simh-list readers:

@String{j-TOSEM                 = "ACM Transactions on Software Engineering and
                                   Methodology"}

@Article{Martignoni:2013:MTC,
  author =       "Lorenzo Martignoni and Roberto Paleari and Alessandro
                 Reina and Giampaolo Fresi Roglia and Danilo Bruschi",
  title =        "A methodology for testing {CPU} emulators",
  journal =      j-TOSEM,
  volume =       "22",
  number =       "4",
  pages =        "29:1--29:??",
  month =        oct,
  year =         "2013",
  CODEN =        "ATSMER",
  DOI =          "http://dx.doi.org/10.1145/2522920.2522922",
  ISSN =         "1049-331X (print), 1557-7392 (electronic)",
  ISSN-L =       "1049-331X",
  bibdate =      "Wed Oct 30 12:18:03 MDT 2013",
  bibsource =    "http://www.acm.org/pubs/contents/journals/tosem/;
                 http://www.math.utah.edu/pub/tex/bib/tosem.bib",
  abstract =     "A CPU emulator is a software system that simulates a
                 hardware CPU. Emulators are widely used by computer
                 scientists for various kind of activities (e.g.,
                 debugging, profiling, and malware analysis). Although
                 no theoretical limitation prevents developing an
                 emulator that faithfully emulates a physical CPU,
                 writing a fully featured emulator is a very challenging
                 and error prone task. Modern CISC architectures have a
                 very rich instruction set, some instructions lack
                 proper specifications, and others may have undefined
                 effects in corner cases. This article presents a
                 testing methodology specific for CPU emulators, based
                 on fuzzing. The emulator is ``stressed'' with specially
                 crafted test cases, to verify whether the CPU is
                 properly emulated or not. Improper behaviors of the
                 emulator are detected by running the same test case
                 concurrently on the emulated and on the physical CPUs
                 and by comparing the state of the two after the
                 execution. Differences in the final state testify
                 defects in the code of the emulator. We implemented
                 this methodology in a prototype (named as EmuFuzzer),
                 analyzed five state-of-the-art IA-32 emulators (QEMU,
                 Valgrind, Pin, BOCHS, and JPC), and found several
                 defects in each of them, some of which can prevent
                 proper execution of programs.",
  acknowledgement = ack-nhfb,
  articleno =    "29",
  fjournal =     "ACM Transactions on Software Engineering and
                 Methodology",
}

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe at math.utah.edu  -
- 155 S 1400 E RM 233                       beebe at acm.org  beebe at computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------
_______________________________________________
Simh mailing list
Simh at trailing-edge.com
http://mailman.trailing-edge.com/mailman/listinfo/simh