[Simh] Way out idea for simh

[Ken Cornetet]

This is a long post, so I'll ask everyone to read it carefully before you start formulating your response to tell me I'm an idiot  (I don't need you to tell me this - my wife reminds me on a regular basis).

I'd like to ask everyone to back up a bit and look at the problem: how to get files into and out of simh guest OSes. Today's simh users are in one of four camps


1.       If you are running Altair, you use the host integration IO (does this feature have an official name?).

2.       If you are running a guest OS that supports networking, and you can get simh networking set up on your host (apparently not trivial) great, you use the network.

3.       If you have a working Kermit, you connect to a Kermit server via the serial port/telnet wedge, and you are off to the races.

4.       If you lack any of those, you are stuck with these options:

A.      Paper tape

B.      Mag tape

C.      Line printer

D.      Cut-and-paste

E.       Cracking the native file system with a host utility

F.       Find or write a Kermit

G.     Implement Altair style host IO in your emulator

Before I go any further, I'd like to mention metadata and binary file transfers. Yes, metadata and the notion of binary files complicates file transfers tremendously. Even text transfers have EOL issues. That has always been a thorn, and always will. Let's set it aside for a bit.

Next, I'd like to share a little story. My grandfather used a reel type mower to mow his half-acre lawn the whole time he owned the house because that is what was available to him when he bought the house and for several years afterward. When my father inherited the house, it became my responsibility to mow this lawn. My father told me that since grandpa had successfully mowed the lawn with a reel mower for years, it was good enough for me. I immediately went and used my hard earned money to buy a power mower.

So, if you are the type to tell me that a reel mower is good enough because that's what grandpa used, you can probably stop reading now.

Ok, back to transfer mechanisms.

If you are in camps 1,2, or 3 above. You are covered. If you are in camp 4, let's look at what we have, and where we might go:

Paper tape, mag tape, line printer: These are workable, but far from convenient. You have to continually break the emulator and attach your source or destination files, then go back to the guest and issue the IO commands. Does it work? Yes. Is it a royal PITA? Yes.  This is the method I use when working with RTE, and it is a pain. In fact, I'd say this is the biggest reason I don't play with RTE any more than I do.

Kermit - Kermit is great, if you can get it (or write it) and compile it. And that is assuming your OS and machine can handle the requirements. Remember though,  not every simh user is a programmer, and even if they are, they may not know the language and architectural  quirks of the guest OS. RTE is a good example - even if you are an expert FORTRAN or Pascal programmer, you aren't going to write a Kermit for it because of the quirks of RTE in general, and specifically the bizarre way RTE deals with serial ports. More on Kermit later.

A host utility to crack to native file system. I know a few exist for some of the simh OSes.  I've written one for RTE. While better than the mag tape tango, it still has limitations. Even a simple real life file system is complex to code for. FMGR disks under RTE can be pretty simple. It is well documented. It still took me 1200 lines of C code and probably 100 hours just to write it. I still haven't tested it extensively. By the time it is done, it will have been a fairly extensive project, and it only benefits one guest OS.

Aside from the implementation difficulty,  you can't copy files in with the disk online to the guest OS, so copying files in still requires a bit of a shuffle. Under RTE, I have to dismount the disc LU, copy a file in, then mount the disc. If a guest OS doesn't support the notion of dismounting disks, you'd have to shut it down for every copy in. And again, not every simh user is a programmer, and even the ones who are may not have the skill set required to write code that can manipulate a file system that probably isn't documented.

So that leaves camp 4 users with a reel mower.  For the types who might argue that if the reel mower was good enough for grandpa, it's  good enough for us, I'd like to point out that file transfers are probably the number two most popular subject on the simh mailing list (number one would be DEC trivia). Clearly, there is interest in something better. The question is what can be done to help the simh users who are stuck with the paper or mag tape shuffle to transfer files?

>From an end user point of view, the best option would be to implement some sort of direct host IO similar to the Altair emulator. While this provides a great amount of flexibility, it has the disadvantage that it is highly dependent on the guest architecture and probably not possible to generalize such that simh could provide it as a framework it the way, say, the paper tape device is. Again, this is a great mechanism, and I intend to implement it in the hp2100 emulator, but it will be very specific to that architecture - custom microcoded instructions. Other architectures will need to use a magic memory location, or a magic IO space location, or extra instructions, or some other feature.  On top of all that, the host IO method still requires non-trivial code written in the guest. Either a user space program, or an OS driver (which additionally requires wizard level knowledge of the OS), or both.

OK, so host IO integration is cool, but a lot of work. Is there anything that would give us roughly the same benefit with less work? I've floated my idea for a shared generic file system disk, but I think I have an idea which is better. It is more convenient, and much easier to implement.

I'll call it "CU mode" for paper and mag tape.  At a conceptual  level, it is similar to how cu (the unix terminal emulator program) transfers files. When it sees the magic string "~>filename", it redirects IO from the data stream to a file. A "~<filename" tells it to start reading from the file and sending it to the remote.

To implement, modify the paper tape reader/punch and/or the mag tape device so that with CU mode enabled in the simh config file, the device would work as it does today, with the exception of when it sees a magic string of text being written, it switches to CU mode. In CU mode, the character after the magic string would be interpreted as a command (open as text, open as binary, close and exit magic mode), and the characters following the magic string and command would be used as a host file name. At that point, the simh device would close whatever host file was attached to that device, and attach the file specified in the magic string. All special processing would be turned off except text translation (for text transfers) No record blocking, no file marks, etc. Data would be written in a form the host can understand (more on that later).

The beauty of this is that it is easy to code, and you only need to implement it once in simh, and all existing and future emulated machines get the capability. Also, there is no, or very little code to write on the client side. For example a unix guest os would do copy a guest file to the host thusly (using cu's "~" as the magic string:

echo "~W/some/host/path/" > /dev/mt0
cat myguestfile >/dev/mt0
echo "~C" > /dev/mt0

The down side? Well, if we are using paper tape, for some OSes, this may be limited to text. I know that paper tape CAN handle 8 bit data (I had an hp 2100 system that loaded its OS from paper tape). From an OS point of view, I don't know. But even if we are limited to text, that's still going to make life a lot easier, and push comes to shove, there's always mechanisms like uuencode (which have the additional benefit of recording the metadata if needed).

For emulated architectures that don't have the notion of paper or mag tape, perhaps the punch/reader device could be mapped to something the architecture does handle, like a serial port or card reader and printer combo

In summary, "CU mode" offers a tremendous amount of power for simh users, and requires very little effort to implement.

Back to the other issues:

Binary and metadata. This has, is, and always will be problematic when doing file transfers between different architectures. Fortunately, we don't have to worry about it at the simh level. The notion of what a binary file is, and its associated metadata is something simh can't in general handle, simply because it doesn't have access to that information. The best thing simh can do is to simply pass the "b" option in the fopen call so that the host doesn't mangle the data and assume the guest is formatting the binary data in a manner that the desired host app can handle. This problem isn't unique to CU mode. Any file transfer mechanism must deal with this.

Text. Architectures that aren't ascii would need to have the native character set translated to ascii. When magic mode is on, and the "T" option was used in the magic string, simh would translate the native character set to/from ascii. Text records would be terminated with a "\n" which would then translated to the correct host EOL sequence by the C runtime (when simh CU mode sees the "T" command after the magic string, it passes the "t" option in the fopen of the host data file).

Kermit. Kermit is a constant topic in this thread, so I'll share my thoughts. Kermit is a great tool. As I've mentioned before, I've used Kermit extensively, and I've even attempted two implementations (one successfully). The first thing you need to know about Kermit is that it isn't one protocol. It is several. It goes everywhere from the original protocol of very short records and no unprintable characters except ^A (and that can be changed) and using  a simple ack mechanism all the way to full binary transfers of large packets using sliding windows which looks a lot more like zmodem than the original kermit protocol.

In theory, any given Kermit implementation is supposed to be able to talk to any other Kermit by negotiating the best set of options that both clients support. In practice, Joe Blow's random terminal emulator supporting  Kermit file transfers probably skips some of the negotiations and makes some assumptions about what the other side can do. Hey, He tested it against a unix system running the reference ckermit, and it worked just fine, so it's all good, right?

On top of those potential problems, we also have the issue of running Kermit over telnet. If you think this isn't fraught with peril in and of itself, go read Columbia's Kermit over telnet page http://www.columbia.edu/kermit/telnet80.html and note all the "ifs", "ands", "buts", and "bugs". Now throw in on top of that the fact that the emulated machines have to wedge all of the possible serial IO options into the telnet protocol (which itself is a bit of a slippery beast) and hope whatever is speaking telnet on the other end does the right thing. The simh serial-to-telnet mapping is not perfect. Witness the recent simh mailing list discussions pertaining to cut and paste problems and serial BREAK conditions.

Hallelujah! Holy shit! Where's the Tylenol?







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.trailing-edge.com/pipermail/simh/attachments/20160421/cd5695e4/attachment-0001.html>