[Simh] Error record marks

[Timothe Litt]

On 22-May-15 15:13, Timothe Litt wrote:
> On 22-May-15 14:12, Al Kossow wrote:
>> On 5/21/15 7:55 PM, Timothe Litt wrote:
>>
>>> This is incorrect.  A tape mark is 32-bits of zero.  0x80000000 is an
>>> invalid encoding and should be reported as
>>> a format error.  (It indicates a length of zero with an error, but
>>> the record length field must be non-zero.)
>>>
>> This is what you get if the tape drive has a read error, but returns
>> no data, which is generally what happens on
>> SCSI tape drives, and points out a serious problem with the format
>> when you are dealing with reading tapes with bad blocks.
>>
>> There is nothing in the format that gives any hints what the tape
>> drive did when it hit a bad block, and different drives
>> do different things. Some return nothing, some return the bad data,
>> some retry the blocks returning data on every retry,
>> some don't.
>>
>> My program keeps a text log which I include if an image I make has bad
>> blocks in it, but I never wrote out the scsi ident
>> string of the drive in it.
>>
> Sounds like we need to define what should happen. 
>
> I do agree with your implicit observation that the transcriber should
> log the drive
> ident string... Actually, there is a larger issue of how to record and
> manage
> metadata that's not in the container.  I'm working on that, so stay tuned.
>
> For this case, the spec is clear: the length must be non-zero.
>
> mtdump's idea that 0x80000000 is a tape mark is clearly wrong.
>
> We can consider updating the spec, but how?  Several options:
>
> 1) We could define this code as "error, no data returned, record length
> is unknown". 
> I'm pretty sure there is code that will break, but code can be fixed. 
> This means making
> sure that all the emulated drives return the "right" thing; e.g. what
> their hardware would
> have for this condition. 
>
> 2) We could state that if the drive returns no data, the reader should
> insert a noise record
> ; e.g. a 1 or 2 byte record with some well-known value.  (Obviously, and
> the error flag.)
>  Tape drives/controllers, OS drivers (and yes, my code) will ignore
> noise records unless
> specifically instructed to return them. (Even then, some physically
> can't.)  This is enough
> of a marker for emulated drives to return the appropriate equivalent to
> the OS.  And
> existing readers of the format are less likely to break.
>
> 3) We could state that if the drive returns no data, the reader should
> insert a > noise length
>      record with well known data.  This could be better for emulated
> drives that haven't been
>      updated... But how the application will react will vary wildly.
>
> I rather lean toward (1), although it implies a lot of work going
> through the tape drive
> emulations.  for "someone" :-(
>
> The pathological case is the one you mention where the drive keeps
> retrying and the
> reader records each try.  The good news is that if the data is slightly
> different each time,
> a human might have a better chance at reconstruction.  The really bad
> news is that
> this will confuse most applications - they will see more records than
> are on the tape.
>
> Essentially, each retry inserts a specious record.  Even if we knew the
> drive type, that
> wouldn't be enough.  Say the drive retries 4 times and moves on.  The
> next record also
> has an error.  How do we know that it's 4 copies of one record and 2 of
> the next?
>
> Perhaps if we have a table of the drive type, firmware version, OS
> patchlevel we can
> be reasonably sure - if the retry limit is the same for all errors.  If
> not, we also need
> the error bits from the drive.  Perhaps even from each retry, as the
> results can vary. :-( 
> But I don't know how to generate that table. 
>
> Then we may need a bit in the container to delimit retry sequences; e.g.
> perhaps flip bit
> 30 on each error?  Then we can have tools that extract retry sequences
> and let a human
> decide which record (or edited record) goes into the file...  At least
> readers (tools/
> emulated drives) can then know that they have to decide which record to
> return to
> their caller...
>
> It's worth noting that records with errors are useful - even expected.
> a) If they do return data, it may have the only extant bits - often the
> error is recoverable by
> manual inspection.  So if transcribers can possibly use drives that
> return data, they should.
>
> b) there are applications that get upset if records with errors are
> dropped.  For example, TOPS-10
> BACKUP optimizes write speed by not backing up and erasing when a write
> error is detected.
> Instead, it writes another copy of the record (marked as a "repeater")
> and tries again.  The monitor
> cooperates by not writing any records queued to the tape when an error
> occurs.  (This is a special
> OPEN bit - UU.SOE.)  This keeps the tape streaming - and is impressive
> on the real hardware.
> Anyhow, the point is that when reading, if backup sees a repeater record
> without a preceding error,
> it complains.
>
> Sigh.
>
>
One more thing for anyone thinking about this.  If 0x80000000 becomes
"error, no data returned",
we need to decide whether this is a degenerate record, or a marker.

If the former,

0x80000000
0x80000000

represents one record with an error that returned no data.

If the latter, each 0x80000000 represents one record, so this would be
decoded as two records,
each with an error.

Code will have to change either way.   I could make a case for either. 
But more interesting is what
the existing usage of this (currently specified as invalid ) bit patter
is.  If there's a consensus, perhaps
we can avoid having yet another format-fixing tool...for existing tapes.

Al, does your tool generate a record or a marker?
From your description, I'd guess a record, but...  Anyone other than Al
using this?



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4942 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mailman.trailing-edge.com/pipermail/simh/attachments/20150524/08bfc72b/attachment.bin>