[Simh] Unaligned references

[Bob Supnik]

I have a design with probe coding for fixing the unaligned IO space 
problem recently uncovered in the VAX emulator.

Because it tears up the core of unaligned processing, I'd like some 
other eyes to look at it before I start editing code and debugging.

My plan is:

a) verify all cases of unaligned access to memory by simple tests (read 
B, W, L, Q unaligned at every possible offset; write B, W, L, Q 
unaligned at every possible offset)
b) run HCORE, the hard core diagnostic - this will test unaligned 
references with memory management
c) verify that the VAX 780 still runs VMS (unaligned IO references are 
not allowed, so only unaligned memory references need to be working 
correctly)
d) verify all cases of unaligned access to Qbus IO space - a bit tricky, 
as I need to find IO registers that are not "sparse"
e) verify that CVAX still runs VMS
f) turn over the changes to the community for further testing

The design paper is attached, for review.

/Bob

-------------- next part --------------
Design Notes for Fixing VAX Unaligned Access to IO and Register Space

Problem Statement: VAX unaligned accesses are handled by reading the
surrounding longword (or longwords) and

a) for reads, extracting the addressed addressed word or longword
b) for writes, inserting the addressed word or longword and then
   writing the surrounding longword (or longwords) back

This is correct for all memory cases. On the 11/780, the unaligned
access to register or IO space causes an error, as it should. On 
CVAX, it causes incorrect behavior, by either performing too many
QBus references, or performing read-modify-writes instead of pure
writes, or accessing the wrong Qbus locations.

The problem cannot be trivially solved with address manipulation.
The core issues is that on CVAX, unaligned access is done to
exactly as many bytes as are required, using a base longword
address and a byte mask. There are five cases, corresponding to
word and longword lengths, and byte offsets 1, 2 (longword only),
and 3. Further, behavior is different for reads and writes, because
the Qbus always performs word operations on reads, leaving it to
the processor to extract a byte if needed.

Conceptual design: Changes in vax_mmu.c:

Unaligned access is done with two separate physical addresses, pa
and pa1, because if the access crosses a page boundary, pa1 may
not be contiguous with pa. It's worth noting that in an unaligned
access, the low part of the data begins at pa (complete with byte
offset), but the high parts begins at pa1 & ~03 (always in the 
low-order end of the second longword).

To handle unaligned data, we will add two routines for read and
write unaligned:

	data = ReadU (pa, len);
	WriteU (pa, len, val);

Note that the length can be 1, 2, or 3 bytes. For ReadU, data is
return right-aligned and masked. For WriteU, val is expected to
be right-aligned and masked.

The read-unaligned flows are changed as follows:

if (mapen && ((off + lnt) > VA_PAGSIZE)) {              /* cross page? */
    vpn = VA_GETVPN (va + lnt);                         /* vpn 2nd page */
    tbi = VA_GETTBI (vpn);
    xpte = (va & VA_S0)? stlb[tbi]: ptlb[tbi];          /* access tlb */
    if (((xpte.pte & acc) == 0) || (xpte.tag != vpn) ||
        ((acc & TLB_WACC) && ((xpte.pte & TLB_M) == 0)))
        xpte = fill (va + lnt, lnt, acc, NULL);         /* fill if needed */
    pa1 = ((xpte.pte & TLB_PFN) | VA_GETOFF (va + 4)) & ~03;
    }
else pa1 = ((pa + 4) & PAMASK) & ~03;                   /* not cross page */
bo = pa & 3;
if (lnt >= L_LONG) {                                    /* lw unaligned? */
    sc = bo << 3;
    wl = ReadU (pa, L_LONG - bo);                       /* read both fragments */
    wh = ReadU (pa1, bo);                               /* extract */
    return ((wl | (wh << (32 - sc))) & LMASK);
    }
else if (bo == 1)                                       /* read within lw */
    return ReadU (pa, L_WORD);
else {
    wl = ReadU (pa, L_BYTE);                            /* word cross lw */
    wh = ReadU (pa1, L_BYTE);                           /* read, extract */
    return (wl | (wh << 8));
    }

These are not very different, but they do reflect that ReadU returns
right-aligned and properly masked data, rather than the encapsulating
longword.

The write-unaligned flows change rather more drastically:

if (mapen && ((off + lnt) > VA_PAGSIZE)) {
    vpn = VA_GETVPN (va + 4);
    tbi = VA_GETTBI (vpn);
    xpte = (va & VA_S0)? stlb[tbi]: ptlb[tbi];          /* access tlb */
    if (((xpte.pte & acc) == 0) || (xpte.tag != vpn) ||
        ((xpte.pte & TLB_M) == 0))
        xpte = fill (va + lnt, lnt, acc, NULL);
    pa1 = ((xpte.pte & TLB_PFN) | VA_GETOFF (va + 4)) & ~03;
    }
else pa1 = ((pa + 4) & PAMASK) & ~03;
bo = pa & 3;
if (lnt >= L_LONG) {
    sc = bo << 3;
    WriteU (pa, L_LONG - bo, val & insert[L_LONG - bo]);
    WriteU (pa, bo, (val >> (32 - sc)) & insert[bo]);
    }
else if (bo == 1)                                       /* read within lw */
    WriteU (pa, L_WORD, val & WMASK);
else {                                                  /* word cross lw */
    WriteU (pa, L_BYTE, val & BMASK);
    WriteU (pa, L_BYTE, (val >> 8) & BMASK);
    }
return;
}

Note that all the burden here has been thrown on the WriteU routine.

-------------

ReadU is the simpler of the two routines that needs to be written.
It will handle memory reads and defer register and IO space to
model-specific unaligned handlers.

int32 ReadU (uint32 pa, int32 lnt)
{
int32 dat;
int32 sc = (pa & 3) << 3;

if (ADDR_IS_MEM (pa))
    dat = M[pa >> 2];
else {
    mchk = REF_V;
    if (ADDR_IS_IO (pa))
       dat = ReadIOU (pa, lnt);
    else dat = ReadRegU (pa, lnt);
    }
return ((dat >> sc) & insert[lnt]);
}

Note that the ReadIOU and ReadRegU return a "full longword," just
like their aligned counterparts, and ReadU right-aligns the result,
just as ReadB, ReadW, and ReadL do.

WriteU must handle the memory read-modify-write sequence. However,
it defers register and IO space to model-specific unaligned handlers.

void WriteU (uint32 pa, int32 lnt, int32 val)
{
if (ADDR_IS_MEM (pa)) {
    int32 bo = pa & 3;
    int32 sc = bo << 3;
    M[pa >> 2] = (M[pa >> 2] & ~(insert[len] << sc) | (val << sc);
    }
else if ADDR_IS_IO (pa)
    WriteIOU (pa, lnt, val);
else WriteRegU (pa, lnt, val);
return;
}

--------------

For the 11/780, ReadIOU, ReadRegU, WriteIOU, and WriteRegU all do the
same thing: they throw an SBI machine check. We can write explicit
routines to do this (and remove the unaligned checks from all the
normal adapter flows), or leave things as they are and simply define
the four routines as macros that go to the normal routines. So there's
very little to do.

On CVAX, I suspect that ReadRegU and WriteRegU behave like the
normal routines. The CVAX specs don't say much, but CMCTL (the memory
controller) notes that it ignores the byte mask and treats every
access as an aligned longword access. I suspect this is true for
the other CVAX support chips, but I no longer have chip specs.

The Qbus, on the other hand... that's a fun one. Note that all of
these cases are presented to the existing aligned IO routine:

bo = 0, byte, word, or longword length
bo = 2, word
bo = 1, 2, 3, byte length

All the other cases are going to end up at ReadIOU and WriteIOU,
and they must turn the request into the exactly correct number of
Qbus accesses AND NO MORE, because Qbus reads can have side-effects,
and word read-modify-write is NOT the same as a byte write.

The read cases are:

bo = 0, byte or word - read one word
bo = 1, byte - read one word
bo = 2, byte or word - read one word
bo = 3, byte - read one word
bo = 0, triword - read two words
bo = 1, word or triword - read two words

ReadIOU is very similar to the existing ReadIO:

int32 ReadIOU (uint32 pa, int32 lnt)
{
int32 iod;

iod = ReadQb (pa);                                      /* wd from Qbus */
if ((lnt + (pa & 1)) <= 2)                              /* byte or word & even */
    iod = iod << ((pa & 2)? 16: 0);                     /* one op */
else iod = (ReadQb (pa + 2) << 16) | iod;               /* two ops, get 2nd wd */
SET_IRQL;
return iod;
}

The write cases are:

bo = x, lnt = byte - write one byte
bo = 0 or 2, lnt = word - write one word
bo = 1, lnt = word - write two bytes
bo = 0, lnt = triword - write word, byte
bo = 1, lnt = triword - write byte, word

WriteIOU is similar to the existing WriteIO:

void WriteIO (uint32 pa, int32 val, int32 lnt)
{
switch (lnt) {
case L_BYTE:                                            /* byte */
    WriteQb (pa, val & BMASK, WRITEB);
    break;
case L_WORD:                                            /* word */
    if (pa & 1) {                                       /* odd addr? */
        WriteQb (pa, val & BMASK, WRITEB);
        WriteQb (pa + 1, (val >> 8) & BMASK, WRITEB);
        }
    else WriteQb (pa, val, WRITE);
    break;
case 3:                                                 /* triword */
    if (pa & 1) {                                       /* odd addr? */
        WriteQb (pa, val & BMASK, WRITEB);
        WriteQb (pa + 1, (val >> 8) & WMASK, WRITE);
        }
    else {
        WriteQb (pa, val & WMASK, WRITE);
        WriteQb (pa + 2, (val >> 16) & BMASK, WRITEB);
        }
    break;
    }
SET_IRQL;
return;
}

-----------------

I think this handles all the cases.

/Bob Supnik