[Simh] pdp11 i/o addressing

Sat Feb 17 08:15:00 EST 2018

And an even stronger curmudgeon warning here then.

If people today claim that VM was designed to solve the problem of 
having more addressable memory, they are utterly confused, as VM don't 
give you any more addressable memory. Addressable memory is limited by 
the size of an address in the CPU, and that does not change with or 
without virtual memory. Now, physical address space on a machine in the 
end is usually different than the size of an address in the CPU, but 
that is handled by external hardware, such as MMUs, and does not change 
how much addressable memory you have. And with MMUs, or other hardware 
devices, you can change how much physical memory is on a machine, but it 
don't change how much memory you can address in your virtual memory.

Furthermore, if you think that VMs purpose is to automatically manage 
overlays, you are also totally confused about what VM is.

To all your defenses, this is not an uncommon confusion, but really is a 
conflation of two different mechanisms and concepts.

First of all we have VM, which is virtual memory (we're not talking 
about virtual machines here...). The idea is that this gives your 
process the appearance that you have the full memory range, and it is 
private to you. That is, you have address 0 (or whatever address), and 
that appears to be a memory location where you can store data. Another 
process can also refer to address 0, but it will not be the same address 
0 as your process has. Each process has its own (full) memory space, and 
it is unique and private. Unix, or RSX, or RSTS/E, or whatever on a 
PDP-11 have virtual memory. The MMU is the device that then translates 
your virtual address to a physical address, and from there you get some 
actual storage for your virtual memory. Of course, your virtual address 
might also not translate to any physical storage at all, at a given 
time. On machines with a rather small virtual address space, but a large 
physical address space, it usually makes enough sense to either have all 
your memory mapped, or none of it. And when it is not mapped, it might 
not be in memory at all, that is, it is swapped out. Swapping in and out 
is reasonably cheap on these systems, so that you don't have much need 
to implement anything more advanced. And with reasonably cheap I mean 
that reading or writing the whole virtual memory to and from disk can be 
done in a reasonable time, and having the full virtual memory of a 
process mapped to physical memory will not allocate too much physical 
memory on the machine.

Now, if the virtual memory space is large, and/or the physical memory is 
small(ish) compared to virtual memory, then swapping becomes 
impractical, or not even feasible. This is where then second concept 
most people think about comes in: demand paging. Demand paging solves 
both the problem with not having enough physical memory to back up the 
full virtual address space, and having large enough virtual memory that 
reading and writing the full virtual memory space would take 
unreasonable amounts of time. With demand paging, only the actual memory 
pages that you refer to, or need, will be read from disk into memory 
when the process is running. Note that this is commonly used in 
combination with virtual memory. So that each process still has its own 
full, private address space. However, not all of the virtual memory is 
necessarily backed by physical memory at a given point in time. Only 
some select parts are. All other parts of your virtual memory is flagged 
as not having a mapping to physical memory, and any reference to 
unmapped addresses will cause a page fault, which will make the OS read 
in the required page of memory from disk into some part of physical 
memory, and set up the MMU to map those virtual addresses to physical 
addresses at that point. Other parts will still be unmapped, and this 
page could potentially also be dropped again without the rest of the 
process being wiped out from physical memory. But the virtual memory 
still exists.

Which then leads me to overlays. Where does overlays fit into this? 
Well, if you look at it, what overlays is is a mechanism for reading in 
only parts of memory when it actually is needed, and having it reside in 
the same virtual address space as other parts of the code (if we have a 
machine totally without virtual memory, overlays can of course also be 
used directly on the physical address space, works the same way). And 
other parts of the code might be wiped out because of demand for some 
other code, which uses the same memory. But if you look at what this 
implies, it should be obvious that overlays really are a user level 
implementation doing the same thing as demand paging does. It has 
nothing really to do with virtual memory. Demand paging does it to 
reduce the amount of physical memory needed, while overlays does it to 
reduce the amount of virtual memory needed (if you have virtual memory) 
or the amount of physical memory (if you don't have virtual memory). But 
overlays as such have nothing to do with virtual memory as such. Virtual 
memory is just the memory your process has. Overlays just make use of 
your virtual memory, without even being aware that it is virtual.

   Johnny

On 2018-02-16 20:51, Clem Cole wrote:
> curmudgeon warning below.....
> 
> On Fri, Feb 16, 2018 at 11:06 AM, Ethan Dicks <ethan.dicks at gmail.com 
> <mailto:ethan.dicks at gmail.com>> wrote:
> 
> 
>     I started on a VAX with 2MB of physical memory in a 16MB physical
>     address space but with 4GB virtual addresses.  Switching over to the
>     PDP-11 was odd from that.
> 
> 
> Sigh... I fear that is a fault of your education.
> 
> If you ask many (younger) programmers what VM was designed to solve 
> (particularly those that never memory constrained systems such as you 
> get in 8, 12, 16 or 18 bit processors), they will tell you 'So can have 
> more addressable memory.'  The problem is said programmers never 
> experienced or learned about overlays.   Conceptually, a PDP-11 can 
> allow a lot more than the 64Ks physical limit by 'swapping out' and 
> 'overlaying parts' and calling subroutines through 'thunks' [which to 
> quote my old friend Paul Hilfinger from page 427 of his book: /"an 
> onomatopoetic reference to the sound made by a pointer as it moves 
> instantaneously up and down the stack"/].  A process has be allowed to 
> be larger that 64K, but only 64K (128K on seperate I/D systems) in the 
> set up memory maps at a time.    If you need to call a subroutine to 
> (optionally) bring in the routines and its data into memory if it is not 
> already there, and then set up the map to point the routine in question.
> 
> BTW: If you play with BSD 2.11 or the like, it uses overlays to allow 
> programs to grow in size.   This was needed as people started to try to 
> move features from 4BSD and later back to the PDP-11.   At this point, I 
> believe you must have what was sometimes referred too as '17th address 
> bit - i.e. I/D space which gives you 128K bytes of mapped in memory at a 
> time.    But you can (with care) let you programs grow.
> 
> The point is that VM is a mechanism/to automatically manage overlays/.  
>   The implementation of this management gets easier if there are more 
> address bits than physical address bit, but that is the key item that is 
> happening.
> 
> Sadly, since people stopped learning about overlay management, the 
> context of what it was doing under the covers was lost.
> 
> Clem
> 
> 
> ᐧ
> 
> 
> _______________________________________________
> Simh mailing list
> Simh at trailing-edge.com
> http://mailman.trailing-edge.com/mailman/listinfo/simh
> 

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol