<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<div class="moz-cite-prefix">On 16-Feb-18 14:51, Clem Cole wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAC20D2NgAew9CbkitU6sA14hSNsaA1iaNHRCKdt-q0oikbd8Og@mail.gmail.com">
<div dir="ltr">
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff">curmudgeon warning below.....</font></div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Feb 16, 2018 at 11:06 AM,
Ethan Dicks <span dir="ltr"><<a
href="mailto:ethan.dicks@gmail.com" target="_blank"
moz-do-not-send="true">ethan.dicks@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
I started on a VAX with 2MB of physical memory in a 16MB
physical<br>
address space but with 4GB virtual addresses. Switching
over to the<br>
PDP-11 was odd from that.<br>
</blockquote>
<div><br>
</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff">Sigh... I fear that is a fault of your
education. </font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff"><br>
</font></div>
<div class="gmail_default" style=""><font style=""
color="#0000ff"><font face="arial, helvetica,
sans-serif">If you ask many (younger) programmers what
VM was designed to solve (particularly those that
never memory constrained systems such as you get in 8,
12, 16 or 18 bit processors), they will tell you 'So
can have more addressable memory.' The problem is
said programmers never experienced or learned about
overlays. Conceptually, a PDP-11 can allow a lot
more than the 64Ks physical limit by 'swapping out'
and 'overlaying parts' and calling subroutines through
'thunks' [which to quote my old friend Paul Hilfinger
from page 427 of his book: </font><i
style="font-family:arial,helvetica,sans-serif">"an
onomatopoetic reference to the sound made by a pointer
as it moves instantaneously up and down the stack"</i><font
face="arial, helvetica, sans-serif">]. A process has
be allowed to be larger that 64K, but only 64K (128K
on seperate I/D systems) in the set up memory maps at
a time. If you need to call a subroutine to
(optionally) bring in the routines and its data into
memory if it is not already there, and then set up the
map to point the routine in question.</font></font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff"><br>
</font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff">BTW: If you play with BSD 2.11 or the
like, it uses overlays to allow programs to grow in
size. This was needed as people started to try to move
features from 4BSD and later back to the PDP-11. At
this point, I believe you must have what was sometimes
referred too as '17th address bit - i.e. I/D space which
gives you 128K bytes of mapped in memory at a time.
But you can (with care) let you programs grow. </font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff"><br>
</font></div>
<div class="gmail_default" style=""><font style=""
color="#0000ff"><font face="arial, helvetica,
sans-serif">The point is that VM is a mechanism<i> to
automatically manage overlays</i>. The
implementation of this management gets easier if there
are more address bits than physical address bit, but
that is the key item that is happening.</font></font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#330033"><br>
</font></div>
</div>
</div>
</div>
</blockquote>
<font color="#330033">You have to pick a frame of reference. There
were overlays before VM. But people may have experience VM before
overlays, or vice-versa. It is true that overlays aren't well
known or understood by the younger crowds.<br>
<br>
But, your observation that "</font><font color="#330033"><font
style="" color="#0000ff"><font face="arial, helvetica,
sans-serif">VM is a mechanism<i> to automatically manage
overlays</i></font></font>" is an over-simplification.<br>
<br>
Overlays were used to compensate for limited virtual address
space. VM expands the limit. But, more than that, it does it
more efficiently than overlays. Overlays generally are
implemented in user mode by a linker (task builder). They are not
visible to the OS, so limit sharing. In addition, they force code
(in the overlay areas) to be writable. And the overlay loader
(invoked by thunks) makes cross-overlay calls considerably more
expensive than an ordinary call - even when the target call is
frequent. They work better for code than for data; identifying
& swapping out writable data in user mode for an overlay is
non-trivial. (Sane overly systems didn't encourage this.)
Although there were some attempts to automate generating an
overlay structure, getting reasonable performance requires a lot
of analysis and programmer effort. This grows (at least)
exponentially as the size and complexity of a program grows and
the overlay structure moves from a partition or two to a tree
structure. And worse with writable overlays.<br>
<br>
VM expands the address space <b>more efficiently</b>. The OS
knows about it; sharing is more efficient; code can be read-only.
Physical memory can be used for caching recently used pages,
reducing I/O. And a programmer can do less (not to be confused
with no) work.<br>
<br>
</font><font color="#330033"><font color="#330033">Overlays had a
place - but VM does a lot more than automatically manage them.
It provides much finer granularity than is practical for
overlays, does so more efficiently (at both the application and
system level), and used thoughtfully saves programmer effort.
Overlays are, in many respects, a "poor man's VM". When you
don't have paging hardware, overlays can do a lot. But at a
steep price. Initially, paging hardware wasn't cheap, neither
was the OS complexity to implement it well. And it took a long
time to figure out how to do it well. (Er, well enough. We're
still learning.)<br>
<br>
</font>That said, VM used carelessly can perform much, much worse
than an overlay structure. An early version of an operating that
I won't name decided that "everything is a page fault". Including
disk I/O. So to run a program, you associated it with an address
space, and - well. Page fault reading the start address. PF
reading the instruction at the start address. PF reading the
operand of the instruction at the start address.... It was
academically beautiful - but hopelessly non-performant. As were
programs that figured that since memory is free, might as well
allocate GB sparse arrays.<br>
<br>
I remember when VAX's 32-bit VM was supposed to be "infinite" and
overlays "obsolete"; divided into 4 segments, it didn't take long
before 1GB was "too small", and thoughts of overlays had a brief
resurgence. <br>
<br>
Still, I miss telling Link-10 to provide a plot (on a drum pen
plotter) of the overlay structure of a really big program, like
SPSS. It could tell you a lot about how the developer thought it
worked. <br>
<br>
I don't miss the RSX-11/TKB overlays, even though they did force
programmers to think about their code.<br>
<br>
And I cede curmudgeon points and status to no one :-)<br>
<br>
("Thunk" is also used for the code used in call by name - which
has another hollow sound.)<br>
<br>
</font>
<blockquote type="cite"
cite="mid:CAC20D2NgAew9CbkitU6sA14hSNsaA1iaNHRCKdt-q0oikbd8Og@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff">Sadly, since people stopped learning
about overlay management, the context of what it was
doing under the covers was lost.</font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff"><br>
</font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><font
color="#0000ff">Clem</font></div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
</div>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>