Very Advanced Virtual Overlays

ADJUSTING LINK TIME PARAMETERS AT RUN TIME

You can change the global options NOEMS and NOEMSRUN and BUFSIZE at runtime. At this time you can also limit the number of EMS pages the overlay manager is allowed to use. To do this you will declare procedures that the overlay manager initialization will call during its initialization. Simply declaring these procedures is all that is necessary.

Note: Since the overlay manager is initialized before any user code is initialized, you should be careful about what Modules your code uses here since they have not been initialized if they need initialization.

PROGRAMEMS

This feature will allow you to supply the EMS memory if any to the overlay manager. Otherwise it allocates its own EMS memory. In this way you can control the EMS usage of the overlay manager.

PROCEDURE AnyName ['OVLPROGEMSINIT'](emmhandle : CARDINAL;

                                                                  startpage : CARDINAL;

                                                                  numpages : CARDINAL) : CARDINAL;

emmhandle is the EMS handle the overlay manager will use.

startpage is the logical page number the overlay manager can use.

numpages is the number of logical pages after startpage the overlay manager can use.

The word returned is the number of logical pages the overlay manager is not going to use from the pages it was allowed to use. Your program is free to use these pages of memory if needed.

EXTREMELY ADVANCED USAGE

READ: You can break things big time by playing with this!

The overlay manager can have its disk read calls intercepted. Typical usage of this can be to split the overlays into multiple files, multiple disks, compress the overlays and decompress upon loading, etc..., and any combo of the above.

This requires knowledge of how overlays are placed in the EXE file, which is discussed later.

The overlay manager requests overlays by asking for a physical offset from the start of the EXE file, and asks for X number of bytes. It is up to you to translate these addresses into whatever you may have done with the overlay data.

Overlay file structure

The linker places overlays at the end of the program memory image in the EXE file. This is not necessarily the end of the EXE file because CodeView debug information is always placed at the end of an EXE file. The end of the memory image can be computed with the information in the EXE header.

At this location will be the overlay header block. This block is PARAgraph aligned and is structured as follows.

Note: If you wish to alter the overlay data and install your own disk read function, the OvHeaderV block must remain in the location set by the linker, all overlays themselves can be altered and or moved.

OvHeaderV =

RECORD

        ovlSig : ARRAY [0..1] OF CHAR; (* SB *)

        ovlSize : LONGINT;

        ovlCount : CARDINAL;

        firstThunkSeg : CARDINAL;

        ovlGlobalOpts : GlobOverlayAttribSet; (* CARDINAL *)

        reserved1 : CARDINAL;

        reserved2 : CARDINAL;

END;

ovlSig will be the letters SB. This is a safety check.

ovlSize is the size of all overlay data in the EXE file.

ovlCount is the number of overlays in the EXE file.

firstThunkSeg is the segment address of the first thunk

segment. More on thunk segments later.

ovlGlobalOpts are the options set globally in the EDF file.

The thunk segments are where the real action is happening. The data structure is as follows.

    ThunkHeaderRec =

    RECORD

        attr : LocOverlayAttribSet;(* 0 *)

farCall : BYTE;(* 1 *)

        ovlMgrEntry : ADDRESS;(* 2 *)

        status : BYTE;(* 6 *)

        nearCall : BYTE;(* 7 *)

        thunkCount : CARDINAL;(* 8 *)

        retIP : CARDINAL;(* 10 *)

        filePos : LONGINT; (* 12 *)

        codeSize : CARDINAL;(* 16 *)

        fixupSize : CARDINAL;(* 18 *)

        nextThunkSeg : CARDINAL;(* 20 *)

        ovlRunSeg : CARDINAL;(* 22 *)

        emsPageOffs : CARDINAL;(* 24 *)

        emsPageRecSeg : CARDINAL;(* 26 *)

        reserved1 : CARDINAL;(* 28 *)

        reserved2 : CARDINAL;(* 30 *)

    END;(* 32 *)

Each overlay has its own thunk segment. These thunk segments are placed in the memory image immediately after the program PSP segment, thus they always reside in memory. All thunk header records are PARAgraph aligned.

attr are the attributes set locally for this overlay in the EDF file.

farCall is simply the BYTE 9Ah

ovlMgrEntry is the address of the overlay manager.

status is used internally while the program is executing.

nearCall is simply E8h

thunkCount is the number thunks for this overlay. NOTE: This field is overwritten with other data after the overlay manager is initialized. Each thunk is 5 bytes of data, as follows.

thunkRec =

RECORD

nearCallFarJump : BYTE; (* E8h or EAh *)

nearOffsFarSeg  : CARDINAL;

actualOvlOffset : CARDINAL;

END;

actualOvlOffset is the actual offset of the procedure in the overlay.

nearCallFarJump and the following CARDINAL will be in one of two states.

nearCall will be a near call to the farCall field in the thunkHeader for this overlay. This is the state when the overlay is not loaded in memory and ready to execute.

FarJump will be a far jump to the overlaid procedure when the overlay is loaded in memory and ready to execute.

retIP is used during execution.

filePos is the file position from the start of the EXE file to the overlay data. The overlay data is PARAgraph aligned and contains the executable code first followed by the fixup data for the overlay. Unlike EXE fixups, these fixups contain only an offset since the overlay cannot exceed 64k.

Note: This filePos is the filePos passed as a parameter to the disk read function.

codeSize is the size in bytes of the executable code.

fixupSize is the size in bytes of the fixups. The number of fixups is (fixupSize DIV 2).

nextThunkSeg is the segment address of the next thunk segment. If there are no more thunk segments the value will be zero. NOTE: This field is overwritten after the overlay manager is initialized.

ovlRunSeg is the segment the overlay is running in. If the value is zero the overlay is currently not in memory. This is true even if overlay are executing in EMS memory.

emsPageOffs is used if the overlay is loaded in EMS.

emsPageRecSeg is a value indexing an internal data structure used when the overlay is loaded in EMS.

Samples for advanced usage.

Setting overlay options normally set at link time.What you need to do is declare a public procedure, and this procedure is called by the overlay manager while it is initializing.

Note: You should be careful what your code calls, since none of your code has been initialized.

The functions of the virtual overlay manager that are used in this procedure, OVLGETBUF, OVLSETBUF and OVLSETEMSBUF can only be called in this procedure. If a direct call is made to any of them after the program code begins, each one will return an error value of -1. These functions reference values used by the overlay manager only during its initialization process, so there is no point to calling them after program code begins.

This is how you should declare the initialization procedure.

PROCEDURE OvlCmdOpts ['OVLCMDOPTS'] () : CARDINAL;

 

CONST

    NOCHANGE = 0000;

    NOEMS = 0003h;

    NOEMSRUN = 0001h;

The Return value for OvlCmdOpts is zero if no change is to bemade to EMS usage. RETURN NOEMS for disable EMS usage. NOEMSRUN to disable that feature.These options are set globally.

PROCEDURE OvlGetBuf ['OVLGETBUF'] () : LONGINT;

    Returns -1 if there was an error

PROCEDURE OvlSetBuf ['OVLSETBUF'] (BufSize : LONGINT): CARDINAL;

    Returns MAX(CARDINAL) if there was an error

PROCEDURE OvlConvAvail ['OVLCONVAVAIL'] () : LONGINT;

    Returns 0 if there an error

PROCEDURE OvlSetEMSBuf ['OVLSETEMSBUF'] (NumPages: CARDINAL): CARDINAL;

These are procedures you can call inside the overlay manager. Their declarations are as above. GetBuf and SetBuf deal with the conventional memory overlay buffer size. ConvAvail returns the memory available for the conventional overlay buffer. You cannot get this information from DOS as the overlay manager owns all DOS memory at this time.

SetEmsBuf sets a maximum number of EMS pages the overlay manager is allowed to use. It will allocate the EMS memory for itself.

WARNING:

When a program will use EMS for its own purposes, care must be taken to avoid a conflict between the Virtual Overlay Manager's use of EMS and that of the program.  The simple solution is to include the global option NOEMSRUN in the .EDF file at link-time.  This will allow the overlay manager to use EMS for caching overlays to minimize disk access, but no overlaid code will execute in EMS.  The manager saves and restores the EMS mapping context each time it accesses EMS when the NOEMSRUN option is in effect. Thus, at runtime the program can forget that the overlay manager is even using EMS.

If it is desirable to allow the overlay manager to execute overlays in EMS while the program is using EMS for its own needs, some careful planning of how the executable is built will be needed.  Of course, no program usage of EMS can coincide with overlay execution in EMS.  This means no code involved in the program's usage of EMS can be allowed to execute in EMS. In addition, the program's EMS management code must save and restore the mapping context each time it accesses EMS.  This will ensure that overlays executing there will still be properly mapped into the EMS standard page frame when code within them must run.

The Virtual Overlay Manager does not 'hook' int 67h (the EMS device driver's interface to programs). Therefore, it is not aware of any usage of EMS other than its own.  Currently, it only uses the standard EMS page frame (64k) and works fine with EMS versions 3.2 and higher.

It is feasible for programs that use EMS directly and require at least version 4.0, to use pages that are mappable outside the standard page frame for their own EMS needs.  This would avoid the necessity of the program's EMS management code having to save and restore the mapping context of the standard page frame each time it accesses EMS.  Then the overlay manager can continue to execute overlaid code in EMS without fear of conflict.  This 'space diversity' technique is worth considering. Be aware, however, that memory managers such as QEMM-386 and 386 MAX often are configured to use the mappable pages outside the standard page frame for UMBs (upper memory blocks).

PROCEDURE OvlProgEmsInit ['OVLPROGEMSINIT'] (emmhandle : CARDINAL;

startpage : CARDINAL;

numpages : CARDINAL): CARDINAL;

 

PROCEDURE OvlEmsNeed ['OVLEMSNEED'] (): CARDINAL;

OvlEmsNeed returns the amount of EMS memory need if the overlay manager is you put all overlays in EMS memory. Use this to determine how much memory to allocate for both your program and the overlay manager.

OvlProgEmsInit gives the overlay manager the EMS memory handle starting EMS page and number of pages it can use. This function will only have effect *IF* you specified PROGRAMEMS in the linker overlay definition.

These functions can only be called in OvlCmdOpts overlay initialization procedure.

--------------------

Note:

Most users of virtual overlays will not need to supply a custom disk read procedure. To write one requires intimate knowledge of how overlays are structured by the linker in the executable file.

  

The primary use of this capability is to split the overlay data across multiple files, or to compress the overlay data to save space.

TYPE

OvlReadFunc = PROCEDURE(OvlyFilePos: LONGINT;

LoadAddr: ADDRESS;

NumBytes: CARDINAL) : CARDINAL;

 

PROCEDURE OvlSetReadFunc ['OVLSETREADFUNC'] () : ADDRESS;

When you declare a function of this name, it will automatically be called by the overlay manager during its initialization. The return value is the address of the function you supply that will supply the overlay data to the overlay manager.

Your disk read function should be of type OvlRealFunc.

OvlyFilePos - is the file position the overlay had when the linker originally created the file.

LoadAddr -  is the address your are supposed to place the data.

NumBytes - is the amount of data your are supposed to supply.

The return value is the number of byte of data actually given by your procedure. If less than requested the overlay manager will treat the as a disk read error and raise an exception.