Author:         Kevin Kenny <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        01-Oct-2009
Post-History:   
Tcl-Version:    8.6
Tcl-Ticket:     2891616

Abstract

This TIP proposes to promote the internal calls, TclLoadFile and TclpFindSymbol to the external API, making them available to C extensions. In addition, it proposes to introduce a Tcl_FSUnloadFile to the VFS layer, removing the weird duplicate client data from TclLoadFile and wrapping all necessary data into the Tcl_LoadHandle

Rationale

In developing TDBC [308], the author of this TIP was advised to look at the way that the 'oratcl' extension contrives to build on a system where Oracle is not installed as a model for TDBC extensions. Examination of the code revealed that it operates essentially by constructing at run time a Stubs table for the routines in the Oracle client library. There is a maze of #if directives selecting whether this process is accomplished by the system calls that manage Unix .so files (dlopen and friends), Windows .DLL files (LoadLibrary and related calls), HP-UX .shl files (shl_load, etc.), and so on.

Tcl already has functionality so that a caller can abstract away all this complexity. It provides the capability in the TclLoadFile call, but this call is MODULE_SCOPE and not exported even in the internal Stubs table. For this reason, it is entirely unavailable to TEA-compliant extensions.

If this call were available, it would be feasible, in the 8.6 time frame, to bundle all the database-specific TDBC drivers with the core TDBC distribution, since things could be set up so that they will build anywhere, even in the absence of the databases where they connect.

However, this call was never fully rationalized in the VFS world. Its strange API (with both a Tcl_LoadHandle and a ClientData, and a function pointer for unloading the library) is, as several reviewers pointed out, not somehting that we want to make available to general callers. Hence, a few more routines need rework.

Specification

The TclLoadFile call shall be renamed Tcl_LoadFile and exported in the external Stubs. Its call signature shall be changed to:

EXTERN int Tcl_LoadFile( Tcl_Interp *interp, Tcl_Obj *pathPtr, const char *symbols[], int flags, void *procPtrs, Tcl_LoadHandle *handlePtr);

In this call, interp designates an interpreter for error reporting. pathPtr is an object containing the path of the library to be loaded. If pathPtr is a single name, the library search path of the current environment will be used to resolve it.
The symbols array contains a NULL-terminated set of character strings that are the names of the imported symbols. symbols may be NULL if the library is to be loaded without resolving any symbols.

The return value of Tcl_LoadFile is a standard Tcl result code. If the result is TCL_ERROR, the interpreter result will contain an appropriate error message.

On return, Tcl_LoadFile fills in the block of memory commencing at the address given by procPtrs with the addresses that correspond to the names in symbols. If a name cannot be resolved in the given library, an error will be returned and all entries in procPtrs will be invalid. procPtrs may be NULL if no symbols are to be resolved.

On a successful return, the loadHandle pointer will be a handle suitable for passing to Tcl_FindSymbol for resolving additional symbols in the library, or to Tcl_FSUnloadFile to unload the library.

The TclpFindSymbol call shall be renamed Tcl_FindSymbol and exported in the external Stubs. Its call signature shall be:

EXTERN void* Tcl_FindSymbol( Tcl_Interp *interp, Tcl_LoadHandle *loadHandle, const char *symbol);

This call searches for the given symbol in the already-loaded library identified by loadHandle. If the symbol is found, a pointer to its memory address is returned. Otherwise, NULL is returned, and an error message is left in interp (if interp is not NULL).

A new call, Tcl_FSUnloadFile shall be introduced and exported in the external Stubs.Its call signature shall be:

EXTERN int Tcl_FSUnloadFile( Tcl_Interp *interp, Tcl_LoadHandle *loadHandle);

This call unloads the library identified by loadHandle. It differs from the [unload] command in that no 'unload' procedure is called; the library is simply unloaded without first being given a chance to clean itself up. (This function is a lower-level interface used by [unload] as part of doing its work.) The return value is either TCL_OK or TCL_ERROR; when TCL_ERROR is returned, an appropriate message is left in the result of interp (if interp is not NULL).

Internals

The Tcl_LoadHandle object shall be represented internally by a structure, declared in <tclInt.h>, looking like:

 struct Tcl_LoadHandle_ {
     ClientData clientData;	/* Client data is the load handle in the
 				 * native filesystem if a module was loaded
 				 * there, or an opaque pointer to a structure
 				 * for further bookkeeping on load-from-VFS
 				 * and load-from-memory */
     TclFindSymbolProc* findSymbolProcPtr;
 				/* Procedure that resolves symbols in a
 				 * loaded module */
     Tcl_FSUnloadFileProc* unloadFileProcPtr;
 				/* Procedure that unloads a loaded module */
 }

The Tcl_FindSymbolProc and Tcl_FSUnloadFileProc data types are declared to be functions of the same type signature as Tcl_FindSymbol and Tcl_FSUnloadFile.

Virtual file systems that implement the loadFileProc are responsible for ensuring that their **Tcl_LoadHandle_ that the loadFileProc returns conforms with this convention. As far as the author has been able to determine, no non-Core filesystem provides anything but NULL for the loadFileProc. Certainly, tclvfs and trofs do not. Most other virtual filesystems layer atop tclvfs.

Reference Implementation

A reference implementation is nearing completion, and the draft code is available attached to Patch 2891616 at SourceForge.

License