Tcl Database Connectivity (TDBC)

Obviously the first thing that any connectivity layer has to offer is the ability to select a database. The way databases are named is quite specific to the database manager, as is the way access is negotiated (credentials such as user name and password may be required, session keys may be negotiated for privacy and authentication, and so on). All of this machinery is formally out of scope for this specification. Similarly, the machinery of database administration (at least at the level of creating/deleting entire databases, managing the physical layer, and authorizing clients) is presumed to be already taken care of. We need merely specify that a connectivity layer must provide at least one command that accepts arguments describing the desired connection and returns a database handle - defined to be an ensemble through which interactions with the given database instance will take place. Here, database instance means the database, or databases, that the given handle can access; rather a circular definition. In many SQL systems, it is possible for a single connection to access several "databases" managed by SQL CREATE DATABASE statments, or several "tablespaces" or similar constructs. We presume that database module implementors will know what is appropriate for their systems, and intentionally leave this particular matter somewhat vague. Database handles are Tcl ensembles, meaning that they are commands that support subcommands. Other ensembles, such as statement handles, are also defined in this specification. Any of the ensembles MAY support abbreviation of its subcommands according to the rules defined by Tcl_GetIndexFromObj; nevertherless, code that uses the database interface SHOULD spell out subcommands in full. Many of the subcommands are expected to take options in Tcl's usual syntax of: ?-option ?value?? ?-option value?... In all of the places where this syntax is expected, a database module MAY support abbreviation of options according to the rules of Tcl_GetIndexFromObj(); once again, code that uses the interface SHOULD spell out options in full. All the database objects (connections, statements and result sets) are "duck typed" - that is, "If it walks like a duck and quacks like a duck, I would call it a duck. (James Whitcomb Riley)." In other words, the ensembles may be implemented using any available functionality as long as the result is that they use the interfaces described. Nevertheless, as a convenience to implementors, a set of base classes, called tdbc::connection, tdbc::statement, and tdbc::resultset, SHALL be provided using Tcl's native object orientation as described in . Certain advantages will accrue to database implementors by using these base classes. In particular, the tdbc::* classes SHALL do all the bookkeeping needed to determine what statements and result sets are open, SHALL provide the internal iterators allrows and foreach, SHALL implement the transaction method on connections, and SHALL ensure that the close method on the objects functions the same as renaming the object to the null string. Once a handle is returned, there are a number of session-level attributes that may be controllable. Every database handle MUST provide a configure subcommand that takes the form: dbHandle configure ?-option ?value?? ?-option value?... This configuration process is analogous to configuring a Tk widget. If there are no arguments presented to configure, the return value MUST be a list of alternating options and values describing the configuration parameters currently in effect. If a single argument is presented, it MUST be the name of a configuration parameter, and the return value MUST be current value for that parameter. Finally, if more than one argument is presented, they MUST be a list of alternating parameter names and values. This last form is an order to set the given parameters to the given values. The connectivity layer SHOULD implement the following parameters, and MAY implement others: (Note: an earlier draft of this TIP specified a -autocommit option; this option has been removed because it is redundant with the transaction management primitives below.) -encoding nameRequests that the encoding to be used in database communication protocol be changed to the one given by name, which MAY be any name acceptable to the [encoding] command. A well-designed database interface SHOULD NOT require this command; however, some backends make it virtually inevitable that mid-stream changes of encodings will be required.-isolation levelRequests that transactions performed on the database be executed at the given isolation level. The acceptable values for level are:readuncommittedAllows the transaction to read "dirty", that is, uncommitted data. This isolation level may compromise data integrity, does not guarantee that foreign keys or uniqueness constraints are satisfied, and in generall does not guarantee data consistency.readcommittedForbids the transaction from reading "dirty" data, but does not guarantee repeatable reads; if a transaction reads a row of a database at a given time, there is no guarantee that the same row will be available at a later time in the same transaction.repeatablereadGuarantees that any row of the database, once read, will have the same values for the life of a transaction. Still permits "phantom reads" (that is, newly-added rows appearing if a table is queried a second time).serializableThe most restrictive (and most expensive) level of transaction isolation. Any query to the database, if repeated, will return precisely the same results for the life of the transaction, exactly as if the transaction is the only user of the database.readonlyBehaves like serializable in that the only results visible to the transaction are those that were committed prior to the start of the transaction, but forbids the transaction from modifying the database.A database that does not implement one of these isolation levels SHOULD instead use the next more restrictive isolation level. If the given level of isolation cannot be obtained, the database interface MUST throw an error reporting the fact. The default isolation level SHOULD be at least readcommitted.A database interface MAY forbid changing the isolation level when a transaction is in progress.-timeout msRequests that operations requested on the database SHOULD time out after the given number of milliseconds, if such an option is supported by the underlying connectivity layer.-readonly booleanNotifies that the application will, or will not, limit its activity to operations that do not modify the content of the database. This option MAY have the effect of adjusting the transaction isolation level. The command that returns a database handle SHOULD also accept these options. A database handle MUST implement the three subcommands starttransaction, commit and rollback: dbHandle starttransaction Begins an atomic transaction on the database. If the underlying database does not implement atomic transactions or rollback, the starttransaction subcommand MUST throw an error reporting the fact. If the underlying database does not implement nested transactions, a starttransaction command that is executed when there is a transaction already in progress (started, but neither committed nor rolled back) MUST result in an error. dbHandle commit Commits a transaction to the database, making the changes durable. dbHandle rollback Rolls back a transaction against the database, cancelling any changes made during the transaction. Statements executed against the database when no transaction is in progress (before the first starttransaction or after all started transactions have been either committed or rolled back) SHOULD be auto-committed; that is, each such statement SHOULD be executed as if a starttransaction command preceded the statement and a commit command followed it (assuming that the statement succeeded; errors should result in rollback of course). These commands are provided primarily to support the construction of higher-level operations. In particular, most simple transactions against a database can be handled using the transaction command: dbHandle transaction script Executes the given script with transaction isolation. In this command, the dbHandle argument is a handle to a database connection, and the script argument is a Tcl script to be evaluated in the calling scope. The script is treated as a single atomic database transaction. The starttransaction command is executed against the given database connection, and then the script is evaluated. If it completes successfully (TCL_OK), the transaction SHALL be committed to the database. If it fails, (TCL_ERROR), the transaction SHALL be rolled back and not visible to other users of the database. TCL_BREAK, TCL_CONTINUE and TCL_RETURN SHALL result in a commit and subsequently rethrow the same exception status outside the transaction. Exception status codes other than these five SHALL rollback the transaction and be rethrown. (Note: Scripts inside a transaction command SHOULD avoid use of the return -code or return -level operations. If a script returns from a transaction, with any combination of return options, the transaction SHALL be committed.) Just as with starttransaction, if a [dbHandle transaction] command is executed while another transaction is already in progress, it is requesting nested transaction semantics. A database handle to an engine that supports nested transactions MUST treat this case correctly; a database handle to an engine that does not support nested transactions (including one that does not support transactions at all) MUST throw an error. The transaction subcommand SHALL be provided by the tdbc::connection base class; database interfaces that use the TclOO features and the TDBC base classes do not need to implement it. A database handle MUST implement the command: dbHandle close This command MUST dismiss the connection to the database and is expected to clean up the system resources associated with it. If there is an uncommitted transaction, it SHOULD be rolled back. Any handles to other objects associated with the database SHOULD become invalid. A database interface also SHOULD perform the same actions if a handle is deleted by means of the rename command. (Interfaces that are implemented in Tcl may be notified of this action by creating a deletion trace with trace add command.) It is recognized that command deletion traces present difficulties in situations like namespace and interpreter deletion; the close subcommand shall therefore be considered the preferred way to terminate connections. A database interface SHOULD attempt to arrange, if possible, to rollback unfinished transactions and clean up on process exit. In particular, if the underlying database engine supports transactions, it SHOULD be considered an error to commit any work that remains uncommitted on process exit. The close command SHALL be provided by the tdbc::connection base class; database interfaces that use the TDBC base classes do not need to implement it. The base class implementation destroys the object using my destroy. As a result, any statements obtained from the connection are also destroyed, since they are stored in a namespace that is subordinate to the connection's namespace. The destructor of the connection object is expected to close the underlying database connection and release any system resources associated with it. A database handle must support the prepare command, which has the syntax: dbHandle prepare SQL-code The SQL-code argument is a SQL statement that is to be executed against the given database connection. This command does not execute the statement directly; rather, it prepares to execute the statement, possibly performing tasks such as code compilation and query optimisation. The database interface MUST support substitutions in SQL-code. Each substitution request has the form :variableName. That is, each substitution request begins with a literal colon (:), followed by a letter or underscore, followed by zero or more letters, digits, or underscores. The database interface is responsible for translating from this syntax to whatever the underlying engine requires. Typical strings required in database interfaces are :name, :number, @name, @number, and ?. The return value from the prepare command is a statement handle, discussed under "The statement interface" below. Rationale. The choice of the colon deserves some discussion. It would surely be more natural for Tcl to use a literal dollar sign to introduce a variable name. This choice, however, seems unwise, since several databases (most notably Oracle) allow the use of table and view names that contain dollar signs. While it might be possible to continue to use these while allowing for variable substitution (for instance, by mandating that table or view names with dollar signs be enclosed in double quotes), it seems unnatural. The colon is syntax that is recognized by JDBC, ODBC, and Oracle's native API, and as such will be familiar to most SQL programmers and unlikely to collide with native syntax. The requirement to support prepared statements is intended to guard against SQL insertion attacks. An interface to a database whose native API does not support prepared statements MUST simulate them. In particular, when the execute command is executed on a statement, substitution must be performed in a safe fashion with whatever magic quoting is required. In any case, magic quoting should be regarded as an infelicitous expedient and avoided if at all possible. If a database interface uses the tdbc::connection base class, then a prepare method will be provided for it. If this method is not overridden, then the database interface MUST arrange that the constructor of the connection sets the instance variable, statementClass, to the fully qualified name of the command that constructs statements. The prepare method SHALL invoke that command with a call of the form: statementClass create handle connectionHandle sql where handle is the name of the new statement being created, connectionHandle is the handle to the connection creating it, and sql is the SQL statement being prepared. A second way to prepare statements is to prepare a stored procedure call. If a database interface supports stored procedures, it MUST support the preparecall command: dbHandle preparecall call call is a string that describes a call to a stored procedure. It takes the form: ?:varName =? procName ( ?:varName? ?, varName?... ) The result of the preparecall command is a statement handle. The statement handle may be used just as any other statement handle. The preparecall method SHALL not be provided in the tdbc::connection base class; individual database interfaces are expected to do so. They MAY do so by rewriting the call to whatever syntax the native database requires, and delegating to the prepare method to prepare that, or they MAY instead prepare another ensemble. (See "The TDBC base classes" below for details of integrating this mechanism with the base classes.) A database handle MUST support the following two calls: dbHandle allrows ?-as lists|dicts? ?-columnsvariable varName? ?--? sql ?dictionary? This command prepares the SQL statement given by the sql parameter, and immediately executes it. Variable substitutions inside the sql parameter are satisfied from the given dictionary, if one is supplied, and from variables in the caller's scope otherwise. The -as option determines the form of the result, and the -columnsvariable option provides an optional variable in which the names of the result columns will be stored. Upon termination of the command, whether successful or unsuccessful, the prepared statement is closed. The command returns a list of the rows returned by the affected statement. (If the affected statement does not yield a set of rows, the return value from the allrows command is an empty list.) This command MUST function the same way as preparing the statement explicitly, executing the statement allrows call (see below) on the resulting statement handle, and then (irrespective of whether the operation succeeeded) destroying the statement handle. dbHandle foreach?-as lists|dicts? ?-columnsvariable varName? ?--? sql ?dictionary? varName script This command prepares the SQL statement given by the sql parameter, and immediately executes it. Variable substitutions inside the sql parameter are satisfied from the given dictionary, if one is supplied, and from variables in the caller's scope otherwise. The -as option determines the form of the result, and the -columnsvariable option provides an optional variable in which the names of the result columns will be stored. For each row returned by the given statement, the given varName is set to a list or dictionary containing the returned row, and the given script is executed in the caller's scope. Upon termination of the command, whether successful or unsuccessful, the prepared statement is closed. This command MUST function the same way as preparing the statement explicitly and then executing the statement foreach call on the resulting statement handle. Both of these commands SHALL be provided in the tdbc::connection base class. A database handle MUST support the statements command: dbHandle statements This command MUST return a list of the statements that have been prepared by means of [dbHandle prepare] but not yet dismissed using [statementHandle close]. Likewise, a database handle MUST support the resultsets command: dbHandle resultsets This command MUST return a list of the result sets that have been returned (by executing statements, or by querying metadata) and have not yet been dismissed using [resultSetHandle close]. Both of these commands SHALL be provided in the tdbc::connection base class. Using the base class implementations imposes certain restrictions on derived classes. (See "The TDBC base classes" below for details of integrating this mechanism with the base classes.) A database interface MUST provide a way of enumerating the tables in the database. The syntax for querying tables MUST be: dbHandle tables ?matchPattern? The optional argument matchPattern, if supplied, is a pattern against which the table names are to be matched. The database interface MUST recognize the SQL wildcards % and _ in the pattern. A database interface MUST provide a way of enumerating the columns in a database table. The syntax for querying columns MUST be: dbHandle columns tableName ?matchPattern? The return value from the tables and columns commands MUST be a dictionary. The keys of the dictionary MUST be the names of the tables in the database, or respectively the columns in the given table. The values stored in the dictionary returned from the tables command MUST be dictionaries. The keys and values of these dictionaries, nevertheless, are implementation-defined; only the keys are mandated in this specification. The values stored in the dictionary returned from the columns command MUST themselves be dictionaries. These subdictionaries MUST include the keys, type, precision, scale, and nullable. The type value MUST be the data type of the column, and SHOULD be chosen from among the standard types bigint, binary, bit, char, date, decimal, double, float, integer, longvarbinary, longvarchar, numeric, real, time, timestamp, smallint, tinyint, varbinary, and ''varchar'. The precision and scale values SHOULD give the precision and scale of the column, and the nullable value SHOULD give a boolean value that represents whether the given column can contain NULL values. Other keys MAY be included in the subdictionaries returned from tables and columns, and SHALL be added to this document (as optional columns) on request from the implementors of database interfaces. The statement handle returned from the prepare command on a database interface must itself be an ensemble. The following subcommands MUST be accepted: statementHandle params Requests a description of the names and expected data types of the parameters to the given statement. The return value from the params command MUST be a dictionary whose keys are the names of the parameters and whose values are themselves dictionaries. The keys of the subdictionaries MUST include name, type, precision, scale, and nullable. They are interpreted in the same way as those of the columns subcommand to a database interface (shown above). The subdictionaries also MUST include the key, direction, whose value identifies the direction of parameter transmission, and MUST be chosen from among in, out and inout. statementHandle execute ?dictionary? Executes a statement against a database. Any variable substitution present in the SQL that was provided when the statement was created MUST be performed at this time. The variable values MUST be obtained from the given dictionary, if one is supplied. If the dictionary does not contain a key equal to a variable name in the statement, a NULL value MUST be provided. If the dictionary argument is omitted, the variable values MUST be obtained from the scope in which the execute command was evaluated. Any variable that is undefined in that scope must be replaced with a NULL value. An array variable provided to a substituent MUST result in an error. Read traces against the substituted variables SHOULD fire, in left-to-right order as they appeared in the SQL statement. The result of the execute command SHOULD be a result set, as defined under "The result set interface" below. This method is provided by the tdbc::connection base class. In the base class, it works by creating an instance of the class whose name appears in the statementClass instance variable. See "The TDBC base classes" below for the details of how the derived classes should be implemented to avail themselves of this method. statementHandle close Announces that a statement is no longer required, and frees all system resources associated with it. The close command MAY invalidate any result sets that were obtained by the params and execute commands. As with database connections, the database interface SHOULD also clean up if a statement handle is removed with [rename $statement {}]. Once again, it is recognized that the strange side effects of namespace and interpreter deletion may make this cleanup impossible in some interfaces, so close SHALL be considered the standard means of discarding statements. The close command SHALL be provided in the tdbc::statement base class. Database interfaces that use the TDBC base classes do not need to implement it. The base class implementation destroys the object using my destroy. As a result, any result sets obtained from the statement are also destroyed, since they are stored in a namespace that is subordinate to the statement's namespace. The destructor of the statement object is expected to release any system resources associated with it. The syntax described so far presumes that the database interface can determine the expected types of the variables that appear in a prepared statement, or at the very least can accept some sort of variant type and perform automatic type coercion. This requirement does not seem horribly onerous at first inspection, since SQLite allows for "everything is a string" parameters; ODBC offers parameter introspection via the SQLDescribeParam call; and JDBC offers it via the getParameterMetaData method of the PreparedStatement interface. Nevertheless, a deeper examination discovers that in at least ODBC, a driver is allowed to fail to offer SQLDescribeParam. Inspection of the JDBC-ODBC bridge reveals that in this case, JDBC will return a ParameterMetaData object that throws a SQLException on any attempt to query specific data. The result is that, while the APIs to introspect parameter types are available, they may be unusable against a particular database engine. In these cases, a backup is needed. For this reason, a database interface MUST support allowing the user to specify types of the parameters of a prepared statement. The syntax for doing so MUST be: statementHandle paramtype paramName ?direction? type ?precision? ?scale? Defines that the parameter identified by paramName in the given statement is to be of type type. The type MUST be chosen from among the names bigint, binary, bit, char, date, decimal, double, float, integer, longvarbinary, longvarchar, numeric, real, time, timestamp, smallint, tinyint, varbinary, and varchar. (Rationale: These types appear to suffice for ODBC, and we can always come back and extend them later if needed.) The precision of a parameter defines the number of characters or digits that it requires, and its scale defines the number of digits after the decimal point, if neeeded. A database interface MAY allow negative numbers for scale in contexts where they make sense. For example, a scale of -3, if allowed, SHOULD indicate that quantities in the given column are all multiples of 1000. The precision and scale are not required by all types. A direction must be one of the words, in, out or inout. It specifies that the given parameter is an input to the statement, an output from the statement, or both. It is usually meaningful only in stored procedure calls. Default is in, unless the parameter appears on the left-hand side of an equal side in a stored procedure call, in which case the default is out. ICRzdGF0ZW1lbnQgcGFyYW10eXBlIG5hbWUgdmFyY2hhciA0MA==ICRzdGF0ZW1lbnQgcGFyYW10eXBlIGJhbGFuY2UgaW4gZGVjaW1hbCAxMCAyICRzdGF0ZW1lbnQgcGFyYW10eXBlIHRyYW5zYWN0aW9uRGF0ZSB0aW1lc3RhbXA= Implementors of database APIs SHOULD make every effort to do appropriate type introspection so that programmers can avoid needing to include explicit type information in their SQL statements. A statement handle MUST support the following two calls: statement allrows ?-as lists|dicts? ?-columnsvariable varName? ?--? ?dictionary? This command executes the given statement. Variable substitutions inside the statement are satisfied from the given dictionary, if one is supplied, and from variables in the caller's scope otherwise. The -as option determines the form of the result, and the -columnsvariable option provides an optional variable in which the names of the result columns will be stored. Upon termination of the command, whether successful or unsuccessful, the prepared statement is closed. The command returns a list of the rows returned by the affected statement. (If the affected statement does not yield a set of rows, the return value from the allrows command is an empty list.) This command MUST function the same way as executing the statement explicitly (with the given dictionary argument if one is supplied), executing the resultset allrows call (see below) on the resulting result set, and then (irrespective of whether the operation succeeeded) destroying the result set. statement foreach?-as lists|dicts? ?-columnsvariable varName? ?--? ?dictionary? varName script This command executes the given statement. Variable substitutions inside the statement are satisfied from the given dictionary, if one is supplied, and from variables in the caller's scope otherwise. The -as option determines the form of the result, and the -columnsvariable option provides an optional variable in which the names of the result columns will be stored. For each row in the result set, the given varName is set to a list or dictionary containing the returned row, and the given script is executed in the caller's scope. Upon termination of the command, whether successful or unsuccessful, the result set is closed. This command MUST function the same way as executing the statement explicitly, executing the resultset foreach call on the resulting statement handle, and then (irrespective of whether the operation succeeded) closing the result set. Both of these commands SHALL be provided in the tdbc::statement base class. Result sets represent the results of operations performed on the database. A preferred implementation for large result sets is that they be implemented as database cursors, so that it is possible to iterate over result sets that will not fit in memory. A result set MUST be an ensemble. The following subcommands MUST be accepted: resultSetHandle rowcount Determines the number of rows affected by a SQL statement such as INSERT, DELETE or UPDATE. This count MUST be returned as an integer. It should not be confused with the number of rows in the result set. A database interface need not provide any interface to determine the latter number (often, the only way to determine it is to read all the rows). For this reason, the rowcount command MAY return an empty string, or a non-positive number, for SELECT operations (and any other operations that do not modify rows of the database). resultSetHandle columns Determines the set of columns contained in the result set. The set of columns is returned simply as a list of column names, in the order in which they appear in the results. resultSetHandle nextrow ?-as lists|dicts? ?--? variableName (This interface SHALL be provided by the tdbc::resultset base class. The default implementation SHALL delegate to either the nextlist or nextdict methods, below. Fetches a row of data from the result set and stores it in the given variable in the caller's context. If -as dicts is specified (the default), the row MUST be represented as a dictionary suitable for use with the dict command. The keys in the dictionary SHALL be the column names, and the values SHALL be the values of the cells. If no rows remain, the 'nextrow command MUST store an empty dictionary. If a cell in the row is NULL, the key MUST be omitted from the dictionary. A database interface MUST NOT use a special value of any kind to represent a NULL in a dictionary. If -as lists is specified, the row MUST be represented as a list of values, in the order in which they appear in the query. (If the statement is a stored procedure call, the values comprise all the out or inout parameters.) If no rows remain, the nextrow command MUST store an empty list. If a cell in the row is NULL, an empty string MUST be stored as its value. The return value of nextrow MUST be 1 if a row has been returned, and 0 if no rows remain in the result set. In the result set, values of type bigint, bit, decimal, double, float, integer, numeric, real, smallint, and tinyint MUST receive their natural representation as decimal numbers. Ideally, they should be returned as "pure" numbers with their string representations generated only on demand. Values of type char, longvarchar and varchar MUST be returned as Tcl strings. A database interface implemented in C MUST take care that all strings are well-formed UTF-8. Values of type date and timestamp MUST be returned as a numeric count of seconds from the Tcl epoch; if necessary, this count may have a decimal point and an appropriate number of additional decimal places appended to it. Values of type time MUST be returned as a integer count of seconds since midnight, to which MAY be appended a decimal point and a fraction of a second. Values of type binary, longvarbinary and varbinary MUST be returned as Tcl byte arrays. Rationale: Dictionaries and lists are both useful in representing the result set rows. Dictionaries allow for a ready distinction between NULL values in a database and any other string. With any scheme where values that can include NULLs can appear in Tcl objects, the problem arises that NULL must be distinguished from any other string, particularly including the empty string and the word "NULL". The lack of such a distinction has led to several ill-advised proposals, such as , for representing NULLs in Tcl. These alternatives founder on the principle of "everything is a string". The NULL value is not any string. Dictionaries also have the advantage that results can be addressed by name rather than by position. On the other hand, lists are convenient when formatting tabular results from ad hoc queries. The brevity of code that can be achieved with them is also attractive. For this reason, this TIP requires both formats to be made available. resultSetHandle nextdict variableName resultSetHandle nextlist variableName These two calls are precisely equivalent to calls to the nextrow command with the -as dicts and -as lists option respectively. A database interface MUST provide both of these, and they are the fundamental means for retrieving rows from the result set. resultSetHandle close Dismisses a result set and releases any system resources associated with it. As with statements and database connections, the database interface SHOULD also clean up if a resut set handle is removed with [rename $statement {}]. Once again, it is recognized that the strange side effects of namespace and interpreter deletion may make this cleanup impossible in some interfaces, so close SHALL be considered the standard means of discarding result sets. The close command SHALL be provided by the tdbc::resultset base class. The base class implementation destroys the object using my destroy. The destructor of the result object is expected to release any system resources associated with it. A result set handle MUST support the following two calls: resultset allrows ?-as lists|dicts? ?-columnsvariable varName ?--? This command executes the nextrow command repeatedly, producing a list of dictonaries or of lists (according to the value of the -as option). The allrowscommand returns the resulting list. Optionally, the names of the columns of the result set are also stored in the named variable given by the -columnsvariable option. statement foreach?-as lists|dicts? ?-columnsvariable varName? ?--? varName script This command optionally stores the names of the columns of the result set in the variable designated by the -columnsvariable option. It then executes the nextrow command repeatedly until all rows of the result set have been processed. The nextrow command receives the given varName and -as option, and stores the row in the named variable. For each row processed, the given script is executed in the caller's scope. Both of these commands SHALL be provided in the tdbc::resultset base class. Most implementations of database drivers SHOULD, as mentioned before, use Tcl objects (as in ) that inherit from the tdbc::connection, tdbc::statement and tdbc::resultset classes. The foregoing discussion has described the user-visible methods that are provided by doing so (and must otherwise be implemented). This section is directed to the driver implementor, and discusses certain necessary housekeeping issues. However a database connection object is constructed, its constructor will need to seize resources (such as opening a database connection to the underlying database system). If the bookkeeping done by the base classes is to work correctly, initialization of the tdbc::connection base class needs to happen before external resources are seized. In addition, if the prepare method is not overloaded (and the driver SHOULD NOT have to overload it), the name of the class that implemements the statement interface needs to be provided at this time. The recommended sequence for connection construction is: ICBjb25zdHJ1Y3RvciBhcmdzIHs=ICAgICAgbmV4dDsgICAgICAgICAgICAgICAgICAgICAgICAgIyBJbml0aWFsaXplIHRkYmM6OmNvbm5lY3Rpb24=ICAgICAgbXkgdmFyaWFibGUgc3RhdGVtZW50Q2xhc3M=ICAgICAgc2V0IHN0YXRlbWVudENsYXNzIDo6d2hhdGV2ZXI7IyBUZWxsIHRkYmM6OmNvbm5lY3Rpb24gd2hhdA==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIyBjbGFzcyBtdXN0IGJlIGluc3RhbnRpYXRlZCBieQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIyB0aGUgJ3ByZXBhcmUnIG1ldGhvZA==ICAgICAgbXkgaW5pdCB7Kn0kYXJncyAgICAgICAgICAgICAgIyBQZXJmb3JtIGltcGxlbWVudGF0aW9uLXNwZWNpZmljICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIyBpbml0aWFsaXphdGlvbg==ICB9 Some database interfaces have a different API to stored procedures than to ordinary SQL statements. These databases may need a separate type of statement object from the one that implements ordinary statements. This object can be managed as a statement owned by the connection by using a prepareCall method that looks like: ICBtZXRob2QgcHJlcGFyZUNhbGwge2NhbGx9IHs=ICAgICAgbXkgdmFyaWFibGUgc3RhdGVtZW50U2VxOyAgICMgUHJvdmlkZWQgaW4gdGhlICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICMgdGRiYzo6Y29ubmVjdGlvbiBiYXNlIGNsYXNzICAgICAgcmV0dXJuIFtwcmVwYXJlZFN0YXRlbWVudENsYXNzIGNyZWF0ZSBcICAgICAgICAgICAgICAgICAgU3RtdDo6W2luY3Igc3RhdGVtZW50U2VxXSBbc2VsZl0gJGNhbGxdICB9 In this call, preparedStatementClass is the name of the class that implements prepared statements. Its constructor is expected to accept two arguments: the handle to the database connection, and the prepared statement that was passed to prepareCall. Placing the resulting object inside the Stmt namespace under the current object (this namespace is created by the constructor of tdbc::connection) allows for its destruction to be sequenced correctly when the connection is destroyed. The methods that a derived class from tdbc::connection MUST implement are prepareCall, begintransaction, commit, and rollback. In addition, system resources belonging to the connection itself MUST be cleaned up by a destructor or by a deletion callback at C level. (Statements and result sets MUST not be deleted then; the base classes take care of that.) See "Best practices for memory management" below for further discussion. The class that implements a statement SHOULD normally inherit from the tdbc::statement base class. Its constructor accepts the connection handle and the SQL statement to prepare. The constructor is responsible for invoking the base class constructor with next, setting an instance variable resultSetClass to the name of the class that implements its result set, and then preparing the statement. (The constructor is invoked by the prepare method of '''tdbc::connection.) A sample constructor looks like: ICBjb25zdHJ1Y3RvciB7Y29ubmVjdGlvbiBzcWx9IHs=ICAgICAgbmV4dDsgICAgICAgICAgICAgICAgICAgICAgICAjIGluaXRpYWxpemUgdGhlIGJhc2UgY2xhc3M=ICAgICAgbXkgdmFyaWFibGUgcmVzdWx0U2V0Q2xhc3MgICAgICAgc2V0IHJlc3VsdFNldENsYXNzIHdoYXRldmVyOyAjIFRlbGwgdGhlIGJhc2UgY2xhc3Mgd2hhdCBjbGFzcw==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjIHRvIHVzZSBmb3IgcmVzdWx0IHNldHM=ICAgICAgbXkgaW5pdCAkY29ubmVjdGlvbiAkc3FsOyAgICAjIFRoZSBbW2luaXRdXSBtZXRob2Qgc2hvdWxkIGRvICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjIHdoYXRldmVyIGlzIG5lY2Vzc2FyeSB0byBwcmVwYXJlICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjIHRoZSBzdGF0ZW1lbnQ=ICB9 Derived classes from tdbc::statement MUST also implement the params and paramtype methods. In addition, system resources belonging to the statement itself MUST be cleaned up by a destructor or by a deletion callback at C level. (Result sets MUST not be deleted then; the base classes take care of that.) See "Best practices for memory management" below for further discussion. The class that implements a result set SHOULD normally inherit from the tdbc::resultset base class. Its constructor accepts the statement handle and the arguments to the execute method. The constructor is responsible for invoking the base class constructor with next, and executing the statement. A sample constructor looks like: ICBjb25zdHJ1Y3RvciB7c3RhdGVtZW50IGFyZ3N9IHs=ICAgICAgbmV4dA==ICAgICAgdXBsZXZlbCAxIFtsaXN0IHsqfVtuYW1lc3BhY2UgY29kZSB7bXkgaW5pdH1dICRzdGF0ZW1lbnQgeyp9JGFyZ3NdICB9 Note the peculiar form of invocation for the init method in the example above. Since the init method needs access to local variables in the caller's context to do variable substitution, it needs to be executed at the same stack level as the constructor itself. The [namespace code {my init}] call gives a command prefix that can be used to invoke the method in a foreign context, and this command is then executed with [uplevel 1] to do the initialization. Besides the constructor and init, the other methods that a result set class MUST implement are columns, nextrow, and rowcount. In addition, a destructor (or a C deletion callback) MUST clean up any system resources belonging to the result set. Since the TclOO interfaces are so new, it seems wise to give developers of database interfaces written in C some guidance about effective ways to manage memory. A C-level extension, if written correctly, gets considerable assistance in releasing memory at the appropriate times from TclOO and the tdbc base classes. When a database interface is first loaded as an extension, it is entered through its PackageName_Init function. It will call, in order, Tcl_InitStubs, Tcloo_InitStubs, and Tdbc_InitStubs so that Tcl, the TclOO system, and the TDBC base classes are all available. Its next task is to allocate any per-interpreter data that may be required. (In the case of the tdbc::odbc bridge, the per-interpreter data include an ODBC environment handle and a string literal pool.) The per-interpreter data structure SHOULD be reference counted, since the order of destruction of the objects that refer to it is unpredictable. Next, the initialization function creates the classes, usually by evaluating an initialization script containing a call to tcl_findLibrary, where the Tcl code contains the skeletons of the class definitions. With the class definitions in hand, methods that are implemented in C can be attached to them. Any methods that need the per-interpreter data can receive it as ClientData. The reference count of the per-interpreter data SHOULD be incremented for these, and the method delete procedures should be responsible for decrementing the reference count. Each of the three classes that make up a database interface SHOULD have a reference-counted data structure to hold any instance data. This structure SHOULD be created within the init method, and attached to the object with Tcl_ObjectSetMetadata. The metadata type structure SHOULD designate a delete procedure that decrements the reference count. The type structure MAY designate a clone procedure that returns TCL_ERROR; it is entirely permissible for TDBC objects not to be clonable. Generally speaking, each object's instance data structure will contain a pointer to (and hold a counted reference to) the next higher object in the ownership hierarchy. A result set will refer to the statement that produced it; a statement will refer to the connection in which it executes, and a connection will refer to the per-interp data. With this infrastructure in place, object destruction becomes strictly a local matter. Any object, when its reference count becomes zero, MUST release any system resources that belong to it, and decrement the reference count of the next object up. There is no need for a connection to track its statements, or a statement to track its result sets. This happens automatically because the prepare and execute methods create statements in a namespace subordinate to the namespace of the owning connection, and create result sets in a namespace subordinate to that of the owning statement. When the owning objects are destroyed, the subordinate namespaces are also destroyed, invoking the destructors of the objects within them. This whole scheme is simpler than it sounds, and is observed to work well for the tdbc::odbc bridge (see the source code of the bridge for further details). Closing a connection gracefully deletes the statement and result class objects (in Tcl) from top to bottom, and then deletes the corresponding C data structures from bottom to top, finally cleaning up the connection data itself. Note that, since TclOO does not guarantee to run destructors on exit, if a database interface needs to always close the underlying connection on termination, the implementation code should install an exit handler with Tcl_CreateExitHandler if it needs to. In addition to the convenience commands discussed above, the Tcl system SHALL provide certain commands to aid the job of database implementors. The task of mapping variable substituions in the form, :varName into whatever form that a native database API can handle is a somewhat tricky one. For instance, substitutions that appear inside quoted strings MUST NOT be mapped. In order to aid in this task, the Tcl system SHALL provide a command, ::tdbc::tokenize. This command SHALL accept a SQL statement as its sole parameter, and return a list of tokens. The lexical value of the tokens can be distinguished by their first characters: '$', ':' and '@' are all variable substitutions; the remainder of the token string is a variable name.';' is a statement separator, for databases that allow multiple statements to be prepared together.'-' is a commentAnything else is literal text to be copied into a SQL statement. Assuming that a native database's lexical structure conforms with standard SQL, the variable names can be substituted with parameter numbers, question marks, or whatever the database needs, to yield the native SQL that must be prepared. Tokenisation is also available at the C level; to access it, a C extension MUST first call Tdbc_InitStubs; it is a macro that behaves as if it is a function with the type signature int Tdbc_InitStubs(Tcl_Interp *interp); where interp is a Tcl interpreter. The function returns TCL_OK if successful, and TCL_ERROR (with an error message left in the interpreter) in the case of failure. The tokenisation is then available by calling Tcl_Obj *Tdbc_TokenizeSql(Tcl_Interp *interp, const char *sqlCode); In this call, interp is a Tcl interpreter, and sqlCode is a SQL statement to parse. If the parse is successful, the return value is a Tcl object with a reference count of zero that contains a list of token strings as with the 'tdbc::tokenize call.