Andreas KupriesAndreas Kupries$Revision: 1.13 $Tcl

This TIP proposes adding a new subcommand to the info command to get a list of all the frames on the current stack, with additional information about command location, type of execution, etc., rather than the limited list returned by info level. It is also related to Peter MacDonald's , or rather, to the info linenum feature proposed there. The base feature of providing location information for a command was extended, allowing the user to ask for the current location in all stack levels. Enough information is returned to consider this as an extended .

The motivation for this feature is the debugging of scripts in situations where an error stack is wanted, to see where the problem occurred, but aborting the execution is not desired. To enable this a means of providing the essential information shown in an error stack is needed, which does not require an abort (and subsequent unwinding of the C stack) to assemble this information. An example is the testing of scripts where an unexpected error should not fail the test case, nor the framework, yet still allow the recording of the problematic location and how it was reached in some log. The original motivation for was that there is currently no way to get a list of all the frames in the current stack managed by Tcl_PushCallFrame() and Tcl_PopCallFrame(). The info level command does not contain frames that are callers of uplevel, reporting only the frames that are accessible via another uplevel command. There are times when the lack of information can have a negative impact on code design. This motivation asks in essence for an error stack as well, but limited itself to the returning commands themselves, and not the other information, like the line the command is on, its context, etc. Other use cases, also found in : tcltest, and other testing frameworks.The first case is with the core's Tcltest package, where the complete lack of ability to gain access to that information means it is impossible to gain information about a test without modifying the Tcltest code itself. Being able to find out the caller info would be very useful, especially for logging information. Currently, there is no way to get the caller's info, due to the fact that the code for the test is upleveled and, hence, not visible via info level.TestStubs PackageThe TestStubs package provides the ability to temporarily redefine commands, in particular for stubbing out or replacing functionality in a test case. There is a command in the package called chain, which is used within the code replacing a command (or part of a command) to call the original definition of the command. For example, one could do:c3R1YnM6OnN0dWIgZW5zZW1ibGUgYXJyYXkgbmFtZXMgew==ICAgIHJldHVybiBbbHNvcnQgW3VwbGV2ZWwgMSBjaGFpbiBuYW1lcyAkYXJnc11dfQ==However, since the chain command is (and should be) limited to only running from within a stub definition, it needs to call info level to find out if its caller is one of the stubbed commands, and what the name of that command is. With info level, it would not have access to the level that is running inside the stubbed procedure. Hence, either it cannot check this constraint, or stubs cannot be allowed to use uplevel when calling it (which means things like the above either cannot work, or need to be rewritten in a considerably less clear manner).

The builtin command info is extended to accept a new subcommand, frame. When this subcommand is called it returns information about the current command and its location. This information is available not only for the current stack level, but also the higher stack levels used to reach the current location. The syntax of the new subcommand is info frame ?level? The new functionality will provide access to all frames on the stack rather than the current limited subset. This TIP does not propose to alter uplevel or upvar so that they can see these hidden levels. If level is not specified, this command returns a number giving the frame level of the command. This is 1 if the command is invoked at top-level. If level is specified, then the result is a dictionary containing the location information for the command at the level on the stack. If level is positive (> 0) then it selects a particular stack level (1 refers to the top-most active command, i.e., info frame itself, 2 to the command it was called from, and so on); otherwise it gives a level relative to the current command (0 refers to the current command, i.e., info frame itself, -1 to its caller, and so on). This is similar to how info level works, except that this subcommand reports all frames, like source'd scripts, evals, uplevels, etc. Note that for nested commands, like "foo [bar [x]]" only "x" will be seen by an info frame invoked within "x". This is the same as for info level and error stack traces. The result dictionary may contain the keys listed below, with the specified meanings for their values. typeThis entry is always present and describes the nature of the location for the command. The recognized values are source, proc, eval, and precompiled.In some circumstances it makes sense to have this information extensible, i.e. to allow user-defined type names. For more about this topic see the discussion at the end of the document.source means that the command is found in a script loaded by the source command.proc means that the command is found in dynamically created procedure body.eval means that the command is executed by eval or uplevel.precompiled means that the command is found in a precompiled script (loadable by the package tbcload), and no further information will be available.lineThis entry provides the number of the line the command is at inside of the script it is a part of. This information is not present for type precompiled. For type source this information is counted relative to the beginning of the file, whereas for the last two types the line is counted relative to the start of the script.fileThis entry is present only for type source. It provides the normalized path of the file the command is in.cmdThis entry provides the string representation of the command. This is usually the unsubstituted form, however for commands which are a pure list executed by eval it is the substituted form as they have no other string representation. Care is taken that the pure-List property of the latter is not spoiled.procThis entry is present only if the command is found in the body of a regular Tcl procedure. It then provides the name of that procedure.lambdaThis entry is present only if the command is found in the body of an anonymous Tcl procedure, i.e. a lambda. It then provides the entire definition of the lambda in question.levelThis entry is present only if the queried frame has a corresponding frame returned by info level. It provides the index of this frame, relative to the current level (0 and negative numbers). A thing of note is that for procedures statically defined in files the locations of commands in their bodies will be reported with type source and absolute line numbers, and not as type proc. The same is true for procedures nested in statically defined procedures, and literal eval scripts in files or statically defined procedures. In contrast, for a procedure definition or eval within a dynamically eval'uated environment count linenumbers relative to the start of their script, even if they would be able to count relative to the start of the outer dynamic script. That type of number usually makes more sense. A different way of describing this behaviour is that we track file based locations as deeply as we can, and where we cannot the lines are counted based on the smallest possible eval or procbody scope, as that scope is usually easier to find than any dynamic outer scope. The syntactic form {expand} is handled like eval. This means that if it is given a literal list argument the system tracks the line-number within the list words as well, and otherwise all line-numbers are counted relative to the start of each word (smallest scope) The following other builtin commands are changed as well to support the tracking of line numbers: catchdict fordict withevalforforeachifinterp evalnamespace evalprocsourceswitchwhile No changes are made to the public C API.

Note that this is not a complete set of examples covering all possible cases. Let us assume that the file is named EX. The "cmd" is always the "info frame ..." command, and "level" is 0 too, always, in this situation. This is left out of the result dictionaries to keep them small. IHB1dHMgW2luZm8gZnJhbWUgMF0gICAgICAgICAgIDsjIDAxKiAtIG91dHB1dDoge2NtZCB7aW5mbyBmcmFtZSAwfSBsaW5lIDEgZmlsZSBFWCB0eXBlIHNvdXJjZX0=ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDAyIHByb2MgZm9vIHt9IHsgICAgICAgICAgICAgICAgIDsjIDAzICAvZm9vIHJlbWVtYmVycyAzICAgICBwdXRzIFtpbmZvIGZyYW1lIDBdICAgICAgIDsjIDA0Kg==IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDA1IGZvbyAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDA2ICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDQgZmlsZSBFWCB0eXBlIHNvdXJjZX0=ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDA3IHNldCBzY3JpcHQgeyAgICAgICAgICAgICAgICAgIDsjIDA4ICAxICAgICBwdXRzIFtpbmZvIGZyYW1lIDBdICAgICAgIDsjIDA5ICAyKg==IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDEwICAzIGV2YWwgJHNjcmlwdCAgICAgICAgICAgICAgICAgIDsjIDExICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDIgdHlwZSBldmFsfQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDEyIGV2YWwgeyAgICAgICAgICAgICAgICAgICAgICAgIDsjIDEzICAxICAgICBwdXRzIFtpbmZvIGZyYW1lIDBdICAgICAgIDsjIDE0KiAyKg==IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDE1ICAzICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDE0IHR5cGUgc291cmNlfQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDE2IHByb2MgZm94IHt9IHsgICAgICAgICAgICAgICAgIDsjIDE3ICAvZm94IHJlbWVtYmVycyAxNw==ICAgICBldmFsICQ6OnNjcmlwdCAgICAgICAgICAgIDsjIDE4IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDE5IGZveCAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDIwICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDIgdHlwZSBldmFsfQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDIxIHByb2Mgc3F1aXJyZWwge30geyAgICAgICAgICAgIDsjIDIyICAvc3F1aXJyZWwgcmVtZW1iZXJzIDIyICAgICBldmFsIHsgICAgICAgICAgICAgICAgICAgIDsjIDIzICAxICAgICAgICAgcHV0cyBbaW5mbyBmcmFtZSAwXSAgIDsjIDI0KiAyKg==ICAgICB9ICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDI1ICAzIH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDI2IHNxdWlycmVsICAgICAgICAgICAgICAgICAgICAgIDsjIDI3ICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDI0IHR5cGUgc291cmNlfQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDI4IHByb2MgZnVjaHMge30geyAgICAgICAgICAgICAgIDsjIDI5ICAgIC9mdWNocyByZW1lbWJlcnMgMjk=ICAgICBwcm9jIGRvZyB7fSB7ICAgICAgICAgICAgIDsjIDMwICAxIC9kb2cgICByZW1lbWJlcnMgMzA=ICAgICAgICAgcHV0cyBbaW5mbyBmcmFtZSAwXSAgIDsjIDMxKiAyKg==ICAgICB9ICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDMyICAzIH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDMzIGZ1Y2hzICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDM0IGRvZyAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDM1ICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDMxIHR5cGUgc291cmNlfQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDM2IGV2YWwgeyAgICAgICAgICAgICAgICAgICAgICAgIDsjIDM3ICAxICAgICBwcm9jIHdvbGYgeyAgICAgICAgICAgICAgIDsjIDM4ICAyIC93b2xmIHJlbWVtYmVycyAzOA==ICAgICAgICAgcHV0cyBbaW5mbyBmcmFtZSAwXSAgIDsjIDM5KiAzKg==ICAgICB9ICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDQwICA0IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDQxICA1IHdvbGYgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDQyICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDM5IHR5cGUgc291cmNlfQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDQzIHNldCBzY3JpcHRlZCB7ICAgICAgICAgICAgICAgIDsjIDQ0ICAxICAgICBwcm9jIGRlZXIge30geyAgICAgICAgICAgIDsjIDQ1ICAyIC9kZWVyIHJlbWVtYmVycyAyICAgICAgICAgcHV0cyBbaW5mbyBmcmFtZSAwXSAgIDsjIDQ2ICAzKg==ICAgICB9ICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDQ3ICA0IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDQ4ICA1IGV2YWwgJHNjcmlwdGVkICAgICAgICAgICAgICAgIDsjIDQ5IGRlZXIgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDUwICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDMgdHlwZSBldmFsfQ==ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDUxIHB1dHMgW3NldCBhIGIgICAgICAgICAgICAgICAgIDsjIDUyICAgICAgIGluZm8gZnJhbWUgMF0gICAgICAgICAgIDsjIDUzKiAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDUzIHR5cGUgc291cmNlIGZpbGUgRVh9ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDU0IHB1dHMgW2luZm8gXA==ICAgICAgIGZyYW1lIDBdICAgICAgICAgICAgICAgIDsjIDU2ICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDU1IHR5cGUgc291cmNlIGZpbGUgRVh9ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDU3IHByb2Mgc2FsbW9uIHt9IFw=IHsgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDU5ICAgICBwdXRzIFtpbmZvIGZyYW1lIDBdICAgICAgIDsjIDYwKg==IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDYxIHNhbG1vbiAgICAgICAgICAgICAgICAgICAgICAgIDsjIDYyICAtIG91dHB1dDoge2NtZCB7Li4ufSBsaW5lIDYwIHR5cGUgc291cmNlIGZpbGUgRVh9ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDYzIHNldCBtZXRob2QgZnJhbWUgICAgICAgICAgICAgIDsjIDY0IHB1dHMgW2luZm8gJG1ldGhvZCAwXSAgICAgICAgIDsjIDY1KiAtIG91dHB1dDoge2NtZCB7aW5mbyAkbWV0aG9kIDB9IGxpbmUgNjUgdHlwZSBzb3VyY2UgZmlsZSBFWH0=ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDY2IHByb2MgdHJvdXQge30geyAgICAgICAgICAgICAgIDsjIDY3ICAgICBwdXRzIFtpbmZvICRtZXRob2QgMF0gICAgIDsjIDY4Kg==IH0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDY5IHRyb3V0ICAgICAgICAgICAgICAgICAgICAgICAgIDsjIDcwICAtIG91dHB1dDoge2NtZCB7aW5mbyAkbWV0aG9kIDB9IGxpbmUgNjggdHlwZSBzb3VyY2UgZmlsZSBFWH0= Another example showing how to query the whole stack of frames. It assumed that the file is named EX. The output shown after the example was manually reformatted to explicitly ordered the keys and the columns vertically aligned, for readability and better comparability of the lines. The returned level information was removed, and level numbers for info frame were added at the beginning of the lines. IHByb2Mgc2V0UmVzIHtyZXN1bHR9IHsgICAgICAgICAgICAgICA7IyAwMQ==ICAgaWYgeyRyZXN1bHQgPT0gImZhaWwifSB7ICAgICAgICAgICA7IyAwMg==ICAgICAgc2V0IGxldmVsIFtpbmZvIGZyYW1lXSAgICAgICAgICA7IyAwMw==ICAgICAgd2hpbGUgeyRsZXZlbH0geyAgICAgICAgICAgICAgICA7IyAwNA==ICAgICAgICAgICBwdXRzIFtpbmZvIGZyYW1lICRsZXZlbF0gICA7IyAwNQ==ICAgICAgICAgICBpbmNyIGxldmVsIC0xICAgICAgICAgICAgICA7IyAwNg==ICAgICAgIH0gOyMgZW5kIG9mIHdoaWxlICAgICAgICAgICAgICA7IyAwNw==ICAgfSA7IyBlbmQgb2YgaWYgICAgICAgICAgICAgICAgICAgICA7IyAwOA==IH0gOyMgZW5kIG9mIHByb2Mgc2V0UmVzICAgICAgICAgICAgICA7IyAwOQ==IHByb2MgcnVuVGVzdCB7dGN9IHsgICAgICAgICAgICAgICAgICA7IyAxMA==ICAgICAgIyBydW4gdGhlIHRlc3RjYXNlICAgICAgICAgICAgICA7IyAxMQ==ICAgICAgdXBsZXZlbCAxICBzZXRSZXMgZmFpbCAgICAgICAgICA7IyAxMg==IH0gOyMgZW5kIG9mIHByb2MgcnVuVGVzdCAgICAgICAgICAgICA7IyAxMw==IHJ1blRlc3QgVEMwMDAxICAgICAgICAgICAgICAgICAgICAgICA7IyAxNA== Output: IDQge2NtZCB7cnVuVGVzdCBUQzAwMDF9ICAgICAgICBsaW5lIDE0IHR5cGUgc291cmNlIGZpbGUgRVh9IDMge2NtZCB7dXBsZXZlbCAxIHNldFJlcyBmYWlsfSBsaW5lIDEyIHR5cGUgc291cmNlIGZpbGUgRVggcHJvYyBydW5UZXN0fQ==IDIge2NtZCB7c2V0UmVzIGZhaWx9ICAgICAgICAgICBsaW5lICAxIHR5cGUgZXZhbH0=IDEge2NtZCB7aW5mbyBmcmFtZSAkbGV2ZWx9ICAgICBsaW5lICA1IHR5cGUgc291cmNlIGZpbGUgRVggcHJvYyBzZXRSZXN9 To see the connection between the new feature and error stack traces replace the info frame command on line 3 of setRes with "return -code error X". This will generate an error trace and with a bit of reformatting the relationship can be seen easily: Stack trace: CVg=CSAgICB3aGlsZSBleGVjdXRpbmc=CSJzZXRSZXMgZmFpbCI=CSAgICAoInVwbGV2ZWwiIGJvZHkgbGluZSAxKQ==CSAgICBpbnZva2VkIGZyb20gd2l0aGluCSJ1cGxldmVsIDEgIHNldFJlcyBmYWlsIg==CSAgICAocHJvY2VkdXJlICJydW5UZXN0IiBsaW5lIDMpCSAgICBpbnZva2VkIGZyb20gd2l0aGluCSJydW5UZXN0IFRDMDAwMSI=CSAgICAoZmlsZSAiRVgudGNsIiBsaW5lIDE0KQ== Reformat the trace, with all output from a single command on one line, and some noise removed. CVg=CSJzZXRSZXMgZmFpbCIgKCJ1cGxldmVsIiBib2R5IGxpbmUgMSk=CSJ1cGxldmVsIDEgIHNldFJlcyBmYWlsIiAocHJvY2VkdXJlICJydW5UZXN0IiBsaW5lIDMpCSJydW5UZXN0IFRDMDAwMSIgKGZpbGUgIkVYLnRjbCIgbGluZSAxNCk= Now revert the order of the lines, and add vertical alignment. Now have in essence the ouput of the unmodified example, with same difference in the line numbers. But that is only because the error stack trace always counts lines relative to the script, and doesn't attempt to determine an absolute location in a file. CSJydW5UZXN0IFRDMDAwMSIgICAgICAgICAoZmlsZSAiRVgudGNsIiBsaW5lIDE0KQ==CSJ1cGxldmVsIDEgIHNldFJlcyBmYWlsIiAocHJvY2VkdXJlICJydW5UZXN0IiBsaW5lIDMpCSJzZXRSZXMgZmFpbCIgICAgICAgICAgICAoInVwbGV2ZWwiIGJvZHkgbGluZSAxKQ==CVg= The current implementation of info level only returns levels up to the top of the stack for the current interpreter. Such an approach puts limitations on what information can be retrieved, but allows for a certain level of "security" when running code in child interps, especially safe interps. Given the security considerations of safe interps, and consistancy with regards to what information is returned across multiple circumstances, the stack trace returned will only return information up to the top level of the current interp, the same limit info level is bound by.

A point noted by Lars Hellstrom is that this TIP makes the builtin source command special. It is not possible anymore to re-create source purely in Tcl, doing so will loose the name of the file. We are still tracking line numbers, but they are then relative to the start of the eval'd script, withut a connection to the file. In a similar way, user-defined control structures lose the ability of the builtin if, etc. to track line numbers in a literal code block relative to the enclosing file. The user-defined commands are again reduced to providing line numbers relative to the start of the code block. To bring user-defined commands back to the same level as the builtins we need: An extension of info frame which delivers the line information for all words of the command we are inside of. As each word can start on a different line (because of continuation lines and preceding multi-line literals).This information is actually already available internally. The current implementation needs it for the tracking into literals.Extensions to the uplevel and eval commands which allow a script to provide them with location information for the script they are evaluating. For example: IHByb2MgbmV3c291cmNlIHtmb28gc2NyaXB0fSB7ICAgLi4uIHByZXByb2Nlc3Mgd2hhdGV2ZXIgLi4uICAgc2V0IGxvYyBbaW5mbyBmcmFtZSBkYXRhIGZvciB3b3JkIDJdICAgdXBsZXZlbCAtbG9jYXRpb24gJGxvYyAxICRzY3JpcHQ=IH0= Internally we also need quite a bit more flexible data structures. The type names for example are defined through a C enum, and can be checked quickly and efficiently. The file information, i.e. path, is stored and used dependent on that, only for type source. To keep this TIP and its implementation reasonably small the actual detailed specification and implementation of such extensibility is defered to a future TIP.

An implementation patch is available on SourceForge[].

This document has been placed in the public domain. Please note that any correspondence to the author concerning this TIP is considered in the public domain unless otherwise specifically requested by the individual(s) authoring said correspondence. This is to allow information about the TIP to be placed in a public forum for discussion.