Stanford Lecture: The Internal Details of TeX82 - Session 9 (July 30, 1982)

Key Concepts

• System Dependency: Features of TeX that rely on the specific operating system and its conventions.
• File Naming Conventions: The diverse and often inconsistent ways operating systems refer to files.
• Portability: The ability of software to run on different systems with minimal or no modification.
• TeX 82: A version of TeX designed to factor out system dependency into specific routines.
• File Name Structure: The proposed three-part structure for file names: name, area, and extension.
• Begin Name, More Name, End Name: A set of three procedures that TeX calls to parse file names in a system-independent way.
• Cur Area, Cur Name, Cur Ext: Global variables that store the parsed components of a file name.
• Pack File Name: A system-dependent procedure that converts TeX's internal string representations of file name components into the format required by the operating system.
• Pack Buffer Name: A system-dependent procedure used during TeX's bootstrapping phase to construct file names when TeX's string pool is not yet initialized.
• Make Name String: A system-dependent procedure to retrieve the full, actual name of an opened file.
• Prompt File Name: A system-independent routine that handles user interaction for obtaining file names.
• Print File Name: A system-dependent procedure to display file names in error messages.
• Interaction Levels: TeX's modes of operation (batch, nonstop, scroll, error stop) that affect how file-not-found errors are handled.
• Reset and Rewrite: Pascal procedures used to open files for input and output, respectively.

File Naming and System Dependency in TeX

This session focuses on system-dependent aspects of TeX, specifically file naming, which is identified as a major challenge for software portability. The inherent differences in how various operating systems handle file names necessitate system-specific implementations within TeX.

The Problem of Diverse File Naming Conventions

The core issue is that operating systems have vastly different conventions for referring to files. While simple file names composed entirely of letters (e.g., "paper") are generally handled consistently across TeX installations, more complex names involving special characters, directories, or network locations vary significantly. Examples include:

• se/paper on one system.
• paper--se on another.
• paper[1,se] on yet another.

The speaker notes that an attempt to create a universal file naming language for TeX was abandoned because users naturally prefer to adhere to their system's conventions. This leads to the unavoidable reality that some TeX programs will contain syntax that is not parsable on all installations.

TeX 82's Approach to Portability

To mitigate this, TeX 82 was designed to "factor out" system dependency into a small number of routines that can be called in a system-independent way. This aims to achieve as much portability as possible despite the inherent differences in file naming.

Historical Context and the "Mess" of File Naming

The speaker attributes the current "mess" of file naming to a historical lack of attention from computer scientists, who often relegated it to "hackers" rather than treating it as a scientific problem. This, along with a similar lack of clean solutions for general input/output in early programming languages like ALGOL, has contributed to the complexity.

Proposed File Name Structure

TeX 82 imposes a conceptual three-part structure for file names:

1. Name Part: The primary identifier of the file (e.g., "paper").
2. Area Part: Indicates the location or directory of the file within a larger file system.
3. Extension Part: Denotes the type or class of the file.

This structure is exemplified by common practices where related files share the same name but have different extensions:

• paper.TeX (source file) -> paper.dvi (output), paper.err (error log).
• basic.TeX (macro package) -> basic.fmt (compiled format).
• TeX.web (source) -> TeX.pscl (Pascal code), TeX.pol (string pool).

The use of a dot (.) as a separator for extensions is common but not strictly required by TeX itself.

Default Areas and Font Handling

• Default Area: If the area part is not specified, a default is used. For user files, this is typically the user's login area.
• Font Area: For font files, the default area is the system font area. This distinction is crucial:
  • Specifying no area for a user file means the user's own area.
  • Specifying no area for a font file means the system font area.
  • To use a custom font, the user's area must be explicitly specified.

Font File Naming and DVI Output

When TeX reads a font file, the resulting DVI (Device Independent) file must contain the font's name for external communication. This name is represented as a sequence of ASCII codes for both the area and the name, without the extension or .tfm suffix.

System-Independent File Name Parsing: Begin Name, More Name, End Name

To handle the diverse file naming conventions, TeX 82 relies on three system-dependent procedures that are called in a system-independent way:

1. begin_name: Called when TeX encounters a keyword that requires a file name (e.g., \input). It performs initializations for parsing.
2. more_name(c): Called for each character c that is part of the file name. It returns true if the name might not have ended yet, and false if it's definitively the end of the name.
3. end_name: Called when the end of the file name is detected. It signals the completion of parsing.

After end_name is called, three global variables are set: cur_area, cur_name, and cur_ext, which hold the parsed area, name, and extension as TeX strings. The null string is supported for cases where an area or extension is not provided.

Example Logic: On MIT's system, where spaces can be part of file names and are delimiters, more_name would return true for characters within quotes until the closing quote is encountered, at which point it would return false.

System-Dependent Procedures for File Handling

Beyond parsing, several system-dependent procedures are required for TeX to interact with the operating system's file system:

• pack_file_name(n, a, e): This procedure takes TeX string numbers for the name (n), area (a), and extension (e) and constructs the file name in the format expected by the operating system. It populates a global variable, name_of_file, which is then used by the Pascal reset procedure. This procedure can be complex, requiring calculations for specific system requirements (e.g., 6-bit codes on Tenex).
• pack_buffer_name(n, a, b): A specialized procedure used during TeX's bootstrapping phase when the string pool is not yet initialized. It constructs name_of_file from a buffer and a default string, typically for loading format files. It's used when TeX is "born" and cannot yet use its full string handling capabilities.
• print_file_name(n, a, e): A procedure that takes the three file name components and prints them in a human-readable format, usually for error messages.
• make_name_string(file_variable): This procedure is designed to return the actual, full name of a file that has just been opened. It's useful for logging and error reporting. It has typed versions for different file types (alpha for text, byte for TFM/DVI, word for FMT).
• reset and rewrite (Pascal procedures): These are fundamental for opening files. The speaker emphasizes that these procedures should not attempt to handle file-not-found errors themselves by prompting the user. Instead, they should return a clear indication of success or failure (e.g., via EOF in Pascal), allowing TeX to manage error messages and user interaction.

Handling File Not Found Errors and Interaction Levels

• prompt_file_name(description, default_extension): A system-independent routine that prompts the user for a file name when one cannot be found. It uses begin_name, more_name, and end_name to parse the user's input.
• Interaction Levels: TeX has four interaction levels:
  • Batch Mode: No user interaction; job aborts if a file is not found.
  • Nonstop Mode: Similar to batch mode; job aborts on file not found.
  • Scroll Mode: User is prompted for a new file name.
  • Error Stop Mode: User is prompted for a new file name.

If a file is not found in batch or nonstop modes, the job is aborted. In scroll or error stop modes, the user is given a chance to provide a correct file name.

Other System-Dependent Routines

• clear_terminal: A system-dependent routine to cancel any "typing ahead" by the user when an error occurs, preventing unintended input from being processed as the new file name.
• open_a_file (and similar): Procedures that wrap reset and rewrite to provide a boolean return value indicating success or failure, allowing TeX to manage error reporting.

Conclusion

The session highlights the significant challenges in achieving software portability due to system-dependent file naming conventions. TeX 82 attempts to address this by abstracting system interactions into specific procedures (begin_name, more_name, end_name, pack_file_name, pack_buffer_name, etc.). While the core logic of TeX's file handling is designed to be system-independent, the implementation of these specific routines requires careful adaptation to each operating system. The speaker concludes by noting that solving these issues for TeX will also benefit Metafont, simplifying future development.