How to default a file's external format

[rmkaplan]

There is a discussion in #2290 about the fact that a dribble file in Medley is open in :MCCS and therefore is not necessarily correctly interpretable by non-Medley tools. This led to a larger discussion, in that PR and also in the 1-21-2026 meeting, about different conventions for determining the external format of a file. For example, that files opened on UNIX might default to :UTF-8 but files on DSK would be :MCCS.

Currently, each FDEV holds its DEFAULTEXTERNALFORMAT, and both UNIX and DSK specify :MCCS (which is also the general system default). That's the external format that is selected by default when FORMAT is not specified as a parameter in the call to OPENSTREAM, and is available from the stream (and can be changed after opening) by STREAMPROP.

But the external format of the stream is not available as a property of a file--a file has no explicit marking to say what format was used at its creation and that can therefore be asserted every time the file is opened (unless there is an explicitly specified FORMAT parameter). (I assume that Unix file systems don't allow arbitrary properties to be associated with files).

We could change the default for the UNIX FDEV, as was suggested. In that case, opening a dribble file as {UNIX}... would produce a file that is interpretable externally, and Medley (presumably Tedit, or SEE) would be able to interpret it internally. The characters would look right in both contexts. If opened on {DSK}, it would look right in Medley but maybe not in other programs.

Another possibility, instead of or in addition to the device, is to key this off the extension, with a built-in list that maps extensions to externalformats. Essentially the same way that the TEXT/BINARY TYPE property is implemented. As an example, if (DRIB :UTF-8) was on the extension-to-format list, (GETFILEINFO '{DSK}xxx.DRIB 'EXTERNAL-FORMAT) would return :UTF-8, and that would be the default format when a .DRIB file is created and when it is later opened. But (GETFILEINFO '{DSK}xxx.txt 'EXTERNAL-FORMAT) would return :MCCS.

[nbriggs]

I assume that Unix file systems don't allow arbitrary properties to be associated with files

Many flavors of Unix system do have something like this, but with a different API for doing so in each one.
It's not easy to implement (in all its variations) in Maiko.

While I think there are some file extensions where one could make a good guess at the external format, I think that it introduces complexity to the code (or errors if you don't compensate) when the guess is not what you expected.

I think that basing it on the FDEV unless there's an explicit override on the open makes it clearer.

[masinter]

i don't like making {DSK} and {UNIX} have different defaults. If the files are really the same, you can't be both :UTF-8 and at the same time MCCS. I think relying on file extensions (or in Common LIsp cl:pathname-type ) may be a necessary evil. We already have TEXT and BINARY designations, and I don't see getting rid of that any time. I could see using the destination file system as a key (so a .TXT file on a Tops-20 file would mean (9 bit byte character codes) but {DSK} and {UNIX} are just naming overlays. https://LarryMasinter.net https://interlisp.org

…

On Wed, Jan 21, 2026 at 11:36 PM rmkaplan ***@***.***> wrote: *rmkaplan* created an issue (Interlisp/medley#2452) <#2452> There is a discussion in #2290 <#2290> about the fact that a dribble file in Medley is open in :MCCS and therefore is not necessarily correctly interpretable by non-Medley tools. This led to a larger discussion, in that PR and also in the 1-21-2026 meeting, about different conventions for determining the external format of a file. For example, that files opened on UNIX might default to :UTF-8 but files on DSK would be :MCCS. Currently, each FDEV holds its DEFAULTEXTERNALFORMAT, and both UNIX and DSK specify :MCCS (which is also the general system default). That's the external format that is selected by default when FORMAT is not specified as a parameter in the call to OPENSTREAM, and is available from the stream (and can be changed after opening) by STREAMPROP. But the external format of the stream is not available as a property of a file--a file has no explicit marking to say what format was used at its creation and that can therefore be asserted every time the file is opened (unless there is an explicitly specified FORMAT parameter). (I assume that Unix file systems don't allow arbitrary properties to be associated with files). We could change the default for the UNIX FDEV, as was suggested. In that case, opening a dribble file as {UNIX}... would produce a file that is interpretable externally, and Medley (presumably Tedit, or SEE) would be able to interpret it internally. The characters would look right in both contexts. If opened on {DSK}, it would look right in Medley but maybe not in other programs. Another possibility, instead of or in addition to the device, is to key this off the extension, with a built-in list that maps extensions to externalformats. Essentially the same way that the TEXT/BINARY TYPE property is implemented. As an example, if (DRIB :UTF-8) was on the extension-to-format list, (GETFILEINFO '{DSK}xxx.DRIB 'EXTERNAL-FORMAT) would return :UTF-8, and that would be the default format when a .DRIB file is created and when it is later opened. But (GETFILEINFO '{DSK}xxx.txt 'EXTERNAL-FORMAT) would return :MCCS. — Reply to this email directly, view it on GitHub <#2452>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIQTK5OE4EWY5HRHSDOAXL4IB4XVAVCNFSM6AAAAACSQD3HCGVHI2DSMVQWIX3LMV43ASLTON2WKOZTHA2DCNRVGQZTQMQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[MattHeffron]

I could see using the destination file system as a key (so a .TXT file on a Tops-20 file would mean (9 bit byte character codes)

On TOPS-10/20, text files were five 7-bit ascii characters per 36-bit word, with the low-order bit reserved for special purposes.
For example, in LINED (line editor), the line numbers were stored with the low-order bit set. This allowed other programs to ignore them when processing depended on specific columns, like the Fortran compiler.