[LISPWORKS][Common Lisp HyperSpec (TM)] [Previous][Up][Next]


Issue PATHNAME-WILD Writeup

Status: Passed, as amended, Jun 89 X3j13

Issue: PATHNAME-WILD

Forum: Cleanup

References: Pathnames (pp410-413)

Related issues: PATHNAME-COMPONENT-VALUE, PATHNAME-LOGICAL

Category: ADDITION

Edit history: 21-Jul-88, Version 1 by Pitman

06-Oct-88, Version 2 by Pitman

9-May-89, Version 3 by Moon (small fixes)

10-May-89, Version 4 by Moon (add two more functions)

13-May-89, Version 5 by Moon (minor cleanups, add clarification)

19-Jun-89, Version 6 by Moon (revise based on extensive

discussion in the cleanup subcommittee; rewrite

the description of TRANSLATE-PATHNAME so it is

possible to understand it)

23-Jun-89, Version 7 by Moon (simplify TRANSLATE-PATHNAME

based on last minute discussion of logical pathnames)

2-Jul-89, Version 8 by Masinter (T -> True as per Jun89 X3J13)

Problem Description:

Some file systems provide more complex conventions for wildcards than

simple component-wise wildcards (:WILD). For example,

"F*O" might mean:

- a normal three character name

- a three-character name, with the middle char wild

- at least a two-character name, with the middle 0 or more chars wild

- a wild match spanning multiple directories

">foo>*>bar" might imply:

- the middle directory is named "*"

- the middle directory is :WILD

- there may be zero or more :WILD middle directories

- the middle directory name matches any one-letter name

">foo>**>bar" might mean

- the middle directory is named "**"

- the middle directory is :WILD

- there may be zero or more :WILD middle directories

- the middle directory name matches any two-letter name

Some file systems support even more complex wildcards, for example

regular expressions.

The CL pathname model does not specify a way to represent complex

wildcards, which means, for example, that (MAKE-PATHNAME :NAME "F*O")

cannot be recognized by portable code as containing a wildcard.

Common Lisp provides only the first of these four common operations

on wildcard pathnames:

(1) Enumerate the set of existing files that match the pathname;

this is provided by the DIRECTORY function.

(2) Test whether a pathname contains wildcards.

(3) Test whether a pathname matches a wildcard pathname.

(4) Translate one pathname into another according to a mapping specified

by a pair of wildcard pathnames.

Proposal (PATHNAME-WILD:NEW-FUNCTIONS):

Introduce the following three functions:

1. WILD-PATHNAME-P pathname &optional field-key

Tests a pathname for the presence of wildcard components. If the first

argument is not a pathname, string, or file stream an error of type

TYPE-ERROR is signalled.

If no <field-key> is provided, or the <field-key> is NIL, the result is

true if <pathname> has any wildcard components, NIL if <pathname> has none.

If a non-null <field-key> is provided, it must be one of :HOST, :DEVICE,

:DIRECTORY, :NAME, :TYPE, or :VERSION. In this case, the result is true

if the indicated component of <pathname> is a wildcard, NIL if the

component is not a wildcard. Note that not all implementations

support wildcards in all fields, according to PATHNAME-COMPONENT-VALUE.

2. PATHNAME-MATCH-P pathname wildcard

true if <pathname> matches <wildcard>, otherwise NIL. The matching rules

are implementation-defined but should be consistent with the

DIRECTORY function. Missing components of <wildcard> default to :WILD.

If either argument is not a pathname, string, or file stream an error

of type TYPE-ERROR is signalled. It is valid for <pathname> to be a

wild pathname; a wildcard field in <pathname> will only match a

wildcard field in <wildcard>, i.e. the function is not commutative.

It is valid for <wildcard> to be a non-wild pathname.

3. TRANSLATE-PATHNAME source from-wildcard to-wildcard &key

Translates the pathname <source>, which matches <from-wildcard>, into

a corresponding pathname <result>, which matches <to-wildcard>, and

returns <result>.

The pathname <result> is <to-wildcard> with each wildcard or missing

field replaced by a portion of <source>. A "wildcard field" is a

pathname component with a value of :WILD, a :WILD element of a

list-valued directory component, or an implementation-defined portion

of a component, such as the "*" in the complex wildcard string

"foo*bar" that some implementations support. An implementation that

adds other wildcard features, such as regular expressions, must define

how TRANSLATE-PATHNAME extends to those features. A "missing field" is

a pathname component with a value of NIL.

The portion of <source> that is copied into <result> is implementation

defined. Typically it is determined by the user interface conventions

of the file systems involved. Usually it is the portion of <source>

that matches a wildcard field of <from-wildcard> that is in the same

position as the wildcard or missing field of <to-wildcard>. If there

is no wildcard field in <from-wildcard> at that position, then usually

it is the entire corresponding pathname component of <source>, or in

the case of a list-valued directory component, the entire corresponding

list element. For example, if the name components of <source>,

<from-wildcard>, and <to-wildcard> are "gazonk", "gaz*", and "h*"

respectively, then in most file systems, the wildcard fields of the

name component of <from-wildcard> and <to-wildcard> are each "*", the

matching portion of <source> is "onk", and the name component of

<result> is "honk". However, the exact behavior of TRANSLATE-PATHNAME

cannot be dictated by the Common Lisp language and must be allowed to

vary, depending on the user interface conventions of the file systems

involved.

During the copying of a portion of <source> into <result>, additional

implementation-defined translations of alphabetic case or file naming

conventions might occur, especially when <from-wildcard> and

<to-wildcard> are for different hosts.

If any of the first three arguments is not a pathname, string, or file

stream an error of type TYPE-ERROR is signalled. It is valid for

<source> to be a wild pathname; in general this will produce a wild

result. It is valid for <from-wildcard> and/or <to-wildcard> to be

non-wild pathnames. (PATHNAME-MATCH-P <source> <from-wildcard>) must

be true or an error is signalled.

There are no specified keyword arguments for TRANSLATE-PATHNAME, but

implementations are permitted to extend it by adding keyword arguments.

There is one specified return value from TRANSLATE-PATHNAME;

implementations are permitted to extend it by returning additional

values.

Implementation guideline: one file system performs this operation by

examining each piece of the three pathnames in turn, where a piece is a

pathname component or a list element of a structured component such as

a hierarchical directory. Hierarchical directory elements in

<from-wildcard> and <to-wildcard> are matched by whether they are

wildcards, not by depth in the directory hierarchy. If the piece in

<to-wildcard> is present and not wild, it is copied into the result.

If the piece in <to-wildcard> is :WILD or NIL, the piece in <source> is

copied into the result. Otherwise, the piece is <to-wildcard> might be

a complex wildcard such as "foo*bar" and the piece in <from-wildcard>

should be wild; the portion of the piece in <source> that matches the

wildcard portion of the piece in <from-wildcard> replaces the wildcard

portion of the piece in <to-wildcard> and the value produced is used in

the result.

4. Clarify that the functions OPEN (and WITH-OPEN-FILE), PROBE-FILE,

FILE-WRITE-DATE, FILE-AUTHOR, and TRUENAME only accept non-wildcard

pathnames and signal an error if given a pathname for which

WILD-PATHNAME-P returns true.

5. Clarify that the functions RENAME-FILE, DELETE-FILE, LOAD, and

COMPILE-FILE have implementation-defined consequences when given a

wildcard pathname. Each function might signal an error or might operate

on all files that match the wildcard pathname.

Examples:

;The following examples are not portable. They are written to run

;with particular file systems and particular wildcard conventions.

;Other implementations will behave differently. These examples are

;intended to be illustrative, not to be prescriptive.

(WILD-PATHNAME-P (MAKE-PATHNAME :NAME :WILD)) => T

(WILD-PATHNAME-P (MAKE-PATHNAME :NAME :WILD) :NAME) => T

(WILD-PATHNAME-P (MAKE-PATHNAME :NAME :WILD) :TYPE) => NIL

(WILD-PATHNAME-P (PATHNAME "S:>foo>**>")) => T ;Lispm

(WILD-PATHNAME-P (PATHNAME :NAME "F*O")) => T ;Most places

;This example assumes one particular set of wildcard conventions

;Not all file systems will run this example exactly as written

(DEFUN RENAME-FILES (FROM TO)

(DOLIST (FILE (DIRECTORY FROM))

(RENAME-FILE FILE (TRANSLATE-PATHNAME FILE FROM TO))))

(RENAME-FILES "/usr/me/*.lisp" "/dev/her/*.l")

;Renames /usr/me/init.lisp to /dev/her/init.l

(RENAME-FILES "/usr/me/pcl*/*" "/sys/pcl/*/")

;Renames /usr/me/pcl-5-may/low.lisp to /sys/pcl/pcl-5-may/low.lisp

;In some file systems the result might be /sys/pcl/5-may/low.lisp

(RENAME-FILES "/usr/me/pcl*/*" "/sys/library/*/")

;Renames /usr/me/pcl-5-may/low.lisp to /sys/library/pcl-5-may/low.lisp

;In some file systems the result might be /sys/library/5-may/low.lisp

(RENAME-FILES "/usr/me/foo.bar" "/usr/me2/")

;Renames /usr/me/foo.bar to /usr/me2/foo.bar

(RENAME-FILES "/usr/joe/*-recipes.text" "/usr/jim/cookbook/joe's-*-rec.text")

;Renames /usr/joe/lamb-recipes.text to /usr/jim/cookbook/joe's-lamb-rec.text

;Renames /usr/joe/pork-recipes.text to /usr/jim/cookbook/joe's-pork-rec.text

;Renames /usr/joe/veg-recipes.text to /usr/jim/cookbook/joe's-veg-rec.text

;This example assumes one particular set of wildcard conventions

(PATHNAME-NAME (TRANSLATE-PATHNAME "foobar" "foo*" "*baz")) => "barbaz"

(PATHNAME-NAME (TRANSLATE-PATHNAME "foobar" "foo*" "*")) => "foobar"

(PATHNAME-NAME (TRANSLATE-PATHNAME "foobar" "*" "foo*")) => "foofoobar"

(PATHNAME-NAME (TRANSLATE-PATHNAME "bar" "*" "foo*")) => "foobar"

;Using Unix syntax and the wildcard conventions used by the

;particular version of Unix on which I tried this:

(NAMESTRING

(TRANSLATE-PATHNAME "/usr/dmr/hacks/frob.l"

"/usr/d*/hacks/*.l"

"/usr/d*/backup/hacks/backup-*.*"))

=> "/usr/dmr/backup/hacks/backup-frob.l"

(NAMESTRING

(TRANSLATE-PATHNAME "/usr/dmr/hacks/frob.l"

"/usr/d*/hacks/fr*.l"

"/usr/d*/backup/hacks/backup-*.*"))

=> "/usr/dmr/backup/hacks/backup-ob.l"

;This is similar to the above example but uses two different hosts,

;U: which is a Unix and V: which is a VMS. Note the translation

;of file type and alphabetic case conventions.

(NAMESTRING

(TRANSLATE-PATHNAME "U:/usr/dmr/hacks/frob.l"

"U:/usr/d*/hacks/*.l"

"V:SYS$DISK:[D*.BACKUP.HACKS]BACKUP-*.*"))

=> "V:SYS$DISK:[DMR.BACKUP.HACKS]BACKUP-FROB.LSP"

(NAMESTRING

(TRANSLATE-PATHNAME "U:/usr/dmr/hacks/frob.l"

"U:/usr/d*/hacks/fr*.l"

"V:SYS$DISK:[D*.BACKUP.HACKS]BACKUP-*.*"))

=> "V:SYS$DISK:[DMR.BACKUP.HACKS]BACKUP-OB.LSP"

;This example presumes background information described in PATHNAME-LOGICAL

(DEFUN TRANSLATE-LOGICAL-PATHNAME-1 (PATHNAME RULES)

(LET ((RULE (ASSOC PATHNAME RULES :TEST #'PATHNAME-MATCH-P)))

(UNLESS RULE (ERROR "No translation rule for ~A" PATHNAME))

(TRANSLATE-PATHNAME PATHNAME (FIRST RULE) (SECOND RULE))))

(TRANSLATE-LOGICAL-PATHNAME-1 "FOO:CODE;BASIC.LISP"

'(("FOO:DOCUMENTATION;" "MY-UNIX:/doc/foo/")

("FOO:CODE;" "MY-UNIX:/lib/foo/")

("FOO:PATCHES;*;" "MY-UNIX:/lib/foo/patch/*/")))

=> the pathname MY-UNIX:/lib/foo/basic.l

Rationale:

1,2,3. These three functions provide a standardized interface to the

idiosyncratic wildcard functionality of each host file system.

1. WILD-PATHNAME-P makes it possible to detect wild pathnames reliably and

do something useful (give up, merge out the bothersome components, call

DIRECTORY for a list of matching pathnames, etc.)

2,3. TRANSLATE-PATHNAME is needed by many application programs that deal

with wildcard pathnames. PATHNAME-MATCH-P and TRANSLATE-PATHNAME are

needed by logical pathnames. The PATHNAME-LOGICAL proposal cannot be

implemented without these features. Implementing PATHNAME-LOGICAL could

involve adding additional capabilities to TRANSLATE-PATHNAME, depending

on the type of file system used, but those capabilities do not need to

be in the standard.

4. Since these functions return a value connected with one file, there

is no meaningful way to extend them to work on wildcard pathnames. It

seems best to specify that they signal an error, rather than leaving

the consequences undefined.

5. The consequences are proposed to be implementation-defined because

current practice varies and no one wants to change.

Current Practice:

Presumably no implementation supports the proposal exactly as stated.

Symbolics Genera has had similar features under different names for many

years:

(SEND pathname :WILD-P) returns a value such as NIL, :NAME, :TYPE,

etc., indicating the first wild field.

(SEND pathname :NAME-WILD-P), (SEND pathname :DIRECTORY-WILD-P),

etc. test individual fields.

The :TRANSLATE-WILD-PATHNAME, :TRANSLATE-WILD-PATHNAME-REVERSIBLE, and

:PATHNAME-MATCH messages resemble TRANSLATE-PATHNAME and

PATHNAME-MATCH-P.

The Explorer also supports the messages :WILD-P (although it only

returns NIL or T), :NAME-WILD-P, etc., :TRANSLATE-WILD-PATHNAME, and

:PATHNAME-MATCH.

Points 4 and 5 are current practice as far as the authors are aware.

The Explorer permits DELETE-FILE on a wild pathname, meaning to delete

all files that match.

Cost to Implementors:

Many implementations probably have a substrate which is capable of this

or something similar already. In such cases, it's a relatively small

matter to add the proposed interface.

Even in cases where an implementation doesn't have ready code, it's clearly

better for the implementor to write that code once and for all than to ask

each user of wildcards to write it.

Since the detailed behavior is at the implementor's discretion, the cost

is unlikely to be large. Some file systems will do all the work and the

implementor need only provide an interface to the file system or to a

standard library routine. For other file systems the implementor has to

write the actual matching and translation algorithms.

Cost to Users:

None. This change is upward compatible.

Cost of Non-Adoption:

Wild pathnames would continue to be mistaken for ordinary pathnames in

many situations. User programs that deal with wildcard pathnames would

have to operate on implementation-dependent representations and hence

would not be easily portable.

The biggest cost is that the logical pathnames proposal would be stymied.

Performance Impact:

None.

Benefits:

A more complete set of wildcard pathname operations. Portable user

programs that deal with wildcard pathnames will be more consistent

and reliable. A portable system construction tool can be written

and the foundations are laid for a `logical pathname' facility

(proposed separately in PATHNAME-LOGICAL).

Aesthetics:

This change would make some portable code less kludgey.

Discussion:

There was some question about the name. The name PATHNAME-WILD-P

suggests a ``slot'' of a pathname (like PATHNAME-HOST),

while WILD-PATHNAME-P suggests a type (like INPUT-STREAM-P).

The committee was split on what to call it. Since it is more

like a type than a slot, the name WILD-PATHNAME-P was chosen.

It's been suggested that WILD-PATHNAME-P and PATHNAME-MATCH-P be allowed

to return a value other than T to represent "truth", which would

somehow encode some additional information.


[Starting Points][Contents][Index][Symbols][Glossary][Issues]
Copyright 1996-2005, LispWorks Ltd. All rights reserved.