Mechanics of Filesystem Rules

This page describes in detail how filesystem rules work in Pathwinder, both in isolation and in tandem with one another. It is accordingly divided into two main sections, the first covering how individual rules work in isolation, and the second showing how they fit together and interact.

Single Rule in Isolation

A single filesystem rule operating alone is powerful in its own right. Depending how it is configured, it can have the effect of presenting an illusionary view of the filesystem that looks like entire directory sub-trees have been introduced or replaced. Just one filesystem rule acting alone is the easiest situation to understand and is a good place to start.

The subsections that follow are example-based and organized roughly in order of increasing complexity. This section's top-level division is based on a filesystem rule's redirection mode, which is a configurable property.

Simple Redirection

When a filesystem rule is configured with "Simple" as its redirection mode, its behavior is oriented towards replacement of files and subdirectories. If an application makes a request to access a particular file using a path located in the origin directory, and that file is within the scope of a filesystem rule that uses "Simple" as its redirection mode, then the request is simply (and unconditionally) redirected to the target directory. If the corresponding file exists in the target directory then the application can access it, otherwise the application is told that the file does not exist.

In more concrete terms, Pathwinder hides from the application anything in the origin directory that falls within the scope of a filesystem rule, replacing any files and directories with whatever exists in the target directory that is also in scope for the same filesystem rule. The examples that follow will make this behavior more clear.

Entire Directory Replacement

Replacing, or introducing, an entire directory is the easiest and most basic way to configure a filesystem rule. Suppose we configure a filesystem rule, as follows.

[FilesystemRule:EntireDirectoryReplacement]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir
; The line below is optional because Simple is the default setting.
RedirectMode = Simple

This rule instructs Pathwinder to replace all files in "C:\AppDir\DataDir", the origin directory, with the files in "C:\TargetDir", the target directory. Because the rule contains no file patterns, it applies to all files in the origin and target directories and hence any real files that exist in "C:\AppDir\DataDir" would be hidden.

There are three valid, possible initial states for the real origin directory in order for the rule above to take effect, and in all three cases the outcome is the same. For now, assume the target directory exists and is not empty.

• Origin directory "C:\AppDir\DataDir" does not exist

  

• Origin directory "C:\AppDir\DataDir" exists but is empty

  

• Origin directory "C:\AppDir\DataDir" exists and is not empty

  

When the application is executed it is presented with the illusionary view of the filesystem shown below. The contents of the origin directory, if any, have been completely replaced with the contents of the target directory. Alternatively, if the origin directory does not exist as a real filesystem directory, Pathwinder still presents it to the application as if it did exist.

If the application attempts to open "C:\AppDir\DataDir\TextFile.txt" the request will transparently be redirected so that it instead opens "C:\TargetDir\TextFile.txt". On the flipside, if the application knows about the target directory and attempts to open "C:\TargetDir\TextFile.txt" then that is the file it will actually open, without any redirection. This is not a problem, because applications generally only care about the files with which they intend to work, which would typically be located on the origin directory side rather than the target directory side.

The same logic applies if the application attempts to create a new file in "C:\AppDir\DataDir". Because of the influence of the filesystem rule, the file creation will be redirected to "C:\TargetDir", and once created it will be presented to the application as part of the illusion. Continuing the same example, suppose the application writes some data to the file "C:\AppDir\DataDir\Data.dat". Once that file is created, the image below shows what the application can see. The new file can be accessed on both the origin and target sides, but generally the application will only do so on the origin side.

Pathwinder works exactly the same way with subdirectories. The application can access "C:\AppDir\DataDir\TargetSub" as well as any files and further subdirectories, and the accesses will be redirected to the corresponding path in the target directory.

At the start of this example we assumed that the target directory already exists and is not empty. However, that was not a requirement, and Pathwinder is perfectly able to handle a different situation. If we instead suppose that the target directory is either empty or does not exist, then:

• When the application first runs, "C:\AppDir\DataDir" appears to be completely empty.
  • The filesystem rule tells Pathwinder to replace the entire contents of the origin directory with the contents of the target directory. Since the target directory does not have any contents, the origin directory likewise appears not to have any contents.
• When the application later creates the file "C:\AppData\DataDir\Data.dat" the behavior is the same as described earlier in this example. It is really saved into the target directory, and the application can see it as being present in the origin directory.
  • If the target directory does not exist at this point, Pathwinder will create it.

Partial Directory Replacement

Adding file patterns to filesystem rules can restrict their scope so that they act on a subset of the files contained in a directory rather than on an entire directory. File patterns can, but are not required to, use wildcard characters to match multiple filenames.

A filesystem rule is not limited to just a single file pattern. If a filesystem rule has multiple file patterns, then a filename is considered a match if it matches any file pattern. Multiple file patterns are used to expand, not restrict, the scope of a filesystem rule.

Suppose we configure a filesystem rule, as follows.

[FilesystemRule:PartialDirectoryReplacement]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir
FilePattern = *.txt
; The line below is optional because Simple is the default setting.
RedirectMode = Simple

This rule instructs Pathwinder to replace all *.txt files in "C:\AppDir\DataDir", the origin directory, with all *.txt files in "C:\TargetDir", the target directory. The file pattern restricts the scope of the rule so that it only operates on files whose names match the "*.txt" pattern.

Pathwinder behaves very similarly as in the previous example, but this time it only acts on *.txt files. Key differences:

• Only *.txt files present in the target directory are visible in the origin directory. Files that exist in the target directory but that do not match the *.txt pattern are not visible in the origin directory.
• Only *.txt files that exist in the origin directory are hidden. Files that exist in the origin directory but that do not match the *.txt pattern are still visible and available for access without any redirection.

To better understand concretely how Pathwinder will behave in this situation we will go through two examples. First we will simplify the scenario and assume that neither the origin nor the target directories contain any subdirectories. Next we will allow subdirectories to exist and discuss how the presence of subdirectories interacts with file patterns in filesystem rules.

Without Subdirectories

Suppose the state of the real filesystem is as shown below. Two files, "1stOrigin.txt" on the origin side and "3rdTarget.txt" on the target side, match the filesystem rule's file pattern and are hence considered in scope for the rule.

Based on the behavior described previously, when the application runs Pathwinder will present it with an illusionary filesystem state, as shown in the image below. Note that, on the origin side, "1stOrigin.txt" is hidden and "3rdTarget.txt" is present. This is how Pathwinder replaces all of the *.txt files in the origin side with the *.txt files on the target side.

Suppose, once again, that the application creates the file "C:\AppDir\DataDir\Data.dat" just as it did before. Because the filename "Data.dat" does not match the file pattern "*.txt" it is unaffected by the filesystem rule, so the file is really written to the origin directory. The image below shows the state of the filesystem that the application sees afterwards. Note that the new file is only visible on the origin side (it really exists there), not on the target side.

Finally, suppose that the application creates the file "C:\AppDir\DataDir\Output.txt". The filename "Output.txt" does match the rule's file pattern, and so this file is redirected. The file will actually be created in the target directory and will be visible to the application in the origin directory. The image below shows the state of the filesystem that the application sees afterwards.

With Subdirectories

Suppose the state of the real filesystem is as shown below.

A filesystem rule's scope is determined by comparing its file patterns to the immediate contents of its origin and target directories, without regard for how deep or nested the actual requested path is. As before, the two files "1stOrigin.txt" on the origin side and "3rdTarget.txt" on the target side match the filesystem rule's file pattern and are hence considered in scope for the rule. Other files, like "2ndOrigin.bin" on the origin side and "4thTarget.log" on the target side, do not match and are therefore out-of-scope.

For subdirectories, it is the name of the subdirectory itself that is checked for a file pattern match because the subdirectory itself is the immediate content of the origin or target directory. If a subdirectory is in scope then all of the files and directories contained within it are also in scope, regardless of their filenames. Conversely, if a subdirectory is out-of-scope, then all of its contents are also out-of-scope.

In this example, subdirectories "OriginSubB.txt" on the origin side and "TargetSubB.txt" on the target side are considered in scope, and the other two subdirectories are out-of-scope. After applying the example filesystem rule, what the application will ultimately see is as shown in the image below.

Changes from the initial state:

• File "1stOrigin.txt" is hidden
• File "3rdTarget.txt" is present in the origin directory
• Directory "OriginSubB.txt" is hidden, along with all of its contents
• Directory "TargetSubB.txt" is present in the origin directory, along with all of its contents

Overlay Redirection

When a filesystem rule is configured with "Overlay" as its redirection mode, its behavior is oriented towards merging the contents of two directories, including both files and subdirectory trees. If an application makes a request to access a particular file using a path located in the origin directory, and that file is within the scope of a filesystem rule, then the request may be redirected to the target directory. Most fundamentally:

• If the file can be accessed on the target side, then the file operation is redirected.
• Otherwise, the file operation is not redirected, and the application is permitted to access the file directly on the origin side.

In other words, if a file can be accessed on both the origin side and the target side, then Pathwinder will prefer the target side. Otherwise Pathwinder will allow the application to access the file wherever it happens to be located. Just like rules that use "Simple" redirect mode, "Overlay" redirect mode rules support file patterns to limit their scope.

There is a bit of nuance to exactly what "can be accessed" means for a given file operation, which depends on whether the application is trying to create a new file, open an existing file, or expresses no preference either way. If the application explicitly requires that a new file be created or explicitly requires that an existing file be opened, then Pathwinder will honor that requirement and try the target-side path before trying the origin-side path. However, if the application expresses no preference ("create the file if it doesn't exist, or open an existing file if it does"), then Pathwinder will make multiple attempts in order, stopping at the first successful one:

1. Open an existing file on the target side.
2. Open an existing file on the origin side.
3. Create a new file on the target side.
4. Create a new file on the origin side.

As an example of how overlay redirection works, suppose we begin with the filesystem state shown in the image below.

The example is separated into two parts, based on whether or not the filesystem rule in question has file patterns.

Without File Patterns

Suppose we create the following filesystem rule that does not use any file patterns and hence matches all files.

[FilesystemRule:OverlayWithoutFilePatterns]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir
RedirectMode = Overlay

This rule instructs Pathwinder to merge all files in "C:\AppDir\DataDir", the origin directory, with the files in "C:\TargetDir", the target directory such that the latter takes precedence. What the application is able to see with this rule active is shown in the image below.

All of the files and subdirectories that really exist in the origin directory are still visible to the application, and the files and subdirectories in the target directory are also visible in the origin directory. One file, "MoreData.txt\OutputB.log", exists on both the origin side and on the target side. If the application opens this file, it will really be opening the version that exists on the target side.

With File Patterns

Suppose we create the following filesystem rule that only matches .txt files.

[FilesystemRule:OverlayWithFilePatterns]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir
FilePattern = *.txt
RedirectMode = Overlay

Because the scope of the rule is limited to the immediate contents of the origin and target directories that match the file pattern, not all of the files and subdirectories on the target side are merged into the origin directory. The image below shows what the application is able to see.

File pattern logic works the same way with "Overlay" as the redirection mode as it does with "Simple" as the redirection mode.

Multiple Rules in Combination

Pathwinder configuration files can define multiple rules, and Pathwinder automatically selects and applies the correct rule for any given situation. Each individual rule behaves as described in the previous section, but there are some behavioral nuances that arise when multiple rules exist simultaneously.

The subsections that follow are example-based and organized roughly in order of increasing complexity. This section's top-level division is based on the relationship between the origin directories of the filesystem rules, either totally unrelated, related by ancestry within the filesystem hierarchy, or exactly the same. Note that Pathwinder constrains target directories such that they are completely unrelated to one another and to all origin directories, which is why target directories are not explicitly covered in this section.

Unrelated Origin Directories

Multiple rules with completely unrelated origin directories present the most straightforward possible situation because they act entirely independently of one another.

Suppose we define two rules, as follows. These two rules operate on entirely different parts of the filesystem hierarchy and thus there is no overlap between their scopes.

[FilesystemRule:UnrelatedOriginDirectories1]
OriginDirectory = C:\OriginSide\Origin1
TargetDirectory = C:\TargetSide\Target1

[FilesystemRule:UnrelatedOriginDirectories2]
OriginDirectory = C:\OriginSide\Origin2
TargetDirectory = C:\TargetSide\Target2
FilePattern = *.txt

If the real state of the filesystem is:

Then the application will see:

The contents of "C:\OriginSide\Origin1" are from first rule, and the contents of "C:\OriginSide\Origin2" are from the second rule.

Related Origin Directories

Two filesystem rules have related origin directories if the origin directory of one rule is a filesystem ancestor of the origin directory of the other. In other words, this means we can reach the origin directory of one rule by starting at the origin directory of the other rule and removing the last directory from its path one or more times. For example, "C:\OriginSide" and "C:\OriginSide\Subdir1\Subdir2\Subdir3" are related because we can repeatedly remove the last directory from the path of the latter (Subdir3, then Subdir2, and finally Subdir1) to arrive at the former. On the other hand, C:\OriginSide and C:\OriginSide1234 are not related.

Suppose we define one filesystem rule as follows.

[FilesystemRule:RelatedOriginDirectories1]
OriginDirectory = C:\OriginSide\Level1
TargetDirectory = C:\TargetSide\Dir1

If we access the file "C:\OriginSide\Level1\Level2\TextFile.txt" then it would be redirected to "C:\TargetSide\Dir1\Level2\TextFile.txt" because it is within the scope of the filesystem rule. But what happens if we had a second filesystem rule, as below?

[FilesystemRule:RelatedOriginDirectories2]
OriginDirectory = C:\OriginSide\Level1\Level2
TargetDirectory = C:\TargetSide\Dir2

The path "C:\OriginSide\Level1\Level2\TextFile.txt" is in scope for both rules, and so both "C:\TargetSide\Dir1\Level2\TextFile.txt" and "C:\TargetSide\Dir2\TextFile.txt" would be valid redirections, but only one can actually be used.

Pathwinder resolves this ambiguity by selecting only the rule that has the deepest origin directory, which is another way of saying that filesystem rules with narrower scopes are given precedence over those with wider scopes. In this example, "C:\OriginSide\Level1\Level2" is deeper in the filesystem (and hence encompasses a narrower scope) than "C:\OriginSide\Level1" so the second rule is selected and the redirection occurs to "C:\TargetSide\Dir2\TextFile.txt".

File patterns add further nuance to the rule selection process. Once Pathwinder has selected a particular origin directory for rule evaluation it will not try any other origin directories, even if no file patterns match and a rule higher up in the origin directory hierarchy might match. To see how this nuance affects rule selection, suppose we redefine the second rule by adding a file pattern to it, as follows.

[FilesystemRule:RelatedOriginDirectories2]
OriginDirectory = C:\OriginSide\Level1\Level2
TargetDirectory = C:\TargetSide\Dir2
FilePattern = *.bin

The path "C:\OriginSide\Level1\Level2\BinFile.bin" works exactly as before because it is in scope for both rules. However, the path "C:\OriginSide\Level1\Level2\TextFile.txt" does not match the file pattern for the second rule and so is out of its scope. At this point Pathwinder has already selected for evaluation the origin directory "C:\OriginSide\Level1\Level2" and it will not try any others, so the conclusion is that no rules match and hence no redirection takes place.

Conceptually, the key point of this subsection is that origin directories define scopes of ownership. Filesystem rules with a particular origin directory own all of the possible redirections for that origin directory and its descendents in the filesystem hierarchy. When two filesystem rules with different origin directories have overlapping potential ownership claims, the filesystem rule with the deeper origin directory wins.

Same Origin Directories

Multiple filesystem rules can share the same origin directory. The intention of this feature is that they would be differentiated by their file patterns so that different sets of files and subdirectories could be redirected to separate locations. However, Pathwinder does not check or attempt to enforce file patterns being non-overlapping or different in any way. This is left entirely to the user.

Pathwinder follows a specific evaluation order when selecting which rule, from among those that share the same origin directory, it should use to redirect a file operation. Rules are sorted first in descending order based on how many file patterns they define and second in ascending order by their names. This is entirely irrespective of the order in which they appear in a configuration file.

As an example, suppose we define the following four filesystem rules.

[FilesystemRule:CatchAll]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir\CatchAll

[FilesystemRule:TxtFilesOnly]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir\TxtFilesOnly
FilePattern = *.txt
RedirectMode = Overlay

[FilesystemRule:BinAndLogFilesOnly]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir\BinAndLogFilesOnly
FilePattern = *.bin
FilePattern = *.log

[FilesystemRule:ExeFilesOnly]
OriginDirectory = C:\AppDir\DataDir
TargetDirectory = C:\TargetDir\ExeFilesOnly
FilePattern = *.exe

The order in which Pathwinder would evaluate these rules:

1. BinAndLogFilesOnly
   • This rule has 2 file patterns
2. ExeFilesOnly
   • This rule has 1 file pattern, and its name is alphabetically first among the rules with 1 file pattern
3. TxtFilesOnly
   • This rule has 1 file pattern, and its name is alphabetically second among the rules with 1 file pattern
4. CatchAll
   • This rule has 0 file patterns

All of the file patterns themselves are non-overlapping, so precedence between all rules except CatchAll has no real effect. However, the scope of CatchAll includes all files in the origin directory, but because it is last in the evaluation order, it only acts on those paths that do not match any of the other three rules.

The same logic applies to determining which files present in the origin directory are visible and accessible to the application. In the example above, TxtFilesOnly uses the "Overlay" as its redirection mode, whereas all of the other rules default to "Simple". Normally, CatchAll would cause all files in the origin directory to be hidden from the application, including those that match the *.txt file pattern. However, because TxtFilesOnly takes precedence over CatchAll, and TxtFilesOnly uses "Overlay" as its redirection mode, any *.txt files really present in the origin directory will be made visible to the application.