generate.copy.outer option question

[gdelory]

Hi all,

I don't really understand how the generate.copy.outer option is supposed to work. From what the documentation says, it is supposed to control if content out of the input/map's directory (or what I like to call out-of-scope) is generated or not, and if the map's folder is pushed down in the output hierarchy so all content can be included.

However, I've tried this with a very small sample with an <xref href="../other_folder/task_in_another_folder.dita" scope="local"> (I know local is the default for relative link, just wanted to make it obvious) on both DITA-OT 3.3.4 (which is what we use today), and the latest (4.2.3), and got the exact same output with -Dgenerate.copy.outer=1 and -Dgenerate.copy.outer=3. The equivalent of the map's folder in the output folder is pushed down one level, and other_folder/task_in_another_folder.html is always generated. The only difference I see is that -Dgenerate.copy.outer=1 issues warnings in the logs, and -Dgenerate.copy.outer=3 doesn't:

Warning: file:/home/gui/dcs_doc/test-oos/minimum_sample/xref_local/task.dita:9:83: [DOTJ036W] The 'file:/home/gui/dcs_doc/test-oos/minimum_sample/other_folder/task_in_another_folder.dita' resource is outside the scope of the input directory.

Here is the result of those 4 runs of the dita-ot:

/home/gui/aem_dita/dita-ot-3.3.4-ootb/bin/dita -t temp -f html5 -o out_334_outer1 -v -i map.ditamap -Dgenerate.copy.outer=1 | tee logs/out_334_outer1.txt
/home/gui/aem_dita/dita-ot-4.2.3-ootb/bin/dita -t temp -f html5 -o out_423_outer1 -v -i map.ditamap -Dgenerate.copy.outer=1 | tee logs/out_423_outer1.txt
/home/gui/aem_dita/dita-ot-3.3.4-ootb/bin/dita -t temp -f html5 -o out_334_outer3 -v -i map.ditamap -Dgenerate.copy.outer=3 | tee logs/out_334_outer3.txt
/home/gui/aem_dita/dita-ot-4.2.3-ootb/bin/dita -t temp -f html5 -o out_423_outer3 -v -i map.ditamap -Dgenerate.copy.outer=3 | tee logs/out_423_outer3.txt

Is it the expected behavior? If this is, then what does the option do? Am I supposed to use it together with outer.control to change the behavior and to not generate content outside the map's folder?

Here is my minimum sample to reproduce. I left the logs of all 4 runs, and the output folders so you don't have to run it yourself 😉

Edit: I also tried with an image (<image href="../other_folder/dcs.png" scope="local"><alt>DCS icon</alt></image>) and got the same behavior, ./other_folder/dcs.png is included regardless of the value of the generate.copy.outer option.

Thanks for your help.

[gdelory]

I had a little bit more time to look at that, and wanted to see how the generate.copy.outer option is affecting the processing. I initially looked a bit at the gen-list target in the preprocess (1), because that's what the html5 transform is using in 3.3.4, and it seems that the GenMapAndTopicListModule class is using the GenListModuleReader, which here seems to just issue a warning if the file is out of scope, and fail if the outer.control is set to fail. But if set to warn (default), I don't think it excludes anything, it basically just issues a warning (DOTJ036W).

But after that, I quickly moved on because I thought it would make more sense to investigate this in the dita-ot 4.3.2 and the html5 transform, which uses preprocess2, which is not using gen-list. This is a bit more tricky for me to follow, but I think basically the preprocess2.maps target, and more exactly the map-reader target seems to be where files are read and processed for the first time, and this seems to be done in the MapReaderModule Java class, which I think eventually also uses the GenListModuleReader when processing files (via waitlist).

I'm not so sure yet how preprocess2 really handles all filesets, so I'll keep doing some investigation to understand how files are passed, references followed, and filesets bult. I did see here that a set of outDitaFiles is built, but I'm not so sure yet if it's supposed to always be processed regardless of the value of generate.copy.outer.

Sorry, lots of information dump here without really any logic, but I'm just tracking my investigation here, and sharing in case someone is wondering the same thing on this mysterious option 😉

[gdelory]

So looking more into that. I think that when GenListModuleReader is parsing attributes, if this is an href attribute, this is where it checks if the file is out of the map's folder by calling toOutFile and issue warning/fail depending on generate.copy.outer and outer.control, and adds it to the outDitaFilesSet if necessary. The parsing is reached from the waitlist above, so all files are parsed that way I assume.

Except this, nothing else happens to the out-of-scope topics, so I'll check if this outDitaFilesSet has any influence later on, and is supposed to stop the generation of those files if generate.copy.outer is set to 1 (default)