Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for an alternative current dir mode in pkldoc #824

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion docs/modules/pkl-doc/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -232,10 +232,22 @@ Relative URIs are resolved against the working directory.
[%collapsible]
====
Default: (none) +
Example: `pkldoc`
Example: `pkldoc` +
The directory where generated documentation is placed.
====

.--current-directory-mode
Copy link
Contributor

@odenix odenix Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless you foresee a third mode, a flag would be easier to use and understand.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think that it might be possible to eventually have a mode to disable current generation at all.

Also, I'm not very sure about a binary flag to be easier to understand to be honest. It is not disabling or enabling something here, it is changing the logic of operation between two states. Enums are supposed to be the tool for this (one of discussions).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @translatenix here; I think this should be called --no-symlinks, and default to false. It's not necessarily about how the "current" dir is treated; the intention is to disable symlinks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point that I had failed to notice.

[%collapsible]
====
Default: `symlink` +
Example: `copy` +
How the "current" directories containing documentation content for the last generated version
should be created. By default, in the symlink mode, a symbolic link is created pointing to the
last generated version. In the copy mode, a full copy of the last generated version
is created. Symbolic links are not processed correctly by some systems (e.g. GitHub Pages),
so the copy mode is occasionally useful.
====

Common CLI options:

include::../../pkl-cli/partials/cli-common-options.adoc[]
Expand Down
12 changes: 12 additions & 0 deletions docs/modules/pkl-gradle/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -515,6 +515,18 @@ Example: `outputDir = layout.projectDirectory.dir("pkl-docs")` +
The directory where generated documentation is placed.
====

.currentDirectoryMode: Property<CurrentDirectoryMode>
[%collapsible]
====
Default: `DocsiteGenerator.CurrentDirectoryMode.SYMLINK` +
Example: `currentDirectoryMode = DocsiteGenerator.CurrentDirectoryMode.COPY` +
How the "current" directories containing documentation content for the last generated version
should be created. By default, in the symlink mode, a symbolic link is created pointing to the
last generated version. In the copy mode, a full copy of the last generated version
is created. Symbolic links are not processed correctly by some systems (e.g. GitHub Pages),
so the copy mode is occasionally useful.
====

Common properties:

include::../partials/gradle-modules-properties.adoc[]
Expand Down
14 changes: 14 additions & 0 deletions pkl-commons/src/main/kotlin/org/pkl/commons/Paths.kt
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import java.nio.charset.Charset
import java.nio.file.*
import java.nio.file.attribute.FileAttribute
import java.util.stream.Stream
import kotlin.io.path.copyTo
import kotlin.io.path.createDirectories
import kotlin.io.path.deleteIfExists
import kotlin.io.path.exists
Expand Down Expand Up @@ -72,6 +73,19 @@ fun Path.deleteRecursively() {
}
}

@Throws(IOException::class)
fun Path.copyRecursively(target: Path) {
if (exists()) {
target.createParentDirectories()
walk().use { paths ->
paths.forEach { src ->
val dst = target.resolve([email protected](src))
src.copyTo(dst, overwrite = true)
}
}
}
}

private val isWindows by lazy { System.getProperty("os.name").contains("Windows") }

/** Copy implementation from IoUtils.toNormalizedPathString */
Expand Down
3 changes: 2 additions & 1 deletion pkl-doc/src/main/kotlin/org/pkl/doc/CliDocGenerator.kt
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,8 @@ class CliDocGenerator(private val options: CliDocGeneratorOptions) : CliCommand(
importedModules::getValue,
versionComparator,
options.normalizedOutputDir,
options.isTestMode
options.isTestMode,
options.currentDirectoryMode
)
.run()
} catch (e: DocGeneratorException) {
Expand Down
13 changes: 12 additions & 1 deletion pkl-doc/src/main/kotlin/org/pkl/doc/CliDocGeneratorOptions.kt
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,18 @@ constructor(
* Generates source URLs with fixed line numbers `#L123-L456` to avoid churn in expected output
* files (e.g., when stdlib line numbers change).
*/
val isTestMode: Boolean = false
val isTestMode: Boolean = false,

/**
* Determines how to create the "current" directory which contains documentation for the latest
* version of the package.
*
* [DocGenerator.CurrentDirectoryMode.SYMLINK] will make the current directory into a symlink to
* the actual version directory. [DocGenerator.CurrentDirectoryMode.COPY], however, will create a
* full copy instead.
*/
var currentDirectoryMode: DocGenerator.CurrentDirectoryMode =
DocGenerator.CurrentDirectoryMode.SYMLINK
) {
/** [outputDir] after undergoing normalization. */
val normalizedOutputDir: Path = base.normalizedWorkingDir.resolveSafely(outputDir)
Expand Down
50 changes: 42 additions & 8 deletions pkl-doc/src/main/kotlin/org/pkl/doc/DocGenerator.kt
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ import java.io.IOException
import java.net.URI
import java.nio.file.Path
import kotlin.io.path.createSymbolicLinkPointingTo
import kotlin.io.path.deleteIfExists
import kotlin.io.path.exists
import kotlin.io.path.isSameFileAs
import org.pkl.commons.copyRecursively
import org.pkl.commons.deleteRecursively
import org.pkl.core.ModuleSchema
import org.pkl.core.PClassInfo
Expand Down Expand Up @@ -53,6 +53,7 @@ class DocGenerator(

/** A comparator for package versions. */
versionComparator: Comparator<String>,

/** The directory where generated documentation is placed. */
private val outputDir: Path,

Expand All @@ -62,9 +63,20 @@ class DocGenerator(
* Generates source URLs with fixed line numbers `#L123-L456` to avoid churn in expected output
* files (e.g., when stdlib line numbers change).
*/
private val isTestMode: Boolean = false
private val isTestMode: Boolean = false,

/**
* Determines how to create the "current" directory which contains documentation for the latest
* version of the package.
*
* [CurrentDirectoryMode.SYMLINK] will make the current directory into a symlink to the actual
* version directory. [CurrentDirectoryMode.COPY], however, will create a full copy instead.
*/
private val currentDirectoryMode: CurrentDirectoryMode = CurrentDirectoryMode.SYMLINK
) {
companion object {
const val CURRENT_DIRECTORY_NAME = "current"

internal fun List<PackageData>.current(
versionComparator: Comparator<String>
): List<PackageData> {
Expand Down Expand Up @@ -105,7 +117,7 @@ class DocGenerator(
val packagesData = packageDataGenerator.readAll()
val currentPackagesData = packagesData.current(descendingVersionComparator)

createSymlinks(currentPackagesData)
createCurrentDirectories(currentPackagesData)

htmlGenerator.generateSite(currentPackagesData)
searchIndexGenerator.generateSiteIndex(currentPackagesData)
Expand All @@ -120,16 +132,38 @@ class DocGenerator(
outputDir.resolve("$name/$version").deleteRecursively()
}

private fun createSymlinks(currentPackagesData: List<PackageData>) {
private fun createCurrentDirectories(currentPackagesData: List<PackageData>) {
for (packageData in currentPackagesData) {
val basePath = outputDir.resolve(packageData.ref.pkg.pathEncoded)
val src = basePath.resolve(packageData.ref.version)
val dest = basePath.resolve("current")
if (dest.exists() && dest.isSameFileAs(src)) continue
dest.deleteIfExists()
dest.createSymbolicLinkPointingTo(IoUtils.relativize(src, basePath))
val dst = basePath.resolve(CURRENT_DIRECTORY_NAME)

when (currentDirectoryMode) {
CurrentDirectoryMode.SYMLINK -> {
if (!dst.exists() || !dst.isSameFileAs(src)) {
dst.deleteRecursively()
dst.createSymbolicLinkPointingTo(IoUtils.relativize(src, basePath))
}
}
CurrentDirectoryMode.COPY -> {
dst.deleteRecursively()
src.copyRecursively(dst)
}
}
}
}

/**
* Determines how to create the "current" directory with the contents of the latest generated
* version of the package docs.
*/
enum class CurrentDirectoryMode {
/** Create a symlink pointing to the directory with the latest version of documentation. */
SYMLINK,

/** Make a full copy of the directory with the latest version of documentation. */
COPY
}
}

internal class DocPackage(val docPackageInfo: DocPackageInfo, val modules: List<ModuleSchema>) {
Expand Down
14 changes: 13 additions & 1 deletion pkl-doc/src/main/kotlin/org/pkl/doc/Main.kt
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,10 @@ import com.github.ajalt.clikt.parameters.arguments.argument
import com.github.ajalt.clikt.parameters.arguments.convert
import com.github.ajalt.clikt.parameters.arguments.multiple
import com.github.ajalt.clikt.parameters.groups.provideDelegate
import com.github.ajalt.clikt.parameters.options.default
import com.github.ajalt.clikt.parameters.options.option
import com.github.ajalt.clikt.parameters.options.required
import com.github.ajalt.clikt.parameters.types.enum
import com.github.ajalt.clikt.parameters.types.path
import java.net.URI
import java.nio.file.Path
Expand Down Expand Up @@ -57,6 +59,15 @@ class DocCommand :
.path()
.required()

private val currentDirectoryMode: DocGenerator.CurrentDirectoryMode by
option(
names = arrayOf("--current-directory-mode"),
metavar = "<mode>",
help = "How current directory should be created (as a symlink or as a full copy)"
)
.enum<DocGenerator.CurrentDirectoryMode>()
.default(DocGenerator.CurrentDirectoryMode.SYMLINK)

private val projectOptions by ProjectOptions()

override fun run() {
Expand All @@ -67,7 +78,8 @@ class DocCommand :
projectOptions,
),
outputDir,
true
true,
currentDirectoryMode
)
CliDocGenerator(options).run()
}
Expand Down
40 changes: 34 additions & 6 deletions pkl-doc/src/test/kotlin/org/pkl/doc/CliDocGeneratorTest.kt
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ import org.pkl.commons.test.FileTestUtils
import org.pkl.commons.test.PackageServer
import org.pkl.commons.test.listFilesRecursively
import org.pkl.commons.toPath
import org.pkl.commons.walk
import org.pkl.core.Version
import org.pkl.core.util.IoUtils
import org.pkl.doc.DocGenerator.Companion.current
Expand Down Expand Up @@ -113,7 +114,12 @@ class CliDocGeneratorTest {
"svg",
)

private fun runDocGenerator(outputDir: Path, cacheDir: Path?) {
private fun runDocGenerator(
outputDir: Path,
cacheDir: Path?,
currentDirectoryMode: DocGenerator.CurrentDirectoryMode =
DocGenerator.CurrentDirectoryMode.SYMLINK
) {
CliDocGenerator(
CliDocGeneratorOptions(
CliBaseOptions(
Expand All @@ -130,7 +136,8 @@ class CliDocGeneratorTest {
moduleCacheDir = cacheDir
),
outputDir = outputDir,
isTestMode = true
isTestMode = true,
currentDirectoryMode = currentDirectoryMode
)
)
.run()
Expand Down Expand Up @@ -260,15 +267,36 @@ class CliDocGeneratorTest {
}

@Test
fun `creates a symlink called current`(@TempDir tempDir: Path) {
fun `creates a symlink called current by default`(@TempDir tempDir: Path) {
PackageServer.populateCacheDir(tempDir)
runDocGenerator(actualOutputDir, tempDir)

val expectedSymlink = actualOutputDir.resolve("com.package1/current")
val expectedDestination = actualOutputDir.resolve("com.package1/1.2.3")
org.junit.jupiter.api.Assertions.assertTrue(Files.isSymbolicLink(expectedSymlink))
org.junit.jupiter.api.Assertions.assertTrue(
Files.isSameFile(expectedSymlink, expectedDestination)

assertThat(expectedSymlink).isSymbolicLink().matches {
Files.isSameFile(it, expectedDestination)
}
}

@Test
fun `creates a copy of the latest output called current when configured`(@TempDir tempDir: Path) {
PackageServer.populateCacheDir(tempDir)
runDocGenerator(
actualOutputDir,
tempDir,
currentDirectoryMode = DocGenerator.CurrentDirectoryMode.COPY
)

val currentDirectory = actualOutputDir.resolve("com.package1/current")
val sourceDirectory = actualOutputDir.resolve("com.package1/1.2.3")

assertThat(currentDirectory).isDirectory()

val expectedFiles = sourceDirectory.walk().map(sourceDirectory::relativize).toList()
val actualFiles = currentDirectory.walk().map(currentDirectory::relativize).toList()

assertThat(actualFiles).hasSameElementsAs(expectedFiles)
}

@Test
Expand Down
9 changes: 8 additions & 1 deletion pkl-gradle/src/main/java/org/pkl/gradle/PklPlugin.java
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
import org.pkl.core.util.IoUtils;
import org.pkl.core.util.LateInit;
import org.pkl.core.util.Nullable;
import org.pkl.doc.DocGenerator.CurrentDirectoryMode;
import org.pkl.gradle.spec.AnalyzeImportsSpec;
import org.pkl.gradle.spec.BasePklSpec;
import org.pkl.gradle.spec.CodeGenSpec;
Expand Down Expand Up @@ -249,8 +250,14 @@ private void configurePkldocTasks(NamedDomainObjectContainer<PkldocSpec> specs)
.getBuildDirectory()
.map(it -> it.dir("pkldoc").dir(spec.getName())));

spec.getCurrentDirectoryMode().convention(CurrentDirectoryMode.SYMLINK);

createModulesTask(PkldocTask.class, spec)
.configure(task -> task.getOutputDir().set(spec.getOutputDir()));
.configure(
task -> {
task.getOutputDir().set(spec.getOutputDir());
task.getCurrentDirectoryMode().set(spec.getCurrentDirectoryMode());
});
});
}

Expand Down
4 changes: 4 additions & 0 deletions pkl-gradle/src/main/java/org/pkl/gradle/spec/PkldocSpec.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,12 @@
package org.pkl.gradle.spec;

import org.gradle.api.file.DirectoryProperty;
import org.gradle.api.provider.Property;
import org.pkl.doc.DocGenerator.CurrentDirectoryMode;

/** Configuration options for Pkldoc generators. Documented in user manual. */
public interface PkldocSpec extends ModulesSpec {
DirectoryProperty getOutputDir();

Property<CurrentDirectoryMode> getCurrentDirectoryMode();
}
11 changes: 10 additions & 1 deletion pkl-gradle/src/main/java/org/pkl/gradle/task/PkldocTask.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,28 @@
package org.pkl.gradle.task;

import org.gradle.api.file.DirectoryProperty;
import org.gradle.api.provider.Property;
import org.gradle.api.tasks.Input;
import org.gradle.api.tasks.OutputDirectory;
import org.pkl.doc.CliDocGenerator;
import org.pkl.doc.CliDocGeneratorOptions;
import org.pkl.doc.DocGenerator.CurrentDirectoryMode;

public abstract class PkldocTask extends ModulesTask {
@OutputDirectory
public abstract DirectoryProperty getOutputDir();

@Input
public abstract Property<CurrentDirectoryMode> getCurrentDirectoryMode();

@Override
protected void doRunTask() {
new CliDocGenerator(
new CliDocGeneratorOptions(
getCliBaseOptions(), getOutputDir().get().getAsFile().toPath()))
getCliBaseOptions(),
getOutputDir().get().getAsFile().toPath(),
false,
getCurrentDirectoryMode().get()))
.run();
}
}