Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stdlib serialization redux #1205

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Conversation

dabrahams
Copy link
Collaborator

@dabrahams dabrahams commented Dec 9, 2023

Currently blocked on swiftlang/swift#70979

{
let sourceDirectory = URL(fileURLWithPath: #filePath)
.deletingLastPathComponent().deletingLastPathComponent().deletingLastPathComponent()
/ "StandardLibrary" / "Sources"
Copy link
Contributor

@kyouko-taiga kyouko-taiga Dec 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I loathe operators for pretty much anything other than arithmetic operations on numerics. It is also not at all obvious to me that / has to be understood as a path separator when its operands are arbitrary expressions.

@kyouko-taiga
Copy link
Contributor

The problem causing the "unexpected nil" error due to the logic used to filter freestanding sources. See #0a9bc7a. My other changes were necessary to build and (successfully) run the tests on my system.

Comment on lines 20 to 24
.compactMap { $0 as? URL }
.filter { $0.pathExtension == "hylo" }

let freestrandingDirectory = sourceDirectory / "Core"
let freestandingSources = hostedSources.filter(freestrandingDirectory.contains)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going over these files 3 times! I think we're starting to have enough sources in the standard library for that to become noticeably inefficient.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How'd you get 3?

Copy link
Contributor

@kyouko-taiga kyouko-taiga Jan 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are:

  • .compactMap { $0 as? URL }
  • .filter { $0.pathExtension == "hylo" }
  • .filter(freestrandingDirectory.contains)

We should merge the two first ones in a single compactMap. Then maybe we might also want to declare hostedSources and freestandingSources as lazy collections.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no way this code will be performance critical, and I thought to prioritize readability. Do you really think those changes would improve it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not as confident as you that there can't be any performance impact here here but regardless I think the changes would improve readability. This kind of code will invariably make me pause and look for a reason why the filters weren't expressed that way. At the very least I think there's a missing abstraction here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

performance impact

Seriously, in a build tool plugin? The cost of reading directory contents (to say nothing of reading actual files off disk) would surely swamp these costs.

I think the changes would improve readability.

Then you should suggest the change exactly as you would like to see it written, and I will accept it without argument. I would not be able to judge any such change to be a readability improvement, and I don't want to guess at what will satisfy you.

The addition of .lazy at the beginning would not hurt readability too much but you have often found that to be a performance cost. If lazy is effective, it's likely in principle that the other changes couldn't have any effect on performance in a release build (in some world where the performance of this code actually matters) because the lazy layers collapse. So I don't see the point at all, but if you want it changed, just say how and I will do it.

.filter { $0.pathExtension == "hylo" }

let freestrandingDirectory = sourceDirectory / "Core"
let freestandingSources = hostedSources.filter(freestrandingDirectory.contains)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thar's a algorithm for that.

Suggested change
let freestandingSources = hostedSources.filter(freestrandingDirectory.contains)
let freestandingSources = hostedSources.filter { $0.pathComponents.starts(with: freestrandingDirectory.pathComponents) }

I didn't put it in contains because that function doesn't do what it says on the tin if you read the comment carefully.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, which function doesn't do what it says it does, and on what?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URL.contains, whose documentation says “returns true if self is the path of a directory containing other.”. In fact it doesn't tell you anything whether self refers to a directory (which requires verifying that it's a file:// URL and then looking at disk) or whether it contains other. in fact it only tells you whether the path of self is a parent of the path of other. Potential for little inaccuracies (or at the very least, ambiguities) like that one, is among the many problems with URL that make me want to write a first-class path abstraction.

@dabrahams dabrahams force-pushed the stdlib-serialization-redux branch from 40aa49d to 7036839 Compare January 19, 2024 01:39
Apply swift-format

Attempt to actually serialize

Fix the gathering of standard library sources

Fix the loading of serialized resources

Swift format
@dabrahams dabrahams force-pushed the stdlib-serialization-redux branch from 7036839 to 92df416 Compare January 19, 2024 02:31
@dabrahams
Copy link
Collaborator Author

@kyouko-taiga now that we're not really dependent on SPM anymore we should discuss whether this PR is going in the right direction and whether I should try to land it or close it.

@kyouko-taiga
Copy link
Contributor

Sorry for the radio silence. I will look into it after my vacation as part of my effort to bring modules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants