Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable embedding sources inside the sourcemaps json as sourcesContent #9936

Merged
merged 4 commits into from
Jul 17, 2024

Conversation

vegegoku
Copy link
Contributor

@vegegoku vegegoku commented Mar 10, 2024

This PR updates the rebased version of sourcemaps jar and other dependencies to support embedding the source files content into the source maps sourcesContent field.

requires : gwtproject/tools#31

Fixes #9752 #9778 #9790

@vegegoku vegegoku force-pushed the source-content-2-11-test branch 2 times, most recently from a80d0a6 to ddf1364 Compare March 13, 2024 19:41
Copy link
Member

@niloc132 niloc132 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some updates from private chat working on this feature:

  • We've learned that SDM won't be improved by this, as the various linking activity doesn't copy source files for in SDM, only in prod code with sourcemaps enabled. The "sourcesContent" part of this patch will only improve SDM in cases where the debugger loads many many sources, to the point where that network traffic could be time better spent reading a giant sourcemap file.
  • This is still a legit way to slightly improve prod build times for projects that deploy with sourcemaps (not many, but they exist), though it might be counterproductive for those only using sourcemaps for stack trace deobfuscation
  • For these reasons, we're leaning towards a config property in CompilerProperties.gwt.xml that would activate this new feature.
  • The patch still has value simply by updating the sourcemap library: both to potentially fix any issues in the last 8ish years since this was updated, and to drop the server-side runtime dependency on protobuf once and for all.

There is also other cleanup possible for compiler.useSourceMaps (since it should work in all browsers today, and not need extra wiring in CoreWithUserAgent), but let's leave that to another patch.

servlet/build.xml Outdated Show resolved Hide resolved
dev/build.xml Show resolved Hide resolved
@vegegoku vegegoku force-pushed the source-content-2-11-test branch 4 times, most recently from 11a0f27 to b39f264 Compare March 18, 2024 23:14
@vegegoku vegegoku requested a review from niloc132 March 19, 2024 01:30
@niloc132
Copy link
Member

niloc132 commented Mar 19, 2024

Description should be updated to indicate that this fixes #9752.

Edit: also #9778, #9790

dev/build.xml Outdated Show resolved Hide resolved
@vegegoku vegegoku force-pushed the source-content-2-11-test branch from b39f264 to 7584780 Compare April 29, 2024 11:19
@vegegoku vegegoku requested a review from niloc132 April 29, 2024 11:19
@niloc132 niloc132 added this to the 2.12 milestone May 11, 2024
@vegegoku vegegoku force-pushed the source-content-2-11-test branch from 7584780 to 8f42b06 Compare May 12, 2024 11:35
@vegegoku vegegoku requested a review from niloc132 May 12, 2024 11:35
@vegegoku vegegoku force-pushed the source-content-2-11-test branch from 8f42b06 to 66f39c9 Compare May 12, 2024 11:42
Copy link
Member

@niloc132 niloc132 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there may be more classpaths that should be expanded, or code that needs to be corrected to stop walking classes it should ignore (annotations mostly?)

Example from the build:

requestfactory-server:
     [java] Could not find class file for com/google/auto/value/AutoValue
     [java] Could not find class file for com/google/auto/value/AutoValue$Builder
     [java] May 12, 2024 4:55:17 AM com.google.auto.value.AutoValue
     [java] SEVERE: Could not find class file for com/google/auto/value/AutoValue
     [java] May 12, 2024 4:55:17 AM com.google.auto.value.AutoValue.Builder
     [java] SEVERE: Could not find class file for com/google/auto/value/AutoValue$Builder
     [java] Could not find class file for com/google/errorprone/annotations/CanIgnoreReturnValue
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.CanIgnoreReturnValue
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/CanIgnoreReturnValue
     [java] Could not find class file for com/google/errorprone/annotations/DoNotCall
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.DoNotCall
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/DoNotCall
     [java] Could not find class file for com/google/errorprone/annotations/InlineMe
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.InlineMe
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/InlineMe
     [java] Could not find class file for com/google/errorprone/annotations/DoNotMock
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.DoNotMock
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/DoNotMock
     [java] Could not find class file for com/google/errorprone/annotations/InlineMeValidationDisabled
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.InlineMeValidationDisabled
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/InlineMeValidationDisabled
     [java] Could not find class file for com/google/errorprone/annotations/CompatibleWith
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.CompatibleWith
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/CompatibleWith
     [java] Could not find class file for com/google/errorprone/annotations/Immutable
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.Immutable
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/Immutable
     [java] Could not find class file for com/google/errorprone/annotations/ForOverride
     [java] Could not find class file for com/google/errorprone/annotations/CheckReturnValue
     [java] Could not find class file for com/google/errorprone/annotations/RestrictedApi
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.ForOverride
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/ForOverride
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.CheckReturnValue
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/CheckReturnValue
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.RestrictedApi
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/RestrictedApi
     [java] Could not find class file for com/google/errorprone/annotations/concurrent/GuardedBy
     [java] May 12, 2024 4:55:17 AM com.google.errorprone.annotations.concurrent.GuardedBy
     [java] SEVERE: Could not find class file for com/google/errorprone/annotations/concurrent/GuardedBy

@vegegoku vegegoku force-pushed the source-content-2-11-test branch from 66f39c9 to 26428f4 Compare May 19, 2024 15:48
@vegegoku vegegoku requested a review from niloc132 May 19, 2024 16:41
Copy link
Member

@niloc132 niloc132 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, let's merge the tools PR, and remove the CI changes here, and give everyone else another day or so to see if they have any last minute feedback.

@niloc132
Copy link
Member

niloc132 commented Jun 2, 2024

Sorry, still lots of build warnings, I think more dependencies need to be added (or the RF jar extractor tool updated to otherwise handle this case): https://github.com/gwtproject/gwt/actions/runs/9196862375/job/25295956029?pr=9936#step:6:592

error_prone will have the needed dependencies to include AutoValue and other annotations classes to the classpath and prevent the build warnings.
@niloc132
Copy link
Member

Confirmed that this is fixed, waiting for full build to merge.

@niloc132
Copy link
Member

@niloc132
Copy link
Member

Full build after dropping gwtproject/tools workaround https://github.com/vegegoku/gwt/actions/runs/9978374423

@niloc132 niloc132 merged commit 4aee3bd into gwtproject:main Jul 17, 2024
3 checks passed
@niloc132 niloc132 mentioned this pull request Jul 30, 2024
@jnehlmeier
Copy link
Member

Running GWT including this change in production sometimes results in a NPE. See comment in #9931 (comment)

niloc132 added a commit to niloc132/gwt that referenced this pull request Aug 28, 2024
Improves accuracy of deobfuscated stack traces by using the best answer
between original JS names, symbol maps, and source maps, by both
avoiding taking an identifier from the sourcemap if none is provided,
and normalizing away chromium specific prefixes in stack trace elements.

Fixes gwtproject#9996
Follow-up gwtproject#9936
niloc132 added a commit to niloc132/gwt that referenced this pull request Aug 28, 2024
Improves accuracy of deobfuscated stack traces by using the best answer
between original JS names, symbol maps, and source maps, by both
avoiding taking an identifier from the sourcemap if none is provided,
and normalizing away chromium specific prefixes in stack trace elements.

Fixes gwtproject#9996
Follow-up gwtproject#9936
niloc132 added a commit to niloc132/gwt that referenced this pull request Aug 28, 2024
Improves accuracy of deobfuscated stack traces by using the best answer
between original JS names, symbol maps, and source maps, by both
avoiding taking an identifier from the sourcemap if none is provided,
and normalizing away chromium specific prefixes in stack trace elements.

Fixes gwtproject#9996
Follow-up gwtproject#9936
niloc132 added a commit that referenced this pull request Sep 5, 2024
Improves accuracy of deobfuscated stack traces by using the best answer
between original JS names, symbol maps, and source maps, by both
avoiding taking an identifier from the sourcemap if none is provided,
and normalizing away chromium specific prefixes in stack trace elements.

Fixes #9996
Follow-up #9936
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Please remove protobuf pom from gwt-servlet
3 participants