Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic highlight and document symbols on slang 0.10 #523

Merged
merged 18 commits into from
Jan 9, 2024

Conversation

antico5
Copy link
Collaborator

@antico5 antico5 commented Oct 24, 2023

This PR includes support for two new language features, semantic highlighting and document symbols (project outline).

It achieves so by making use of the latest version of slang (0.10.1)

These are some key points from this PR:

  • I noticed that accessing nodes and properties from the CST through slang takes a lot of time, probably because it interfaces through napi. So I store some of this information on a SlangNodeWrapper object that keeps the data on the node process after reading it just once.
  • Features are enabled only on supported platforms (os and architecture). Regardless, all user's platforms are now registered in sentry so we can have some insight on platforms slang should be targeting.
  • The features have been rewritten using the new Cursor api from slang, versus the old custom walking method implemented on the extension side.
  • Feature flags were implemented. The way they work is by fetching a json file in a given github repo/branch (currently https://raw.githubusercontent.com/antico5/hardhat-vscode/flags/flags.json). The numbers should be between 0 (0%) and 1 (100%). These flags are re-fetched on each server initialization. To determine if a given user should have the feature enabled, their machine id is hashed using MD5 and the last 2 bytes are taken. Then they are divided by 65535 and the percentile is used. This way, when a given flag is changed e.g. from 0.1 to 0.2, all users that have it previously enabled will continue to do so.
  • The features are now error tolerant because of the new version of slang.

Closes #217
Closes #42
Closes #491

@socket-security
Copy link

socket-security bot commented Oct 24, 2023

New dependencies detected. Learn more about Socket for GitHub ↗︎

Packages Version New capabilities Transitives Size Publisher
@nomicfoundation/slang 0.10.1 None +9 41.8 MB nomic-foundation-publisher

@codecov-commenter
Copy link

codecov-commenter commented Oct 27, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (development@aac6f3a). Click here to learn what that means.

❗ Current head 4ce5653 differs from pull request most recent head 76b210f. Consider uploading reports for the commit 76b210f to get more accurate results

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@              Coverage Diff               @@
##             development     #523   +/-   ##
==============================================
  Coverage               ?   52.66%           
==============================================
  Files                  ?      228           
  Lines                  ?     5317           
  Branches               ?      835           
==============================================
  Hits                   ?     2800           
  Misses                 ?     2279           
  Partials               ?      238           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@antico5 antico5 changed the title WIP: semantic highlight and document symbols on slang 0.10 Semantic highlight and document symbols on slang 0.10 Nov 17, 2023
@antico5 antico5 force-pushed the migrate_to_slang_0_10 branch from c93765e to bcc6c11 Compare November 20, 2023 22:42
package.json Outdated Show resolved Hide resolved
@antico5 antico5 force-pushed the migrate_to_slang_0_10 branch 2 times, most recently from ab6120a to d0fb8f4 Compare December 8, 2023 18:16
@antico5 antico5 force-pushed the migrate_to_slang_0_10 branch from d0fb8f4 to f3095bd Compare December 8, 2023 18:35
flags.json Show resolved Hide resolved
Copy link

🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎

To accept the risk, merge this PR and you will not be notified again.

Issue Package Version Note Source
Invalid package.json @nomicfoundation/slang 0.10.1

Next steps

What is an invalid package.json?

Package has an invalid package.json and can cause installation problems if you try to use it.

Fix syntax errors in the invalid package.json and publish a new version with a valid package.json. Consumers can use npm overrides to force a version that does not have this problem if one exists.

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of package-name@version specifiers. e.g. @SocketSecurity ignore [email protected] bar@* or ignore all packages with @SocketSecurity ignore-all

@OmarTawfik OmarTawfik merged commit 12bb234 into NomicFoundation:development Jan 9, 2024
4 of 5 checks passed
kanej pushed a commit that referenced this pull request Mar 25, 2024
* feat: semantic highlighting

* feat: documentSymbol - project outline

* chore: migrate to slang 0.10.1

semantic highlight

* track performance spans

* error tolerance for document symbols

* decouple onSemanticTokens from solidity-parser

* optimize walking by storing references

* leverage trackTimingSync error handling

* enable documentSymbol and semanticHighlight only on slang-supported platforms

* feature flags

* use finders for walking the cst

* rework semantic highlighting to use cursor finders

* fix linter

* Changes based on PR feedback (part 1)

* Support unnamed function definition on documentSymbols

* refactor based on feedback

* moving version selection to slangHelpers

* add comment to semantic tokens test
@kanej kanej mentioned this pull request Mar 25, 2024
kanej pushed a commit that referenced this pull request Apr 17, 2024
* feat: semantic highlighting

* feat: documentSymbol - project outline

* chore: migrate to slang 0.10.1

semantic highlight

* track performance spans

* error tolerance for document symbols

* decouple onSemanticTokens from solidity-parser

* optimize walking by storing references

* leverage trackTimingSync error handling

* enable documentSymbol and semanticHighlight only on slang-supported platforms

* feature flags

* use finders for walking the cst

* rework semantic highlighting to use cursor finders

* fix linter

* Changes based on PR feedback (part 1)

* Support unnamed function definition on documentSymbols

* refactor based on feedback

* moving version selection to slangHelpers

* add comment to semantic tokens test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Document Symbol - Outline provider Support advanced syntax highlighting Add OUTLINE
4 participants