Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tree-sitter rolling fixes: 1.121 edition #1085

Merged
merged 28 commits into from
Sep 17, 2024
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
4a701e2
Move indentation-related tasks to their own class…
savetheclocktower Aug 17, 2024
338f688
[language-css] Bump `tree-sitter-css` to `0.21.1`
savetheclocktower Aug 17, 2024
74ef36a
Fix bugs in invocations of `resolveMatch`
savetheclocktower Aug 17, 2024
6eee25f
Tweaks
savetheclocktower Aug 17, 2024
c5f87a6
Pass `comparisonRow` for `match` capture interpretation
savetheclocktower Aug 23, 2024
3834c3f
Add comments and make further indentation tweaks…
savetheclocktower Aug 24, 2024
50dbbd1
Add a spec for `@match.next`…
savetheclocktower Aug 24, 2024
738536f
[language-php] Handle case-insensitive constants like `TRUE`/`FALSE`…
savetheclocktower Aug 27, 2024
02f1563
Fail `(starts|ends)OnSameRowAs` when the node position descriptor fails
savetheclocktower Aug 27, 2024
9db65c0
[language-java] Fix the scope on `foo` in `foo.bar()`
savetheclocktower Aug 27, 2024
26be874
[language-typescript] Scope `!` in variable declarators
savetheclocktower Sep 4, 2024
87a9322
Upgrade `web-tree-sitter` to 0.23.0…
savetheclocktower Sep 7, 2024
f64b44f
(Whoops, left some logging statements in there)
savetheclocktower Sep 8, 2024
d433753
Fix regression with indentation
savetheclocktower Sep 8, 2024
4d696a7
Rewrite warning message in `web-tree-sitter`
savetheclocktower Sep 8, 2024
72b1a6e
[language-yaml] Update to newest `tree-sitter-yaml`
savetheclocktower Sep 8, 2024
c674d8f
Update README for `web-tree-sitter`
savetheclocktower Sep 8, 2024
265d7e0
[language-(java|type)script] Scope `??=`, `++`, etc., as assignments
savetheclocktower Sep 8, 2024
fbdf18a
`coverShallowerScopes` shouldn't affect “sibling” injections…
savetheclocktower Sep 8, 2024
03ba902
[language-(java|type)script]: Update Tree-sitter parsers to 0.23.0…
savetheclocktower Sep 9, 2024
44ad31f
[symbol-provider-tree-sitter] Fix specs…
savetheclocktower Sep 10, 2024
cdad4ff
[language-(java|type)script] Update injections…
savetheclocktower Sep 10, 2024
44d7734
[language-gfm] Migrate to `tree-sitter-markdown`…
savetheclocktower Sep 10, 2024
0f0050f
[language-html] Update to latest Tree-sitter parsers…
savetheclocktower Sep 14, 2024
99521c7
Prevent race condition during startup…
savetheclocktower Sep 14, 2024
986c697
[language-html] Update `parserSource` fields
savetheclocktower Sep 14, 2024
74479ba
Fix failing tests
savetheclocktower Sep 14, 2024
c1d3a8c
Address feedback
savetheclocktower Sep 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/language-css/grammars/modern-tree-sitter-css.cson
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ fileTypes: [
]

treeSitter:
parserSource: 'github:tree-sitter/tree-sitter-css#98c7b3dceb24f1ee17f1322f3947e55638251c37'
parserSource: 'github:tree-sitter/tree-sitter-css#9af0bdd9d225edee12f489cfa8284e248321959b'
grammar: 'tree-sitter/tree-sitter-css.wasm'
highlightsQuery: 'tree-sitter/queries/highlights.scm'
foldsQuery: 'tree-sitter/queries/folds.scm'
Expand Down
Binary file modified packages/language-css/grammars/tree-sitter/tree-sitter-css.wasm
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ parser: 'tree-sitter-markdown_inline'
injectionRegex: '^(markdown-inline-internal)$'

treeSitter:
parserSource: 'github:MDeiml/tree-sitter-markdown/tree-sitter-markdown-inline#28aa3baef73bd458d053b613b8bd10fd102b4405'
parserSource: 'github:tree-sitter-grammars/tree-sitter-markdown/tree-sitter-markdown-inline#a3481e1db0b5f9aecfa7124d362a6485286a8e72'
grammar: 'tree-sitter-markdown-inline/tree-sitter-markdown-inline.wasm'
highlightsQuery: 'tree-sitter-markdown-inline/highlights.scm'
2 changes: 1 addition & 1 deletion packages/language-gfm/grammars/modern-tree-sitter-gfm.cson
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ parser: 'tree-sitter-markdown'
injectionRegex: '^(markdown|MARKDOWN|gfm|GFM)$'

treeSitter:
parserSource: 'github:MDeiml/tree-sitter-markdown/tree-sitter-markdown#28aa3baef73bd458d053b613b8bd10fd102b4405'
parserSource: 'github:tree-sitter-grammars/tree-sitter-markdown/tree-sitter-markdown#a3481e1db0b5f9aecfa7124d362a6485286a8e72'
grammar: 'tree-sitter-markdown/tree-sitter-markdown.wasm'
highlightsQuery: 'tree-sitter-markdown/highlights.scm'
foldsQuery: 'tree-sitter-markdown/folds.scm'
Expand Down
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ fileTypes: [
injectionRegex: '^(ejs|EJS)$'

treeSitter:
parserSource: 'github:tree-sitter/tree-sitter-embedded-template#203f7bd3c1bbfbd98fc19add4b8fcb213c059205'
parserSource: 'github:tree-sitter/tree-sitter-embedded-template#v0.23.0'
grammar: 'tree-sitter-embedded-template/tree-sitter-embedded-template.wasm'
highlightsQuery: 'tree-sitter-embedded-template/ejs/highlights.scm'
foldsQuery: 'tree-sitter-embedded-template/ejs/folds.scm'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ fileTypes: [
injectionRegex: '^(erb|ERB)$'

treeSitter:
parserSource: 'github:tree-sitter/tree-sitter-embedded-template#203f7bd3c1bbfbd98fc19add4b8fcb213c059205'
parserSource: 'github:tree-sitter/tree-sitter-embedded-template#v0.23.0'
grammar: 'tree-sitter-embedded-template/tree-sitter-embedded-template.wasm'
highlightsQuery: 'tree-sitter-embedded-template/erb/highlights.scm'
foldsQuery: 'tree-sitter-embedded-template/erb/folds.scm'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ parser: 'tree-sitter-html'
injectionRegex: '(HTML|html|Html)$'

treeSitter:
parserSource: 'github:tree-sitter/tree-sitter-html#d742025fa2d8e6100f134a6ea990443aa1f074b3'
parserSource: 'github:tree-sitter/tree-sitter-html#v0.23.0'
grammar: 'tree-sitter-html/tree-sitter-html.wasm'
highlightsQuery: 'tree-sitter-html/highlights.scm'
foldsQuery: 'tree-sitter-html/folds.scm'
Expand Down
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,10 @@

(throws "throws" @storage.modifier.throws.java)

(method_invocation (identifier)
(method_invocation object: (identifier)
@support.other.object.java)

(method_invocation name: (identifier)
@support.other.function.java)

(field_access (identifier) @constant.other.java
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ parser: 'tree-sitter-javascript'
injectionRegex: '^(js|javascript|JS|JAVASCRIPT)$'

treeSitter:
parserSource: 'github:tree-sitter/tree-sitter-javascript#f1e5a09b8d02f8209a68249c93f0ad647b228e6e'
parserSource: 'github:tree-sitter/tree-sitter-javascript#v0.23.0'
grammar: 'tree-sitter/tree-sitter-javascript.wasm'
highlightsQuery: 'tree-sitter/highlights.scm'
localsQuery: 'tree-sitter/locals.scm'
Expand Down
43 changes: 32 additions & 11 deletions packages/language-javascript/grammars/tree-sitter/highlights.scm
Original file line number Diff line number Diff line change
@@ -1,23 +1,29 @@
; MISC
; ====

; In the JSX construct `<FOO.Bar>`, `FOO` should not be marked as
; `constant.other.js`. Block off identifiers within complex JSX tag names early
; to prevent this.
; In the JSX construct `<Foo.Bar>`, `Foo.Bar` is treated as a
; `member_expression`. We don't want the ordinary rules for member expressions
; to apply, so we block them off.
;
; TODO: If we wanted to give these segments individual scopes, we'd do that
; here — replacing the `@_IGNORE_`s with scope names.

(jsx_opening_element
(member_expression
(identifier) @_IGNORE_
(property_identifier) @_IGNORE_
(#set! capture.final)))

(jsx_closing_element
(member_expression
(identifier) @_IGNORE_
(property_identifier) @_IGNORE_
(#set! capture.final)))

(jsx_self_closing_element
(member_expression
(identifier) @_IGNORE_
(property_identifier) @_IGNORE_
(#set! capture.final)))


Expand Down Expand Up @@ -98,7 +104,7 @@
(assignment_expression
left: (member_expression
property: (property_identifier) @variable.other.assignment.property.js)
right: [(arrow_function) (function)] @_IGNORE_
right: [(arrow_function) (function_expression)] @_IGNORE_
(#set! isFunctionProperty true))

; The "bar" in `foo.bar = true`.
Expand All @@ -117,10 +123,25 @@
(augmented_assignment_expression
left: (identifier) @variable.other.assignment.js)

; The "bar" in `foo.bar += 1`.
(augmented_assignment_expression
left: (member_expression
property: (property_identifier) @variable.other.assignment.property.js)
(#is-not? test.rangeWithData isFunctionProperty)
(#set! capture.final))

; The "foo" in `foo++`.
(update_expression
argument: (identifier) @variable.other.assignment.js)

; The "bar" in `foo.bar++`.
(update_expression
argument: (member_expression
property: (property_identifier) @variable.other.assignment.property.js)
(#is-not? test.rangeWithData isFunctionProperty)
(#set! capture.final))


; Public field definition in a class body:
; The "foo" in `foo = "bar";`
(field_definition
Expand Down Expand Up @@ -264,7 +285,7 @@

; Named function expressions:
; the "foo" in `let bar = function foo () {`
(function
(function_expression
name: (identifier) @entity.name.function.definition.js)

; Function definitions:
Expand Down Expand Up @@ -298,27 +319,27 @@
left: (member_expression
property: (property_identifier) @entity.name.function.definition.js
(#set! capture.final true))
right: [(arrow_function) (function)])
right: [(arrow_function) (function_expression)])

; Function variable assignment:
; The "foo" in `let foo = function () {`
(variable_declarator
name: (identifier) @entity.name.function.definition.js
value: [(function) (arrow_function)])
value: [(function_expression) (arrow_function)])

; Function variable reassignment:
; The "foo" in `foo = function () {`
(assignment_expression
left: (identifier) @function
right: [(function) (arrow_function)])
right: [(function_expression) (arrow_function)])

; Object key-value pair function:
; The "foo" in `{ foo: function () {} }`
(pair
key: (property_identifier) @entity.name.function.method.definition.js
value: [(function) (arrow_function)])
value: [(function_expression) (arrow_function)])

(function "function" @storage.type.function.js)
(function_expression "function" @storage.type.function.js)
(function_declaration "function" @storage.type.function.js)

(generator_function "function" @storage.type.function.js)
Expand Down Expand Up @@ -995,7 +1016,7 @@
(#set! adjust.endAt lastChild.startPosition)
(#set! capture.final true))

(function
(function_expression
body: (statement_block) @meta.block.function.js
(#set! adjust.startAt firstChild.endPosition)
(#set! adjust.endAt lastChild.startPosition)
Expand Down
10 changes: 5 additions & 5 deletions packages/language-javascript/grammars/tree-sitter/tags.scm
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
(comment)* @doc
.
[
(function
(function_expression
name: (identifier) @name)
(function_declaration
name: (identifier) @name)
Expand All @@ -47,7 +47,7 @@
(lexical_declaration
(variable_declarator
name: (identifier) @name
value: [(arrow_function) (function)]) @definition.function)
value: [(arrow_function) (function_expression)]) @definition.function)
(#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
(#select-adjacent! @doc @definition.function)
)
Expand All @@ -58,7 +58,7 @@
(variable_declaration
(variable_declarator
name: (identifier) @name
value: [(arrow_function) (function)]) @definition.function)
value: [(arrow_function) (function_expression)]) @definition.function)
(#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
(#select-adjacent! @doc @definition.function)
)
Expand All @@ -69,12 +69,12 @@
(member_expression
property: (property_identifier) @name)
]
right: [(arrow_function) (function)]
right: [(arrow_function) (function_expression)]
) @definition.function

(pair
key: (property_identifier) @name
value: [(arrow_function) (function)]) @definition.function
value: [(arrow_function) (function_expression)]) @definition.function

(
(call_expression
Expand Down
Binary file not shown.
18 changes: 12 additions & 6 deletions packages/language-javascript/lib/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,27 +21,27 @@ exports.activate = function () {
content(callExpression) {
const { lastChild } = callExpression;
if (lastChild.type === 'template_string') {
return lastChild;
return stringFragmentsOfTemplateString(lastChild);
}
}
});

atom.grammars.addInjectionPoint('source.js', {
type: 'assignment_expression',

language(callExpression) {
const { firstChild } = callExpression;
language(expression) {
const { firstChild } = expression;
if (firstChild.type === 'member_expression') {
if (firstChild.lastChild.text === 'innerHTML') {
return 'html';
}
}
},

content(callExpression) {
const { lastChild } = callExpression;
content(expression) {
const { lastChild } = expression;
if (lastChild.type === 'template_string') {
return lastChild;
return stringFragmentsOfTemplateString(lastChild);
}
}
});
Expand Down Expand Up @@ -95,3 +95,9 @@ function languageStringForTemplateTag(tag) {
return tag;
}
}

function stringFragmentsOfTemplateString(templateStringNode) {
return templateStringNode.children.filter(
c => c.type === 'string_fragment'
);
}
Original file line number Diff line number Diff line change
Expand Up @@ -408,7 +408,11 @@
(const_declaration (const_element) @variable.other.constant.php)

((name) @constant.language.php
(#match? @constant.language.php "^__[A-Z][A-Z\d_]+__$"))
(#match? @constant.language.php "\\b(__(FILE|DIR|FUNCTION|CLASS|METHOD|LINE|NAMESPACE)__|ON|OFF|YES|NO|NL|BR|TAB)\\b")
(#set! capture.final))

((name) @constant.language.php
(#match? @constant.language.php "^__[A-Z][A-Z\d_]+__$"))

(argument
name: (_) @variable.other.named-argument.php
Expand Down Expand Up @@ -489,8 +493,34 @@
; =========

(boolean) @constant.language.boolean._TEXT_.php

; In PHP, `true` is technically no different from `TRUE` or `True`; they appear
; to be implemented as case-insensitive language constants. `tree-sitter-php`
; treats `true` as a `boolean` node, but `TRUE` and `True` as `name` nodes.
;
; This is silly — but, as usual, the origin of the silliness is PHP.
;
; `#match?` is case-sensitive and there's no way to opt out of that. At some
; point in the future, we might be able to fix that by defining a custom
; predicate for case-insensitive `#match?`; for now we'll just write a silly
; regex.

((name) @constant.language.boolean.true.php
(#match? @constant.language.boolean.true.php "^[Tt][Rr][Uu][Ee]$")
(#set! capture.final))

((name) @constant.language.boolean.false.php
(#match? @constant.language.boolean.false.php "^[Ff][Aa][Ll][Ss][Ee]$")
(#set! capture.final))

(null) @constant.language.null.php

; Likewise, `null` is a case-insensitive constant. `NULL` and `null` are common;
; others are less common, but we might as well cover them.
((name) @constant.language.null.php
(#match? @constant.language.null.php "^[Nn][Uu][Ll][Ll]")
(#set! capture.final))

(integer) @constant.numeric.decimal.integer.php
(float) @constant.numeric.decimal.float.php

Expand Down
Loading
Loading