Support modern ECMAScript regexp syntax

[bd82]

For example:

• unicode flag
• lookbehind assertions.

The biggest question is to decide if to update regexp-to-ast to the newst spec
or replace it with regexpp.

At the moment I am leaning towards updating regexp-to-ast as it seems regexpp is only minimally maintained.

• See Investigate using regexpp Instead of regexp-to-ast #777

[likern]

@bd82 Hello, I've started using this awesome parser in React Native environment. Please, consider also supporting React Native Hermes engine, it probably has some limitations in RegEx compared to V8.

More information is on Hermes site.

[0xThiebaut]

For anyone ending up here and needing modern ECMAScript syntax, here is a quick workaround (in TypeScript) wrapping a RegExp into a Custom Token Pattern.

import { createToken, CustomPatternMatcherFunc } from "chevrotain"

function wrap(regex: RegExp): CustomPatternMatcherFunc {
    return (text: string, offset: number): RegExpExecArray | null => {
        const re = new RegExp(regex, 'y');
        re.lastIndex = offset
        return re.exec(text)
    }
}

export const ComplexIdentifier = createToken({
    name: 'variable.other.identifier.complex',
    pattern: wrap(/(?<=\[['"])[\w. -]{1,1024}(?=['"]\])/), // Wrap the modern ECMAScript RegExp syntax in the function
    line_breaks: false // A Custom Token Pattern should specify the <line_breaks> option
})

[bd82]

I've started using this awesome parser in React Native environment. Please, consider also supporting React Native Hermes engine, it probably has some limitations in RegEx compared to V8.

Hello @likern

Chevrotain is a JavaScript library, so it should not need any special support to run inside of hermes.
Generally it should run on any ES6 compatible JavaScript engine which I assume Hermes is...

[bd82]

Should evaluate the effort in updating regexp-to-ast to ECMAScript 2024 standard