Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Exploration] Translate Prism nodes directly into AST::Expression nodes #238

Closed
wants to merge 152 commits into from

Commits on Sep 4, 2024

  1. Configuration menu
    Copy the full SHA
    7553261 View commit details
    Browse the repository at this point in the history
  2. Simplify Prism BUILD file

    Instead of listing all the `srcs` and `hdrs` for the Prism C library,
    use a glob to pull all `.c` and `.h` files, excluding the ones that are
    generated by the genrule so that we can specify that the library
    depends on that genrule without listing those files twice.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    7b61184 View commit details
    Browse the repository at this point in the history
  3. Add --parser option and use it to decide whether to parse with Sorb…

    …et or prism
    
    If `--parser=prism`, diverge in the `pipeline.cc` file and run a separate method that parses the source with prism.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    32d9e64 View commit details
    Browse the repository at this point in the history
  4. Implement parsing a program with just an integer

    This involves setting up a `convertPrismToSorbet` method that checks against every
    possible node type. Program and Statements are basically skipped because Sorbet doesn't
    use them, and then Integer is converted from a prism node to a Sorbet node.
    
    Co-authored-by: Vinicius Stock <[email protected]>
    egiurleo and vinistock committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    60cd0a3 View commit details
    Browse the repository at this point in the history
  5. Implement string parsing

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    006bb09 View commit details
    Browse the repository at this point in the history
  6. Implement float parsing

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    fe9e0f6 View commit details
    Browse the repository at this point in the history
  7. Implement rational parsing

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e70f237 View commit details
    Browse the repository at this point in the history
  8. Add missing src file

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    bd066fe View commit details
    Browse the repository at this point in the history
  9. Add a prism test suite

    1. Add a new test suite that only tests files in the `test/prism_regression` folder
    2. Modify Sorbet's test code to allow the user to specify a parser
    3. Add logic to the pipeline test runner to call `runPrismParser` when specified
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    413267a View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    ffb7744 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    63d0972 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    802522c View commit details
    Browse the repository at this point in the history
  13. Implement prism -> sorbet conversion for multi-statement programs

    Sorbet constructs slightly different ASTs depending on whether a program contains
    one statement or more than one statements. Correctly parsing programs with more
    than one statement will make it easier to benchmark this project.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    175c173 View commit details
    Browse the repository at this point in the history
  14. Modify runPrismParser method to respect stopAfter option

    In order to compare the performance of Prism with the Sorbet parser,
    we need to be able to stop AST generation after Prism has run and
    before we translate the Prism AST into the Sorbet AST.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    379e8a2 View commit details
    Browse the repository at this point in the history
  15. Prism benchmarking script

    These benchmarks will help us measure the progress of the prism in Sorbet project.
    While they can be run from any machine, for "official" results, they should be run
    on an AWS bare metal instance. Results should be added to the prism_benchmarks/data
    directory.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    b504145 View commit details
    Browse the repository at this point in the history
  16. Run benchmarks

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e441403 View commit details
    Browse the repository at this point in the history
  17. Remove .gitkeep files

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    caa790b View commit details
    Browse the repository at this point in the history
  18. Refactor statements node handling

    Relocate it to the `PM_STATEMENTS_NODE` case so it can be reused
    by multiple parent nodes.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    3a7e398 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    a72aac7 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    85de2c8 View commit details
    Browse the repository at this point in the history
  21. Remove extra print messages

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    27dfa18 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    b8dcf94 View commit details
    Browse the repository at this point in the history
  23. Use span to simplify looping

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    d227e9b View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    a42b3f4 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    63e0cff View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    a32896c View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    379ca5c View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    97613da View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    fb08ed6 View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    867137f View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    9211871 View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    e17683f View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    549f59b View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    736e5ad View commit details
    Browse the repository at this point in the history
  35. Extract Prism::Node class

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    36095d3 View commit details
    Browse the repository at this point in the history
  36. Extract Prism::Translator class

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    5963229 View commit details
    Browse the repository at this point in the history
  37. Remove using namespace std

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    b4503ee View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    faf1a73 View commit details
    Browse the repository at this point in the history
  39. Configuration menu
    Copy the full SHA
    d563255 View commit details
    Browse the repository at this point in the history
  40. Configuration menu
    Copy the full SHA
    a6a3673 View commit details
    Browse the repository at this point in the history
  41. Configuration menu
    Copy the full SHA
    1270582 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    4eac333 View commit details
    Browse the repository at this point in the history
  43. Rename convertPrismToSorbet

    Now that it’s own by the `Prism::Translator` class, it’s association to Prism is implied, and doesn’t need to be spelled out in its name.
    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e8bf53d View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    f995d51 View commit details
    Browse the repository at this point in the history
  45. Configuration menu
    Copy the full SHA
    2e95061 View commit details
    Browse the repository at this point in the history
  46. Implement nested constant reads

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    683039d View commit details
    Browse the repository at this point in the history
  47. Configuration menu
    Copy the full SHA
    421122d View commit details
    Browse the repository at this point in the history
  48. Configuration menu
    Copy the full SHA
    ea3e76c View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    c11d3f5 View commit details
    Browse the repository at this point in the history
  50. Configuration menu
    Copy the full SHA
    c237f2c View commit details
    Browse the repository at this point in the history
  51. Configuration menu
    Copy the full SHA
    0229656 View commit details
    Browse the repository at this point in the history
  52. Configuration menu
    Copy the full SHA
    7f70a22 View commit details
    Browse the repository at this point in the history
  53. Configuration menu
    Copy the full SHA
    046f880 View commit details
    Browse the repository at this point in the history
  54. Configuration menu
    Copy the full SHA
    c3595e6 View commit details
    Browse the repository at this point in the history
  55. Configuration menu
    Copy the full SHA
    5dffce8 View commit details
    Browse the repository at this point in the history
  56. Configuration menu
    Copy the full SHA
    4f21c52 View commit details
    Browse the repository at this point in the history
  57. Configuration menu
    Copy the full SHA
    2610df1 View commit details
    Browse the repository at this point in the history
  58. Configuration menu
    Copy the full SHA
    bee6043 View commit details
    Browse the repository at this point in the history
  59. Configuration menu
    Copy the full SHA
    9fc2d00 View commit details
    Browse the repository at this point in the history
  60. Test everything together

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    6923a0f View commit details
    Browse the repository at this point in the history
  61. Clean up nullptr checks

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    2b761a7 View commit details
    Browse the repository at this point in the history
  62. Configuration menu
    Copy the full SHA
    e19a44a View commit details
    Browse the repository at this point in the history
  63. Implement translation for &&

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e471921 View commit details
    Browse the repository at this point in the history
  64. Implement translation for ||

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    c6c3563 View commit details
    Browse the repository at this point in the history
  65. Implement translation for return

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    5a0e549 View commit details
    Browse the repository at this point in the history
  66. Extract translateArguments helper

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    87a112c View commit details
    Browse the repository at this point in the history
  67. Implement translation for yield

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    94863e0 View commit details
    Browse the repository at this point in the history
  68. Configuration menu
    Copy the full SHA
    cba60c5 View commit details
    Browse the repository at this point in the history
  69. Configuration menu
    Copy the full SHA
    f22c57b View commit details
    Browse the repository at this point in the history
  70. Extract translateStatements helper

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    865e57d View commit details
    Browse the repository at this point in the history
  71. Handle empty interpolation segments

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    ef025d4 View commit details
    Browse the repository at this point in the history
  72. Configuration menu
    Copy the full SHA
    5b9dd87 View commit details
    Browse the repository at this point in the history
  73. Configuration menu
    Copy the full SHA
    70589fd View commit details
    Browse the repository at this point in the history
  74. Configuration menu
    Copy the full SHA
    e554392 View commit details
    Browse the repository at this point in the history
  75. Configuration menu
    Copy the full SHA
    a8a3d49 View commit details
    Browse the repository at this point in the history
  76. Implement translation for super

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    0ca51ba View commit details
    Browse the repository at this point in the history
  77. Configuration menu
    Copy the full SHA
    fe7d939 View commit details
    Browse the repository at this point in the history
  78. Fix operator_or test

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e15439e View commit details
    Browse the repository at this point in the history
  79. Configuration menu
    Copy the full SHA
    c044afa View commit details
    Browse the repository at this point in the history
  80. Configuration menu
    Copy the full SHA
    c10812d View commit details
    Browse the repository at this point in the history
  81. Configuration menu
    Copy the full SHA
    6d9f1ab View commit details
    Browse the repository at this point in the history
  82. Configuration menu
    Copy the full SHA
    8d9a0f7 View commit details
    Browse the repository at this point in the history
  83. Implement translation for self

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    3f220eb View commit details
    Browse the repository at this point in the history
  84. Configuration menu
    Copy the full SHA
    3750d7b View commit details
    Browse the repository at this point in the history
  85. Configuration menu
    Copy the full SHA
    cf914d0 View commit details
    Browse the repository at this point in the history
  86. Configuration menu
    Copy the full SHA
    b997c13 View commit details
    Browse the repository at this point in the history
  87. Configuration menu
    Copy the full SHA
    3fed6ce View commit details
    Browse the repository at this point in the history
  88. Configuration menu
    Copy the full SHA
    215ddb6 View commit details
    Browse the repository at this point in the history
  89. Configuration menu
    Copy the full SHA
    6e3a6d3 View commit details
    Browse the repository at this point in the history
  90. Configuration menu
    Copy the full SHA
    f340ba5 View commit details
    Browse the repository at this point in the history
  91. Configuration menu
    Copy the full SHA
    11e9975 View commit details
    Browse the repository at this point in the history
  92. Misc. cleanup

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    93e2017 View commit details
    Browse the repository at this point in the history
  93. Configuration menu
    Copy the full SHA
    784ed61 View commit details
    Browse the repository at this point in the history
  94. Configuration menu
    Copy the full SHA
    ed81d82 View commit details
    Browse the repository at this point in the history
  95. Configuration menu
    Copy the full SHA
    f107047 View commit details
    Browse the repository at this point in the history
  96. Configuration menu
    Copy the full SHA
    c7fbdc5 View commit details
    Browse the repository at this point in the history
  97. Configuration menu
    Copy the full SHA
    077c57c View commit details
    Browse the repository at this point in the history
  98. Bump prism to v0.30.0

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    997f294 View commit details
    Browse the repository at this point in the history
  99. Updated Rational handling

    Prism now represents rationals as numerator/denominator, which
    doesn't preserve the original representation of the numerical value.
    We just pull the value as a string from source and pass it
    to Sorbet.
    
    Co-authored-by: Alex Momchilov <[email protected]>
    egiurleo and amomchilov committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    d15e43a View commit details
    Browse the repository at this point in the history
  100. Configuration menu
    Copy the full SHA
    a5176f7 View commit details
    Browse the repository at this point in the history
  101. Configuration menu
    Copy the full SHA
    d3d5a86 View commit details
    Browse the repository at this point in the history
  102. Configuration menu
    Copy the full SHA
    1e3697b View commit details
    Browse the repository at this point in the history
  103. Translation for while node

    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e853642 View commit details
    Browse the repository at this point in the history
  104. Create extractString method in Prism parser

    To pull out the process of taking a `pm_string_t` and using it to
    extract a string from the source code.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    a95cb39 View commit details
    Browse the repository at this point in the history
  105. Configuration menu
    Copy the full SHA
    74f08ea View commit details
    Browse the repository at this point in the history
  106. Configuration menu
    Copy the full SHA
    d0016ae View commit details
    Browse the repository at this point in the history
  107. Configuration menu
    Copy the full SHA
    cc681d9 View commit details
    Browse the repository at this point in the history
  108. Configuration menu
    Copy the full SHA
    77a5c2b View commit details
    Browse the repository at this point in the history
  109. Extract translateMulti helper

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e4fdb0a View commit details
    Browse the repository at this point in the history
  110. Configuration menu
    Copy the full SHA
    2139489 View commit details
    Browse the repository at this point in the history
  111. Extract translateMultiInto helper

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    9fbf2af View commit details
    Browse the repository at this point in the history
  112. Configuration menu
    Copy the full SHA
    6b3aa60 View commit details
    Browse the repository at this point in the history
  113. Fix incorrect cast type

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    6b297b3 View commit details
    Browse the repository at this point in the history
  114. Configuration menu
    Copy the full SHA
    5bc9837 View commit details
    Browse the repository at this point in the history
  115. Configuration menu
    Copy the full SHA
    3345ffc View commit details
    Browse the repository at this point in the history
  116. Configuration menu
    Copy the full SHA
    aef6cbd View commit details
    Browse the repository at this point in the history
  117. Rename keyword-related tests

    amomchilov authored and egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    f66ed98 View commit details
    Browse the repository at this point in the history
  118. Configuration menu
    Copy the full SHA
    7403a1f View commit details
    Browse the repository at this point in the history
  119. Configuration menu
    Copy the full SHA
    22b6d78 View commit details
    Browse the repository at this point in the history
  120. Configuration menu
    Copy the full SHA
    dd96bc8 View commit details
    Browse the repository at this point in the history
  121. Configuration menu
    Copy the full SHA
    953ad93 View commit details
    Browse the repository at this point in the history
  122. Create template to generalize assignment translations

    The template takes the Prism and Sorbet node types, as well as Sorbet's LHS
    node type. Then, it performs the assignment translation correctly given
    all of those types.
    egiurleo committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    288243f View commit details
    Browse the repository at this point in the history
  123. Configuration menu
    Copy the full SHA
    8b30a32 View commit details
    Browse the repository at this point in the history
  124. Configuration menu
    Copy the full SHA
    9e6be53 View commit details
    Browse the repository at this point in the history
  125. Configuration menu
    Copy the full SHA
    1d92202 View commit details
    Browse the repository at this point in the history
  126. Configuration menu
    Copy the full SHA
    76bb644 View commit details
    Browse the repository at this point in the history
  127. Configuration menu
    Copy the full SHA
    2fef998 View commit details
    Browse the repository at this point in the history
  128. Configuration menu
    Copy the full SHA
    9a3905c View commit details
    Browse the repository at this point in the history
  129. Configuration menu
    Copy the full SHA
    9fa4e01 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. Configuration menu
    Copy the full SHA
    bab8574 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4971f02 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    92f44f3 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Configuration menu
    Copy the full SHA
    359e1c4 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #234 from Shopify/emily/index-assign

    Implement Prism -> Sorbet translation for index assign nodes
    egiurleo authored Sep 6, 2024
    Configuration menu
    Copy the full SHA
    ff3d674 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    09e8803 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4b3c666 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    3521058 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2024

  1. Merge pull request #235 from Shopify/Alex/translate-anonymous-params

    Implement missed translations for anonymous `*`, `**` and `&` params
    amomchilov authored Sep 7, 2024
    Configuration menu
    Copy the full SHA
    db760a8 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Create a template abstracting variable assignment logic

    And use the template on all nodes where we create an `Assign` Sorbet node.
    egiurleo committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    5bea295 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. Merge pull request #236 from Shopify/emily/const-write

    Implement Prism -> Sorbet translation for `PM_CONSTANT_WRITE_NODE`
    egiurleo authored Sep 12, 2024
    Configuration menu
    Copy the full SHA
    db94397 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5d371a4 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #237 from Shopify/emily/const-path-write

    Implement translation for `PM_CONSTANT_PATH_WRITE_NODE`
    egiurleo authored Sep 12, 2024
    Configuration menu
    Copy the full SHA
    0ca762b View commit details
    Browse the repository at this point in the history
  4. Fix call_kw_rest to actually test method calls

    It was still a copy of `def_kw_rest_params`, and were still testing method defs, not calls.
    amomchilov committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    77b2343 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b3bc14e View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. Configuration menu
    Copy the full SHA
    7fd1860 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    74b5caa View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    082c0c9 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    98b1fc8 View commit details
    Browse the repository at this point in the history
  5. Simplify translateMulti

    amomchilov committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c59eaea View commit details
    Browse the repository at this point in the history
  6. Merge pull request #232 from Shopify/Alex/translate-call-kwrest

    Implement Prism -> Sorbet translation for anonymous Array and Hash splats
    amomchilov authored Sep 13, 2024
    Configuration menu
    Copy the full SHA
    0afadaa View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. Merge pull request #228 from Shopify/Alex/translate-keyword-it

    Implement Prism -> Sorbet translation for `it` keyword
    amomchilov authored Sep 16, 2024
    Configuration menu
    Copy the full SHA
    9c010bb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c833b4b View commit details
    Browse the repository at this point in the history