From 92ceab1ff2aa1d6c7cc673b711a4468c3dc03856 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B4natas=20Davi=20Paganini?= Date: Fri, 17 Nov 2023 19:35:31 -0300 Subject: [PATCH] Deployed cb25d52 with MkDocs version: 1.3.1 --- search/search_index.json | 2 +- sitemap.xml | 28 +++++++++--------- sitemap.xml.gz | Bin 207 -> 208 bytes sql-support/index.html | 61 +++++++++++++++++++++++++++++---------- stylesheets/custom.css | 2 +- 5 files changed, 62 insertions(+), 31 deletions(-) diff --git a/search/search_index.json b/search/search_index.json index 317207f..8d90bd7 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Fast \u00b6 Fast is a \"Find AST\" tool to help you search in the code abstract syntax tree. Ruby allow us to do the same thing in a few ways then it's hard to check how the code is written. Using the AST will be easier than try to cover the multiple ways we can write the same code. You can define a string like %|| or '' or \"\" but they will have the same AST representation. AST representation \u00b6 Each detail of the ruby syntax have a equivalent identifier and some content. The content can be another expression or a final value. Fast uses parser gem behind the scenes to parse the code into nodes. First get familiar with parser gem and understand how ruby code is represented. When you install parser gem, you will have access to ruby-parse and you can use it with -e to parse an expression directly from the command line. Example: ruby-parse -e 1 It will print the following output: (int 1) And trying a number with decimals: ruby-parse -e 1.1 (float 1) Building a regex that will match decimals and integer looks like something easy and with fast you use a node pattern that reminds the syntax of regular expressions. Syntax for find in AST \u00b6 The current version cover the following elements: () to represent a node search {} is for any matches like union conditions with or operator [] is for all matches like intersect conditions with and operator $ is for capture current expression _ is something not nil nil matches exactly nil ... is a node with children ^ is to get the parent node of an expression ? is for maybe \\1 to use the first previous captured element \"\" surround the value with double quotes to match literal strings Jump to Syntax . ast \u00b6 Use Fast.ast to convert simple code to AST objects. You can use it as ruby-parse but directly from the console. Fast . ast ( \"1\" ) # => s(:int, 1) Fast . ast ( \"method\" ) # => s(:send, nil, :method) Fast . ast ( \"a.b\" ) # => s(:send, s(:send, nil, :a), :b) Fast . ast ( \"1 + 1\" ) # => s(:send, s(:int, 1), :+, s(:int, 1)) Fast . ast ( \"a = 2\" ) # => s(:lvasgn, :a, s(:int, 2)) Fast . ast ( \"b += 2\" ) # => s(:op_asgn, s(:lvasgn, :b), :+, s(:int, 2)) It uses astrolable gem behind the scenes: Fast . ast ( Fast . ast ( \"1\" )) . class => Astrolabe :: Node Fast . ast ( Fast . ast ( \"1\" )) . type => :int Fast . ast ( Fast . ast ( \"1\" )) . children => [ 1 ] See also ast_from_file . match? \u00b6 Fast.match? is the most granular function that tries to compare a node with an expression. It returns true or false and some node captures case it find something. Let's start with a simple integer in Ruby: 1 The AST can be represented with the following expression: (int 1) The ast representation holds node type and children . Let's build a method s to represent Parser::AST::Node with a #type and #children . def s ( type , * children ) Parser :: AST :: Node . new ( type , children ) end A local variable assignment: value = 42 Can be represented with: ast = s ( :lvasgn , :value , s ( :int , 42 )) Now, lets find local variable named value with an value 42 : Fast . match? ( '(lvasgn value (int 42))' , ast ) # true Lets abstract a bit and allow some integer value using _ as a shortcut: Fast . match? ( '(lvasgn value (int _))' , ast ) # true Lets abstract more and allow float or integer: Fast . match? ( '(lvasgn value ({float int} _))' , ast ) # true Or combine multiple assertions using [] to join conditions: Fast . match? ( '(lvasgn value ([!str !hash !array] _))' , ast ) # true Matches all local variables not string and not hash and not array. We can match \"a node with children\" using ... : Fast . match? ( '(lvasgn value ...)' , ast ) # true You can use $ to capture a node: Fast . match? ( '(lvasgn value $...)' , ast ) # => [s(:int), 42] Or match whatever local variable assignment combining both _ and ... : Fast . match? ( '(lvasgn _ ...)' , ast ) # true You can also use captures in any levels you want: Fast . match? ( '(lvasgn $_ $...)' , ast ) # [:value, s(:int), 42] Keep in mind that _ means something not nil and ... means a node with children. Then, if do you get a method declared: def my_method call_other_method end It will be represented with the following structure: ast = s ( :def , :my_method , s ( :args ), s ( :send , nil , :call_other_method )) Keep an eye on the node (args) . Then you know you can't use ... but you can match with (_) to match with such case. Let's test a few other examples. You can go deeply with the arrays. Let's suppose we have a hardcore call to a.b.c.d and the following AST represents it: ast = s ( :send , s ( :send , s ( :send , s ( :send , nil , :a ), :b ), :c ), :d ) You can search using sub-arrays with pure values , or shortcuts or procs : Fast . match? ( [ :send , [ :send , '...' ] , :d ] , ast ) # => true Fast . match? ( [ :send , [ :send , '...' ] , :c ] , ast ) # => false Fast . match? ( [ :send , [ :send , [ :send , '...' ] , :c ] , :d ] , ast ) # => true Shortcuts like ... and _ are just literals for procs. Then you can use procs directly too: Fast . match? ( [ :send , [ -> ( node ) { node . type == :send }, [ :send , '...' ] , :c ] , :d ] , ast ) # => true And also work with expressions: Fast . match? ( '(send (send (send (send nil $_) $_) $_) $_)' , ast ) # => [:a, :b, :c, :d] If something does not work you can debug with a block: Fast . debug { Fast . match? ( [ :int , 1 ] , s ( :int , 1 )) } It will output each comparison to stdout: int == (int 1) # => true 1 == 1 # => true search \u00b6 Search allows you to go deeply in the AST, collecting nodes that matches with the expression. It also returns captures if they exist. Fast . search ( '(int _)' , Fast . ast ( 'a = 1' )) # => s(:int, 1) If you use captures, it returns the node and the captures respectively: Fast . search ( '(int $_)' , Fast . ast ( 'a = 1' )) # => [s(:int, 1), 1] You can also bind external parameters in the search using extra arguments: Fast . search ( '(int %1)' , Fast . ast ( 'a = 1' ), 1 ) # => [s(:int, 1)] capture \u00b6 To pick just the captures and ignore the nodes, use Fast.capture : Fast . capture ( '(int $_)' , Fast . ast ( 'a = 1' )) # => 1 replace \u00b6 And if I want to refactor a code and use delegate , to: , try with replace: Fast . replace '(def $_ ... (send (send nil $_) \\1 ))' , ast do | node , captures | attribute , object = captures replace ( node . location . expression , \"delegate : #{ attribute } , to: : #{ object } \" ) end replace_file \u00b6 Now let's imagine we have real files like sample.rb with the following code: def good_bye message = [ \"good\" , \"bye\" ] puts message . join ( ' ' ) end And we decide to remove the message variable and put it inline with the puts . Basically, we need to find the local variable assignment, store the value in memory. Remove the assignment expression and use the value where the variable is being called. assignment = nil Fast . replace_file ( '({ lvasgn lvar } message )' , 'sample.rb' ) do | node , _ | if node . type == :lvasgn assignment = node . children . last remove ( node . location . expression ) elsif node . type == :lvar replace ( node . location . expression , assignment . location . expression . source ) end end It will return an output of the new source code with the changes but not save the file. You can use ()[#rewrite_file] if you're confident about the changes. capture_file \u00b6 Fast.capture_file can be used to combine capture and file system. Fast . capture_file ( \"$(casgn)\" , \"lib/fast/version.rb\" ) # => s(:casgn, nil, :VERSION, s(:str, \"0.1.3\")) Fast . capture_file ( \"(casgn nil _ (str $_))\" , \"lib/fast/version.rb\" ) # => \"0.1.3\" capture_all \u00b6 Fast.capture_all can be used to combine capture_file from multiple sources: Fast . capture_all ( \"(casgn nil $_)\" ) # => { \"./lib/fast/version.rb\"=>:VERSION, \"./lib/fast.rb\"=>[:LITERAL, :TOKENIZER], ...} The second parameter can also be passed with to filter specific folders: Fast . capture_all ( \"(casgn nil $_)\" , \"lib/fast\" ) # => {\"lib/fast/shortcut.rb\"=>:LOOKUP_FAST_FILES_DIRECTORIES, \"lib/fast/version.rb\"=>:VERSION} rewrite_file \u00b6 Fast.rewrite_file works exactly as the replace but it will override the file from the input. ast_from_file \u00b6 This method parses the code and load into a AST representation. Fast . ast_from_file ( 'sample.rb' ) search_file \u00b6 You can use search_file and pass the path for search for expressions inside files. Fast . search_file ( expression , 'file.rb' ) It's simple combination of Fast.ast_from_file with Fast.search . ruby_files_from \u00b6 You'll be probably looking for multiple ruby files, then this method fetches all internal .rb files Fast . ruby_files_from ( [ 'lib' ] ) # => [\"lib/fast.rb\"] search_all \u00b6 Combines the search_file with ruby_files_from multiple locations and returns tuples with files and results. Fast . search_all ( \"(def ast_from_file)\" ) => { \"./lib/fast.rb\" =>[ s ( :def , :ast_from_file , s ( :args , s ( :arg , :file )), s ( :begin , You can also override the second param and pass target files or folders: Fast . search_all ( \"(def _)\" , '../other-folder' )","title":"Introduction"},{"location":"#fast","text":"Fast is a \"Find AST\" tool to help you search in the code abstract syntax tree. Ruby allow us to do the same thing in a few ways then it's hard to check how the code is written. Using the AST will be easier than try to cover the multiple ways we can write the same code. You can define a string like %|| or '' or \"\" but they will have the same AST representation.","title":"Fast"},{"location":"#ast-representation","text":"Each detail of the ruby syntax have a equivalent identifier and some content. The content can be another expression or a final value. Fast uses parser gem behind the scenes to parse the code into nodes. First get familiar with parser gem and understand how ruby code is represented. When you install parser gem, you will have access to ruby-parse and you can use it with -e to parse an expression directly from the command line. Example: ruby-parse -e 1 It will print the following output: (int 1) And trying a number with decimals: ruby-parse -e 1.1 (float 1) Building a regex that will match decimals and integer looks like something easy and with fast you use a node pattern that reminds the syntax of regular expressions.","title":"AST representation"},{"location":"#syntax-for-find-in-ast","text":"The current version cover the following elements: () to represent a node search {} is for any matches like union conditions with or operator [] is for all matches like intersect conditions with and operator $ is for capture current expression _ is something not nil nil matches exactly nil ... is a node with children ^ is to get the parent node of an expression ? is for maybe \\1 to use the first previous captured element \"\" surround the value with double quotes to match literal strings Jump to Syntax .","title":"Syntax for find in AST"},{"location":"#ast","text":"Use Fast.ast to convert simple code to AST objects. You can use it as ruby-parse but directly from the console. Fast . ast ( \"1\" ) # => s(:int, 1) Fast . ast ( \"method\" ) # => s(:send, nil, :method) Fast . ast ( \"a.b\" ) # => s(:send, s(:send, nil, :a), :b) Fast . ast ( \"1 + 1\" ) # => s(:send, s(:int, 1), :+, s(:int, 1)) Fast . ast ( \"a = 2\" ) # => s(:lvasgn, :a, s(:int, 2)) Fast . ast ( \"b += 2\" ) # => s(:op_asgn, s(:lvasgn, :b), :+, s(:int, 2)) It uses astrolable gem behind the scenes: Fast . ast ( Fast . ast ( \"1\" )) . class => Astrolabe :: Node Fast . ast ( Fast . ast ( \"1\" )) . type => :int Fast . ast ( Fast . ast ( \"1\" )) . children => [ 1 ] See also ast_from_file .","title":"ast"},{"location":"#match","text":"Fast.match? is the most granular function that tries to compare a node with an expression. It returns true or false and some node captures case it find something. Let's start with a simple integer in Ruby: 1 The AST can be represented with the following expression: (int 1) The ast representation holds node type and children . Let's build a method s to represent Parser::AST::Node with a #type and #children . def s ( type , * children ) Parser :: AST :: Node . new ( type , children ) end A local variable assignment: value = 42 Can be represented with: ast = s ( :lvasgn , :value , s ( :int , 42 )) Now, lets find local variable named value with an value 42 : Fast . match? ( '(lvasgn value (int 42))' , ast ) # true Lets abstract a bit and allow some integer value using _ as a shortcut: Fast . match? ( '(lvasgn value (int _))' , ast ) # true Lets abstract more and allow float or integer: Fast . match? ( '(lvasgn value ({float int} _))' , ast ) # true Or combine multiple assertions using [] to join conditions: Fast . match? ( '(lvasgn value ([!str !hash !array] _))' , ast ) # true Matches all local variables not string and not hash and not array. We can match \"a node with children\" using ... : Fast . match? ( '(lvasgn value ...)' , ast ) # true You can use $ to capture a node: Fast . match? ( '(lvasgn value $...)' , ast ) # => [s(:int), 42] Or match whatever local variable assignment combining both _ and ... : Fast . match? ( '(lvasgn _ ...)' , ast ) # true You can also use captures in any levels you want: Fast . match? ( '(lvasgn $_ $...)' , ast ) # [:value, s(:int), 42] Keep in mind that _ means something not nil and ... means a node with children. Then, if do you get a method declared: def my_method call_other_method end It will be represented with the following structure: ast = s ( :def , :my_method , s ( :args ), s ( :send , nil , :call_other_method )) Keep an eye on the node (args) . Then you know you can't use ... but you can match with (_) to match with such case. Let's test a few other examples. You can go deeply with the arrays. Let's suppose we have a hardcore call to a.b.c.d and the following AST represents it: ast = s ( :send , s ( :send , s ( :send , s ( :send , nil , :a ), :b ), :c ), :d ) You can search using sub-arrays with pure values , or shortcuts or procs : Fast . match? ( [ :send , [ :send , '...' ] , :d ] , ast ) # => true Fast . match? ( [ :send , [ :send , '...' ] , :c ] , ast ) # => false Fast . match? ( [ :send , [ :send , [ :send , '...' ] , :c ] , :d ] , ast ) # => true Shortcuts like ... and _ are just literals for procs. Then you can use procs directly too: Fast . match? ( [ :send , [ -> ( node ) { node . type == :send }, [ :send , '...' ] , :c ] , :d ] , ast ) # => true And also work with expressions: Fast . match? ( '(send (send (send (send nil $_) $_) $_) $_)' , ast ) # => [:a, :b, :c, :d] If something does not work you can debug with a block: Fast . debug { Fast . match? ( [ :int , 1 ] , s ( :int , 1 )) } It will output each comparison to stdout: int == (int 1) # => true 1 == 1 # => true","title":"match?"},{"location":"#search","text":"Search allows you to go deeply in the AST, collecting nodes that matches with the expression. It also returns captures if they exist. Fast . search ( '(int _)' , Fast . ast ( 'a = 1' )) # => s(:int, 1) If you use captures, it returns the node and the captures respectively: Fast . search ( '(int $_)' , Fast . ast ( 'a = 1' )) # => [s(:int, 1), 1] You can also bind external parameters in the search using extra arguments: Fast . search ( '(int %1)' , Fast . ast ( 'a = 1' ), 1 ) # => [s(:int, 1)]","title":"search"},{"location":"#capture","text":"To pick just the captures and ignore the nodes, use Fast.capture : Fast . capture ( '(int $_)' , Fast . ast ( 'a = 1' )) # => 1","title":"capture"},{"location":"#replace","text":"And if I want to refactor a code and use delegate , to: , try with replace: Fast . replace '(def $_ ... (send (send nil $_) \\1 ))' , ast do | node , captures | attribute , object = captures replace ( node . location . expression , \"delegate : #{ attribute } , to: : #{ object } \" ) end","title":"replace"},{"location":"#replace_file","text":"Now let's imagine we have real files like sample.rb with the following code: def good_bye message = [ \"good\" , \"bye\" ] puts message . join ( ' ' ) end And we decide to remove the message variable and put it inline with the puts . Basically, we need to find the local variable assignment, store the value in memory. Remove the assignment expression and use the value where the variable is being called. assignment = nil Fast . replace_file ( '({ lvasgn lvar } message )' , 'sample.rb' ) do | node , _ | if node . type == :lvasgn assignment = node . children . last remove ( node . location . expression ) elsif node . type == :lvar replace ( node . location . expression , assignment . location . expression . source ) end end It will return an output of the new source code with the changes but not save the file. You can use ()[#rewrite_file] if you're confident about the changes.","title":"replace_file"},{"location":"#capture_file","text":"Fast.capture_file can be used to combine capture and file system. Fast . capture_file ( \"$(casgn)\" , \"lib/fast/version.rb\" ) # => s(:casgn, nil, :VERSION, s(:str, \"0.1.3\")) Fast . capture_file ( \"(casgn nil _ (str $_))\" , \"lib/fast/version.rb\" ) # => \"0.1.3\"","title":"capture_file"},{"location":"#capture_all","text":"Fast.capture_all can be used to combine capture_file from multiple sources: Fast . capture_all ( \"(casgn nil $_)\" ) # => { \"./lib/fast/version.rb\"=>:VERSION, \"./lib/fast.rb\"=>[:LITERAL, :TOKENIZER], ...} The second parameter can also be passed with to filter specific folders: Fast . capture_all ( \"(casgn nil $_)\" , \"lib/fast\" ) # => {\"lib/fast/shortcut.rb\"=>:LOOKUP_FAST_FILES_DIRECTORIES, \"lib/fast/version.rb\"=>:VERSION}","title":"capture_all"},{"location":"#rewrite_file","text":"Fast.rewrite_file works exactly as the replace but it will override the file from the input.","title":"rewrite_file"},{"location":"#ast_from_file","text":"This method parses the code and load into a AST representation. Fast . ast_from_file ( 'sample.rb' )","title":"ast_from_file"},{"location":"#search_file","text":"You can use search_file and pass the path for search for expressions inside files. Fast . search_file ( expression , 'file.rb' ) It's simple combination of Fast.ast_from_file with Fast.search .","title":"search_file"},{"location":"#ruby_files_from","text":"You'll be probably looking for multiple ruby files, then this method fetches all internal .rb files Fast . ruby_files_from ( [ 'lib' ] ) # => [\"lib/fast.rb\"]","title":"ruby_files_from"},{"location":"#search_all","text":"Combines the search_file with ruby_files_from multiple locations and returns tuples with files and results. Fast . search_all ( \"(def ast_from_file)\" ) => { \"./lib/fast.rb\" =>[ s ( :def , :ast_from_file , s ( :args , s ( :arg , :file )), s ( :begin , You can also override the second param and pass target files or folders: Fast . search_all ( \"(def _)\" , '../other-folder' )","title":"search_all"},{"location":"command_line/","text":"Command line \u00b6 When you install the ffast gem, it will also create an executable named fast and you can use it to search and find code using the concept: $ fast '(def match?)' lib/fast.rb Use -d or --debug for enable debug mode. Use --ast to output the AST instead of the original code Use --pry to jump debugging the first result with pry Use -c to search from code example Use -s to search similar code Use -p to or --parallel to use multi core search --pry \u00b6 $ fast '(block (send nil it))' spec --pry And inside pry session, you can use result as the first result or results to use all occurrences found. results . map { | e | e . children [ 0 ]. children [ 2 ] } # => [s(:str, \"parses ... as Find\"), # s(:str, \"parses $ as Capture\"), # s(:str, \"parses quoted values as strings\"), # s(:str, \"parses {} as Any\"), # s(:str, \"parses [] as All\"), ...] Getting all it blocks without description: $ fast '(block (send nil it (nil)) (args ) (!str)) ) )' spec # spec/fast_spec.rb:166 it { expect ( described_class ) . to be_match ( '(...)' , s ( :int , 1 )) } ... --debug \u00b6 This option will print all matching details while validating each node. $ echo 'object.method' > sample.rb $ fast -d '(send (send nil _) _)' sample.rb It will bring details of the expression compiled and each node being validated: Expression: f[send] [#, #, #] f[_] send == (send (send nil :object) :method) # => true f[send] == (send (send nil :object) :method) # => true send == (send nil :object) # => true f[send] == (send nil :object) # => true == # => true f[nil] == # => true # == object # => true f[_] == object # => true [#, #, #] == (send nil :object) # => true # == method # => true f[_] == method # => true # sample.rb:1 object.method -s for similarity \u00b6 Sometimes you want to search for some similar code like (send (send (send nil _) _) _) and we could simply say a.b.c . The option -s build an expression from the code ignoring final values. $ echo 'object.method' > sample.rb $ fast -s 'a.b' sample.rb # sample.rb:1 object . method See also Code Similarity tutorial. -c to search from code example \u00b6 You can search for the exact expression with -c $ fast -c 'object.method' sample.rb # sample.rb:1 object . method Combining with -d , in the header you can see the generated expression. $ fast -d -c 'object.method' sample.rb | head -n 3 The generated expression from AST was: (send (send nil :object) :method) Fastfile \u00b6 Fastfile will loaded when you start a pattern with a dot. It means the pattern will be a shortcut predefined on these Fastfiles. It will make three attempts to load Fastfile defined in $PWD , $HOME or checking if the $FAST_FILE_DIR is configured. You can define a Fastfile in any project with your custom shortcuts and easy check some code or run some task. Shortcut examples \u00b6 Create shortcuts with blocks enables introduce custom coding in the scope of the Fast module. Print library version. \u00b6 Let's say you'd like to show the version of your library. Your regular params in the command line will look like: $ fast '(casgn nil VERSION)' lib/*/version.rb It will output but the command is not very handy. In order to just say fast .version you can use the previous snippet in your Fastfile . Fast . shortcut ( :version , '(casgn nil VERSION)' , 'lib/fast/version.rb' ) And calling fast .version it will output something like this: # lib/fast/version.rb:4 VERSION = '0.1.2' We can also always override the files params passing some other target file like fast .version lib/other/file.rb and it will reuse the other arguments from command line but replace the target files. Bumping a gem version \u00b6 While releasing a new gem version, we always need to mechanical go through the lib//version.rb and change the string value to bump the version of your library. It's pretty mechanical and here is an example that allow you to simple use fast .bump_version : Fast . shortcut :bump_version do rewrite_file ( 'lib/fast/version.rb' , '(casgn nil VERSION (str _)' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( \".\" ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( \".\" ) } '\" ) end end Note the shortcut scope The shortcut calls rewrite_file from Fast scope as it use Fast.instance_exec for shortcuts that yields blocks. Checking the version: $ fast .version 13 :58:40 # lib/fast/version.rb:4 VERSION = '0.1.2' Bumping the version: $ fast .bump_version 13 :58:43 No output because we don't print anything. Checking version again: $ fast .version 13 :58:54 # lib/fast/version.rb:4 VERSION = '0.1.3' And now a fancy shortcut to report the other shortcuts :) Fast . shortcut :shortcuts do report ( shortcuts . keys ) end Or we can make it a bit more friendly and also use Fast to process the shortcut positions and pick the comment that each shortcut have in the previous line: # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _))' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And it will be printing all loaded shortcuts with comments: $ fast .shortcuts fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments You can find more examples in the Fastfile .","title":"Command Line"},{"location":"command_line/#command-line","text":"When you install the ffast gem, it will also create an executable named fast and you can use it to search and find code using the concept: $ fast '(def match?)' lib/fast.rb Use -d or --debug for enable debug mode. Use --ast to output the AST instead of the original code Use --pry to jump debugging the first result with pry Use -c to search from code example Use -s to search similar code Use -p to or --parallel to use multi core search","title":"Command line"},{"location":"command_line/#-pry","text":"$ fast '(block (send nil it))' spec --pry And inside pry session, you can use result as the first result or results to use all occurrences found. results . map { | e | e . children [ 0 ]. children [ 2 ] } # => [s(:str, \"parses ... as Find\"), # s(:str, \"parses $ as Capture\"), # s(:str, \"parses quoted values as strings\"), # s(:str, \"parses {} as Any\"), # s(:str, \"parses [] as All\"), ...] Getting all it blocks without description: $ fast '(block (send nil it (nil)) (args ) (!str)) ) )' spec # spec/fast_spec.rb:166 it { expect ( described_class ) . to be_match ( '(...)' , s ( :int , 1 )) } ...","title":"--pry"},{"location":"command_line/#-debug","text":"This option will print all matching details while validating each node. $ echo 'object.method' > sample.rb $ fast -d '(send (send nil _) _)' sample.rb It will bring details of the expression compiled and each node being validated: Expression: f[send] [#, #, #] f[_] send == (send (send nil :object) :method) # => true f[send] == (send (send nil :object) :method) # => true send == (send nil :object) # => true f[send] == (send nil :object) # => true == # => true f[nil] == # => true # == object # => true f[_] == object # => true [#, #, #] == (send nil :object) # => true # == method # => true f[_] == method # => true # sample.rb:1 object.method","title":"--debug"},{"location":"command_line/#-s-for-similarity","text":"Sometimes you want to search for some similar code like (send (send (send nil _) _) _) and we could simply say a.b.c . The option -s build an expression from the code ignoring final values. $ echo 'object.method' > sample.rb $ fast -s 'a.b' sample.rb # sample.rb:1 object . method See also Code Similarity tutorial.","title":"-s for similarity"},{"location":"command_line/#-c-to-search-from-code-example","text":"You can search for the exact expression with -c $ fast -c 'object.method' sample.rb # sample.rb:1 object . method Combining with -d , in the header you can see the generated expression. $ fast -d -c 'object.method' sample.rb | head -n 3 The generated expression from AST was: (send (send nil :object) :method)","title":"-c to search from code example"},{"location":"command_line/#fastfile","text":"Fastfile will loaded when you start a pattern with a dot. It means the pattern will be a shortcut predefined on these Fastfiles. It will make three attempts to load Fastfile defined in $PWD , $HOME or checking if the $FAST_FILE_DIR is configured. You can define a Fastfile in any project with your custom shortcuts and easy check some code or run some task.","title":"Fastfile"},{"location":"command_line/#shortcut-examples","text":"Create shortcuts with blocks enables introduce custom coding in the scope of the Fast module.","title":"Shortcut examples"},{"location":"command_line/#print-library-version","text":"Let's say you'd like to show the version of your library. Your regular params in the command line will look like: $ fast '(casgn nil VERSION)' lib/*/version.rb It will output but the command is not very handy. In order to just say fast .version you can use the previous snippet in your Fastfile . Fast . shortcut ( :version , '(casgn nil VERSION)' , 'lib/fast/version.rb' ) And calling fast .version it will output something like this: # lib/fast/version.rb:4 VERSION = '0.1.2' We can also always override the files params passing some other target file like fast .version lib/other/file.rb and it will reuse the other arguments from command line but replace the target files.","title":"Print library version."},{"location":"command_line/#bumping-a-gem-version","text":"While releasing a new gem version, we always need to mechanical go through the lib//version.rb and change the string value to bump the version of your library. It's pretty mechanical and here is an example that allow you to simple use fast .bump_version : Fast . shortcut :bump_version do rewrite_file ( 'lib/fast/version.rb' , '(casgn nil VERSION (str _)' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( \".\" ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( \".\" ) } '\" ) end end Note the shortcut scope The shortcut calls rewrite_file from Fast scope as it use Fast.instance_exec for shortcuts that yields blocks. Checking the version: $ fast .version 13 :58:40 # lib/fast/version.rb:4 VERSION = '0.1.2' Bumping the version: $ fast .bump_version 13 :58:43 No output because we don't print anything. Checking version again: $ fast .version 13 :58:54 # lib/fast/version.rb:4 VERSION = '0.1.3' And now a fancy shortcut to report the other shortcuts :) Fast . shortcut :shortcuts do report ( shortcuts . keys ) end Or we can make it a bit more friendly and also use Fast to process the shortcut positions and pick the comment that each shortcut have in the previous line: # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _))' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And it will be printing all loaded shortcuts with comments: $ fast .shortcuts fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments You can find more examples in the Fastfile .","title":"Bumping a gem version"},{"location":"editors-integration/","text":"Editors' integration \u00b6 We don't have any proper integration or official plugins for editors yet. Here are a few ideas you can use to make your own flow. Vim \u00b6 Split terminal vertically and open fast focused on build the expression. nnoremap < Leader > ff : vsplit \\ | terminal fast \"()\" % < Left >< Left >< Left >< Left >< Left > Or you can build a function: function ! s:Fast ( args ) let cmd = '' if ! empty ( b :ruby_project_root ) let cmd . = 'cd ' . b :ruby_project_root . ' && ' endif let cmd . = 'fast --no-color ' . a :args let custom_maker = neomake#utils#MakerFromCommand ( cmd ) let custom_maker.name = cmd let custom_maker.cwd = b :ruby_project_root let custom_maker.remove_invalid_entries = 0 \" e.g.: \" # path/to/file.rb:1141 \" my_method( \" :boom, \" arg1: 1, \" ) \" %W# %f:%l -> start a multiline warning when the line matches '# path/file.rb:1234' \" %-Z# end multiline warning on the next line that starts with '#' \" %C%m continued multiline warning message let custom_maker. errorformat = '%W# %f:%l, %-Z#, %C%m' let enabled_makers = [custom_maker] update | call neomake#Make ( 0 , enabled_makers ) | echom \"running: \" . cmd endfunction command ! - complete = file - nargs = 1 Fast call s:Fast (< q - args >) Check the conversation about vim integration here .","title":"Editors' Integration"},{"location":"editors-integration/#editors-integration","text":"We don't have any proper integration or official plugins for editors yet. Here are a few ideas you can use to make your own flow.","title":"Editors' integration"},{"location":"editors-integration/#vim","text":"Split terminal vertically and open fast focused on build the expression. nnoremap < Leader > ff : vsplit \\ | terminal fast \"()\" % < Left >< Left >< Left >< Left >< Left > Or you can build a function: function ! s:Fast ( args ) let cmd = '' if ! empty ( b :ruby_project_root ) let cmd . = 'cd ' . b :ruby_project_root . ' && ' endif let cmd . = 'fast --no-color ' . a :args let custom_maker = neomake#utils#MakerFromCommand ( cmd ) let custom_maker.name = cmd let custom_maker.cwd = b :ruby_project_root let custom_maker.remove_invalid_entries = 0 \" e.g.: \" # path/to/file.rb:1141 \" my_method( \" :boom, \" arg1: 1, \" ) \" %W# %f:%l -> start a multiline warning when the line matches '# path/file.rb:1234' \" %-Z# end multiline warning on the next line that starts with '#' \" %C%m continued multiline warning message let custom_maker. errorformat = '%W# %f:%l, %-Z#, %C%m' let enabled_makers = [custom_maker] update | call neomake#Make ( 0 , enabled_makers ) | echom \"running: \" . cmd endfunction command ! - complete = file - nargs = 1 Fast call s:Fast (< q - args >) Check the conversation about vim integration here .","title":"Vim"},{"location":"experiments/","text":"Experiments \u00b6 Experiments allow us to play with AST and do some code transformation, execute some code and continue combining successful transformations. The major idea is try a new approach without any promise and if it works continue transforming the code. Replace FactoryBot#create with build_stubbed . \u00b6 Let's look into the following spec example: describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's say we're amazed with FactoryBot#build_stubbed and want to build a small bot to make the changes in a entire code base. Skip some database touches while testing huge test suites are always a good idea. First we can hunt for the cases we want to find: $ ruby-parse -e \"create(:user)\" (send nil :create (sym :user)) Using fast in the command line to see real examples in the spec folder: $ fast \"(send nil create)\" spec If you don't have a real project but want to test, just create a sample ruby file with the code example above. Running it in a big codebase will probably find a few examples of blocks. The next step is build a replacement of each independent occurrence to use build_stubbed instead of create and combine the successful ones, run again and combine again, until try all kind of successful replacements combined. Considering we have the following code in sample_spec.rb : describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's create the experiment that will contain the nodes that are target to be executed and what we want to do when we find the node. experiment = Fast . experiment ( 'RSpec/ReplaceCreateWithBuildStubbed' ) do search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } end If we use Fast.replace_file it will replace all occurrences in the same run and that's one of the motivations behind create the ExperimentFile class. Executing a partial replacement of the first occurrence: experiment_file = Fast :: ExperimentFile . new ( 'sample_spec.rb' , experiment ) } puts experiment_file . partial_replace ( 1 ) The command will output the following code: describe \"my spec\" do let ( :user ) { build_stubbed ( :user ) } let ( :address ) { create ( :address ) } # ... end Remove useless before block \u00b6 Imagine the following code sample: describe \"my spec\" do before { create ( :user ) } # ... after { User . delete_all } end And now, we can define an experiment that removes the entire code block and run the experimental specs. experiment = Fast . experiment ( 'RSpec/RemoveUselessBeforeAfterHook' ) do lookup 'spec' search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end To run the experiment you can simply say: experiment . run Or drop the code into experiments folder and use the fast-experiment command line tool. $ fast-experiment RSpec/RemoveUselessBeforeAfterHook spec DSL \u00b6 In the lookup you can pass files or folders. The search contains the expression you want to match With edit block you can apply the code change And the policy is executed to check if the current change is valuable If the file contains multiple before or after blocks, each removal will occur independently and the successfull removals will be combined as a secondary change. The process repeates until find all possible combinations. See more examples in experiments folder. To run multiple experiments, use fast-experiment runner: fast-experiment You can limit experiments or file escope: fast-experiment RSpec/RemoveUselessBeforeAfterHook spec/models/**/*_spec.rb Or a single file: fast-experiment RSpec/ReplaceCreateWithBuildStubbed spec/models/my_spec.rb","title":"Experiments"},{"location":"experiments/#experiments","text":"Experiments allow us to play with AST and do some code transformation, execute some code and continue combining successful transformations. The major idea is try a new approach without any promise and if it works continue transforming the code.","title":"Experiments"},{"location":"experiments/#replace-factorybotcreate-with-build_stubbed","text":"Let's look into the following spec example: describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's say we're amazed with FactoryBot#build_stubbed and want to build a small bot to make the changes in a entire code base. Skip some database touches while testing huge test suites are always a good idea. First we can hunt for the cases we want to find: $ ruby-parse -e \"create(:user)\" (send nil :create (sym :user)) Using fast in the command line to see real examples in the spec folder: $ fast \"(send nil create)\" spec If you don't have a real project but want to test, just create a sample ruby file with the code example above. Running it in a big codebase will probably find a few examples of blocks. The next step is build a replacement of each independent occurrence to use build_stubbed instead of create and combine the successful ones, run again and combine again, until try all kind of successful replacements combined. Considering we have the following code in sample_spec.rb : describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's create the experiment that will contain the nodes that are target to be executed and what we want to do when we find the node. experiment = Fast . experiment ( 'RSpec/ReplaceCreateWithBuildStubbed' ) do search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } end If we use Fast.replace_file it will replace all occurrences in the same run and that's one of the motivations behind create the ExperimentFile class. Executing a partial replacement of the first occurrence: experiment_file = Fast :: ExperimentFile . new ( 'sample_spec.rb' , experiment ) } puts experiment_file . partial_replace ( 1 ) The command will output the following code: describe \"my spec\" do let ( :user ) { build_stubbed ( :user ) } let ( :address ) { create ( :address ) } # ... end","title":"Replace FactoryBot#create with build_stubbed."},{"location":"experiments/#remove-useless-before-block","text":"Imagine the following code sample: describe \"my spec\" do before { create ( :user ) } # ... after { User . delete_all } end And now, we can define an experiment that removes the entire code block and run the experimental specs. experiment = Fast . experiment ( 'RSpec/RemoveUselessBeforeAfterHook' ) do lookup 'spec' search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end To run the experiment you can simply say: experiment . run Or drop the code into experiments folder and use the fast-experiment command line tool. $ fast-experiment RSpec/RemoveUselessBeforeAfterHook spec","title":"Remove useless before block"},{"location":"experiments/#dsl","text":"In the lookup you can pass files or folders. The search contains the expression you want to match With edit block you can apply the code change And the policy is executed to check if the current change is valuable If the file contains multiple before or after blocks, each removal will occur independently and the successfull removals will be combined as a secondary change. The process repeates until find all possible combinations. See more examples in experiments folder. To run multiple experiments, use fast-experiment runner: fast-experiment You can limit experiments or file escope: fast-experiment RSpec/RemoveUselessBeforeAfterHook spec/models/**/*_spec.rb Or a single file: fast-experiment RSpec/ReplaceCreateWithBuildStubbed spec/models/my_spec.rb","title":"DSL"},{"location":"git/","text":"You can overload the AST node with extra methods to get information from Git. Let's start with some basic setup to reuse in the next examples: Git require \u00b6 By default, this extension is not loaded in the fast environment, so you should require it. require 'fast/git' Then it will work with any AST node. ast = Fast . ast_from_file ( 'lib/fast.rb' ) Log \u00b6 First commit from git: ast . git_log . first . author . name # => \"Jonatas Davi Paganini\" It uses ruby-git gem, so all methods are available: ast . git_log . since ( Time . mktime ( 2019 )) . entries . map ( & :message ) Counting commits per year: ast . git_log . entries . group_by { | t | t . date . year } . transform_values ( & :size ) # => {2020=>4, 2019=>22, 2018=>4} Counting commits per contributor: ast . git_log . entries . group_by { | t | t . author . name } . transform_values ( & :size ) # => {\"J\u00f4natas Davi Paganini\"=>29, ...} Selecting last commit message: ast . last_commit . message # => \"Add node extensions for extracting info from git (#21)\" Remote git URL: ast . remote_url # => \"git@github.com:jonatas/fast.git\" ast . project_url # => \"https://github.com/jonatas/fast\" The sha from last commit: ast . sha # => \"cd1c036b55ec1d41e5769ad73b282dd6429a90a6\" Pick a link from the files to master version: ast . link # => \"https://github.com/jonatas/fast/blob/master/lib/fast.rb#L3-L776\" Getting permalink from current commit: ast . permalink # => \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L3-L776\" Markdown link \u00b6 Let's say you'd like to capture a list of class names that inherits the Find class: puts ast . capture ( \"(class $(const nil _) (const nil Find)\" ) . map ( & :md_link ) . join ( \" \\n * \" ) It will output the following links: FindString MethodCall InstanceMethodCall FindWithCapture FindFromArgument Capture Parent Any All Not Maybe Permalink \u00b6 If you need to get a permanent link to the code, use the permalink method: ast . search ( \"(class (const nil _) (const nil Find)\" ) . map ( & :permalink ) # => [\"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L524-L541\", # \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L551-L571\", ...]","title":"Git Integration"},{"location":"git/#git-require","text":"By default, this extension is not loaded in the fast environment, so you should require it. require 'fast/git' Then it will work with any AST node. ast = Fast . ast_from_file ( 'lib/fast.rb' )","title":"Git require"},{"location":"git/#log","text":"First commit from git: ast . git_log . first . author . name # => \"Jonatas Davi Paganini\" It uses ruby-git gem, so all methods are available: ast . git_log . since ( Time . mktime ( 2019 )) . entries . map ( & :message ) Counting commits per year: ast . git_log . entries . group_by { | t | t . date . year } . transform_values ( & :size ) # => {2020=>4, 2019=>22, 2018=>4} Counting commits per contributor: ast . git_log . entries . group_by { | t | t . author . name } . transform_values ( & :size ) # => {\"J\u00f4natas Davi Paganini\"=>29, ...} Selecting last commit message: ast . last_commit . message # => \"Add node extensions for extracting info from git (#21)\" Remote git URL: ast . remote_url # => \"git@github.com:jonatas/fast.git\" ast . project_url # => \"https://github.com/jonatas/fast\" The sha from last commit: ast . sha # => \"cd1c036b55ec1d41e5769ad73b282dd6429a90a6\" Pick a link from the files to master version: ast . link # => \"https://github.com/jonatas/fast/blob/master/lib/fast.rb#L3-L776\" Getting permalink from current commit: ast . permalink # => \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L3-L776\"","title":"Log"},{"location":"git/#markdown-link","text":"Let's say you'd like to capture a list of class names that inherits the Find class: puts ast . capture ( \"(class $(const nil _) (const nil Find)\" ) . map ( & :md_link ) . join ( \" \\n * \" ) It will output the following links: FindString MethodCall InstanceMethodCall FindWithCapture FindFromArgument Capture Parent Any All Not Maybe","title":"Markdown link"},{"location":"git/#permalink","text":"If you need to get a permanent link to the code, use the permalink method: ast . search ( \"(class (const nil _) (const nil Find)\" ) . map ( & :permalink ) # => [\"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L524-L541\", # \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L551-L571\", ...]","title":"Permalink"},{"location":"ideas/","text":"Ideas I want to build with Fast \u00b6 I don't have all the time I need to develop all the ideas I have to build around this tool, so here is a dump of a few brainstormings: Inline target code \u00b6 I started fast-inline that can be useful to try to see how much every library is used in a project. My idea is try to inline some specific method call to understand if it makes sense to have an entire library in the stock. Understanding dependencies and how the code works can be a first step to get an \"algorithm as a service\". Instead of loading everything from the library, it would facilitate the cherry pick of only the proper dependencies necessaries to run the code you have and not the code that is overloading the project. Neo4J adapter \u00b6 Easy pipe fast results to Neo4J. It would facilitate to explore more complex scenarios and combine data from other sources. Ast Diff \u00b6 Allow to compare and return a summary of differences between two trees. It would be useful to identify renamings or other small changes, like only changes in comments that does not affect the file and possibly be ignored for some operations like run or not run tests. Transition synapses \u00b6 Following the previous idea, it would be great if we can understand the transition synapses and make it easily available to catch up with previous learnings. https://github.com/jonatas/chewy-diff/blob/master/lib/chewy/diff.rb This example, shows adds and removals from specific node targets between two different files. If we start tracking AST transition synapses and associating with \"Fixes\" or \"Reverts\" we can predict introduction of new bugs by inpecting if the introduction of new patterns that can be possibly reverted or improved. Fast Rewriter with pure strings \u00b6 As the AST rewriter adopts a custom block that needs to implement ruby code, we can expand the a query language for rewriting files without need to take the custom Ruby block. Example: Fast . gsub_expression ( 'remove(@expression)' ) # (node) => { remove(node.location.expression) } And later we can bind it in the command line to allow implement custom replacements without need to write a ruby file. fast (def my_target_method) lib spec --rewrite \"remove(@expression)\" or fast (def my_target_method) lib spec --rewrite \"replace(@name, 'renamed_method')\"","title":"Ideas"},{"location":"ideas/#ideas-i-want-to-build-with-fast","text":"I don't have all the time I need to develop all the ideas I have to build around this tool, so here is a dump of a few brainstormings:","title":"Ideas I want to build with Fast"},{"location":"ideas/#inline-target-code","text":"I started fast-inline that can be useful to try to see how much every library is used in a project. My idea is try to inline some specific method call to understand if it makes sense to have an entire library in the stock. Understanding dependencies and how the code works can be a first step to get an \"algorithm as a service\". Instead of loading everything from the library, it would facilitate the cherry pick of only the proper dependencies necessaries to run the code you have and not the code that is overloading the project.","title":"Inline target code"},{"location":"ideas/#neo4j-adapter","text":"Easy pipe fast results to Neo4J. It would facilitate to explore more complex scenarios and combine data from other sources.","title":"Neo4J adapter"},{"location":"ideas/#ast-diff","text":"Allow to compare and return a summary of differences between two trees. It would be useful to identify renamings or other small changes, like only changes in comments that does not affect the file and possibly be ignored for some operations like run or not run tests.","title":"Ast Diff"},{"location":"ideas/#transition-synapses","text":"Following the previous idea, it would be great if we can understand the transition synapses and make it easily available to catch up with previous learnings. https://github.com/jonatas/chewy-diff/blob/master/lib/chewy/diff.rb This example, shows adds and removals from specific node targets between two different files. If we start tracking AST transition synapses and associating with \"Fixes\" or \"Reverts\" we can predict introduction of new bugs by inpecting if the introduction of new patterns that can be possibly reverted or improved.","title":"Transition synapses"},{"location":"ideas/#fast-rewriter-with-pure-strings","text":"As the AST rewriter adopts a custom block that needs to implement ruby code, we can expand the a query language for rewriting files without need to take the custom Ruby block. Example: Fast . gsub_expression ( 'remove(@expression)' ) # (node) => { remove(node.location.expression) } And later we can bind it in the command line to allow implement custom replacements without need to write a ruby file. fast (def my_target_method) lib spec --rewrite \"remove(@expression)\" or fast (def my_target_method) lib spec --rewrite \"replace(@name, 'renamed_method')\"","title":"Fast Rewriter with pure strings"},{"location":"pry-integration/","text":"You can create a custom command in pry to reuse fast in any session. Start simply dropping it on your .pryrc : Pry :: Commands . block_command \"fast\" , \"Fast search\" do | expression , file | require \"fast\" files = Fast . ruby_files_from ( file || '.' ) files . each do | f | results = Fast . search_file ( expression , f ) next if results . nil? || results . empty? output . puts Fast . highlight ( \"# #{ f } \" ) results . each do | result | output . puts Fast . highlight ( result ) end end end And use it in the console: fast '(def match?)' lib/fast.rb","title":"Pry Integration"},{"location":"research/","text":"Research \u00b6 I love to research about codebase as data and prototyping ideas several times doesn't fit in simple shortcuts . Here is my first research that worth sharing: Combining Runtime metadata with AST complex searches \u00b6 This example covers how to find RSpec allow combined with and_return missing the with clause specifying the nested parameters. Here is the gist if you want to go straight and run it. Scenario for simple example: Given I have the following class: class Account def withdraw ( value ) if @total >= value @total -= value :ok else :not_allowed end end end And I'm testing it with allow and some possibilities: # bad allow ( Account ) . to receive ( :withdraw ) . and_return ( :ok ) # good allow ( Account ) . to receive ( :withdraw ) . with ( 100 ) . and_return ( :ok ) Objective: find all bad cases of any class that does not respect the method parameters signature. First, let's understand the method signature of a method: Account . instance_method ( :withdraw ) . parameters # => [[:req, :value]] Now, we can build a small script to use the node pattern to match the proper specs that are using such pattern and later visit their method signatures. Fast . class_eval do # Captures class and method name when find syntax like: # `allow(...).to receive(...)` that does not end with `.with(...)` pattern_with_captures = <<~ FAST (send (send nil allow (const nil $_)) to (send (send nil receive (sym $_)) !with)) FAST pattern = expression ( pattern_with_captures . tr ( '$' , '' )) ruby_files_from ( 'spec' ) . each do | file | results = search_file ( pattern , file ) || [] rescue next results . each do | n | clazz , method = capture ( n , pattern_with_captures ) if klazz = Object . const_get ( clazz . to_s ) rescue nil if klazz . respond_to? ( method ) params = klazz . method ( method ) . parameters if params . any? { | e | e . first == :req } code = n . loc . expression range = [ code . first_line , code . last_line ]. uniq . join ( \",\" ) boom_message = \"BOOM! #{ clazz } . #{ method } does not include the REQUIRED parameters!\" puts boom_message , \" #{ file } : #{ range } \" , code . source end end end end end end Preload your environment before run the script Keep in mind that you should run it with your environment preloaded otherwise it will skip the classes. You can add elses for const_get and respond_to and report weird cases if your environment is not preloading properly.","title":"Research"},{"location":"research/#research","text":"I love to research about codebase as data and prototyping ideas several times doesn't fit in simple shortcuts . Here is my first research that worth sharing:","title":"Research"},{"location":"research/#combining-runtime-metadata-with-ast-complex-searches","text":"This example covers how to find RSpec allow combined with and_return missing the with clause specifying the nested parameters. Here is the gist if you want to go straight and run it. Scenario for simple example: Given I have the following class: class Account def withdraw ( value ) if @total >= value @total -= value :ok else :not_allowed end end end And I'm testing it with allow and some possibilities: # bad allow ( Account ) . to receive ( :withdraw ) . and_return ( :ok ) # good allow ( Account ) . to receive ( :withdraw ) . with ( 100 ) . and_return ( :ok ) Objective: find all bad cases of any class that does not respect the method parameters signature. First, let's understand the method signature of a method: Account . instance_method ( :withdraw ) . parameters # => [[:req, :value]] Now, we can build a small script to use the node pattern to match the proper specs that are using such pattern and later visit their method signatures. Fast . class_eval do # Captures class and method name when find syntax like: # `allow(...).to receive(...)` that does not end with `.with(...)` pattern_with_captures = <<~ FAST (send (send nil allow (const nil $_)) to (send (send nil receive (sym $_)) !with)) FAST pattern = expression ( pattern_with_captures . tr ( '$' , '' )) ruby_files_from ( 'spec' ) . each do | file | results = search_file ( pattern , file ) || [] rescue next results . each do | n | clazz , method = capture ( n , pattern_with_captures ) if klazz = Object . const_get ( clazz . to_s ) rescue nil if klazz . respond_to? ( method ) params = klazz . method ( method ) . parameters if params . any? { | e | e . first == :req } code = n . loc . expression range = [ code . first_line , code . last_line ]. uniq . join ( \",\" ) boom_message = \"BOOM! #{ clazz } . #{ method } does not include the REQUIRED parameters!\" puts boom_message , \" #{ file } : #{ range } \" , code . source end end end end end end Preload your environment before run the script Keep in mind that you should run it with your environment preloaded otherwise it will skip the classes. You can add elses for const_get and respond_to and report weird cases if your environment is not preloading properly.","title":"Combining Runtime metadata with AST complex searches"},{"location":"shortcuts/","text":"Shortcuts \u00b6 Shortcuts are defined on a Fastfile inside any ruby project. Use ~/Fastfile You can also add one extra in your $HOME if you want to have something loaded always. By default, the command line interface does not load any Fastfile if the first param is not a shortcut. It should start with . . I'm building several researches and I'll make the examples open here to show several interesting cases in action. List your fast shortcuts \u00b6 As the interface is very rudimentar, let's build a shortcut to print what shortcuts are available. This is a good one to your $HOME/Fastfile : # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And using it on fast project that loads both ~/Fastfile and the Fastfile from the project: fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments Search for references \u00b6 I always miss bringing something simple as grep keyword where I can leave a simple string and it can search in all types of nodes and report interesting things about it. Let's consider a very flexible search that can target any code related to some keyword. Considering that we're talking about code indentifiers: # Search all references about some keyword or regular expression Fast . shortcut ( :ref ) do require 'fast/cli' Kernel . class_eval do def matches_args? identifier search = ARGV . last regex = Regexp . new ( search , Regexp :: IGNORECASE ) case identifier when Symbol , String regex . match? ( identifier ) || identifier . to_s . include? ( search ) when Astrolabe :: Node regex . match? ( identifier . to_sexp ) end end end pattern = <<~ FAST { ({class def sym str} #matches_args?)' ({const send} nil #matches_args?)' } FAST Fast :: Cli . run! ( [ pattern , '.' , '--parallel' ] ) end Rails: Show validations from models \u00b6 If the shortcut does not define a block, it works as a holder for arguments from the command line. Let's say you always use fast \"(send nil {validate validates})\" app/models to check validations in the models. You can define a shortcut to hold the args and avoid retyping long lines: # Show validations from app/models Fast . shortcut ( :validations , \"(send nil {validate validates})\" , \"app/models\" ) And you can reuse the search with the shortcut starting with a . : fast .validations And it will also accept params if you want to filter a specific file: fast .validations app/models/user.rb Note that you can also use flags in the command line shortcuts Let's say you also want to use fast --headless you can add it to the params: Fast.shortcut(:validations, \"(send nil {validate validates})\", \"app/models\", \"--headless\") Automated Refactor: Bump version \u00b6 Let's start with a real usage to bump a new version of the gem. Fast . shortcut :bump_version do rewrite_file ( '(casgn nil VERSION (str _)' , 'lib/fast/version.rb' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( '.' ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( '.' ) } '\" ) end end And then the change is done in the lib/fast/version.rb : module Fast - VERSION = '0.1.6' + VERSION = '0.1.7' end RSpec: Find unused shared contexts \u00b6 If you build shared contexts often, probably you can forget some left overs. The objective of the shortcut is find leftovers from shared contexts. First, the objective is capture all names of the RSpec.shared_context or shared_context declared in the spec/support folder. Fast . capture_all ( '(block (send {nil,_} shared_context (str $_)))' , Fast . ruby_files_from ( 'spec/support' )) Then, we need to check all the specs and search for include_context usages to confirm if all defined contexts are being used: specs = Fast . ruby_files_from ( 'spec' ) . select { | f | f !~ %r{spec/support/} } Fast . search_all ( \"(send nil include_context (str #register_usage)\" , specs ) Note that we created a new reference to #register_usage and we need to define the method too: @used = [] def register_usage context_name @used << context_name end Wrapping up everything in a shortcut: # Show unused shared contexts Fast . shortcut ( :unused_shared_contexts ) do puts \"Checking shared contexts\" Kernel . class_eval do @used = [] def register_usage context_name @used << context_name end def show_report! defined_contexts unused = defined_contexts . values . flatten - @used if unused . any? puts \"Unused shared contexts\" , unused else puts \"Good job! all the #{ defined_contexts . size } contexts are used!\" end end end specs = ruby_files_from ( 'spec/' ) . select { | f | f !~ %r{spec/support/} } search_all ( \"(send nil include_context (str #register_usage)\" , specs ) defined_contexts = capture_all ( '(block (send {nil,_} shared_context (str $_)))' , ruby_files_from ( 'spec' )) Kernel . public_send ( :show_report! , defined_contexts ) end Why #register_usage is defined on the Kernel ? Yes! note that the #register_usage was forced to be inside Kernel because of the shortcut block that takes the Fast context to be easy to access in the default functions. As I can define multiple shortcuts I don't want to polute my Kernel module with other methods that are not useful. RSpec: Remove unused let \u00b6 First shortcut with experiments If you're not familiar with automated experiments, you can read about it here . The current scenario is similar in terms of search with the previous one, but more advanced because we're going to introduce automated refactoring. The idea is simple, if it finds a let in a RSpec scenario that is not referenced, it tries to experimentally remove the let and run the tests: # Experimental remove `let` that are not referenced in the spec Fast . shortcut ( :exp_remove_let ) do require 'fast/experiment' Kernel . class_eval do file = ARGV . last defined_lets = Fast . capture_file ( '(block (send nil let (sym $_)))' , file ) . uniq @unreferenced = defined_lets . select do | identifier | Fast . search_file ( \"(send nil #{ identifier } )\" , file ) . empty? end def unreferenced_let? ( identifier ) @unreferenced . include? identifier end end experiment ( 'RSpec/RemoveUnreferencedLet' ) do lookup ARGV . last search '(block (send nil let (sym #unreferenced_let?)))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end And it will run with a single file from command line: fast .exp_remove_let spec/my_file_spec.rb FactoryBot: Replace create with build_stubbed \u00b6 For performance reasons, if we can avoid touching the database the test will always be faster. # Experimental switch from `create` to `build_stubbed` Fast . shortcut ( :exp_build_stubbed ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/UseBuildStubbed' ) do lookup ARGV . last search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end RSpec: Use let_it_be instead of let \u00b6 The let_it_be is a simple helper from TestProf gem that can speed up the specs by caching some factories using like a before_all approach. This experiment hunts for let(...) { create(...) } and switch the let to let_it_be : # Experimental replace `let(_)` with `let_it_be` case it calls `create` inside the block Fast . shortcut ( :exp_let_it_be ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/LetItBe' ) do lookup ARGV . last search '(block (send nil let (sym _)) (args) (send nil create))' edit { | node | replace ( node . children . first . loc . selector , 'let_it_be' ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end RSpec: Remove before or after blocks \u00b6 From time to time, we forget some left overs like before or after blocks that even removing from the code, the tests still passes. This experiment removes the before/after blocks and check if the test passes. # Experimental remove `before` or `after` blocks. Fast . shortcut ( :exp_remove_before_after ) do require 'fast/experiment' Fast . experiment ( 'RSpec/RemoveBeforeAfter' ) do lookup ARGV . last search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end RSpec: Show message chains \u00b6 I often forget the syntax and need to search for message chains on specs, so I created an shortcut for it. # Show RSpec message chains Fast . shortcut ( :message_chains , '^^(send nil receive_message_chain)' , 'spec' ) RSpec: Show nested assertions \u00b6 I love to use nested assertions and I often need examples to refer to them: # Show RSpec nested assertions with .and Fast . shortcut ( :nested_assertions , '^^(send ... and)' , 'spec' )","title":"Shortcuts"},{"location":"shortcuts/#shortcuts","text":"Shortcuts are defined on a Fastfile inside any ruby project. Use ~/Fastfile You can also add one extra in your $HOME if you want to have something loaded always. By default, the command line interface does not load any Fastfile if the first param is not a shortcut. It should start with . . I'm building several researches and I'll make the examples open here to show several interesting cases in action.","title":"Shortcuts"},{"location":"shortcuts/#list-your-fast-shortcuts","text":"As the interface is very rudimentar, let's build a shortcut to print what shortcuts are available. This is a good one to your $HOME/Fastfile : # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And using it on fast project that loads both ~/Fastfile and the Fastfile from the project: fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments","title":"List your fast shortcuts"},{"location":"shortcuts/#search-for-references","text":"I always miss bringing something simple as grep keyword where I can leave a simple string and it can search in all types of nodes and report interesting things about it. Let's consider a very flexible search that can target any code related to some keyword. Considering that we're talking about code indentifiers: # Search all references about some keyword or regular expression Fast . shortcut ( :ref ) do require 'fast/cli' Kernel . class_eval do def matches_args? identifier search = ARGV . last regex = Regexp . new ( search , Regexp :: IGNORECASE ) case identifier when Symbol , String regex . match? ( identifier ) || identifier . to_s . include? ( search ) when Astrolabe :: Node regex . match? ( identifier . to_sexp ) end end end pattern = <<~ FAST { ({class def sym str} #matches_args?)' ({const send} nil #matches_args?)' } FAST Fast :: Cli . run! ( [ pattern , '.' , '--parallel' ] ) end","title":"Search for references"},{"location":"shortcuts/#rails-show-validations-from-models","text":"If the shortcut does not define a block, it works as a holder for arguments from the command line. Let's say you always use fast \"(send nil {validate validates})\" app/models to check validations in the models. You can define a shortcut to hold the args and avoid retyping long lines: # Show validations from app/models Fast . shortcut ( :validations , \"(send nil {validate validates})\" , \"app/models\" ) And you can reuse the search with the shortcut starting with a . : fast .validations And it will also accept params if you want to filter a specific file: fast .validations app/models/user.rb Note that you can also use flags in the command line shortcuts Let's say you also want to use fast --headless you can add it to the params: Fast.shortcut(:validations, \"(send nil {validate validates})\", \"app/models\", \"--headless\")","title":"Rails: Show validations from models"},{"location":"shortcuts/#automated-refactor-bump-version","text":"Let's start with a real usage to bump a new version of the gem. Fast . shortcut :bump_version do rewrite_file ( '(casgn nil VERSION (str _)' , 'lib/fast/version.rb' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( '.' ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( '.' ) } '\" ) end end And then the change is done in the lib/fast/version.rb : module Fast - VERSION = '0.1.6' + VERSION = '0.1.7' end","title":"Automated Refactor: Bump version"},{"location":"shortcuts/#rspec-find-unused-shared-contexts","text":"If you build shared contexts often, probably you can forget some left overs. The objective of the shortcut is find leftovers from shared contexts. First, the objective is capture all names of the RSpec.shared_context or shared_context declared in the spec/support folder. Fast . capture_all ( '(block (send {nil,_} shared_context (str $_)))' , Fast . ruby_files_from ( 'spec/support' )) Then, we need to check all the specs and search for include_context usages to confirm if all defined contexts are being used: specs = Fast . ruby_files_from ( 'spec' ) . select { | f | f !~ %r{spec/support/} } Fast . search_all ( \"(send nil include_context (str #register_usage)\" , specs ) Note that we created a new reference to #register_usage and we need to define the method too: @used = [] def register_usage context_name @used << context_name end Wrapping up everything in a shortcut: # Show unused shared contexts Fast . shortcut ( :unused_shared_contexts ) do puts \"Checking shared contexts\" Kernel . class_eval do @used = [] def register_usage context_name @used << context_name end def show_report! defined_contexts unused = defined_contexts . values . flatten - @used if unused . any? puts \"Unused shared contexts\" , unused else puts \"Good job! all the #{ defined_contexts . size } contexts are used!\" end end end specs = ruby_files_from ( 'spec/' ) . select { | f | f !~ %r{spec/support/} } search_all ( \"(send nil include_context (str #register_usage)\" , specs ) defined_contexts = capture_all ( '(block (send {nil,_} shared_context (str $_)))' , ruby_files_from ( 'spec' )) Kernel . public_send ( :show_report! , defined_contexts ) end Why #register_usage is defined on the Kernel ? Yes! note that the #register_usage was forced to be inside Kernel because of the shortcut block that takes the Fast context to be easy to access in the default functions. As I can define multiple shortcuts I don't want to polute my Kernel module with other methods that are not useful.","title":"RSpec: Find unused shared contexts"},{"location":"shortcuts/#rspec-remove-unused-let","text":"First shortcut with experiments If you're not familiar with automated experiments, you can read about it here . The current scenario is similar in terms of search with the previous one, but more advanced because we're going to introduce automated refactoring. The idea is simple, if it finds a let in a RSpec scenario that is not referenced, it tries to experimentally remove the let and run the tests: # Experimental remove `let` that are not referenced in the spec Fast . shortcut ( :exp_remove_let ) do require 'fast/experiment' Kernel . class_eval do file = ARGV . last defined_lets = Fast . capture_file ( '(block (send nil let (sym $_)))' , file ) . uniq @unreferenced = defined_lets . select do | identifier | Fast . search_file ( \"(send nil #{ identifier } )\" , file ) . empty? end def unreferenced_let? ( identifier ) @unreferenced . include? identifier end end experiment ( 'RSpec/RemoveUnreferencedLet' ) do lookup ARGV . last search '(block (send nil let (sym #unreferenced_let?)))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end And it will run with a single file from command line: fast .exp_remove_let spec/my_file_spec.rb","title":"RSpec: Remove unused let"},{"location":"shortcuts/#factorybot-replace-create-with-build_stubbed","text":"For performance reasons, if we can avoid touching the database the test will always be faster. # Experimental switch from `create` to `build_stubbed` Fast . shortcut ( :exp_build_stubbed ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/UseBuildStubbed' ) do lookup ARGV . last search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end","title":"FactoryBot: Replace create with build_stubbed"},{"location":"shortcuts/#rspec-use-let_it_be-instead-of-let","text":"The let_it_be is a simple helper from TestProf gem that can speed up the specs by caching some factories using like a before_all approach. This experiment hunts for let(...) { create(...) } and switch the let to let_it_be : # Experimental replace `let(_)` with `let_it_be` case it calls `create` inside the block Fast . shortcut ( :exp_let_it_be ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/LetItBe' ) do lookup ARGV . last search '(block (send nil let (sym _)) (args) (send nil create))' edit { | node | replace ( node . children . first . loc . selector , 'let_it_be' ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end","title":"RSpec: Use let_it_be instead of let"},{"location":"shortcuts/#rspec-remove-before-or-after-blocks","text":"From time to time, we forget some left overs like before or after blocks that even removing from the code, the tests still passes. This experiment removes the before/after blocks and check if the test passes. # Experimental remove `before` or `after` blocks. Fast . shortcut ( :exp_remove_before_after ) do require 'fast/experiment' Fast . experiment ( 'RSpec/RemoveBeforeAfter' ) do lookup ARGV . last search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end","title":"RSpec: Remove before or after blocks"},{"location":"shortcuts/#rspec-show-message-chains","text":"I often forget the syntax and need to search for message chains on specs, so I created an shortcut for it. # Show RSpec message chains Fast . shortcut ( :message_chains , '^^(send nil receive_message_chain)' , 'spec' )","title":"RSpec: Show message chains"},{"location":"shortcuts/#rspec-show-nested-assertions","text":"I love to use nested assertions and I often need examples to refer to them: # Show RSpec nested assertions with .and Fast . shortcut ( :nested_assertions , '^^(send ... and)' , 'spec' )","title":"RSpec: Show nested assertions"},{"location":"similarity_tutorial/","text":"Research for code similarity \u00b6 This is a small tutorial to explore code similarity. Check the code example here . The major idea is register all expression styles and see if we can find some similarity between the structures. First we need to create a function that can analyze AST nodes and extract a pattern from the expression. The expression needs to generalize final node values and recursively build a pattern that can be used as a search expression. def expression_from ( node ) case node when Parser :: AST :: Node if node . children . any? children_expression = node . children . map ( & method ( :expression_from )) . join ( ' ' ) \"( #{ node . type } #{ children_expression } )\" else \"( #{ node . type } )\" end when nil , 'nil' 'nil' when Symbol , String , Integer '_' when Array , Hash '...' else node end end The pattern generated only flexibilize the search allowing us to group similar nodes. Example: expression_from ( code [ '1' ] ) # =>'(int _)' expression_from ( code [ 'nil' ] ) # =>'(nil)' expression_from ( code [ 'a = 1' ] ) # =>'(lvasgn _ (int _))' expression_from ( code [ 'def name; person.name end' ] ) # =>'(def _ (args) (send (send nil _) _))' The current method can translate all kind of expressions and the next step is observe some specific node types and try to group the similarities using the pattern generated. Fast . search_file ( 'class' , 'lib/fast.rb' ) Capturing the constant name and filtering only for symbols is easy and we can see that we have a few classes defined in the the same file. Fast . search_file ( '(class (const nil $_))' , 'lib/fast.rb' ) . grep ( Symbol ) => [ :Rewriter , :ExpressionParser , :Find , :FindString , :FindWithCapture , :Capture , :Parent , :Any , :All , :Not , :Maybe , :Matcher , :Experiment , :ExperimentFile ] The idea of this inspecton is build a proof of concept to show the similarity of matcher classes because they only define a match? method. patterns = Fast . search_file ( 'class' , 'lib/fast.rb' ) . map { | n | Fast . expression_from ( n )} A simple comparison between the patterns size versus .uniq.size can proof if the idea will work. patterns . size == patterns . uniq . size It does not work for the matcher cases but we can go deeper and analyze all files required by bundler. similarities = {} Gem . find_files ( '*.rb' ) . each do | file | Fast . search_file ( '{block send if while case def defs class module}' , file ) . map do | n | key = Fast . expression_from ( n ) similarities [ key ] ||= Set . new similarities [ key ] << file end end similarities . delete_if { | k , v | v . size < 2 } The similarities found are the following: { \"(class (const nil _) (const nil _) nil)\" => # , \"(class (const nil _) nil nil)\" => #} And now we can test the expression using the command line tool through the files and observe the similarity: \u22ca> ~ fast \"(class (const nil _) (const nil _) nil)\" /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb Output: # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:8 class DeadWorker < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:11 class Break < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:14 class Kill < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb:16 class SourceNotFoundError < StandardError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb:63 class Error < RuntimeError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb:338 class Abort < Exception ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb:125 class Cyclic < StandardError end It works and now we can create a method to do what the command line tool did, grouping the patterns and inspecting the occurrences. def similarities . show pattern files = self [ pattern ] files . each do | file | nodes = Fast . search_file ( pattern , file ) nodes . each do | result | Fast . report ( result , file : file ) end end end And calling the method exploring some \"if\" similarities, it prints the following results: similarities . show \"(if (send (const nil _) _ (lvar _)) nil (return (false)))\" # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/resolv.rb:1248 return false unless Name === other # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/fileutils.rb:138 return false unless File . exist? ( new ) # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/matrix.rb:1862 return false unless Vector === other","title":"Code Similarity"},{"location":"similarity_tutorial/#research-for-code-similarity","text":"This is a small tutorial to explore code similarity. Check the code example here . The major idea is register all expression styles and see if we can find some similarity between the structures. First we need to create a function that can analyze AST nodes and extract a pattern from the expression. The expression needs to generalize final node values and recursively build a pattern that can be used as a search expression. def expression_from ( node ) case node when Parser :: AST :: Node if node . children . any? children_expression = node . children . map ( & method ( :expression_from )) . join ( ' ' ) \"( #{ node . type } #{ children_expression } )\" else \"( #{ node . type } )\" end when nil , 'nil' 'nil' when Symbol , String , Integer '_' when Array , Hash '...' else node end end The pattern generated only flexibilize the search allowing us to group similar nodes. Example: expression_from ( code [ '1' ] ) # =>'(int _)' expression_from ( code [ 'nil' ] ) # =>'(nil)' expression_from ( code [ 'a = 1' ] ) # =>'(lvasgn _ (int _))' expression_from ( code [ 'def name; person.name end' ] ) # =>'(def _ (args) (send (send nil _) _))' The current method can translate all kind of expressions and the next step is observe some specific node types and try to group the similarities using the pattern generated. Fast . search_file ( 'class' , 'lib/fast.rb' ) Capturing the constant name and filtering only for symbols is easy and we can see that we have a few classes defined in the the same file. Fast . search_file ( '(class (const nil $_))' , 'lib/fast.rb' ) . grep ( Symbol ) => [ :Rewriter , :ExpressionParser , :Find , :FindString , :FindWithCapture , :Capture , :Parent , :Any , :All , :Not , :Maybe , :Matcher , :Experiment , :ExperimentFile ] The idea of this inspecton is build a proof of concept to show the similarity of matcher classes because they only define a match? method. patterns = Fast . search_file ( 'class' , 'lib/fast.rb' ) . map { | n | Fast . expression_from ( n )} A simple comparison between the patterns size versus .uniq.size can proof if the idea will work. patterns . size == patterns . uniq . size It does not work for the matcher cases but we can go deeper and analyze all files required by bundler. similarities = {} Gem . find_files ( '*.rb' ) . each do | file | Fast . search_file ( '{block send if while case def defs class module}' , file ) . map do | n | key = Fast . expression_from ( n ) similarities [ key ] ||= Set . new similarities [ key ] << file end end similarities . delete_if { | k , v | v . size < 2 } The similarities found are the following: { \"(class (const nil _) (const nil _) nil)\" => # , \"(class (const nil _) nil nil)\" => #} And now we can test the expression using the command line tool through the files and observe the similarity: \u22ca> ~ fast \"(class (const nil _) (const nil _) nil)\" /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb Output: # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:8 class DeadWorker < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:11 class Break < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:14 class Kill < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb:16 class SourceNotFoundError < StandardError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb:63 class Error < RuntimeError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb:338 class Abort < Exception ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb:125 class Cyclic < StandardError end It works and now we can create a method to do what the command line tool did, grouping the patterns and inspecting the occurrences. def similarities . show pattern files = self [ pattern ] files . each do | file | nodes = Fast . search_file ( pattern , file ) nodes . each do | result | Fast . report ( result , file : file ) end end end And calling the method exploring some \"if\" similarities, it prints the following results: similarities . show \"(if (send (const nil _) _ (lvar _)) nil (return (false)))\" # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/resolv.rb:1248 return false unless Name === other # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/fileutils.rb:138 return false unless File . exist? ( new ) # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/matrix.rb:1862 return false unless Vector === other","title":"Research for code similarity"},{"location":"sql-support/","text":"SQL Support \u00b6 Fast is partially supporting SQL syntax. Behind the scenes it parses SQL using pg_query and simplifies it to AST Nodes using the same Ruby interface. It's using Postgresql parser behind the scenes, but probably could be useful for other SQL similar diallects . The plan is that Fast would auto-detect file extensions and choose the sql path in case the file relates to sql. By default, this module is not included into the main library as it still very experimental. require 'fast/sql' Parsing a sql content \u00b6 ast = Fast . parse_sql ( 'select 1' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:val, # s(:integer, # s(:ival, 1)))))))) Use match? with your node pattern to traverse the abstract syntax tree. Fast . match? ( \"(select_stmt ...)\" , ast ) # => true Use $ to capture elements from the AST: Fast . match? ( \"(select_stmt $...)\" , ast ) => [ s ( :target_list , s ( :res_target , s ( :val , s ( :a_const , s ( :val , s ( :integer , s ( :ival , 1 ))))))) ] You can dig deeper into the AST specifying nodes: Fast . match? ( \"(select_stmt (target_list (res_target (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))] And ignoring node types or values using _ . Check all syntax options. Fast . match? ( \"(select_stmt (_ (_ (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))] Examples \u00b6 Capturing fields and where clause \u00b6 Let's dive into a more complex example capturing fields and from clause of a condition. Let's start parsing the sql: ast = Fast . parse_sql ( 'select name from customer' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:column_ref, # s(:fields, # s(:string, # s(:str, \"name\"))))))), # s(:from_clause, # s(:range_var, # s(:relname, \"customer\"), # s(:inh, true), # s(:relpersistence, \"p\")))) Now, let's build the expression to get the fields and from_clause. cols_and_from = \" (select_stmt (target_list (res_target (val (column_ref (fields $...))))) (from_clause (range_var $(relname _)))) \" Now, we can use Fast.capture or Fast.match? to extract the values from the AST. Fast . capture ( cols_and_from , ast ) # => [s(:string, # s(:str, \"name\")), s(:relname, \"customer\")] Search inside \u00b6 relname = Fast . parse_sql ( 'select name from customer' ) . search ( 'relname' ) . first # => s(:relname, \"customer\") Find the location of a node. relname . location # => #, # @node=s(:relname, \"customer\")> The location can be useful to allow you to do refactorings and find specific delimitations of objects in the string. The attribute expression gives access to the source range. relname . location . expression # => # The source_buffer is shared and can be accessed through the expression. relname . location . expression . source_buffer # => #, # , # , # ]> The tokens are useful to find the proper node location during the build but they're not available for all the nodes, so, it can be very handy as an extra reference. Replace \u00b6 Replace fragments of your SQL based on AST can also be done with all the work inherited from Parser::TreeRewriter components. Fast . parse_sql ( 'select 1' ) . replace ( 'ival' , '2' ) # => \"select 2\" The previous example is a syntax sugar for the following code: Fast . replace_sql ( 'ival' , Fast . parse_sql ( 'select 1' ), &-> ( node ){ replace ( node . location . expression , '2' ) } ) # => \"select 2\" The last argument is a proc that runs on the parser tree rewriter scope. Let's break down the previous code: ast = Fast . parse_sql ( \"select 1\" ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:val, # s(:integer, # s(:ival, 1)))))))) The pattern is simply matching node type that is ival but it could be a complex expression like (val (a_const (val (integer (ival _))))) . Completing the example: Fast . replace_sql ( \"ival\" , ast , &-> ( n ) { replace ( n . loc . expression , \"3\" ) }) # => \"select 3\" loc is a shortcut for location attribute.","title":"SQL Support"},{"location":"sql-support/#sql-support","text":"Fast is partially supporting SQL syntax. Behind the scenes it parses SQL using pg_query and simplifies it to AST Nodes using the same Ruby interface. It's using Postgresql parser behind the scenes, but probably could be useful for other SQL similar diallects . The plan is that Fast would auto-detect file extensions and choose the sql path in case the file relates to sql. By default, this module is not included into the main library as it still very experimental. require 'fast/sql'","title":"SQL Support"},{"location":"sql-support/#parsing-a-sql-content","text":"ast = Fast . parse_sql ( 'select 1' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:val, # s(:integer, # s(:ival, 1)))))))) Use match? with your node pattern to traverse the abstract syntax tree. Fast . match? ( \"(select_stmt ...)\" , ast ) # => true Use $ to capture elements from the AST: Fast . match? ( \"(select_stmt $...)\" , ast ) => [ s ( :target_list , s ( :res_target , s ( :val , s ( :a_const , s ( :val , s ( :integer , s ( :ival , 1 ))))))) ] You can dig deeper into the AST specifying nodes: Fast . match? ( \"(select_stmt (target_list (res_target (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))] And ignoring node types or values using _ . Check all syntax options. Fast . match? ( \"(select_stmt (_ (_ (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))]","title":"Parsing a sql content"},{"location":"sql-support/#examples","text":"","title":"Examples"},{"location":"sql-support/#capturing-fields-and-where-clause","text":"Let's dive into a more complex example capturing fields and from clause of a condition. Let's start parsing the sql: ast = Fast . parse_sql ( 'select name from customer' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:column_ref, # s(:fields, # s(:string, # s(:str, \"name\"))))))), # s(:from_clause, # s(:range_var, # s(:relname, \"customer\"), # s(:inh, true), # s(:relpersistence, \"p\")))) Now, let's build the expression to get the fields and from_clause. cols_and_from = \" (select_stmt (target_list (res_target (val (column_ref (fields $...))))) (from_clause (range_var $(relname _)))) \" Now, we can use Fast.capture or Fast.match? to extract the values from the AST. Fast . capture ( cols_and_from , ast ) # => [s(:string, # s(:str, \"name\")), s(:relname, \"customer\")]","title":"Capturing fields and where clause"},{"location":"sql-support/#search-inside","text":"relname = Fast . parse_sql ( 'select name from customer' ) . search ( 'relname' ) . first # => s(:relname, \"customer\") Find the location of a node. relname . location # => #, # @node=s(:relname, \"customer\")> The location can be useful to allow you to do refactorings and find specific delimitations of objects in the string. The attribute expression gives access to the source range. relname . location . expression # => # The source_buffer is shared and can be accessed through the expression. relname . location . expression . source_buffer # => #, # , # , # ]> The tokens are useful to find the proper node location during the build but they're not available for all the nodes, so, it can be very handy as an extra reference.","title":"Search inside"},{"location":"sql-support/#replace","text":"Replace fragments of your SQL based on AST can also be done with all the work inherited from Parser::TreeRewriter components. Fast . parse_sql ( 'select 1' ) . replace ( 'ival' , '2' ) # => \"select 2\" The previous example is a syntax sugar for the following code: Fast . replace_sql ( 'ival' , Fast . parse_sql ( 'select 1' ), &-> ( node ){ replace ( node . location . expression , '2' ) } ) # => \"select 2\" The last argument is a proc that runs on the parser tree rewriter scope. Let's break down the previous code: ast = Fast . parse_sql ( \"select 1\" ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:val, # s(:integer, # s(:ival, 1)))))))) The pattern is simply matching node type that is ival but it could be a complex expression like (val (a_const (val (integer (ival _))))) . Completing the example: Fast . replace_sql ( \"ival\" , ast , &-> ( n ) { replace ( n . loc . expression , \"3\" ) }) # => \"select 3\" loc is a shortcut for location attribute.","title":"Replace"},{"location":"syntax/","text":"Syntax \u00b6 The syntax is inspired on RuboCop Node Pattern . You can find a great tutorial about RuboCop node pattern in the official documentation . Code example \u00b6 Let's consider the following example.rb code example: class Example ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end end Looking the AST representation we have: $ ruby-parse example.rb (class (const nil :Example) nil (begin (casgn nil :ANSWER (int 42)) (def :magic (args) (send nil :rand (const nil :ANSWER))) (def :duplicate (args (arg :value)) (send (lvar :value) :* (int 2))))) Now, let's explore all details of the current AST, combining with the syntax operators. Fast works with a single word that will be the node type. A simple search of def nodes can be done and will also print the code. $ fast def example.rb # example.rb:3 def magic rand ( ANSWER ) end or check the casgn that will show constant assignments: $ fast casgn example.rb # example.rb:2 ANSWER = 42 () to represent a node search \u00b6 To specify details about a node, the ( means navigate deeply into a node and go deep into the expression. $ fast '(casgn' example.rb # example.rb:2 ANSWER = 42 Fast matcher never checks the end of the expression and close parens are not necessary. We keep them for the sake of specify more node details but the expression works with incomplete parens. $ fast '(casgn)' example.rb # example.rb:2 ANSWER = 42 Closing extra params also don't have a side effect. $ fast '(casgn))' example.rb # example.rb:2 ANSWER = 42 It also automatically flat parens case you put more levels in the beginning. $ fast '((casgn))' example.rb # example.rb:2 ANSWER = 42 For checking AST details while doing some search, you can use --ast in the command line for printing the AST instead of the code: $ fast '((casgn ' example.rb --ast # example.rb:2 ( casgn nil :ANSWER ( int 42 )) _ is something not nil \u00b6 Let's enhance our current expression and specify that we're looking for constant assignments of integers ignoring values and constant names replacing with _ . $ fast '(casgn nil _ (int _))' example.rb # example.rb:2 ANSWER = 42 Keep in mind that _ means not nil and (casgn _ _ (int _)) would not match. Let's search for integer nodes: $ fast int example.rb # example.rb:2 42 # example.rb:7 2 The current search show the nodes but they are not so useful without understand the expression in their context. We need to check their parent . ^ is to get the parent node of an expression \u00b6 By default, Parser::AST::Node does not have access to parent and for accessing it you can say ^ for reaching the parent. $ fast '^int' example.rb # example.rb:2 ANSWER = 42 # example.rb:7 value * 2 And using it multiple times will make the node match from levels up: $ fast '^^int' example.rb # example.rb:2 ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end [] join conditions \u00b6 Let's hunt for integer nodes that the parent is also a method: $ fast '[ ^^int def ]' example.rb The match will filter only nodes that matches all internal expressions. # example.rb:6 def duplicate ( value ) value * 2 end The expression is matching nodes that have a integer granchild and also with type def . ... is a node with children \u00b6 Looking the method representation we have: $ fast def example.rb --ast # example.rb:3 ( def :magic ( args ) ( send nil :rand ( const nil :ANSWER ))) # example.rb:6 ( def :duplicate ( args ( arg :value )) ( send ( lvar :value ) :* ( int 2 ))) And if we want to delimit only methods with arguments: $ fast '(def _ ...)' example.rb # example.rb:6 def duplicate ( value ) value * 2 end If we use (def _ _) instead it will match both methods because (args) does not have children but is not nil. $ is for capture current expression \u00b6 Now, let's say we want to extract some method name from current classes. In such case we don't want to have the node definition but only return the node name. # example.rb:2 def magic rand ( ANSWER ) end # example.rb: magic # example.rb:9 def duplicate ( value ) value * 2 end # example.rb: duplicate One extra method name was printed because of $ is capturing the element. nil matches exactly nil \u00b6 Nil is used in the code as a node type but parser gem also represents empty spaces in expressions with nil. Example, a method call from Kernel is a send from nil calling the method while I can also send a method call from a class. $ ruby-parse -e 'method' (send nil :method) And a method from a object will have the nested target not nil. $ ruby-parse -e 'object.method' (send (send nil :object) :method) Let's build a serch for any calls from nil : $ fast '(_ nil _)' example.rb # example.rb:3 Example # example.rb:4 ANSWER = 42 # example.rb:6 rand ( ANSWER ) Double check the expressions that have matched printing the AST: $ fast '(_ nil _)' example.rb --ast # example.rb:3 ( const nil :Example ) # example.rb:4 ( casgn nil :ANSWER ( int 42 )) # example.rb:6 ( send nil :rand ( const nil :ANSWER )) {} is for any matches like union conditions with or operator \u00b6 Let's say we to add check all occurrencies of the constant ANSWER . We'll need to get both casgn and const node types. For such cases we can surround the expressions with {} and it will return if the node matches with any of the internal expressions. $ fast '({casgn const} nil ANSWER)' example.rb # example.rb:4 ANSWER = 42 # example.rb:6 ANSWER # for custom methods \u00b6 Custom methods can let you into ruby doman for more complicated rules. Let's say we're looking for duplicated methods in the same class. We need to collect method names and guarantee they are unique. def duplicated ( method_name ) @methods ||= [] already_exists = @methods . include? ( method_name ) @methods << method_name already_exists end puts Fast . search_file ( '(def #duplicated)' , 'example.rb' ) The same principle can be used in the node level or for debugging purposes. require 'pry' def debug ( node ) binding . pry end puts Fast . search_file ( '#debug' , 'example.rb' ) If you want to get only def nodes you can also intersect expressions with [] : puts Fast . search_file ( '[ def #debug ]' , 'example.rb' ) Or if you want to debug a very specific expression you can use () to specify more details of the node puts Fast . search_file ( '[ (def a) #debug ]' , 'example.rb' ) . for instance methods \u00b6 You can also call instance methods using . . Example nil is the same of calling nil? and you can also use (int .odd?) to pick only odd integers. The int fragment can also be int_type? . \\1 for first previous capture \u00b6 Imagine you're looking for a method that is just delegating something to another method, like: def name person . name end This can be represented as the following AST: (def :name (args) (send (send nil :person) :name)) Then, let's build a search for methods that calls an attribute with the same name: Fast . match? ( '(def $_ ... (send (send nil _) \\1 ))' , ast ) # => [:name] With the method name being captured with $_ it can be later referenced in the expression with \\1 . If the search contains multiple captures, the \\2 , \\3 can be used as the sequence of captures.","title":"Syntax"},{"location":"syntax/#syntax","text":"The syntax is inspired on RuboCop Node Pattern . You can find a great tutorial about RuboCop node pattern in the official documentation .","title":"Syntax"},{"location":"syntax/#code-example","text":"Let's consider the following example.rb code example: class Example ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end end Looking the AST representation we have: $ ruby-parse example.rb (class (const nil :Example) nil (begin (casgn nil :ANSWER (int 42)) (def :magic (args) (send nil :rand (const nil :ANSWER))) (def :duplicate (args (arg :value)) (send (lvar :value) :* (int 2))))) Now, let's explore all details of the current AST, combining with the syntax operators. Fast works with a single word that will be the node type. A simple search of def nodes can be done and will also print the code. $ fast def example.rb # example.rb:3 def magic rand ( ANSWER ) end or check the casgn that will show constant assignments: $ fast casgn example.rb # example.rb:2 ANSWER = 42","title":"Code example"},{"location":"syntax/#to-represent-a-node-search","text":"To specify details about a node, the ( means navigate deeply into a node and go deep into the expression. $ fast '(casgn' example.rb # example.rb:2 ANSWER = 42 Fast matcher never checks the end of the expression and close parens are not necessary. We keep them for the sake of specify more node details but the expression works with incomplete parens. $ fast '(casgn)' example.rb # example.rb:2 ANSWER = 42 Closing extra params also don't have a side effect. $ fast '(casgn))' example.rb # example.rb:2 ANSWER = 42 It also automatically flat parens case you put more levels in the beginning. $ fast '((casgn))' example.rb # example.rb:2 ANSWER = 42 For checking AST details while doing some search, you can use --ast in the command line for printing the AST instead of the code: $ fast '((casgn ' example.rb --ast # example.rb:2 ( casgn nil :ANSWER ( int 42 ))","title":"() to represent a node search"},{"location":"syntax/#_-is-something-not-nil","text":"Let's enhance our current expression and specify that we're looking for constant assignments of integers ignoring values and constant names replacing with _ . $ fast '(casgn nil _ (int _))' example.rb # example.rb:2 ANSWER = 42 Keep in mind that _ means not nil and (casgn _ _ (int _)) would not match. Let's search for integer nodes: $ fast int example.rb # example.rb:2 42 # example.rb:7 2 The current search show the nodes but they are not so useful without understand the expression in their context. We need to check their parent .","title":"_ is something not nil"},{"location":"syntax/#is-to-get-the-parent-node-of-an-expression","text":"By default, Parser::AST::Node does not have access to parent and for accessing it you can say ^ for reaching the parent. $ fast '^int' example.rb # example.rb:2 ANSWER = 42 # example.rb:7 value * 2 And using it multiple times will make the node match from levels up: $ fast '^^int' example.rb # example.rb:2 ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end","title":"^ is to get the parent node of an expression"},{"location":"syntax/#join-conditions","text":"Let's hunt for integer nodes that the parent is also a method: $ fast '[ ^^int def ]' example.rb The match will filter only nodes that matches all internal expressions. # example.rb:6 def duplicate ( value ) value * 2 end The expression is matching nodes that have a integer granchild and also with type def .","title":"[] join conditions"},{"location":"syntax/#is-a-node-with-children","text":"Looking the method representation we have: $ fast def example.rb --ast # example.rb:3 ( def :magic ( args ) ( send nil :rand ( const nil :ANSWER ))) # example.rb:6 ( def :duplicate ( args ( arg :value )) ( send ( lvar :value ) :* ( int 2 ))) And if we want to delimit only methods with arguments: $ fast '(def _ ...)' example.rb # example.rb:6 def duplicate ( value ) value * 2 end If we use (def _ _) instead it will match both methods because (args) does not have children but is not nil.","title":"... is a node with children"},{"location":"syntax/#is-for-capture-current-expression","text":"Now, let's say we want to extract some method name from current classes. In such case we don't want to have the node definition but only return the node name. # example.rb:2 def magic rand ( ANSWER ) end # example.rb: magic # example.rb:9 def duplicate ( value ) value * 2 end # example.rb: duplicate One extra method name was printed because of $ is capturing the element.","title":"$ is for capture current expression"},{"location":"syntax/#nil-matches-exactly-nil","text":"Nil is used in the code as a node type but parser gem also represents empty spaces in expressions with nil. Example, a method call from Kernel is a send from nil calling the method while I can also send a method call from a class. $ ruby-parse -e 'method' (send nil :method) And a method from a object will have the nested target not nil. $ ruby-parse -e 'object.method' (send (send nil :object) :method) Let's build a serch for any calls from nil : $ fast '(_ nil _)' example.rb # example.rb:3 Example # example.rb:4 ANSWER = 42 # example.rb:6 rand ( ANSWER ) Double check the expressions that have matched printing the AST: $ fast '(_ nil _)' example.rb --ast # example.rb:3 ( const nil :Example ) # example.rb:4 ( casgn nil :ANSWER ( int 42 )) # example.rb:6 ( send nil :rand ( const nil :ANSWER ))","title":"nil matches exactly nil"},{"location":"syntax/#is-for-any-matches-like-union-conditions-with-or-operator","text":"Let's say we to add check all occurrencies of the constant ANSWER . We'll need to get both casgn and const node types. For such cases we can surround the expressions with {} and it will return if the node matches with any of the internal expressions. $ fast '({casgn const} nil ANSWER)' example.rb # example.rb:4 ANSWER = 42 # example.rb:6 ANSWER","title":"{} is for any matches like union conditions with or operator"},{"location":"syntax/#for-custom-methods","text":"Custom methods can let you into ruby doman for more complicated rules. Let's say we're looking for duplicated methods in the same class. We need to collect method names and guarantee they are unique. def duplicated ( method_name ) @methods ||= [] already_exists = @methods . include? ( method_name ) @methods << method_name already_exists end puts Fast . search_file ( '(def #duplicated)' , 'example.rb' ) The same principle can be used in the node level or for debugging purposes. require 'pry' def debug ( node ) binding . pry end puts Fast . search_file ( '#debug' , 'example.rb' ) If you want to get only def nodes you can also intersect expressions with [] : puts Fast . search_file ( '[ def #debug ]' , 'example.rb' ) Or if you want to debug a very specific expression you can use () to specify more details of the node puts Fast . search_file ( '[ (def a) #debug ]' , 'example.rb' )","title":"# for custom methods"},{"location":"syntax/#for-instance-methods","text":"You can also call instance methods using . . Example nil is the same of calling nil? and you can also use (int .odd?) to pick only odd integers. The int fragment can also be int_type? .","title":". for instance methods"},{"location":"syntax/#1-for-first-previous-capture","text":"Imagine you're looking for a method that is just delegating something to another method, like: def name person . name end This can be represented as the following AST: (def :name (args) (send (send nil :person) :name)) Then, let's build a search for methods that calls an attribute with the same name: Fast . match? ( '(def $_ ... (send (send nil _) \\1 ))' , ast ) # => [:name] With the method name being captured with $_ it can be later referenced in the expression with \\1 . If the search contains multiple captures, the \\2 , \\3 can be used as the sequence of captures.","title":"\\1 for first previous capture"},{"location":"videos/","text":"Videos \u00b6 Ruby Kaigi TakeOut 2020: Grepping Ruby code like a boss Also, similar livecoding session at RubyConf Brazil 2019 (Portuguese) . Introduction to inline code . Making local variables inline Making methods inline","title":"Videos"},{"location":"videos/#videos","text":"Ruby Kaigi TakeOut 2020: Grepping Ruby code like a boss Also, similar livecoding session at RubyConf Brazil 2019 (Portuguese) . Introduction to inline code . Making local variables inline Making methods inline","title":"Videos"},{"location":"walkthrough/","text":"Fast walkthrough \u00b6 This is the main interactive tutorial we have on fast . If you're reading it on the web, please consider also try it in the command line: fast .intro in the terminal to get a rapid pace on reading and testing on your own computer. The objective here is give you some insights about how to use ffast gem in the command line. Let's start finding the main fast.rb file for the fast library: $ gem which fast And now, let's combine the previous expression that returns the path to the file and take a quick look into the methods match? in the file using a regular grep: $ grep \"def match\\?\" $(gem which fast) Boring results, no? The code here is not easy to digest because we just see a fragment of the code block that we want. Let's make it a bit more advanced with grep -rn to file name and line number: $ grep -rn \"def match\\?\" $(gem which fast) Still hard to understand the scope of the search. That's why fast exists! Now, let's take a look on how a method like this looks like from the AST perspective. Let's use ruby-parse for it: $ ruby-parse -e \"def match?(node); end\" Now, let's make the same search with fast node pattern: fast \"(def match?)\" $(gem which fast) Wow! in this case you got all the match? methods, but you'd like to go one level upper and understand the classes that implements the method with a single node as argument. Let's first use ^ to jump into the parent: fast \"^(def match?)\" $(gem which fast) As you can see it still prints some match? methods that are not the ones that we want, so, let's add a filter by the argument node (args (arg node)) : fast \"(def match? (args (arg node)))\" $(gem which fast) Now, it looks closer to have some understanding of the scope, filtering only methods that have the name match? and receive node as a parameter. Now, let's do something different and find all methods that receives a node as an argument: fast \"(def _ (args (arg node)))\" $(gem which fast) Looks like almost all of them are the match? and we can also skip the match? methods negating the expression prefixing with ! : fast \"(def !match? (args (arg node)))\" $(gem which fast) Let's move on and learn more about node pattern with the RuboCop project: $ VISUAL=echo gem open rubocop RuboCop contains def_node_matcher and def_node_search . Let's make a search for both method names wrapping the query with {} selector: fast \"(send nil {def_node_matcher def_node_search})\" $(VISUAL=echo gem open rubocop) As you can see, node pattern is widely adopted in the cops to target code. Rubocop contains a few projects with dedicated cops that can help you learn more. How to automate refactor using AST \u00b6 Moving towards to the code automation, the next step after finding some target code is refactor and change the code behavior. Let's imagine that we already found some code that we want to edit or remove. If we get the AST we can also cherry-pick any fragment of the expression to be replaced. As you can imagine, RuboCop also benefits from automatic refactoring offering the --autocorrect option. All the hardcore algorithms are in the parser rewriter, but we can find a lot of great examples on RuboCop project searching for the autocorrect method. fast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop) Look that most part of the methods are just returning a lambda with a corrector. Now, let's use the --ast to get familiar with the tree details for the implementation: fast --ast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop)/lib/rubocop/cop/style As we can see, we have a (send (lvar corrector)) that is the interface that we can get the most interesting calls to overwrite files: fast \"(send (lvar corrector)\" $(VISUAL=echo gem open rubocop) That is all for now! \u00b6 I hope you enjoyed to learn by example searching with fast. If you like it, please star the project ! You can also build your own tutorials simply using markdown files like I did here, you can find this tutorial here .","title":"Walkthrough"},{"location":"walkthrough/#fast-walkthrough","text":"This is the main interactive tutorial we have on fast . If you're reading it on the web, please consider also try it in the command line: fast .intro in the terminal to get a rapid pace on reading and testing on your own computer. The objective here is give you some insights about how to use ffast gem in the command line. Let's start finding the main fast.rb file for the fast library: $ gem which fast And now, let's combine the previous expression that returns the path to the file and take a quick look into the methods match? in the file using a regular grep: $ grep \"def match\\?\" $(gem which fast) Boring results, no? The code here is not easy to digest because we just see a fragment of the code block that we want. Let's make it a bit more advanced with grep -rn to file name and line number: $ grep -rn \"def match\\?\" $(gem which fast) Still hard to understand the scope of the search. That's why fast exists! Now, let's take a look on how a method like this looks like from the AST perspective. Let's use ruby-parse for it: $ ruby-parse -e \"def match?(node); end\" Now, let's make the same search with fast node pattern: fast \"(def match?)\" $(gem which fast) Wow! in this case you got all the match? methods, but you'd like to go one level upper and understand the classes that implements the method with a single node as argument. Let's first use ^ to jump into the parent: fast \"^(def match?)\" $(gem which fast) As you can see it still prints some match? methods that are not the ones that we want, so, let's add a filter by the argument node (args (arg node)) : fast \"(def match? (args (arg node)))\" $(gem which fast) Now, it looks closer to have some understanding of the scope, filtering only methods that have the name match? and receive node as a parameter. Now, let's do something different and find all methods that receives a node as an argument: fast \"(def _ (args (arg node)))\" $(gem which fast) Looks like almost all of them are the match? and we can also skip the match? methods negating the expression prefixing with ! : fast \"(def !match? (args (arg node)))\" $(gem which fast) Let's move on and learn more about node pattern with the RuboCop project: $ VISUAL=echo gem open rubocop RuboCop contains def_node_matcher and def_node_search . Let's make a search for both method names wrapping the query with {} selector: fast \"(send nil {def_node_matcher def_node_search})\" $(VISUAL=echo gem open rubocop) As you can see, node pattern is widely adopted in the cops to target code. Rubocop contains a few projects with dedicated cops that can help you learn more.","title":"Fast walkthrough"},{"location":"walkthrough/#how-to-automate-refactor-using-ast","text":"Moving towards to the code automation, the next step after finding some target code is refactor and change the code behavior. Let's imagine that we already found some code that we want to edit or remove. If we get the AST we can also cherry-pick any fragment of the expression to be replaced. As you can imagine, RuboCop also benefits from automatic refactoring offering the --autocorrect option. All the hardcore algorithms are in the parser rewriter, but we can find a lot of great examples on RuboCop project searching for the autocorrect method. fast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop) Look that most part of the methods are just returning a lambda with a corrector. Now, let's use the --ast to get familiar with the tree details for the implementation: fast --ast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop)/lib/rubocop/cop/style As we can see, we have a (send (lvar corrector)) that is the interface that we can get the most interesting calls to overwrite files: fast \"(send (lvar corrector)\" $(VISUAL=echo gem open rubocop)","title":"How to automate refactor using AST"},{"location":"walkthrough/#that-is-all-for-now","text":"I hope you enjoyed to learn by example searching with fast. If you like it, please star the project ! You can also build your own tutorials simply using markdown files like I did here, you can find this tutorial here .","title":"That is all for now!"}]} \ No newline at end of file +{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Fast \u00b6 Fast is a \"Find AST\" tool to help you search in the code abstract syntax tree. Ruby allow us to do the same thing in a few ways then it's hard to check how the code is written. Using the AST will be easier than try to cover the multiple ways we can write the same code. You can define a string like %|| or '' or \"\" but they will have the same AST representation. AST representation \u00b6 Each detail of the ruby syntax have a equivalent identifier and some content. The content can be another expression or a final value. Fast uses parser gem behind the scenes to parse the code into nodes. First get familiar with parser gem and understand how ruby code is represented. When you install parser gem, you will have access to ruby-parse and you can use it with -e to parse an expression directly from the command line. Example: ruby-parse -e 1 It will print the following output: (int 1) And trying a number with decimals: ruby-parse -e 1.1 (float 1) Building a regex that will match decimals and integer looks like something easy and with fast you use a node pattern that reminds the syntax of regular expressions. Syntax for find in AST \u00b6 The current version cover the following elements: () to represent a node search {} is for any matches like union conditions with or operator [] is for all matches like intersect conditions with and operator $ is for capture current expression _ is something not nil nil matches exactly nil ... is a node with children ^ is to get the parent node of an expression ? is for maybe \\1 to use the first previous captured element \"\" surround the value with double quotes to match literal strings Jump to Syntax . ast \u00b6 Use Fast.ast to convert simple code to AST objects. You can use it as ruby-parse but directly from the console. Fast . ast ( \"1\" ) # => s(:int, 1) Fast . ast ( \"method\" ) # => s(:send, nil, :method) Fast . ast ( \"a.b\" ) # => s(:send, s(:send, nil, :a), :b) Fast . ast ( \"1 + 1\" ) # => s(:send, s(:int, 1), :+, s(:int, 1)) Fast . ast ( \"a = 2\" ) # => s(:lvasgn, :a, s(:int, 2)) Fast . ast ( \"b += 2\" ) # => s(:op_asgn, s(:lvasgn, :b), :+, s(:int, 2)) It uses astrolable gem behind the scenes: Fast . ast ( Fast . ast ( \"1\" )) . class => Astrolabe :: Node Fast . ast ( Fast . ast ( \"1\" )) . type => :int Fast . ast ( Fast . ast ( \"1\" )) . children => [ 1 ] See also ast_from_file . match? \u00b6 Fast.match? is the most granular function that tries to compare a node with an expression. It returns true or false and some node captures case it find something. Let's start with a simple integer in Ruby: 1 The AST can be represented with the following expression: (int 1) The ast representation holds node type and children . Let's build a method s to represent Parser::AST::Node with a #type and #children . def s ( type , * children ) Parser :: AST :: Node . new ( type , children ) end A local variable assignment: value = 42 Can be represented with: ast = s ( :lvasgn , :value , s ( :int , 42 )) Now, lets find local variable named value with an value 42 : Fast . match? ( '(lvasgn value (int 42))' , ast ) # true Lets abstract a bit and allow some integer value using _ as a shortcut: Fast . match? ( '(lvasgn value (int _))' , ast ) # true Lets abstract more and allow float or integer: Fast . match? ( '(lvasgn value ({float int} _))' , ast ) # true Or combine multiple assertions using [] to join conditions: Fast . match? ( '(lvasgn value ([!str !hash !array] _))' , ast ) # true Matches all local variables not string and not hash and not array. We can match \"a node with children\" using ... : Fast . match? ( '(lvasgn value ...)' , ast ) # true You can use $ to capture a node: Fast . match? ( '(lvasgn value $...)' , ast ) # => [s(:int), 42] Or match whatever local variable assignment combining both _ and ... : Fast . match? ( '(lvasgn _ ...)' , ast ) # true You can also use captures in any levels you want: Fast . match? ( '(lvasgn $_ $...)' , ast ) # [:value, s(:int), 42] Keep in mind that _ means something not nil and ... means a node with children. Then, if do you get a method declared: def my_method call_other_method end It will be represented with the following structure: ast = s ( :def , :my_method , s ( :args ), s ( :send , nil , :call_other_method )) Keep an eye on the node (args) . Then you know you can't use ... but you can match with (_) to match with such case. Let's test a few other examples. You can go deeply with the arrays. Let's suppose we have a hardcore call to a.b.c.d and the following AST represents it: ast = s ( :send , s ( :send , s ( :send , s ( :send , nil , :a ), :b ), :c ), :d ) You can search using sub-arrays with pure values , or shortcuts or procs : Fast . match? ( [ :send , [ :send , '...' ] , :d ] , ast ) # => true Fast . match? ( [ :send , [ :send , '...' ] , :c ] , ast ) # => false Fast . match? ( [ :send , [ :send , [ :send , '...' ] , :c ] , :d ] , ast ) # => true Shortcuts like ... and _ are just literals for procs. Then you can use procs directly too: Fast . match? ( [ :send , [ -> ( node ) { node . type == :send }, [ :send , '...' ] , :c ] , :d ] , ast ) # => true And also work with expressions: Fast . match? ( '(send (send (send (send nil $_) $_) $_) $_)' , ast ) # => [:a, :b, :c, :d] If something does not work you can debug with a block: Fast . debug { Fast . match? ( [ :int , 1 ] , s ( :int , 1 )) } It will output each comparison to stdout: int == (int 1) # => true 1 == 1 # => true search \u00b6 Search allows you to go deeply in the AST, collecting nodes that matches with the expression. It also returns captures if they exist. Fast . search ( '(int _)' , Fast . ast ( 'a = 1' )) # => s(:int, 1) If you use captures, it returns the node and the captures respectively: Fast . search ( '(int $_)' , Fast . ast ( 'a = 1' )) # => [s(:int, 1), 1] You can also bind external parameters in the search using extra arguments: Fast . search ( '(int %1)' , Fast . ast ( 'a = 1' ), 1 ) # => [s(:int, 1)] capture \u00b6 To pick just the captures and ignore the nodes, use Fast.capture : Fast . capture ( '(int $_)' , Fast . ast ( 'a = 1' )) # => 1 replace \u00b6 And if I want to refactor a code and use delegate , to: , try with replace: Fast . replace '(def $_ ... (send (send nil $_) \\1 ))' , ast do | node , captures | attribute , object = captures replace ( node . location . expression , \"delegate : #{ attribute } , to: : #{ object } \" ) end replace_file \u00b6 Now let's imagine we have real files like sample.rb with the following code: def good_bye message = [ \"good\" , \"bye\" ] puts message . join ( ' ' ) end And we decide to remove the message variable and put it inline with the puts . Basically, we need to find the local variable assignment, store the value in memory. Remove the assignment expression and use the value where the variable is being called. assignment = nil Fast . replace_file ( '({ lvasgn lvar } message )' , 'sample.rb' ) do | node , _ | if node . type == :lvasgn assignment = node . children . last remove ( node . location . expression ) elsif node . type == :lvar replace ( node . location . expression , assignment . location . expression . source ) end end It will return an output of the new source code with the changes but not save the file. You can use ()[#rewrite_file] if you're confident about the changes. capture_file \u00b6 Fast.capture_file can be used to combine capture and file system. Fast . capture_file ( \"$(casgn)\" , \"lib/fast/version.rb\" ) # => s(:casgn, nil, :VERSION, s(:str, \"0.1.3\")) Fast . capture_file ( \"(casgn nil _ (str $_))\" , \"lib/fast/version.rb\" ) # => \"0.1.3\" capture_all \u00b6 Fast.capture_all can be used to combine capture_file from multiple sources: Fast . capture_all ( \"(casgn nil $_)\" ) # => { \"./lib/fast/version.rb\"=>:VERSION, \"./lib/fast.rb\"=>[:LITERAL, :TOKENIZER], ...} The second parameter can also be passed with to filter specific folders: Fast . capture_all ( \"(casgn nil $_)\" , \"lib/fast\" ) # => {\"lib/fast/shortcut.rb\"=>:LOOKUP_FAST_FILES_DIRECTORIES, \"lib/fast/version.rb\"=>:VERSION} rewrite_file \u00b6 Fast.rewrite_file works exactly as the replace but it will override the file from the input. ast_from_file \u00b6 This method parses the code and load into a AST representation. Fast . ast_from_file ( 'sample.rb' ) search_file \u00b6 You can use search_file and pass the path for search for expressions inside files. Fast . search_file ( expression , 'file.rb' ) It's simple combination of Fast.ast_from_file with Fast.search . ruby_files_from \u00b6 You'll be probably looking for multiple ruby files, then this method fetches all internal .rb files Fast . ruby_files_from ( [ 'lib' ] ) # => [\"lib/fast.rb\"] search_all \u00b6 Combines the search_file with ruby_files_from multiple locations and returns tuples with files and results. Fast . search_all ( \"(def ast_from_file)\" ) => { \"./lib/fast.rb\" =>[ s ( :def , :ast_from_file , s ( :args , s ( :arg , :file )), s ( :begin , You can also override the second param and pass target files or folders: Fast . search_all ( \"(def _)\" , '../other-folder' )","title":"Introduction"},{"location":"#fast","text":"Fast is a \"Find AST\" tool to help you search in the code abstract syntax tree. Ruby allow us to do the same thing in a few ways then it's hard to check how the code is written. Using the AST will be easier than try to cover the multiple ways we can write the same code. You can define a string like %|| or '' or \"\" but they will have the same AST representation.","title":"Fast"},{"location":"#ast-representation","text":"Each detail of the ruby syntax have a equivalent identifier and some content. The content can be another expression or a final value. Fast uses parser gem behind the scenes to parse the code into nodes. First get familiar with parser gem and understand how ruby code is represented. When you install parser gem, you will have access to ruby-parse and you can use it with -e to parse an expression directly from the command line. Example: ruby-parse -e 1 It will print the following output: (int 1) And trying a number with decimals: ruby-parse -e 1.1 (float 1) Building a regex that will match decimals and integer looks like something easy and with fast you use a node pattern that reminds the syntax of regular expressions.","title":"AST representation"},{"location":"#syntax-for-find-in-ast","text":"The current version cover the following elements: () to represent a node search {} is for any matches like union conditions with or operator [] is for all matches like intersect conditions with and operator $ is for capture current expression _ is something not nil nil matches exactly nil ... is a node with children ^ is to get the parent node of an expression ? is for maybe \\1 to use the first previous captured element \"\" surround the value with double quotes to match literal strings Jump to Syntax .","title":"Syntax for find in AST"},{"location":"#ast","text":"Use Fast.ast to convert simple code to AST objects. You can use it as ruby-parse but directly from the console. Fast . ast ( \"1\" ) # => s(:int, 1) Fast . ast ( \"method\" ) # => s(:send, nil, :method) Fast . ast ( \"a.b\" ) # => s(:send, s(:send, nil, :a), :b) Fast . ast ( \"1 + 1\" ) # => s(:send, s(:int, 1), :+, s(:int, 1)) Fast . ast ( \"a = 2\" ) # => s(:lvasgn, :a, s(:int, 2)) Fast . ast ( \"b += 2\" ) # => s(:op_asgn, s(:lvasgn, :b), :+, s(:int, 2)) It uses astrolable gem behind the scenes: Fast . ast ( Fast . ast ( \"1\" )) . class => Astrolabe :: Node Fast . ast ( Fast . ast ( \"1\" )) . type => :int Fast . ast ( Fast . ast ( \"1\" )) . children => [ 1 ] See also ast_from_file .","title":"ast"},{"location":"#match","text":"Fast.match? is the most granular function that tries to compare a node with an expression. It returns true or false and some node captures case it find something. Let's start with a simple integer in Ruby: 1 The AST can be represented with the following expression: (int 1) The ast representation holds node type and children . Let's build a method s to represent Parser::AST::Node with a #type and #children . def s ( type , * children ) Parser :: AST :: Node . new ( type , children ) end A local variable assignment: value = 42 Can be represented with: ast = s ( :lvasgn , :value , s ( :int , 42 )) Now, lets find local variable named value with an value 42 : Fast . match? ( '(lvasgn value (int 42))' , ast ) # true Lets abstract a bit and allow some integer value using _ as a shortcut: Fast . match? ( '(lvasgn value (int _))' , ast ) # true Lets abstract more and allow float or integer: Fast . match? ( '(lvasgn value ({float int} _))' , ast ) # true Or combine multiple assertions using [] to join conditions: Fast . match? ( '(lvasgn value ([!str !hash !array] _))' , ast ) # true Matches all local variables not string and not hash and not array. We can match \"a node with children\" using ... : Fast . match? ( '(lvasgn value ...)' , ast ) # true You can use $ to capture a node: Fast . match? ( '(lvasgn value $...)' , ast ) # => [s(:int), 42] Or match whatever local variable assignment combining both _ and ... : Fast . match? ( '(lvasgn _ ...)' , ast ) # true You can also use captures in any levels you want: Fast . match? ( '(lvasgn $_ $...)' , ast ) # [:value, s(:int), 42] Keep in mind that _ means something not nil and ... means a node with children. Then, if do you get a method declared: def my_method call_other_method end It will be represented with the following structure: ast = s ( :def , :my_method , s ( :args ), s ( :send , nil , :call_other_method )) Keep an eye on the node (args) . Then you know you can't use ... but you can match with (_) to match with such case. Let's test a few other examples. You can go deeply with the arrays. Let's suppose we have a hardcore call to a.b.c.d and the following AST represents it: ast = s ( :send , s ( :send , s ( :send , s ( :send , nil , :a ), :b ), :c ), :d ) You can search using sub-arrays with pure values , or shortcuts or procs : Fast . match? ( [ :send , [ :send , '...' ] , :d ] , ast ) # => true Fast . match? ( [ :send , [ :send , '...' ] , :c ] , ast ) # => false Fast . match? ( [ :send , [ :send , [ :send , '...' ] , :c ] , :d ] , ast ) # => true Shortcuts like ... and _ are just literals for procs. Then you can use procs directly too: Fast . match? ( [ :send , [ -> ( node ) { node . type == :send }, [ :send , '...' ] , :c ] , :d ] , ast ) # => true And also work with expressions: Fast . match? ( '(send (send (send (send nil $_) $_) $_) $_)' , ast ) # => [:a, :b, :c, :d] If something does not work you can debug with a block: Fast . debug { Fast . match? ( [ :int , 1 ] , s ( :int , 1 )) } It will output each comparison to stdout: int == (int 1) # => true 1 == 1 # => true","title":"match?"},{"location":"#search","text":"Search allows you to go deeply in the AST, collecting nodes that matches with the expression. It also returns captures if they exist. Fast . search ( '(int _)' , Fast . ast ( 'a = 1' )) # => s(:int, 1) If you use captures, it returns the node and the captures respectively: Fast . search ( '(int $_)' , Fast . ast ( 'a = 1' )) # => [s(:int, 1), 1] You can also bind external parameters in the search using extra arguments: Fast . search ( '(int %1)' , Fast . ast ( 'a = 1' ), 1 ) # => [s(:int, 1)]","title":"search"},{"location":"#capture","text":"To pick just the captures and ignore the nodes, use Fast.capture : Fast . capture ( '(int $_)' , Fast . ast ( 'a = 1' )) # => 1","title":"capture"},{"location":"#replace","text":"And if I want to refactor a code and use delegate , to: , try with replace: Fast . replace '(def $_ ... (send (send nil $_) \\1 ))' , ast do | node , captures | attribute , object = captures replace ( node . location . expression , \"delegate : #{ attribute } , to: : #{ object } \" ) end","title":"replace"},{"location":"#replace_file","text":"Now let's imagine we have real files like sample.rb with the following code: def good_bye message = [ \"good\" , \"bye\" ] puts message . join ( ' ' ) end And we decide to remove the message variable and put it inline with the puts . Basically, we need to find the local variable assignment, store the value in memory. Remove the assignment expression and use the value where the variable is being called. assignment = nil Fast . replace_file ( '({ lvasgn lvar } message )' , 'sample.rb' ) do | node , _ | if node . type == :lvasgn assignment = node . children . last remove ( node . location . expression ) elsif node . type == :lvar replace ( node . location . expression , assignment . location . expression . source ) end end It will return an output of the new source code with the changes but not save the file. You can use ()[#rewrite_file] if you're confident about the changes.","title":"replace_file"},{"location":"#capture_file","text":"Fast.capture_file can be used to combine capture and file system. Fast . capture_file ( \"$(casgn)\" , \"lib/fast/version.rb\" ) # => s(:casgn, nil, :VERSION, s(:str, \"0.1.3\")) Fast . capture_file ( \"(casgn nil _ (str $_))\" , \"lib/fast/version.rb\" ) # => \"0.1.3\"","title":"capture_file"},{"location":"#capture_all","text":"Fast.capture_all can be used to combine capture_file from multiple sources: Fast . capture_all ( \"(casgn nil $_)\" ) # => { \"./lib/fast/version.rb\"=>:VERSION, \"./lib/fast.rb\"=>[:LITERAL, :TOKENIZER], ...} The second parameter can also be passed with to filter specific folders: Fast . capture_all ( \"(casgn nil $_)\" , \"lib/fast\" ) # => {\"lib/fast/shortcut.rb\"=>:LOOKUP_FAST_FILES_DIRECTORIES, \"lib/fast/version.rb\"=>:VERSION}","title":"capture_all"},{"location":"#rewrite_file","text":"Fast.rewrite_file works exactly as the replace but it will override the file from the input.","title":"rewrite_file"},{"location":"#ast_from_file","text":"This method parses the code and load into a AST representation. Fast . ast_from_file ( 'sample.rb' )","title":"ast_from_file"},{"location":"#search_file","text":"You can use search_file and pass the path for search for expressions inside files. Fast . search_file ( expression , 'file.rb' ) It's simple combination of Fast.ast_from_file with Fast.search .","title":"search_file"},{"location":"#ruby_files_from","text":"You'll be probably looking for multiple ruby files, then this method fetches all internal .rb files Fast . ruby_files_from ( [ 'lib' ] ) # => [\"lib/fast.rb\"]","title":"ruby_files_from"},{"location":"#search_all","text":"Combines the search_file with ruby_files_from multiple locations and returns tuples with files and results. Fast . search_all ( \"(def ast_from_file)\" ) => { \"./lib/fast.rb\" =>[ s ( :def , :ast_from_file , s ( :args , s ( :arg , :file )), s ( :begin , You can also override the second param and pass target files or folders: Fast . search_all ( \"(def _)\" , '../other-folder' )","title":"search_all"},{"location":"command_line/","text":"Command line \u00b6 When you install the ffast gem, it will also create an executable named fast and you can use it to search and find code using the concept: $ fast '(def match?)' lib/fast.rb Use -d or --debug for enable debug mode. Use --ast to output the AST instead of the original code Use --pry to jump debugging the first result with pry Use -c to search from code example Use -s to search similar code Use -p to or --parallel to use multi core search --pry \u00b6 $ fast '(block (send nil it))' spec --pry And inside pry session, you can use result as the first result or results to use all occurrences found. results . map { | e | e . children [ 0 ]. children [ 2 ] } # => [s(:str, \"parses ... as Find\"), # s(:str, \"parses $ as Capture\"), # s(:str, \"parses quoted values as strings\"), # s(:str, \"parses {} as Any\"), # s(:str, \"parses [] as All\"), ...] Getting all it blocks without description: $ fast '(block (send nil it (nil)) (args ) (!str)) ) )' spec # spec/fast_spec.rb:166 it { expect ( described_class ) . to be_match ( '(...)' , s ( :int , 1 )) } ... --debug \u00b6 This option will print all matching details while validating each node. $ echo 'object.method' > sample.rb $ fast -d '(send (send nil _) _)' sample.rb It will bring details of the expression compiled and each node being validated: Expression: f[send] [#, #, #] f[_] send == (send (send nil :object) :method) # => true f[send] == (send (send nil :object) :method) # => true send == (send nil :object) # => true f[send] == (send nil :object) # => true == # => true f[nil] == # => true # == object # => true f[_] == object # => true [#, #, #] == (send nil :object) # => true # == method # => true f[_] == method # => true # sample.rb:1 object.method -s for similarity \u00b6 Sometimes you want to search for some similar code like (send (send (send nil _) _) _) and we could simply say a.b.c . The option -s build an expression from the code ignoring final values. $ echo 'object.method' > sample.rb $ fast -s 'a.b' sample.rb # sample.rb:1 object . method See also Code Similarity tutorial. -c to search from code example \u00b6 You can search for the exact expression with -c $ fast -c 'object.method' sample.rb # sample.rb:1 object . method Combining with -d , in the header you can see the generated expression. $ fast -d -c 'object.method' sample.rb | head -n 3 The generated expression from AST was: (send (send nil :object) :method) Fastfile \u00b6 Fastfile will loaded when you start a pattern with a dot. It means the pattern will be a shortcut predefined on these Fastfiles. It will make three attempts to load Fastfile defined in $PWD , $HOME or checking if the $FAST_FILE_DIR is configured. You can define a Fastfile in any project with your custom shortcuts and easy check some code or run some task. Shortcut examples \u00b6 Create shortcuts with blocks enables introduce custom coding in the scope of the Fast module. Print library version. \u00b6 Let's say you'd like to show the version of your library. Your regular params in the command line will look like: $ fast '(casgn nil VERSION)' lib/*/version.rb It will output but the command is not very handy. In order to just say fast .version you can use the previous snippet in your Fastfile . Fast . shortcut ( :version , '(casgn nil VERSION)' , 'lib/fast/version.rb' ) And calling fast .version it will output something like this: # lib/fast/version.rb:4 VERSION = '0.1.2' We can also always override the files params passing some other target file like fast .version lib/other/file.rb and it will reuse the other arguments from command line but replace the target files. Bumping a gem version \u00b6 While releasing a new gem version, we always need to mechanical go through the lib//version.rb and change the string value to bump the version of your library. It's pretty mechanical and here is an example that allow you to simple use fast .bump_version : Fast . shortcut :bump_version do rewrite_file ( 'lib/fast/version.rb' , '(casgn nil VERSION (str _)' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( \".\" ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( \".\" ) } '\" ) end end Note the shortcut scope The shortcut calls rewrite_file from Fast scope as it use Fast.instance_exec for shortcuts that yields blocks. Checking the version: $ fast .version 13 :58:40 # lib/fast/version.rb:4 VERSION = '0.1.2' Bumping the version: $ fast .bump_version 13 :58:43 No output because we don't print anything. Checking version again: $ fast .version 13 :58:54 # lib/fast/version.rb:4 VERSION = '0.1.3' And now a fancy shortcut to report the other shortcuts :) Fast . shortcut :shortcuts do report ( shortcuts . keys ) end Or we can make it a bit more friendly and also use Fast to process the shortcut positions and pick the comment that each shortcut have in the previous line: # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _))' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And it will be printing all loaded shortcuts with comments: $ fast .shortcuts fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments You can find more examples in the Fastfile .","title":"Command Line"},{"location":"command_line/#command-line","text":"When you install the ffast gem, it will also create an executable named fast and you can use it to search and find code using the concept: $ fast '(def match?)' lib/fast.rb Use -d or --debug for enable debug mode. Use --ast to output the AST instead of the original code Use --pry to jump debugging the first result with pry Use -c to search from code example Use -s to search similar code Use -p to or --parallel to use multi core search","title":"Command line"},{"location":"command_line/#-pry","text":"$ fast '(block (send nil it))' spec --pry And inside pry session, you can use result as the first result or results to use all occurrences found. results . map { | e | e . children [ 0 ]. children [ 2 ] } # => [s(:str, \"parses ... as Find\"), # s(:str, \"parses $ as Capture\"), # s(:str, \"parses quoted values as strings\"), # s(:str, \"parses {} as Any\"), # s(:str, \"parses [] as All\"), ...] Getting all it blocks without description: $ fast '(block (send nil it (nil)) (args ) (!str)) ) )' spec # spec/fast_spec.rb:166 it { expect ( described_class ) . to be_match ( '(...)' , s ( :int , 1 )) } ...","title":"--pry"},{"location":"command_line/#-debug","text":"This option will print all matching details while validating each node. $ echo 'object.method' > sample.rb $ fast -d '(send (send nil _) _)' sample.rb It will bring details of the expression compiled and each node being validated: Expression: f[send] [#, #, #] f[_] send == (send (send nil :object) :method) # => true f[send] == (send (send nil :object) :method) # => true send == (send nil :object) # => true f[send] == (send nil :object) # => true == # => true f[nil] == # => true # == object # => true f[_] == object # => true [#, #, #] == (send nil :object) # => true # == method # => true f[_] == method # => true # sample.rb:1 object.method","title":"--debug"},{"location":"command_line/#-s-for-similarity","text":"Sometimes you want to search for some similar code like (send (send (send nil _) _) _) and we could simply say a.b.c . The option -s build an expression from the code ignoring final values. $ echo 'object.method' > sample.rb $ fast -s 'a.b' sample.rb # sample.rb:1 object . method See also Code Similarity tutorial.","title":"-s for similarity"},{"location":"command_line/#-c-to-search-from-code-example","text":"You can search for the exact expression with -c $ fast -c 'object.method' sample.rb # sample.rb:1 object . method Combining with -d , in the header you can see the generated expression. $ fast -d -c 'object.method' sample.rb | head -n 3 The generated expression from AST was: (send (send nil :object) :method)","title":"-c to search from code example"},{"location":"command_line/#fastfile","text":"Fastfile will loaded when you start a pattern with a dot. It means the pattern will be a shortcut predefined on these Fastfiles. It will make three attempts to load Fastfile defined in $PWD , $HOME or checking if the $FAST_FILE_DIR is configured. You can define a Fastfile in any project with your custom shortcuts and easy check some code or run some task.","title":"Fastfile"},{"location":"command_line/#shortcut-examples","text":"Create shortcuts with blocks enables introduce custom coding in the scope of the Fast module.","title":"Shortcut examples"},{"location":"command_line/#print-library-version","text":"Let's say you'd like to show the version of your library. Your regular params in the command line will look like: $ fast '(casgn nil VERSION)' lib/*/version.rb It will output but the command is not very handy. In order to just say fast .version you can use the previous snippet in your Fastfile . Fast . shortcut ( :version , '(casgn nil VERSION)' , 'lib/fast/version.rb' ) And calling fast .version it will output something like this: # lib/fast/version.rb:4 VERSION = '0.1.2' We can also always override the files params passing some other target file like fast .version lib/other/file.rb and it will reuse the other arguments from command line but replace the target files.","title":"Print library version."},{"location":"command_line/#bumping-a-gem-version","text":"While releasing a new gem version, we always need to mechanical go through the lib//version.rb and change the string value to bump the version of your library. It's pretty mechanical and here is an example that allow you to simple use fast .bump_version : Fast . shortcut :bump_version do rewrite_file ( 'lib/fast/version.rb' , '(casgn nil VERSION (str _)' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( \".\" ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( \".\" ) } '\" ) end end Note the shortcut scope The shortcut calls rewrite_file from Fast scope as it use Fast.instance_exec for shortcuts that yields blocks. Checking the version: $ fast .version 13 :58:40 # lib/fast/version.rb:4 VERSION = '0.1.2' Bumping the version: $ fast .bump_version 13 :58:43 No output because we don't print anything. Checking version again: $ fast .version 13 :58:54 # lib/fast/version.rb:4 VERSION = '0.1.3' And now a fancy shortcut to report the other shortcuts :) Fast . shortcut :shortcuts do report ( shortcuts . keys ) end Or we can make it a bit more friendly and also use Fast to process the shortcut positions and pick the comment that each shortcut have in the previous line: # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _))' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And it will be printing all loaded shortcuts with comments: $ fast .shortcuts fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments You can find more examples in the Fastfile .","title":"Bumping a gem version"},{"location":"editors-integration/","text":"Editors' integration \u00b6 We don't have any proper integration or official plugins for editors yet. Here are a few ideas you can use to make your own flow. Vim \u00b6 Split terminal vertically and open fast focused on build the expression. nnoremap < Leader > ff : vsplit \\ | terminal fast \"()\" % < Left >< Left >< Left >< Left >< Left > Or you can build a function: function ! s:Fast ( args ) let cmd = '' if ! empty ( b :ruby_project_root ) let cmd . = 'cd ' . b :ruby_project_root . ' && ' endif let cmd . = 'fast --no-color ' . a :args let custom_maker = neomake#utils#MakerFromCommand ( cmd ) let custom_maker.name = cmd let custom_maker.cwd = b :ruby_project_root let custom_maker.remove_invalid_entries = 0 \" e.g.: \" # path/to/file.rb:1141 \" my_method( \" :boom, \" arg1: 1, \" ) \" %W# %f:%l -> start a multiline warning when the line matches '# path/file.rb:1234' \" %-Z# end multiline warning on the next line that starts with '#' \" %C%m continued multiline warning message let custom_maker. errorformat = '%W# %f:%l, %-Z#, %C%m' let enabled_makers = [custom_maker] update | call neomake#Make ( 0 , enabled_makers ) | echom \"running: \" . cmd endfunction command ! - complete = file - nargs = 1 Fast call s:Fast (< q - args >) Check the conversation about vim integration here .","title":"Editors' Integration"},{"location":"editors-integration/#editors-integration","text":"We don't have any proper integration or official plugins for editors yet. Here are a few ideas you can use to make your own flow.","title":"Editors' integration"},{"location":"editors-integration/#vim","text":"Split terminal vertically and open fast focused on build the expression. nnoremap < Leader > ff : vsplit \\ | terminal fast \"()\" % < Left >< Left >< Left >< Left >< Left > Or you can build a function: function ! s:Fast ( args ) let cmd = '' if ! empty ( b :ruby_project_root ) let cmd . = 'cd ' . b :ruby_project_root . ' && ' endif let cmd . = 'fast --no-color ' . a :args let custom_maker = neomake#utils#MakerFromCommand ( cmd ) let custom_maker.name = cmd let custom_maker.cwd = b :ruby_project_root let custom_maker.remove_invalid_entries = 0 \" e.g.: \" # path/to/file.rb:1141 \" my_method( \" :boom, \" arg1: 1, \" ) \" %W# %f:%l -> start a multiline warning when the line matches '# path/file.rb:1234' \" %-Z# end multiline warning on the next line that starts with '#' \" %C%m continued multiline warning message let custom_maker. errorformat = '%W# %f:%l, %-Z#, %C%m' let enabled_makers = [custom_maker] update | call neomake#Make ( 0 , enabled_makers ) | echom \"running: \" . cmd endfunction command ! - complete = file - nargs = 1 Fast call s:Fast (< q - args >) Check the conversation about vim integration here .","title":"Vim"},{"location":"experiments/","text":"Experiments \u00b6 Experiments allow us to play with AST and do some code transformation, execute some code and continue combining successful transformations. The major idea is try a new approach without any promise and if it works continue transforming the code. Replace FactoryBot#create with build_stubbed . \u00b6 Let's look into the following spec example: describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's say we're amazed with FactoryBot#build_stubbed and want to build a small bot to make the changes in a entire code base. Skip some database touches while testing huge test suites are always a good idea. First we can hunt for the cases we want to find: $ ruby-parse -e \"create(:user)\" (send nil :create (sym :user)) Using fast in the command line to see real examples in the spec folder: $ fast \"(send nil create)\" spec If you don't have a real project but want to test, just create a sample ruby file with the code example above. Running it in a big codebase will probably find a few examples of blocks. The next step is build a replacement of each independent occurrence to use build_stubbed instead of create and combine the successful ones, run again and combine again, until try all kind of successful replacements combined. Considering we have the following code in sample_spec.rb : describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's create the experiment that will contain the nodes that are target to be executed and what we want to do when we find the node. experiment = Fast . experiment ( 'RSpec/ReplaceCreateWithBuildStubbed' ) do search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } end If we use Fast.replace_file it will replace all occurrences in the same run and that's one of the motivations behind create the ExperimentFile class. Executing a partial replacement of the first occurrence: experiment_file = Fast :: ExperimentFile . new ( 'sample_spec.rb' , experiment ) } puts experiment_file . partial_replace ( 1 ) The command will output the following code: describe \"my spec\" do let ( :user ) { build_stubbed ( :user ) } let ( :address ) { create ( :address ) } # ... end Remove useless before block \u00b6 Imagine the following code sample: describe \"my spec\" do before { create ( :user ) } # ... after { User . delete_all } end And now, we can define an experiment that removes the entire code block and run the experimental specs. experiment = Fast . experiment ( 'RSpec/RemoveUselessBeforeAfterHook' ) do lookup 'spec' search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end To run the experiment you can simply say: experiment . run Or drop the code into experiments folder and use the fast-experiment command line tool. $ fast-experiment RSpec/RemoveUselessBeforeAfterHook spec DSL \u00b6 In the lookup you can pass files or folders. The search contains the expression you want to match With edit block you can apply the code change And the policy is executed to check if the current change is valuable If the file contains multiple before or after blocks, each removal will occur independently and the successfull removals will be combined as a secondary change. The process repeates until find all possible combinations. See more examples in experiments folder. To run multiple experiments, use fast-experiment runner: fast-experiment You can limit experiments or file escope: fast-experiment RSpec/RemoveUselessBeforeAfterHook spec/models/**/*_spec.rb Or a single file: fast-experiment RSpec/ReplaceCreateWithBuildStubbed spec/models/my_spec.rb","title":"Experiments"},{"location":"experiments/#experiments","text":"Experiments allow us to play with AST and do some code transformation, execute some code and continue combining successful transformations. The major idea is try a new approach without any promise and if it works continue transforming the code.","title":"Experiments"},{"location":"experiments/#replace-factorybotcreate-with-build_stubbed","text":"Let's look into the following spec example: describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's say we're amazed with FactoryBot#build_stubbed and want to build a small bot to make the changes in a entire code base. Skip some database touches while testing huge test suites are always a good idea. First we can hunt for the cases we want to find: $ ruby-parse -e \"create(:user)\" (send nil :create (sym :user)) Using fast in the command line to see real examples in the spec folder: $ fast \"(send nil create)\" spec If you don't have a real project but want to test, just create a sample ruby file with the code example above. Running it in a big codebase will probably find a few examples of blocks. The next step is build a replacement of each independent occurrence to use build_stubbed instead of create and combine the successful ones, run again and combine again, until try all kind of successful replacements combined. Considering we have the following code in sample_spec.rb : describe \"my spec\" do let ( :user ) { create ( :user ) } let ( :address ) { create ( :address ) } # ... end Let's create the experiment that will contain the nodes that are target to be executed and what we want to do when we find the node. experiment = Fast . experiment ( 'RSpec/ReplaceCreateWithBuildStubbed' ) do search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } end If we use Fast.replace_file it will replace all occurrences in the same run and that's one of the motivations behind create the ExperimentFile class. Executing a partial replacement of the first occurrence: experiment_file = Fast :: ExperimentFile . new ( 'sample_spec.rb' , experiment ) } puts experiment_file . partial_replace ( 1 ) The command will output the following code: describe \"my spec\" do let ( :user ) { build_stubbed ( :user ) } let ( :address ) { create ( :address ) } # ... end","title":"Replace FactoryBot#create with build_stubbed."},{"location":"experiments/#remove-useless-before-block","text":"Imagine the following code sample: describe \"my spec\" do before { create ( :user ) } # ... after { User . delete_all } end And now, we can define an experiment that removes the entire code block and run the experimental specs. experiment = Fast . experiment ( 'RSpec/RemoveUselessBeforeAfterHook' ) do lookup 'spec' search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end To run the experiment you can simply say: experiment . run Or drop the code into experiments folder and use the fast-experiment command line tool. $ fast-experiment RSpec/RemoveUselessBeforeAfterHook spec","title":"Remove useless before block"},{"location":"experiments/#dsl","text":"In the lookup you can pass files or folders. The search contains the expression you want to match With edit block you can apply the code change And the policy is executed to check if the current change is valuable If the file contains multiple before or after blocks, each removal will occur independently and the successfull removals will be combined as a secondary change. The process repeates until find all possible combinations. See more examples in experiments folder. To run multiple experiments, use fast-experiment runner: fast-experiment You can limit experiments or file escope: fast-experiment RSpec/RemoveUselessBeforeAfterHook spec/models/**/*_spec.rb Or a single file: fast-experiment RSpec/ReplaceCreateWithBuildStubbed spec/models/my_spec.rb","title":"DSL"},{"location":"git/","text":"You can overload the AST node with extra methods to get information from Git. Let's start with some basic setup to reuse in the next examples: Git require \u00b6 By default, this extension is not loaded in the fast environment, so you should require it. require 'fast/git' Then it will work with any AST node. ast = Fast . ast_from_file ( 'lib/fast.rb' ) Log \u00b6 First commit from git: ast . git_log . first . author . name # => \"Jonatas Davi Paganini\" It uses ruby-git gem, so all methods are available: ast . git_log . since ( Time . mktime ( 2019 )) . entries . map ( & :message ) Counting commits per year: ast . git_log . entries . group_by { | t | t . date . year } . transform_values ( & :size ) # => {2020=>4, 2019=>22, 2018=>4} Counting commits per contributor: ast . git_log . entries . group_by { | t | t . author . name } . transform_values ( & :size ) # => {\"J\u00f4natas Davi Paganini\"=>29, ...} Selecting last commit message: ast . last_commit . message # => \"Add node extensions for extracting info from git (#21)\" Remote git URL: ast . remote_url # => \"git@github.com:jonatas/fast.git\" ast . project_url # => \"https://github.com/jonatas/fast\" The sha from last commit: ast . sha # => \"cd1c036b55ec1d41e5769ad73b282dd6429a90a6\" Pick a link from the files to master version: ast . link # => \"https://github.com/jonatas/fast/blob/master/lib/fast.rb#L3-L776\" Getting permalink from current commit: ast . permalink # => \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L3-L776\" Markdown link \u00b6 Let's say you'd like to capture a list of class names that inherits the Find class: puts ast . capture ( \"(class $(const nil _) (const nil Find)\" ) . map ( & :md_link ) . join ( \" \\n * \" ) It will output the following links: FindString MethodCall InstanceMethodCall FindWithCapture FindFromArgument Capture Parent Any All Not Maybe Permalink \u00b6 If you need to get a permanent link to the code, use the permalink method: ast . search ( \"(class (const nil _) (const nil Find)\" ) . map ( & :permalink ) # => [\"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L524-L541\", # \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L551-L571\", ...]","title":"Git Integration"},{"location":"git/#git-require","text":"By default, this extension is not loaded in the fast environment, so you should require it. require 'fast/git' Then it will work with any AST node. ast = Fast . ast_from_file ( 'lib/fast.rb' )","title":"Git require"},{"location":"git/#log","text":"First commit from git: ast . git_log . first . author . name # => \"Jonatas Davi Paganini\" It uses ruby-git gem, so all methods are available: ast . git_log . since ( Time . mktime ( 2019 )) . entries . map ( & :message ) Counting commits per year: ast . git_log . entries . group_by { | t | t . date . year } . transform_values ( & :size ) # => {2020=>4, 2019=>22, 2018=>4} Counting commits per contributor: ast . git_log . entries . group_by { | t | t . author . name } . transform_values ( & :size ) # => {\"J\u00f4natas Davi Paganini\"=>29, ...} Selecting last commit message: ast . last_commit . message # => \"Add node extensions for extracting info from git (#21)\" Remote git URL: ast . remote_url # => \"git@github.com:jonatas/fast.git\" ast . project_url # => \"https://github.com/jonatas/fast\" The sha from last commit: ast . sha # => \"cd1c036b55ec1d41e5769ad73b282dd6429a90a6\" Pick a link from the files to master version: ast . link # => \"https://github.com/jonatas/fast/blob/master/lib/fast.rb#L3-L776\" Getting permalink from current commit: ast . permalink # => \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L3-L776\"","title":"Log"},{"location":"git/#markdown-link","text":"Let's say you'd like to capture a list of class names that inherits the Find class: puts ast . capture ( \"(class $(const nil _) (const nil Find)\" ) . map ( & :md_link ) . join ( \" \\n * \" ) It will output the following links: FindString MethodCall InstanceMethodCall FindWithCapture FindFromArgument Capture Parent Any All Not Maybe","title":"Markdown link"},{"location":"git/#permalink","text":"If you need to get a permanent link to the code, use the permalink method: ast . search ( \"(class (const nil _) (const nil Find)\" ) . map ( & :permalink ) # => [\"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L524-L541\", # \"https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L551-L571\", ...]","title":"Permalink"},{"location":"ideas/","text":"Ideas I want to build with Fast \u00b6 I don't have all the time I need to develop all the ideas I have to build around this tool, so here is a dump of a few brainstormings: Inline target code \u00b6 I started fast-inline that can be useful to try to see how much every library is used in a project. My idea is try to inline some specific method call to understand if it makes sense to have an entire library in the stock. Understanding dependencies and how the code works can be a first step to get an \"algorithm as a service\". Instead of loading everything from the library, it would facilitate the cherry pick of only the proper dependencies necessaries to run the code you have and not the code that is overloading the project. Neo4J adapter \u00b6 Easy pipe fast results to Neo4J. It would facilitate to explore more complex scenarios and combine data from other sources. Ast Diff \u00b6 Allow to compare and return a summary of differences between two trees. It would be useful to identify renamings or other small changes, like only changes in comments that does not affect the file and possibly be ignored for some operations like run or not run tests. Transition synapses \u00b6 Following the previous idea, it would be great if we can understand the transition synapses and make it easily available to catch up with previous learnings. https://github.com/jonatas/chewy-diff/blob/master/lib/chewy/diff.rb This example, shows adds and removals from specific node targets between two different files. If we start tracking AST transition synapses and associating with \"Fixes\" or \"Reverts\" we can predict introduction of new bugs by inpecting if the introduction of new patterns that can be possibly reverted or improved. Fast Rewriter with pure strings \u00b6 As the AST rewriter adopts a custom block that needs to implement ruby code, we can expand the a query language for rewriting files without need to take the custom Ruby block. Example: Fast . gsub_expression ( 'remove(@expression)' ) # (node) => { remove(node.location.expression) } And later we can bind it in the command line to allow implement custom replacements without need to write a ruby file. fast (def my_target_method) lib spec --rewrite \"remove(@expression)\" or fast (def my_target_method) lib spec --rewrite \"replace(@name, 'renamed_method')\"","title":"Ideas"},{"location":"ideas/#ideas-i-want-to-build-with-fast","text":"I don't have all the time I need to develop all the ideas I have to build around this tool, so here is a dump of a few brainstormings:","title":"Ideas I want to build with Fast"},{"location":"ideas/#inline-target-code","text":"I started fast-inline that can be useful to try to see how much every library is used in a project. My idea is try to inline some specific method call to understand if it makes sense to have an entire library in the stock. Understanding dependencies and how the code works can be a first step to get an \"algorithm as a service\". Instead of loading everything from the library, it would facilitate the cherry pick of only the proper dependencies necessaries to run the code you have and not the code that is overloading the project.","title":"Inline target code"},{"location":"ideas/#neo4j-adapter","text":"Easy pipe fast results to Neo4J. It would facilitate to explore more complex scenarios and combine data from other sources.","title":"Neo4J adapter"},{"location":"ideas/#ast-diff","text":"Allow to compare and return a summary of differences between two trees. It would be useful to identify renamings or other small changes, like only changes in comments that does not affect the file and possibly be ignored for some operations like run or not run tests.","title":"Ast Diff"},{"location":"ideas/#transition-synapses","text":"Following the previous idea, it would be great if we can understand the transition synapses and make it easily available to catch up with previous learnings. https://github.com/jonatas/chewy-diff/blob/master/lib/chewy/diff.rb This example, shows adds and removals from specific node targets between two different files. If we start tracking AST transition synapses and associating with \"Fixes\" or \"Reverts\" we can predict introduction of new bugs by inpecting if the introduction of new patterns that can be possibly reverted or improved.","title":"Transition synapses"},{"location":"ideas/#fast-rewriter-with-pure-strings","text":"As the AST rewriter adopts a custom block that needs to implement ruby code, we can expand the a query language for rewriting files without need to take the custom Ruby block. Example: Fast . gsub_expression ( 'remove(@expression)' ) # (node) => { remove(node.location.expression) } And later we can bind it in the command line to allow implement custom replacements without need to write a ruby file. fast (def my_target_method) lib spec --rewrite \"remove(@expression)\" or fast (def my_target_method) lib spec --rewrite \"replace(@name, 'renamed_method')\"","title":"Fast Rewriter with pure strings"},{"location":"pry-integration/","text":"You can create a custom command in pry to reuse fast in any session. Start simply dropping it on your .pryrc : Pry :: Commands . block_command \"fast\" , \"Fast search\" do | expression , file | require \"fast\" files = Fast . ruby_files_from ( file || '.' ) files . each do | f | results = Fast . search_file ( expression , f ) next if results . nil? || results . empty? output . puts Fast . highlight ( \"# #{ f } \" ) results . each do | result | output . puts Fast . highlight ( result ) end end end And use it in the console: fast '(def match?)' lib/fast.rb","title":"Pry Integration"},{"location":"research/","text":"Research \u00b6 I love to research about codebase as data and prototyping ideas several times doesn't fit in simple shortcuts . Here is my first research that worth sharing: Combining Runtime metadata with AST complex searches \u00b6 This example covers how to find RSpec allow combined with and_return missing the with clause specifying the nested parameters. Here is the gist if you want to go straight and run it. Scenario for simple example: Given I have the following class: class Account def withdraw ( value ) if @total >= value @total -= value :ok else :not_allowed end end end And I'm testing it with allow and some possibilities: # bad allow ( Account ) . to receive ( :withdraw ) . and_return ( :ok ) # good allow ( Account ) . to receive ( :withdraw ) . with ( 100 ) . and_return ( :ok ) Objective: find all bad cases of any class that does not respect the method parameters signature. First, let's understand the method signature of a method: Account . instance_method ( :withdraw ) . parameters # => [[:req, :value]] Now, we can build a small script to use the node pattern to match the proper specs that are using such pattern and later visit their method signatures. Fast . class_eval do # Captures class and method name when find syntax like: # `allow(...).to receive(...)` that does not end with `.with(...)` pattern_with_captures = <<~ FAST (send (send nil allow (const nil $_)) to (send (send nil receive (sym $_)) !with)) FAST pattern = expression ( pattern_with_captures . tr ( '$' , '' )) ruby_files_from ( 'spec' ) . each do | file | results = search_file ( pattern , file ) || [] rescue next results . each do | n | clazz , method = capture ( n , pattern_with_captures ) if klazz = Object . const_get ( clazz . to_s ) rescue nil if klazz . respond_to? ( method ) params = klazz . method ( method ) . parameters if params . any? { | e | e . first == :req } code = n . loc . expression range = [ code . first_line , code . last_line ]. uniq . join ( \",\" ) boom_message = \"BOOM! #{ clazz } . #{ method } does not include the REQUIRED parameters!\" puts boom_message , \" #{ file } : #{ range } \" , code . source end end end end end end Preload your environment before run the script Keep in mind that you should run it with your environment preloaded otherwise it will skip the classes. You can add elses for const_get and respond_to and report weird cases if your environment is not preloading properly.","title":"Research"},{"location":"research/#research","text":"I love to research about codebase as data and prototyping ideas several times doesn't fit in simple shortcuts . Here is my first research that worth sharing:","title":"Research"},{"location":"research/#combining-runtime-metadata-with-ast-complex-searches","text":"This example covers how to find RSpec allow combined with and_return missing the with clause specifying the nested parameters. Here is the gist if you want to go straight and run it. Scenario for simple example: Given I have the following class: class Account def withdraw ( value ) if @total >= value @total -= value :ok else :not_allowed end end end And I'm testing it with allow and some possibilities: # bad allow ( Account ) . to receive ( :withdraw ) . and_return ( :ok ) # good allow ( Account ) . to receive ( :withdraw ) . with ( 100 ) . and_return ( :ok ) Objective: find all bad cases of any class that does not respect the method parameters signature. First, let's understand the method signature of a method: Account . instance_method ( :withdraw ) . parameters # => [[:req, :value]] Now, we can build a small script to use the node pattern to match the proper specs that are using such pattern and later visit their method signatures. Fast . class_eval do # Captures class and method name when find syntax like: # `allow(...).to receive(...)` that does not end with `.with(...)` pattern_with_captures = <<~ FAST (send (send nil allow (const nil $_)) to (send (send nil receive (sym $_)) !with)) FAST pattern = expression ( pattern_with_captures . tr ( '$' , '' )) ruby_files_from ( 'spec' ) . each do | file | results = search_file ( pattern , file ) || [] rescue next results . each do | n | clazz , method = capture ( n , pattern_with_captures ) if klazz = Object . const_get ( clazz . to_s ) rescue nil if klazz . respond_to? ( method ) params = klazz . method ( method ) . parameters if params . any? { | e | e . first == :req } code = n . loc . expression range = [ code . first_line , code . last_line ]. uniq . join ( \",\" ) boom_message = \"BOOM! #{ clazz } . #{ method } does not include the REQUIRED parameters!\" puts boom_message , \" #{ file } : #{ range } \" , code . source end end end end end end Preload your environment before run the script Keep in mind that you should run it with your environment preloaded otherwise it will skip the classes. You can add elses for const_get and respond_to and report weird cases if your environment is not preloading properly.","title":"Combining Runtime metadata with AST complex searches"},{"location":"shortcuts/","text":"Shortcuts \u00b6 Shortcuts are defined on a Fastfile inside any ruby project. Use ~/Fastfile You can also add one extra in your $HOME if you want to have something loaded always. By default, the command line interface does not load any Fastfile if the first param is not a shortcut. It should start with . . I'm building several researches and I'll make the examples open here to show several interesting cases in action. List your fast shortcuts \u00b6 As the interface is very rudimentar, let's build a shortcut to print what shortcuts are available. This is a good one to your $HOME/Fastfile : # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And using it on fast project that loads both ~/Fastfile and the Fastfile from the project: fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments Search for references \u00b6 I always miss bringing something simple as grep keyword where I can leave a simple string and it can search in all types of nodes and report interesting things about it. Let's consider a very flexible search that can target any code related to some keyword. Considering that we're talking about code indentifiers: # Search all references about some keyword or regular expression Fast . shortcut ( :ref ) do require 'fast/cli' Kernel . class_eval do def matches_args? identifier search = ARGV . last regex = Regexp . new ( search , Regexp :: IGNORECASE ) case identifier when Symbol , String regex . match? ( identifier ) || identifier . to_s . include? ( search ) when Astrolabe :: Node regex . match? ( identifier . to_sexp ) end end end pattern = <<~ FAST { ({class def sym str} #matches_args?)' ({const send} nil #matches_args?)' } FAST Fast :: Cli . run! ( [ pattern , '.' , '--parallel' ] ) end Rails: Show validations from models \u00b6 If the shortcut does not define a block, it works as a holder for arguments from the command line. Let's say you always use fast \"(send nil {validate validates})\" app/models to check validations in the models. You can define a shortcut to hold the args and avoid retyping long lines: # Show validations from app/models Fast . shortcut ( :validations , \"(send nil {validate validates})\" , \"app/models\" ) And you can reuse the search with the shortcut starting with a . : fast .validations And it will also accept params if you want to filter a specific file: fast .validations app/models/user.rb Note that you can also use flags in the command line shortcuts Let's say you also want to use fast --headless you can add it to the params: Fast.shortcut(:validations, \"(send nil {validate validates})\", \"app/models\", \"--headless\") Automated Refactor: Bump version \u00b6 Let's start with a real usage to bump a new version of the gem. Fast . shortcut :bump_version do rewrite_file ( '(casgn nil VERSION (str _)' , 'lib/fast/version.rb' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( '.' ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( '.' ) } '\" ) end end And then the change is done in the lib/fast/version.rb : module Fast - VERSION = '0.1.6' + VERSION = '0.1.7' end RSpec: Find unused shared contexts \u00b6 If you build shared contexts often, probably you can forget some left overs. The objective of the shortcut is find leftovers from shared contexts. First, the objective is capture all names of the RSpec.shared_context or shared_context declared in the spec/support folder. Fast . capture_all ( '(block (send {nil,_} shared_context (str $_)))' , Fast . ruby_files_from ( 'spec/support' )) Then, we need to check all the specs and search for include_context usages to confirm if all defined contexts are being used: specs = Fast . ruby_files_from ( 'spec' ) . select { | f | f !~ %r{spec/support/} } Fast . search_all ( \"(send nil include_context (str #register_usage)\" , specs ) Note that we created a new reference to #register_usage and we need to define the method too: @used = [] def register_usage context_name @used << context_name end Wrapping up everything in a shortcut: # Show unused shared contexts Fast . shortcut ( :unused_shared_contexts ) do puts \"Checking shared contexts\" Kernel . class_eval do @used = [] def register_usage context_name @used << context_name end def show_report! defined_contexts unused = defined_contexts . values . flatten - @used if unused . any? puts \"Unused shared contexts\" , unused else puts \"Good job! all the #{ defined_contexts . size } contexts are used!\" end end end specs = ruby_files_from ( 'spec/' ) . select { | f | f !~ %r{spec/support/} } search_all ( \"(send nil include_context (str #register_usage)\" , specs ) defined_contexts = capture_all ( '(block (send {nil,_} shared_context (str $_)))' , ruby_files_from ( 'spec' )) Kernel . public_send ( :show_report! , defined_contexts ) end Why #register_usage is defined on the Kernel ? Yes! note that the #register_usage was forced to be inside Kernel because of the shortcut block that takes the Fast context to be easy to access in the default functions. As I can define multiple shortcuts I don't want to polute my Kernel module with other methods that are not useful. RSpec: Remove unused let \u00b6 First shortcut with experiments If you're not familiar with automated experiments, you can read about it here . The current scenario is similar in terms of search with the previous one, but more advanced because we're going to introduce automated refactoring. The idea is simple, if it finds a let in a RSpec scenario that is not referenced, it tries to experimentally remove the let and run the tests: # Experimental remove `let` that are not referenced in the spec Fast . shortcut ( :exp_remove_let ) do require 'fast/experiment' Kernel . class_eval do file = ARGV . last defined_lets = Fast . capture_file ( '(block (send nil let (sym $_)))' , file ) . uniq @unreferenced = defined_lets . select do | identifier | Fast . search_file ( \"(send nil #{ identifier } )\" , file ) . empty? end def unreferenced_let? ( identifier ) @unreferenced . include? identifier end end experiment ( 'RSpec/RemoveUnreferencedLet' ) do lookup ARGV . last search '(block (send nil let (sym #unreferenced_let?)))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end And it will run with a single file from command line: fast .exp_remove_let spec/my_file_spec.rb FactoryBot: Replace create with build_stubbed \u00b6 For performance reasons, if we can avoid touching the database the test will always be faster. # Experimental switch from `create` to `build_stubbed` Fast . shortcut ( :exp_build_stubbed ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/UseBuildStubbed' ) do lookup ARGV . last search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end RSpec: Use let_it_be instead of let \u00b6 The let_it_be is a simple helper from TestProf gem that can speed up the specs by caching some factories using like a before_all approach. This experiment hunts for let(...) { create(...) } and switch the let to let_it_be : # Experimental replace `let(_)` with `let_it_be` case it calls `create` inside the block Fast . shortcut ( :exp_let_it_be ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/LetItBe' ) do lookup ARGV . last search '(block (send nil let (sym _)) (args) (send nil create))' edit { | node | replace ( node . children . first . loc . selector , 'let_it_be' ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end RSpec: Remove before or after blocks \u00b6 From time to time, we forget some left overs like before or after blocks that even removing from the code, the tests still passes. This experiment removes the before/after blocks and check if the test passes. # Experimental remove `before` or `after` blocks. Fast . shortcut ( :exp_remove_before_after ) do require 'fast/experiment' Fast . experiment ( 'RSpec/RemoveBeforeAfter' ) do lookup ARGV . last search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end RSpec: Show message chains \u00b6 I often forget the syntax and need to search for message chains on specs, so I created an shortcut for it. # Show RSpec message chains Fast . shortcut ( :message_chains , '^^(send nil receive_message_chain)' , 'spec' ) RSpec: Show nested assertions \u00b6 I love to use nested assertions and I often need examples to refer to them: # Show RSpec nested assertions with .and Fast . shortcut ( :nested_assertions , '^^(send ... and)' , 'spec' )","title":"Shortcuts"},{"location":"shortcuts/#shortcuts","text":"Shortcuts are defined on a Fastfile inside any ruby project. Use ~/Fastfile You can also add one extra in your $HOME if you want to have something loaded always. By default, the command line interface does not load any Fastfile if the first param is not a shortcut. It should start with . . I'm building several researches and I'll make the examples open here to show several interesting cases in action.","title":"Shortcuts"},{"location":"shortcuts/#list-your-fast-shortcuts","text":"As the interface is very rudimentar, let's build a shortcut to print what shortcuts are available. This is a good one to your $HOME/Fastfile : # List all shortcut with comments Fast . shortcut :shortcuts do fast_files . each do | file | lines = File . readlines ( file ) . map { | line | line . chomp . gsub ( /\\s*#/ , '' ) . strip } result = capture_file ( '(send ... shortcut $(sym _' , file ) result = [ result ] unless result . is_a? Array result . each do | capture | target = capture . loc . expression puts \"fast . #{ target . source [ 1 ..- 1 ]. ljust ( 30 ) } # #{ lines [ target . line - 2 ] } \" end end end And using it on fast project that loads both ~/Fastfile and the Fastfile from the project: fast .version # Let's say you'd like to show the version that is over the version file fast .parser # Simple shortcut that I used often to show how the expression parser works fast .bump_version # Use `fast .bump_version` to rewrite the version file fast .shortcuts # List all shortcut with comments","title":"List your fast shortcuts"},{"location":"shortcuts/#search-for-references","text":"I always miss bringing something simple as grep keyword where I can leave a simple string and it can search in all types of nodes and report interesting things about it. Let's consider a very flexible search that can target any code related to some keyword. Considering that we're talking about code indentifiers: # Search all references about some keyword or regular expression Fast . shortcut ( :ref ) do require 'fast/cli' Kernel . class_eval do def matches_args? identifier search = ARGV . last regex = Regexp . new ( search , Regexp :: IGNORECASE ) case identifier when Symbol , String regex . match? ( identifier ) || identifier . to_s . include? ( search ) when Astrolabe :: Node regex . match? ( identifier . to_sexp ) end end end pattern = <<~ FAST { ({class def sym str} #matches_args?)' ({const send} nil #matches_args?)' } FAST Fast :: Cli . run! ( [ pattern , '.' , '--parallel' ] ) end","title":"Search for references"},{"location":"shortcuts/#rails-show-validations-from-models","text":"If the shortcut does not define a block, it works as a holder for arguments from the command line. Let's say you always use fast \"(send nil {validate validates})\" app/models to check validations in the models. You can define a shortcut to hold the args and avoid retyping long lines: # Show validations from app/models Fast . shortcut ( :validations , \"(send nil {validate validates})\" , \"app/models\" ) And you can reuse the search with the shortcut starting with a . : fast .validations And it will also accept params if you want to filter a specific file: fast .validations app/models/user.rb Note that you can also use flags in the command line shortcuts Let's say you also want to use fast --headless you can add it to the params: Fast.shortcut(:validations, \"(send nil {validate validates})\", \"app/models\", \"--headless\")","title":"Rails: Show validations from models"},{"location":"shortcuts/#automated-refactor-bump-version","text":"Let's start with a real usage to bump a new version of the gem. Fast . shortcut :bump_version do rewrite_file ( '(casgn nil VERSION (str _)' , 'lib/fast/version.rb' ) do | node | target = node . children . last . loc . expression pieces = target . source . split ( '.' ) . map ( & :to_i ) pieces . reverse . each_with_index do | fragment , i | if fragment < 9 pieces [- ( i + 1 ) ] = fragment + 1 break else pieces [- ( i + 1 ) ] = 0 end end replace ( target , \"' #{ pieces . join ( '.' ) } '\" ) end end And then the change is done in the lib/fast/version.rb : module Fast - VERSION = '0.1.6' + VERSION = '0.1.7' end","title":"Automated Refactor: Bump version"},{"location":"shortcuts/#rspec-find-unused-shared-contexts","text":"If you build shared contexts often, probably you can forget some left overs. The objective of the shortcut is find leftovers from shared contexts. First, the objective is capture all names of the RSpec.shared_context or shared_context declared in the spec/support folder. Fast . capture_all ( '(block (send {nil,_} shared_context (str $_)))' , Fast . ruby_files_from ( 'spec/support' )) Then, we need to check all the specs and search for include_context usages to confirm if all defined contexts are being used: specs = Fast . ruby_files_from ( 'spec' ) . select { | f | f !~ %r{spec/support/} } Fast . search_all ( \"(send nil include_context (str #register_usage)\" , specs ) Note that we created a new reference to #register_usage and we need to define the method too: @used = [] def register_usage context_name @used << context_name end Wrapping up everything in a shortcut: # Show unused shared contexts Fast . shortcut ( :unused_shared_contexts ) do puts \"Checking shared contexts\" Kernel . class_eval do @used = [] def register_usage context_name @used << context_name end def show_report! defined_contexts unused = defined_contexts . values . flatten - @used if unused . any? puts \"Unused shared contexts\" , unused else puts \"Good job! all the #{ defined_contexts . size } contexts are used!\" end end end specs = ruby_files_from ( 'spec/' ) . select { | f | f !~ %r{spec/support/} } search_all ( \"(send nil include_context (str #register_usage)\" , specs ) defined_contexts = capture_all ( '(block (send {nil,_} shared_context (str $_)))' , ruby_files_from ( 'spec' )) Kernel . public_send ( :show_report! , defined_contexts ) end Why #register_usage is defined on the Kernel ? Yes! note that the #register_usage was forced to be inside Kernel because of the shortcut block that takes the Fast context to be easy to access in the default functions. As I can define multiple shortcuts I don't want to polute my Kernel module with other methods that are not useful.","title":"RSpec: Find unused shared contexts"},{"location":"shortcuts/#rspec-remove-unused-let","text":"First shortcut with experiments If you're not familiar with automated experiments, you can read about it here . The current scenario is similar in terms of search with the previous one, but more advanced because we're going to introduce automated refactoring. The idea is simple, if it finds a let in a RSpec scenario that is not referenced, it tries to experimentally remove the let and run the tests: # Experimental remove `let` that are not referenced in the spec Fast . shortcut ( :exp_remove_let ) do require 'fast/experiment' Kernel . class_eval do file = ARGV . last defined_lets = Fast . capture_file ( '(block (send nil let (sym $_)))' , file ) . uniq @unreferenced = defined_lets . select do | identifier | Fast . search_file ( \"(send nil #{ identifier } )\" , file ) . empty? end def unreferenced_let? ( identifier ) @unreferenced . include? identifier end end experiment ( 'RSpec/RemoveUnreferencedLet' ) do lookup ARGV . last search '(block (send nil let (sym #unreferenced_let?)))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end And it will run with a single file from command line: fast .exp_remove_let spec/my_file_spec.rb","title":"RSpec: Remove unused let"},{"location":"shortcuts/#factorybot-replace-create-with-build_stubbed","text":"For performance reasons, if we can avoid touching the database the test will always be faster. # Experimental switch from `create` to `build_stubbed` Fast . shortcut ( :exp_build_stubbed ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/UseBuildStubbed' ) do lookup ARGV . last search '(send nil create)' edit { | node | replace ( node . loc . selector , 'build_stubbed' ) } policy { | new_file | system ( \"bundle exec rspec --fail-fast #{ new_file } \" ) } end . run end","title":"FactoryBot: Replace create with build_stubbed"},{"location":"shortcuts/#rspec-use-let_it_be-instead-of-let","text":"The let_it_be is a simple helper from TestProf gem that can speed up the specs by caching some factories using like a before_all approach. This experiment hunts for let(...) { create(...) } and switch the let to let_it_be : # Experimental replace `let(_)` with `let_it_be` case it calls `create` inside the block Fast . shortcut ( :exp_let_it_be ) do require 'fast/experiment' Fast . experiment ( 'FactoryBot/LetItBe' ) do lookup ARGV . last search '(block (send nil let (sym _)) (args) (send nil create))' edit { | node | replace ( node . children . first . loc . selector , 'let_it_be' ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end","title":"RSpec: Use let_it_be instead of let"},{"location":"shortcuts/#rspec-remove-before-or-after-blocks","text":"From time to time, we forget some left overs like before or after blocks that even removing from the code, the tests still passes. This experiment removes the before/after blocks and check if the test passes. # Experimental remove `before` or `after` blocks. Fast . shortcut ( :exp_remove_before_after ) do require 'fast/experiment' Fast . experiment ( 'RSpec/RemoveBeforeAfter' ) do lookup ARGV . last search '(block (send nil {before after}))' edit { | node | remove ( node . loc . expression ) } policy { | new_file | system ( \"bin/spring rspec --fail-fast #{ new_file } \" ) } end . run end","title":"RSpec: Remove before or after blocks"},{"location":"shortcuts/#rspec-show-message-chains","text":"I often forget the syntax and need to search for message chains on specs, so I created an shortcut for it. # Show RSpec message chains Fast . shortcut ( :message_chains , '^^(send nil receive_message_chain)' , 'spec' )","title":"RSpec: Show message chains"},{"location":"shortcuts/#rspec-show-nested-assertions","text":"I love to use nested assertions and I often need examples to refer to them: # Show RSpec nested assertions with .and Fast . shortcut ( :nested_assertions , '^^(send ... and)' , 'spec' )","title":"RSpec: Show nested assertions"},{"location":"similarity_tutorial/","text":"Research for code similarity \u00b6 This is a small tutorial to explore code similarity. Check the code example here . The major idea is register all expression styles and see if we can find some similarity between the structures. First we need to create a function that can analyze AST nodes and extract a pattern from the expression. The expression needs to generalize final node values and recursively build a pattern that can be used as a search expression. def expression_from ( node ) case node when Parser :: AST :: Node if node . children . any? children_expression = node . children . map ( & method ( :expression_from )) . join ( ' ' ) \"( #{ node . type } #{ children_expression } )\" else \"( #{ node . type } )\" end when nil , 'nil' 'nil' when Symbol , String , Integer '_' when Array , Hash '...' else node end end The pattern generated only flexibilize the search allowing us to group similar nodes. Example: expression_from ( code [ '1' ] ) # =>'(int _)' expression_from ( code [ 'nil' ] ) # =>'(nil)' expression_from ( code [ 'a = 1' ] ) # =>'(lvasgn _ (int _))' expression_from ( code [ 'def name; person.name end' ] ) # =>'(def _ (args) (send (send nil _) _))' The current method can translate all kind of expressions and the next step is observe some specific node types and try to group the similarities using the pattern generated. Fast . search_file ( 'class' , 'lib/fast.rb' ) Capturing the constant name and filtering only for symbols is easy and we can see that we have a few classes defined in the the same file. Fast . search_file ( '(class (const nil $_))' , 'lib/fast.rb' ) . grep ( Symbol ) => [ :Rewriter , :ExpressionParser , :Find , :FindString , :FindWithCapture , :Capture , :Parent , :Any , :All , :Not , :Maybe , :Matcher , :Experiment , :ExperimentFile ] The idea of this inspecton is build a proof of concept to show the similarity of matcher classes because they only define a match? method. patterns = Fast . search_file ( 'class' , 'lib/fast.rb' ) . map { | n | Fast . expression_from ( n )} A simple comparison between the patterns size versus .uniq.size can proof if the idea will work. patterns . size == patterns . uniq . size It does not work for the matcher cases but we can go deeper and analyze all files required by bundler. similarities = {} Gem . find_files ( '*.rb' ) . each do | file | Fast . search_file ( '{block send if while case def defs class module}' , file ) . map do | n | key = Fast . expression_from ( n ) similarities [ key ] ||= Set . new similarities [ key ] << file end end similarities . delete_if { | k , v | v . size < 2 } The similarities found are the following: { \"(class (const nil _) (const nil _) nil)\" => # , \"(class (const nil _) nil nil)\" => #} And now we can test the expression using the command line tool through the files and observe the similarity: \u22ca> ~ fast \"(class (const nil _) (const nil _) nil)\" /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb Output: # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:8 class DeadWorker < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:11 class Break < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:14 class Kill < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb:16 class SourceNotFoundError < StandardError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb:63 class Error < RuntimeError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb:338 class Abort < Exception ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb:125 class Cyclic < StandardError end It works and now we can create a method to do what the command line tool did, grouping the patterns and inspecting the occurrences. def similarities . show pattern files = self [ pattern ] files . each do | file | nodes = Fast . search_file ( pattern , file ) nodes . each do | result | Fast . report ( result , file : file ) end end end And calling the method exploring some \"if\" similarities, it prints the following results: similarities . show \"(if (send (const nil _) _ (lvar _)) nil (return (false)))\" # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/resolv.rb:1248 return false unless Name === other # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/fileutils.rb:138 return false unless File . exist? ( new ) # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/matrix.rb:1862 return false unless Vector === other","title":"Code Similarity"},{"location":"similarity_tutorial/#research-for-code-similarity","text":"This is a small tutorial to explore code similarity. Check the code example here . The major idea is register all expression styles and see if we can find some similarity between the structures. First we need to create a function that can analyze AST nodes and extract a pattern from the expression. The expression needs to generalize final node values and recursively build a pattern that can be used as a search expression. def expression_from ( node ) case node when Parser :: AST :: Node if node . children . any? children_expression = node . children . map ( & method ( :expression_from )) . join ( ' ' ) \"( #{ node . type } #{ children_expression } )\" else \"( #{ node . type } )\" end when nil , 'nil' 'nil' when Symbol , String , Integer '_' when Array , Hash '...' else node end end The pattern generated only flexibilize the search allowing us to group similar nodes. Example: expression_from ( code [ '1' ] ) # =>'(int _)' expression_from ( code [ 'nil' ] ) # =>'(nil)' expression_from ( code [ 'a = 1' ] ) # =>'(lvasgn _ (int _))' expression_from ( code [ 'def name; person.name end' ] ) # =>'(def _ (args) (send (send nil _) _))' The current method can translate all kind of expressions and the next step is observe some specific node types and try to group the similarities using the pattern generated. Fast . search_file ( 'class' , 'lib/fast.rb' ) Capturing the constant name and filtering only for symbols is easy and we can see that we have a few classes defined in the the same file. Fast . search_file ( '(class (const nil $_))' , 'lib/fast.rb' ) . grep ( Symbol ) => [ :Rewriter , :ExpressionParser , :Find , :FindString , :FindWithCapture , :Capture , :Parent , :Any , :All , :Not , :Maybe , :Matcher , :Experiment , :ExperimentFile ] The idea of this inspecton is build a proof of concept to show the similarity of matcher classes because they only define a match? method. patterns = Fast . search_file ( 'class' , 'lib/fast.rb' ) . map { | n | Fast . expression_from ( n )} A simple comparison between the patterns size versus .uniq.size can proof if the idea will work. patterns . size == patterns . uniq . size It does not work for the matcher cases but we can go deeper and analyze all files required by bundler. similarities = {} Gem . find_files ( '*.rb' ) . each do | file | Fast . search_file ( '{block send if while case def defs class module}' , file ) . map do | n | key = Fast . expression_from ( n ) similarities [ key ] ||= Set . new similarities [ key ] << file end end similarities . delete_if { | k , v | v . size < 2 } The similarities found are the following: { \"(class (const nil _) (const nil _) nil)\" => # , \"(class (const nil _) nil nil)\" => #} And now we can test the expression using the command line tool through the files and observe the similarity: \u22ca> ~ fast \"(class (const nil _) (const nil _) nil)\" /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb Output: # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:8 class DeadWorker < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:11 class Break < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parallel-1.12.1/lib/parallel.rb:14 class Kill < StandardError end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/method_source-0.9.0/lib/method_source.rb:16 class SourceNotFoundError < StandardError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rdoc.rb:63 class Error < RuntimeError ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/irb.rb:338 class Abort < Exception ; end # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/tsort.rb:125 class Cyclic < StandardError end It works and now we can create a method to do what the command line tool did, grouping the patterns and inspecting the occurrences. def similarities . show pattern files = self [ pattern ] files . each do | file | nodes = Fast . search_file ( pattern , file ) nodes . each do | result | Fast . report ( result , file : file ) end end end And calling the method exploring some \"if\" similarities, it prints the following results: similarities . show \"(if (send (const nil _) _ (lvar _)) nil (return (false)))\" # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/resolv.rb:1248 return false unless Name === other # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/fileutils.rb:138 return false unless File . exist? ( new ) # /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/2.5.0/matrix.rb:1862 return false unless Vector === other","title":"Research for code similarity"},{"location":"sql-support/","text":"SQL Support \u00b6 Fast is partially supporting SQL syntax. Behind the scenes it parses SQL using pg_query and simplifies it to AST Nodes using the same interface. It's using Postgresql parser behind the scenes, but probably could be useful for other SQL similar diallects. fast auto detects SQL files in the command line By default, this module is not included into the main library. Fast can auto-detect file extensions and choose the sql path in case the file relates to sql. Use fast --sql in case you want to force the usage of the SQL parser. ``` ``` Parsing a sql content \u00b6 require 'fast/sql' ast = Fast . parse_sql ( 'select 1' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:val, # s(:integer, # s(:ival, 1)))))))) Why it's interesting to use AST for SQL? \u00b6 Both SQL are available and do the same thing: select * from customers or table customers they have exactly the same objective but written down in very different syntax. Give a try: Fast . parse_sql ( \"select * from customers\" ) == Fast . parse_sql ( \"table customers\" ) # => true Match \u00b6 Use match? with your node pattern to traverse the abstract syntax tree. Fast . match? ( \"(select_stmt ...)\" , ast ) # => true Use $ to capture elements from the AST: Fast . match? ( \"(select_stmt $...)\" , ast ) => [ s ( :target_list , s ( :res_target , s ( :val , s ( :a_const , s ( :val , s ( :integer , s ( :ival , 1 ))))))) ] You can dig deeper into the AST specifying nodes: Fast . match? ( \"(select_stmt (target_list (res_target (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))] And ignoring node types or values using _ . Check all syntax options. Fast . match? ( \"(select_stmt (_ (_ (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))] Search directly from the AST \u00b6 You can also search directly from nodes and keep digging: ast = Fast . parse_sql ( 'select 1' ); ast . search ( 'ival' ) # => [s(:ival, s(:ival, 1))] Use first to return the node directly: ast . first ( '(ival (ival _))' ) #=> s(:ival, s(:ival, 1)) Combine the capture method with $ : ast . capture ( '(ival (ival $_))' ) # => [1] Examples \u00b6 Let's dive into a more complex example capturing fields and from clause of a condition. Let's start parsing the sql: Capturing fields and where clause \u00b6 ast = Fast . parse_sql ( 'select name from customer' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:column_ref, # s(:fields, # s(:string, # s(:str, \"name\"))))))), # s(:from_clause, # s(:range_var, # s(:relname, \"customer\"), # s(:inh, true), # s(:relpersistence, \"p\")))) Now, let's build the expression to get the fields and from_clause. cols_and_from = \" (select_stmt (target_list (res_target (val (column_ref (fields $...))))) (from_clause (range_var $(relname _)))) \" Now, we can use Fast.capture or Fast.match? to extract the values from the AST. Fast . capture ( cols_and_from , ast ) # => [s(:string, # s(:str, \"name\")), s(:relname, \"customer\")] Search inside \u00b6 relname = Fast . parse_sql ( 'select name from customer' ) . search ( 'relname' ) . first # => s(:relname, \"customer\") Find the location of a node. relname . location # => #, # @node=s(:relname, \"customer\")> The location can be useful to allow you to do refactorings and find specific delimitations of objects in the string. The attribute expression gives access to the source range. relname . location . expression # => # The source_buffer is shared and can be accessed through the expression. relname . location . expression . source_buffer # => #, # , # , # ]> The tokens are useful to find the proper node location during the build but they're not available for all the nodes, so, it can be very handy as an extra reference. Replace \u00b6 Replace fragments of your SQL based on AST can also be done with all the work inherited from Parser::TreeRewriter components. Fast . parse_sql ( 'select 1' ) . replace ( 'ival' , '2' ) # => \"select 2\" The previous example is a syntax sugar for the following code: Fast . replace_sql ( 'ival' , Fast . parse_sql ( 'select 1' ), &-> ( node ){ replace ( node . location . expression , '2' ) } ) # => \"select 2\" The last argument is a proc that runs on the parser tree rewriter scope. Let's break down the previous code: ast = Fast . parse_sql ( \"select 1\" ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:ival, # s(:ival, 1))))))) The pattern is simply matching node type that is ival but it could be a complex expression like (val (a_const (val (ival (ival _))))) . Completing the example: Fast . replace_sql ( \"ival\" , ast , &-> ( n ) { replace ( n . loc . expression , \"3\" ) }) # => \"select 3\" loc is a shortcut for location attribute.","title":"SQL Support"},{"location":"sql-support/#sql-support","text":"Fast is partially supporting SQL syntax. Behind the scenes it parses SQL using pg_query and simplifies it to AST Nodes using the same interface. It's using Postgresql parser behind the scenes, but probably could be useful for other SQL similar diallects. fast auto detects SQL files in the command line By default, this module is not included into the main library. Fast can auto-detect file extensions and choose the sql path in case the file relates to sql. Use fast --sql in case you want to force the usage of the SQL parser. ``` ```","title":"SQL Support"},{"location":"sql-support/#parsing-a-sql-content","text":"require 'fast/sql' ast = Fast . parse_sql ( 'select 1' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))))))","title":"Parsing a sql content"},{"location":"sql-support/#why-its-interesting-to-use-ast-for-sql","text":"Both SQL are available and do the same thing: select * from customers or table customers they have exactly the same objective but written down in very different syntax. Give a try: Fast . parse_sql ( \"select * from customers\" ) == Fast . parse_sql ( \"table customers\" ) # => true","title":"Why it's interesting to use AST for SQL?"},{"location":"sql-support/#match","text":"Use match? with your node pattern to traverse the abstract syntax tree. Fast . match? ( \"(select_stmt ...)\" , ast ) # => true Use $ to capture elements from the AST: Fast . match? ( \"(select_stmt $...)\" , ast ) => [ s ( :target_list , s ( :res_target , s ( :val , s ( :a_const , s ( :val , s ( :integer , s ( :ival , 1 ))))))) ] You can dig deeper into the AST specifying nodes: Fast . match? ( \"(select_stmt (target_list (res_target (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))] And ignoring node types or values using _ . Check all syntax options. Fast . match? ( \"(select_stmt (_ (_ (val ($...)))))\" , ast ) # => [s(:a_const, # s(:val, # s(:integer, # s(:ival, 1))))]","title":"Match"},{"location":"sql-support/#search-directly-from-the-ast","text":"You can also search directly from nodes and keep digging: ast = Fast . parse_sql ( 'select 1' ); ast . search ( 'ival' ) # => [s(:ival, s(:ival, 1))] Use first to return the node directly: ast . first ( '(ival (ival _))' ) #=> s(:ival, s(:ival, 1)) Combine the capture method with $ : ast . capture ( '(ival (ival $_))' ) # => [1]","title":"Search directly from the AST"},{"location":"sql-support/#examples","text":"Let's dive into a more complex example capturing fields and from clause of a condition. Let's start parsing the sql:","title":"Examples"},{"location":"sql-support/#capturing-fields-and-where-clause","text":"ast = Fast . parse_sql ( 'select name from customer' ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:column_ref, # s(:fields, # s(:string, # s(:str, \"name\"))))))), # s(:from_clause, # s(:range_var, # s(:relname, \"customer\"), # s(:inh, true), # s(:relpersistence, \"p\")))) Now, let's build the expression to get the fields and from_clause. cols_and_from = \" (select_stmt (target_list (res_target (val (column_ref (fields $...))))) (from_clause (range_var $(relname _)))) \" Now, we can use Fast.capture or Fast.match? to extract the values from the AST. Fast . capture ( cols_and_from , ast ) # => [s(:string, # s(:str, \"name\")), s(:relname, \"customer\")]","title":"Capturing fields and where clause"},{"location":"sql-support/#search-inside","text":"relname = Fast . parse_sql ( 'select name from customer' ) . search ( 'relname' ) . first # => s(:relname, \"customer\") Find the location of a node. relname . location # => #, # @node=s(:relname, \"customer\")> The location can be useful to allow you to do refactorings and find specific delimitations of objects in the string. The attribute expression gives access to the source range. relname . location . expression # => # The source_buffer is shared and can be accessed through the expression. relname . location . expression . source_buffer # => #, # , # , # ]> The tokens are useful to find the proper node location during the build but they're not available for all the nodes, so, it can be very handy as an extra reference.","title":"Search inside"},{"location":"sql-support/#replace","text":"Replace fragments of your SQL based on AST can also be done with all the work inherited from Parser::TreeRewriter components. Fast . parse_sql ( 'select 1' ) . replace ( 'ival' , '2' ) # => \"select 2\" The previous example is a syntax sugar for the following code: Fast . replace_sql ( 'ival' , Fast . parse_sql ( 'select 1' ), &-> ( node ){ replace ( node . location . expression , '2' ) } ) # => \"select 2\" The last argument is a proc that runs on the parser tree rewriter scope. Let's break down the previous code: ast = Fast . parse_sql ( \"select 1\" ) # => s(:select_stmt, # s(:target_list, # s(:res_target, # s(:val, # s(:a_const, # s(:ival, # s(:ival, 1))))))) The pattern is simply matching node type that is ival but it could be a complex expression like (val (a_const (val (ival (ival _))))) . Completing the example: Fast . replace_sql ( \"ival\" , ast , &-> ( n ) { replace ( n . loc . expression , \"3\" ) }) # => \"select 3\" loc is a shortcut for location attribute.","title":"Replace"},{"location":"syntax/","text":"Syntax \u00b6 The syntax is inspired on RuboCop Node Pattern . You can find a great tutorial about RuboCop node pattern in the official documentation . Code example \u00b6 Let's consider the following example.rb code example: class Example ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end end Looking the AST representation we have: $ ruby-parse example.rb (class (const nil :Example) nil (begin (casgn nil :ANSWER (int 42)) (def :magic (args) (send nil :rand (const nil :ANSWER))) (def :duplicate (args (arg :value)) (send (lvar :value) :* (int 2))))) Now, let's explore all details of the current AST, combining with the syntax operators. Fast works with a single word that will be the node type. A simple search of def nodes can be done and will also print the code. $ fast def example.rb # example.rb:3 def magic rand ( ANSWER ) end or check the casgn that will show constant assignments: $ fast casgn example.rb # example.rb:2 ANSWER = 42 () to represent a node search \u00b6 To specify details about a node, the ( means navigate deeply into a node and go deep into the expression. $ fast '(casgn' example.rb # example.rb:2 ANSWER = 42 Fast matcher never checks the end of the expression and close parens are not necessary. We keep them for the sake of specify more node details but the expression works with incomplete parens. $ fast '(casgn)' example.rb # example.rb:2 ANSWER = 42 Closing extra params also don't have a side effect. $ fast '(casgn))' example.rb # example.rb:2 ANSWER = 42 It also automatically flat parens case you put more levels in the beginning. $ fast '((casgn))' example.rb # example.rb:2 ANSWER = 42 For checking AST details while doing some search, you can use --ast in the command line for printing the AST instead of the code: $ fast '((casgn ' example.rb --ast # example.rb:2 ( casgn nil :ANSWER ( int 42 )) _ is something not nil \u00b6 Let's enhance our current expression and specify that we're looking for constant assignments of integers ignoring values and constant names replacing with _ . $ fast '(casgn nil _ (int _))' example.rb # example.rb:2 ANSWER = 42 Keep in mind that _ means not nil and (casgn _ _ (int _)) would not match. Let's search for integer nodes: $ fast int example.rb # example.rb:2 42 # example.rb:7 2 The current search show the nodes but they are not so useful without understand the expression in their context. We need to check their parent . ^ is to get the parent node of an expression \u00b6 By default, Parser::AST::Node does not have access to parent and for accessing it you can say ^ for reaching the parent. $ fast '^int' example.rb # example.rb:2 ANSWER = 42 # example.rb:7 value * 2 And using it multiple times will make the node match from levels up: $ fast '^^int' example.rb # example.rb:2 ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end [] join conditions \u00b6 Let's hunt for integer nodes that the parent is also a method: $ fast '[ ^^int def ]' example.rb The match will filter only nodes that matches all internal expressions. # example.rb:6 def duplicate ( value ) value * 2 end The expression is matching nodes that have a integer granchild and also with type def . ... is a node with children \u00b6 Looking the method representation we have: $ fast def example.rb --ast # example.rb:3 ( def :magic ( args ) ( send nil :rand ( const nil :ANSWER ))) # example.rb:6 ( def :duplicate ( args ( arg :value )) ( send ( lvar :value ) :* ( int 2 ))) And if we want to delimit only methods with arguments: $ fast '(def _ ...)' example.rb # example.rb:6 def duplicate ( value ) value * 2 end If we use (def _ _) instead it will match both methods because (args) does not have children but is not nil. $ is for capture current expression \u00b6 Now, let's say we want to extract some method name from current classes. In such case we don't want to have the node definition but only return the node name. # example.rb:2 def magic rand ( ANSWER ) end # example.rb: magic # example.rb:9 def duplicate ( value ) value * 2 end # example.rb: duplicate One extra method name was printed because of $ is capturing the element. nil matches exactly nil \u00b6 Nil is used in the code as a node type but parser gem also represents empty spaces in expressions with nil. Example, a method call from Kernel is a send from nil calling the method while I can also send a method call from a class. $ ruby-parse -e 'method' (send nil :method) And a method from a object will have the nested target not nil. $ ruby-parse -e 'object.method' (send (send nil :object) :method) Let's build a serch for any calls from nil : $ fast '(_ nil _)' example.rb # example.rb:3 Example # example.rb:4 ANSWER = 42 # example.rb:6 rand ( ANSWER ) Double check the expressions that have matched printing the AST: $ fast '(_ nil _)' example.rb --ast # example.rb:3 ( const nil :Example ) # example.rb:4 ( casgn nil :ANSWER ( int 42 )) # example.rb:6 ( send nil :rand ( const nil :ANSWER )) {} is for any matches like union conditions with or operator \u00b6 Let's say we to add check all occurrencies of the constant ANSWER . We'll need to get both casgn and const node types. For such cases we can surround the expressions with {} and it will return if the node matches with any of the internal expressions. $ fast '({casgn const} nil ANSWER)' example.rb # example.rb:4 ANSWER = 42 # example.rb:6 ANSWER # for custom methods \u00b6 Custom methods can let you into ruby doman for more complicated rules. Let's say we're looking for duplicated methods in the same class. We need to collect method names and guarantee they are unique. def duplicated ( method_name ) @methods ||= [] already_exists = @methods . include? ( method_name ) @methods << method_name already_exists end puts Fast . search_file ( '(def #duplicated)' , 'example.rb' ) The same principle can be used in the node level or for debugging purposes. require 'pry' def debug ( node ) binding . pry end puts Fast . search_file ( '#debug' , 'example.rb' ) If you want to get only def nodes you can also intersect expressions with [] : puts Fast . search_file ( '[ def #debug ]' , 'example.rb' ) Or if you want to debug a very specific expression you can use () to specify more details of the node puts Fast . search_file ( '[ (def a) #debug ]' , 'example.rb' ) . for instance methods \u00b6 You can also call instance methods using . . Example nil is the same of calling nil? and you can also use (int .odd?) to pick only odd integers. The int fragment can also be int_type? . \\1 for first previous capture \u00b6 Imagine you're looking for a method that is just delegating something to another method, like: def name person . name end This can be represented as the following AST: (def :name (args) (send (send nil :person) :name)) Then, let's build a search for methods that calls an attribute with the same name: Fast . match? ( '(def $_ ... (send (send nil _) \\1 ))' , ast ) # => [:name] With the method name being captured with $_ it can be later referenced in the expression with \\1 . If the search contains multiple captures, the \\2 , \\3 can be used as the sequence of captures.","title":"Syntax"},{"location":"syntax/#syntax","text":"The syntax is inspired on RuboCop Node Pattern . You can find a great tutorial about RuboCop node pattern in the official documentation .","title":"Syntax"},{"location":"syntax/#code-example","text":"Let's consider the following example.rb code example: class Example ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end end Looking the AST representation we have: $ ruby-parse example.rb (class (const nil :Example) nil (begin (casgn nil :ANSWER (int 42)) (def :magic (args) (send nil :rand (const nil :ANSWER))) (def :duplicate (args (arg :value)) (send (lvar :value) :* (int 2))))) Now, let's explore all details of the current AST, combining with the syntax operators. Fast works with a single word that will be the node type. A simple search of def nodes can be done and will also print the code. $ fast def example.rb # example.rb:3 def magic rand ( ANSWER ) end or check the casgn that will show constant assignments: $ fast casgn example.rb # example.rb:2 ANSWER = 42","title":"Code example"},{"location":"syntax/#to-represent-a-node-search","text":"To specify details about a node, the ( means navigate deeply into a node and go deep into the expression. $ fast '(casgn' example.rb # example.rb:2 ANSWER = 42 Fast matcher never checks the end of the expression and close parens are not necessary. We keep them for the sake of specify more node details but the expression works with incomplete parens. $ fast '(casgn)' example.rb # example.rb:2 ANSWER = 42 Closing extra params also don't have a side effect. $ fast '(casgn))' example.rb # example.rb:2 ANSWER = 42 It also automatically flat parens case you put more levels in the beginning. $ fast '((casgn))' example.rb # example.rb:2 ANSWER = 42 For checking AST details while doing some search, you can use --ast in the command line for printing the AST instead of the code: $ fast '((casgn ' example.rb --ast # example.rb:2 ( casgn nil :ANSWER ( int 42 ))","title":"() to represent a node search"},{"location":"syntax/#_-is-something-not-nil","text":"Let's enhance our current expression and specify that we're looking for constant assignments of integers ignoring values and constant names replacing with _ . $ fast '(casgn nil _ (int _))' example.rb # example.rb:2 ANSWER = 42 Keep in mind that _ means not nil and (casgn _ _ (int _)) would not match. Let's search for integer nodes: $ fast int example.rb # example.rb:2 42 # example.rb:7 2 The current search show the nodes but they are not so useful without understand the expression in their context. We need to check their parent .","title":"_ is something not nil"},{"location":"syntax/#is-to-get-the-parent-node-of-an-expression","text":"By default, Parser::AST::Node does not have access to parent and for accessing it you can say ^ for reaching the parent. $ fast '^int' example.rb # example.rb:2 ANSWER = 42 # example.rb:7 value * 2 And using it multiple times will make the node match from levels up: $ fast '^^int' example.rb # example.rb:2 ANSWER = 42 def magic rand ( ANSWER ) end def duplicate ( value ) value * 2 end","title":"^ is to get the parent node of an expression"},{"location":"syntax/#join-conditions","text":"Let's hunt for integer nodes that the parent is also a method: $ fast '[ ^^int def ]' example.rb The match will filter only nodes that matches all internal expressions. # example.rb:6 def duplicate ( value ) value * 2 end The expression is matching nodes that have a integer granchild and also with type def .","title":"[] join conditions"},{"location":"syntax/#is-a-node-with-children","text":"Looking the method representation we have: $ fast def example.rb --ast # example.rb:3 ( def :magic ( args ) ( send nil :rand ( const nil :ANSWER ))) # example.rb:6 ( def :duplicate ( args ( arg :value )) ( send ( lvar :value ) :* ( int 2 ))) And if we want to delimit only methods with arguments: $ fast '(def _ ...)' example.rb # example.rb:6 def duplicate ( value ) value * 2 end If we use (def _ _) instead it will match both methods because (args) does not have children but is not nil.","title":"... is a node with children"},{"location":"syntax/#is-for-capture-current-expression","text":"Now, let's say we want to extract some method name from current classes. In such case we don't want to have the node definition but only return the node name. # example.rb:2 def magic rand ( ANSWER ) end # example.rb: magic # example.rb:9 def duplicate ( value ) value * 2 end # example.rb: duplicate One extra method name was printed because of $ is capturing the element.","title":"$ is for capture current expression"},{"location":"syntax/#nil-matches-exactly-nil","text":"Nil is used in the code as a node type but parser gem also represents empty spaces in expressions with nil. Example, a method call from Kernel is a send from nil calling the method while I can also send a method call from a class. $ ruby-parse -e 'method' (send nil :method) And a method from a object will have the nested target not nil. $ ruby-parse -e 'object.method' (send (send nil :object) :method) Let's build a serch for any calls from nil : $ fast '(_ nil _)' example.rb # example.rb:3 Example # example.rb:4 ANSWER = 42 # example.rb:6 rand ( ANSWER ) Double check the expressions that have matched printing the AST: $ fast '(_ nil _)' example.rb --ast # example.rb:3 ( const nil :Example ) # example.rb:4 ( casgn nil :ANSWER ( int 42 )) # example.rb:6 ( send nil :rand ( const nil :ANSWER ))","title":"nil matches exactly nil"},{"location":"syntax/#is-for-any-matches-like-union-conditions-with-or-operator","text":"Let's say we to add check all occurrencies of the constant ANSWER . We'll need to get both casgn and const node types. For such cases we can surround the expressions with {} and it will return if the node matches with any of the internal expressions. $ fast '({casgn const} nil ANSWER)' example.rb # example.rb:4 ANSWER = 42 # example.rb:6 ANSWER","title":"{} is for any matches like union conditions with or operator"},{"location":"syntax/#for-custom-methods","text":"Custom methods can let you into ruby doman for more complicated rules. Let's say we're looking for duplicated methods in the same class. We need to collect method names and guarantee they are unique. def duplicated ( method_name ) @methods ||= [] already_exists = @methods . include? ( method_name ) @methods << method_name already_exists end puts Fast . search_file ( '(def #duplicated)' , 'example.rb' ) The same principle can be used in the node level or for debugging purposes. require 'pry' def debug ( node ) binding . pry end puts Fast . search_file ( '#debug' , 'example.rb' ) If you want to get only def nodes you can also intersect expressions with [] : puts Fast . search_file ( '[ def #debug ]' , 'example.rb' ) Or if you want to debug a very specific expression you can use () to specify more details of the node puts Fast . search_file ( '[ (def a) #debug ]' , 'example.rb' )","title":"# for custom methods"},{"location":"syntax/#for-instance-methods","text":"You can also call instance methods using . . Example nil is the same of calling nil? and you can also use (int .odd?) to pick only odd integers. The int fragment can also be int_type? .","title":". for instance methods"},{"location":"syntax/#1-for-first-previous-capture","text":"Imagine you're looking for a method that is just delegating something to another method, like: def name person . name end This can be represented as the following AST: (def :name (args) (send (send nil :person) :name)) Then, let's build a search for methods that calls an attribute with the same name: Fast . match? ( '(def $_ ... (send (send nil _) \\1 ))' , ast ) # => [:name] With the method name being captured with $_ it can be later referenced in the expression with \\1 . If the search contains multiple captures, the \\2 , \\3 can be used as the sequence of captures.","title":"\\1 for first previous capture"},{"location":"videos/","text":"Videos \u00b6 Ruby Kaigi TakeOut 2020: Grepping Ruby code like a boss Also, similar livecoding session at RubyConf Brazil 2019 (Portuguese) . Introduction to inline code . Making local variables inline Making methods inline","title":"Videos"},{"location":"videos/#videos","text":"Ruby Kaigi TakeOut 2020: Grepping Ruby code like a boss Also, similar livecoding session at RubyConf Brazil 2019 (Portuguese) . Introduction to inline code . Making local variables inline Making methods inline","title":"Videos"},{"location":"walkthrough/","text":"Fast walkthrough \u00b6 This is the main interactive tutorial we have on fast . If you're reading it on the web, please consider also try it in the command line: fast .intro in the terminal to get a rapid pace on reading and testing on your own computer. The objective here is give you some insights about how to use ffast gem in the command line. Let's start finding the main fast.rb file for the fast library: $ gem which fast And now, let's combine the previous expression that returns the path to the file and take a quick look into the methods match? in the file using a regular grep: $ grep \"def match\\?\" $(gem which fast) Boring results, no? The code here is not easy to digest because we just see a fragment of the code block that we want. Let's make it a bit more advanced with grep -rn to file name and line number: $ grep -rn \"def match\\?\" $(gem which fast) Still hard to understand the scope of the search. That's why fast exists! Now, let's take a look on how a method like this looks like from the AST perspective. Let's use ruby-parse for it: $ ruby-parse -e \"def match?(node); end\" Now, let's make the same search with fast node pattern: fast \"(def match?)\" $(gem which fast) Wow! in this case you got all the match? methods, but you'd like to go one level upper and understand the classes that implements the method with a single node as argument. Let's first use ^ to jump into the parent: fast \"^(def match?)\" $(gem which fast) As you can see it still prints some match? methods that are not the ones that we want, so, let's add a filter by the argument node (args (arg node)) : fast \"(def match? (args (arg node)))\" $(gem which fast) Now, it looks closer to have some understanding of the scope, filtering only methods that have the name match? and receive node as a parameter. Now, let's do something different and find all methods that receives a node as an argument: fast \"(def _ (args (arg node)))\" $(gem which fast) Looks like almost all of them are the match? and we can also skip the match? methods negating the expression prefixing with ! : fast \"(def !match? (args (arg node)))\" $(gem which fast) Let's move on and learn more about node pattern with the RuboCop project: $ VISUAL=echo gem open rubocop RuboCop contains def_node_matcher and def_node_search . Let's make a search for both method names wrapping the query with {} selector: fast \"(send nil {def_node_matcher def_node_search})\" $(VISUAL=echo gem open rubocop) As you can see, node pattern is widely adopted in the cops to target code. Rubocop contains a few projects with dedicated cops that can help you learn more. How to automate refactor using AST \u00b6 Moving towards to the code automation, the next step after finding some target code is refactor and change the code behavior. Let's imagine that we already found some code that we want to edit or remove. If we get the AST we can also cherry-pick any fragment of the expression to be replaced. As you can imagine, RuboCop also benefits from automatic refactoring offering the --autocorrect option. All the hardcore algorithms are in the parser rewriter, but we can find a lot of great examples on RuboCop project searching for the autocorrect method. fast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop) Look that most part of the methods are just returning a lambda with a corrector. Now, let's use the --ast to get familiar with the tree details for the implementation: fast --ast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop)/lib/rubocop/cop/style As we can see, we have a (send (lvar corrector)) that is the interface that we can get the most interesting calls to overwrite files: fast \"(send (lvar corrector)\" $(VISUAL=echo gem open rubocop) That is all for now! \u00b6 I hope you enjoyed to learn by example searching with fast. If you like it, please star the project ! You can also build your own tutorials simply using markdown files like I did here, you can find this tutorial here .","title":"Walkthrough"},{"location":"walkthrough/#fast-walkthrough","text":"This is the main interactive tutorial we have on fast . If you're reading it on the web, please consider also try it in the command line: fast .intro in the terminal to get a rapid pace on reading and testing on your own computer. The objective here is give you some insights about how to use ffast gem in the command line. Let's start finding the main fast.rb file for the fast library: $ gem which fast And now, let's combine the previous expression that returns the path to the file and take a quick look into the methods match? in the file using a regular grep: $ grep \"def match\\?\" $(gem which fast) Boring results, no? The code here is not easy to digest because we just see a fragment of the code block that we want. Let's make it a bit more advanced with grep -rn to file name and line number: $ grep -rn \"def match\\?\" $(gem which fast) Still hard to understand the scope of the search. That's why fast exists! Now, let's take a look on how a method like this looks like from the AST perspective. Let's use ruby-parse for it: $ ruby-parse -e \"def match?(node); end\" Now, let's make the same search with fast node pattern: fast \"(def match?)\" $(gem which fast) Wow! in this case you got all the match? methods, but you'd like to go one level upper and understand the classes that implements the method with a single node as argument. Let's first use ^ to jump into the parent: fast \"^(def match?)\" $(gem which fast) As you can see it still prints some match? methods that are not the ones that we want, so, let's add a filter by the argument node (args (arg node)) : fast \"(def match? (args (arg node)))\" $(gem which fast) Now, it looks closer to have some understanding of the scope, filtering only methods that have the name match? and receive node as a parameter. Now, let's do something different and find all methods that receives a node as an argument: fast \"(def _ (args (arg node)))\" $(gem which fast) Looks like almost all of them are the match? and we can also skip the match? methods negating the expression prefixing with ! : fast \"(def !match? (args (arg node)))\" $(gem which fast) Let's move on and learn more about node pattern with the RuboCop project: $ VISUAL=echo gem open rubocop RuboCop contains def_node_matcher and def_node_search . Let's make a search for both method names wrapping the query with {} selector: fast \"(send nil {def_node_matcher def_node_search})\" $(VISUAL=echo gem open rubocop) As you can see, node pattern is widely adopted in the cops to target code. Rubocop contains a few projects with dedicated cops that can help you learn more.","title":"Fast walkthrough"},{"location":"walkthrough/#how-to-automate-refactor-using-ast","text":"Moving towards to the code automation, the next step after finding some target code is refactor and change the code behavior. Let's imagine that we already found some code that we want to edit or remove. If we get the AST we can also cherry-pick any fragment of the expression to be replaced. As you can imagine, RuboCop also benefits from automatic refactoring offering the --autocorrect option. All the hardcore algorithms are in the parser rewriter, but we can find a lot of great examples on RuboCop project searching for the autocorrect method. fast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop) Look that most part of the methods are just returning a lambda with a corrector. Now, let's use the --ast to get familiar with the tree details for the implementation: fast --ast \"(def autocorrect)\" $(VISUAL=echo gem open rubocop)/lib/rubocop/cop/style As we can see, we have a (send (lvar corrector)) that is the interface that we can get the most interesting calls to overwrite files: fast \"(send (lvar corrector)\" $(VISUAL=echo gem open rubocop)","title":"How to automate refactor using AST"},{"location":"walkthrough/#that-is-all-for-now","text":"I hope you enjoyed to learn by example searching with fast. If you like it, please star the project ! You can also build your own tutorials simply using markdown files like I did here, you can find this tutorial here .","title":"That is all for now!"}]} \ No newline at end of file diff --git a/sitemap.xml b/sitemap.xml index eb5b48c..4c52411 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,72 +2,72 @@ None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily None - 2023-11-10 + 2023-11-17 daily \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz index 9d712e5353cfe9a0374a00b4f69720691e4dc7fb..ba1432f6290f418373a6b9cd6f26d7ed270ce7bd 100644 GIT binary patch literal 208 zcmV;>05AU^iwFp_>Q`j~|8r?{Wo=<_E_iKh0PU4M62c%5h5MX>p$~*u_%nobmR5QI zhG4=Bfe2{y_L84*asW%V*|%@s>{l+W-@OJe>AbS0f&>*1NF!{cOk2U*^_(AI%`*CQ z+RFf9*!T)|Aq*#s$8khn1=;h#BkNl1MHdH;R}&ZL2w4VVO$Qb=7u8m|Mi4Qtk^%3rbRC^)@lt3EQf>R}bC@Z9WWdoP~&g$PCZ$4A1Zk&+xy&=azmV{{nJ& J>rq1o007bxTuT4| diff --git a/sql-support/index.html b/sql-support/index.html index 1f60e19..2798d1e 100644 --- a/sql-support/index.html +++ b/sql-support/index.html @@ -458,17 +458,20 @@

SQL Support

Fast is partially supporting SQL syntax. Behind the scenes it parses SQL using pg_query and simplifies it to AST Nodes -using the same Ruby interface. It's using Postgresql parser behind the scenes, -but probably could be useful for other SQL similar diallects .

-

The plan is that Fast would auto-detect file extensions and choose the sql path -in case the file relates to sql.

-

By default, this module is not included into the main library as it still very -experimental.

-
require 'fast/sql'
-
- +using the same interface. It's using Postgresql parser behind the scenes, +but probably could be useful for other SQL similar diallects.

+
+

fast auto detects SQL files in the command line

+

By default, this module is not included into the main library. +Fast can auto-detect file extensions and choose the sql path in case the +file relates to sql.

+

Use fast --sql in case you want to force the usage of the SQL parser.

+

```

+

```

+

Parsing a sql content

-
ast = Fast.parse_sql('select 1')
+
require 'fast/sql'
+ast = Fast.parse_sql('select 1')
 # => s(:select_stmt,
 #     s(:target_list,
 #       s(:res_target,
@@ -479,6 +482,21 @@ 
Parsing a sql content#                 s(:ival, 1))))))))
 

 
+
Why it's interesting to use AST for SQL?

+
Both SQL are available and do the same thing:

+
select * from customers
+

+
+
or

+
table customers
+

+
+
they have exactly the same objective but written down in very different syntax.

+
Give a try:

+
Fast.parse_sql("select * from customers") == Fast.parse_sql("table customers") # => true
+

+
+
Match

 
Use match? with your node pattern to traverse the abstract syntax tree.

 
 Fast.match?("(select_stmt ...)", ast) # => true
 

@@ -510,10 +528,24 @@ 
Parsing a sql content#         s(:ival, 1))))]
+

Search directly from the AST

+

You can also search directly from nodes and keep digging:

+
ast = Fast.parse_sql('select 1');
+ast.search('ival') # => [s(:ival, s(:ival, 1))]
+
+ +

Use first to return the node directly:

+
ast.first('(ival (ival _))')  #=> s(:ival, s(:ival, 1))
+
+ +

Combine the capture method with $:

+
ast.capture('(ival (ival $_))') # => [1]
+
+

Examples

-

Capturing fields and where clause

Let's dive into a more complex example capturing fields and from clause of a condition. Let's start parsing the sql:

+

Capturing fields and where clause

ast = Fast.parse_sql('select name from customer')
 #   => s(:select_stmt,
 #     s(:target_list,
@@ -599,13 +631,12 @@ 
Replace#      s(:res_target,
 #        s(:val,
 #          s(:a_const,
-#            s(:val,
-#              s(:integer,
-#                s(:ival, 1))))))))
+#            s(:ival,
+#              s(:ival, 1)))))))

The pattern is simply matching node type that is ival but it could be a complex expression -like (val (a_const (val (integer (ival _))))).

+like (val (a_const (val (ival (ival _))))).

Completing the example:

 Fast.replace_sql("ival", ast, &-> (n) { replace(n.loc.expression, "3") })
  # => "select 3"
diff --git a/stylesheets/custom.css b/stylesheets/custom.css
index 9cda3be..a6d4bbd 100644
--- a/stylesheets/custom.css
+++ b/stylesheets/custom.css
@@ -1,6 +1,6 @@
 :root {
   --md-primary-fg-color: #355E3B; /* Primary text color to match the green in the logo */
-  --md-primary-bg-color: #FAFAFA; /* Light background, can be adjusted to your preference */
+  --md-primary-bg-color: #FAFAFA; /* Light background */
   --md-accent-fg-color: #58A55C; /* Accent color for buttons and other elements */
   --md-accent-bg-color: #E8F5E9; /* Light accent background, for elements like badges or tags */
 }