Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filter2 Cannot transition token types from STRING to VARIABLE caused by single quotes #174

Closed
tolot27 opened this issue Nov 17, 2021 · 5 comments

Comments

@tolot27
Copy link

tolot27 commented Nov 17, 2021

I'm using csvtk v0.23 and get an error message if I try to filter against a string containing single quotes.

Example:

echo -e "Organism\nHuman 'child'" | csvtk -t filter2 -f '$Organism=="test"'
[ERRO] Cannot transition token types from STRING [Human ] to VARIABLE [child]

The lazy quotes parameter does not help in this case. Single quoting should not matter for tab-separated input and also not for csv because it uses double quotes.

@shenwei356
Copy link
Owner

Fixed. I'm tired of adding patches for these two commands since https://github.com/Knetic/govaluate is not active anymore, I will soon replace it with https://github.com/antonmedv/expr. #172

$ echo -e "Organism\nHuman 'child'\nHuman \"child\"\nHuman 'child\nHuman \"child\ntest\n\"test\"\n'test'\n"
Human 'child'
Human "child"
Human 'child
Human "child
test
"test"
'test'

$ echo -e "Organism\nHuman 'child'\nHuman \"child\"\nHuman 'child\nHuman \"child\ntest\n\"test\"\n'test'\n" \
    | csvtk -t filter2 -f '$Organism == "test"' -l
Organism
test
test

@tolot27
Copy link
Author

tolot27 commented Nov 18, 2021

Many thanks!

@tolot27 tolot27 closed this as completed Nov 18, 2021
@tolot27
Copy link
Author

tolot27 commented Nov 19, 2021

Unfortunately, the new build have not the same flags for "mutate2" anymore. For instance, there is no flag -n to specify a name for the column to create.

Flags of previous build:

  -L, --digits int          number of digits after the decimal dot (default 2)
  -e, --expression string   arithmetic/string expressions. e.g. "'string'", '"abc"', ' $a + "-" + $b ', '$1 + $2', '$a / $b', ' $1 > 100 ? "big" : "small" '
  -h, --help                help for mutate2
  -n, --name string         new column name
  -s, --numeric-as-string   treat even numeric fields as strings to avoid converting big numbers into scientific notation

Flags of new build:

  -w, --decimal-width int   limit floats to N decimal points (default 2)
  -L, --digits int          number of digits after the decimal dot (default 2)
  -e, --expression string   arithmetic/string expressions. e.g. "'string'", '"abc"', ' $a + "-" + $b ', '$1 + $2', '$a / $b', ' $1 > 100 ? "big" : "small" '
  -h, --help                help for mutate2
  -s, --numeric-as-string   treat even numeric fields as strings to avoid converting big numbers into scientific notation

-w is similar to -L.

@tolot27 tolot27 reopened this Nov 19, 2021
@shenwei356
Copy link
Owner

@tolot27
Copy link
Author

tolot27 commented Nov 19, 2021

Many thanks for the hot fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants