only honor IFS for splitting portion of word that had expansions

[grisenti]

The POSIX standard specifies that field splitting should be done before quote removal, so in theory the following

IFS=\'
echo a'b'c

should print a b c. Although it looks like bash doesn't do this even for "simpler" cases, like

IFS=:
echo a:b:c

outputs a:b:c. But it works if we do

x=a:b:c
IFS=:
echo $x

which outputs a b c.

[reubeno]

Interesting observation! Note that the bash doc page on word splitting specifically notes:

Note that if no expansion occurs, no splitting is performed.

I tested with a somewhat recent bash and dash and found similar results. brush looks to behave the same. While experimenting, though, I did note that the following produces different results between bash and brush:

IFS=:
x=a:b:c
echo 0:$x:1

In both dash and bash, the displayed text is:

0:a b c:1

In brush, however, we see:

0 a b c 1

That is, brush appears to do more word splitting. As for your case of IFS=\', I do see that both bash and brush yield the following when expansion does occur:

$ IFS=\'
$ x=a\'b\'c
$ echo $x
a b c

So... with all that said, I see one case here where brush diverges from the behavior of other real-world shells (and bash, in particular); we can use this issue to track that--to make sure we add a test to cover it and work on resolving it.

[grisenti]

interesting, I didn't see that part on the bash docs. Also I checked the POSIX spec again and it says

After parameter expansion, command substitution, and arithmetic expansion, the shell shall scan the results of expansions and substitutions that did not occur in double-quotes for field splitting and multiple fields can result.

which I guess sounds like what bash is doing, I interpreted it differently the first time I read it though.

Also, the POSIX spec says that quote removal is done only on the characters that were present in the original word so its definitely correct to do it during tokenization