Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experiment with shelling out to parse files #474

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Conversation

bkeepers
Copy link
Owner

@bkeepers bkeepers commented Jan 23, 2024

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"
default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"
changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'
default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.


TODO:

  • Solve conditional assignment problem
  • Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
  • Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

@bkeepers bkeepers marked this pull request as draft January 23, 2024 19:58
@bkeepers bkeepers changed the base branch from master to main January 25, 2024 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant