Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

assets:precomile fails, but deploy succeeds #719

Open
PeterVenhuizen opened this issue Mar 2, 2023 · 1 comment
Open

assets:precomile fails, but deploy succeeds #719

PeterVenhuizen opened this issue Mar 2, 2023 · 1 comment

Comments

@PeterVenhuizen
Copy link

The invoke :'rails:assets_precompile' step in my deploy block fails. I know why it fails, because some of the config files are formatted incorrectly. This issue is that there is a rake aborted! message + the SyntaxError, but instead of the GitLab CI deploy job stopping there, mina just moves on to the next task.

The output below is the full output of the deploy job, with certain pieces of information [REDACTED].

How does mina listen for errors when running tasks and why wasn't the error picked up here?

Hope someone can help out with this, since this forms quite a risk for the stability of our production environment.

Thanks!

Running with gitlab-runner 15.8.2 (4d1ca121)
  on [REDACTED]
Preparing the "docker" executor 00:03
Using Docker executor with image [REDACTED] ...
Authenticating with credentials from /root/.docker/config.json
Pulling docker image [REDACTED] ...
Using docker image [REDACTED] for [REDACTED] with digest [REDACTED][REDACTED] ...
Preparing environment 00:01
Running on [REDACTED] via [REDACTED]...
Getting source from Git repository 00:01
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
Executing "step_script" stage of the job script 01:52
Using docker image [REDACTED] for [REDACTED] with digest [REDACTED][REDACTED] ...
$ cd /app
$ . bin/keytab_auth
Generate keytab ticket...
$ mina -f config/deploy.rb $STAGE deploy
! !  Unable to check latest version  ! !
Warning: Permanently added '[REDACTED]' (ECDSA) to the list of known hosts.
-----> Creating a temporary build path
-----> Reading master key
-----> Using git commit "18b60e30"
Cloning into '.'...
remote: Enumerating objects: 9137, done.        
remote: Counting objects: 100% (380/380), done.        
remote: Compressing objects: 100% (19/19), done.        
remote: Total 9137 (delta 369), reused 362 (delta 359), pack-reused 8757        
Receiving objects: 100% (9137/9137), 1.83 MiB | 0 bytes/s, done.
Resolving deltas: 100% (6742/6742), done.
Switched to a new branch 'current_release'
-----> Updating submodules
-----> Using this git commit
Peter Venhuizen (18b60e3):
> chore: test change init_command
-----> Symlinking shared paths
-----> Generate config files from vault
-----> Successfully created /app/config/auth.yml
-----> Successfully created /app/config/database.yml
-----> Successfully created /app/config/mw.yml
-----> Successfully created /app/config/nr.yml
-----> Successfully created /app/config/nr3.yml
-----> Successfully created /app/config/service_availability_database.yml
-----> Successfully created /app/config/sidekiq_redis.yml
-----> Installing gem dependencies using Bundler
[REDACTED]
  -----> DB migrations unchanged; skipping DB migration
-----> Precompiling asset files
Rails couldn't infer whether you are using multiple databases from your database.yml and can't generate the tasks for the non-primary databases. If you'd like to use this feature, please simplify your ERB.
rake aborted!
Psych::SyntaxError: (<unknown>): did not find expected key while parsing a block mapping at line 1 column 1
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/psych-3.3.4/lib/psych.rb:457:in `parse'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/psych-3.3.4/lib/psych.rb:457:in `parse_stream'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/psych-3.3.4/lib/psych.rb:391:in `parse'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/psych-3.3.4/lib/psych.rb:351:in `safe_load'
/[REDACTED]/config/initializers/service_availability_database.rb:5:in `<top (required)>'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/engine.rb:667:in `load'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/engine.rb:667:in `block in load_config_initializer'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.4.2/lib/active_support/notifications.rb:208:in `instrument'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/engine.rb:666:in `load_config_initializer'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/engine.rb:620:in `block (2 levels) in <class:Engine>'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/engine.rb:619:in `each'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/engine.rb:619:in `block in <class:Engine>'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/initializable.rb:32:in `instance_exec'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/initializable.rb:32:in `run'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/initializable.rb:61:in `block in run_initializers'
/data/ruby/installed/3.1.0/tsort.rb:228:in `block in tsort_each'
/data/ruby/installed/3.1.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
/data/ruby/installed/3.1.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
/data/ruby/installed/3.1.0/tsort.rb:431:in `each_strongly_connected_component_from'
/data/ruby/installed/3.1.0/tsort.rb:421:in `block in each_strongly_connected_component_from'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/initializable.rb:50:in `each'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/initializable.rb:50:in `tsort_each_child'
/data/ruby/installed/3.1.0/tsort.rb:415:in `call'
/data/ruby/installed/3.1.0/tsort.rb:415:in `each_strongly_connected_component_from'
/data/ruby/installed/3.1.0/tsort.rb:349:in `block in each_strongly_connected_component'
/data/ruby/installed/3.1.0/tsort.rb:347:in `each'
/data/ruby/installed/3.1.0/tsort.rb:347:in `call'
/data/ruby/installed/3.1.0/tsort.rb:347:in `each_strongly_connected_component'
/data/ruby/installed/3.1.0/tsort.rb:226:in `tsort_each'
/data/ruby/installed/3.1.0/tsort.rb:205:in `tsort_each'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/initializable.rb:60:in `run_initializers'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/application.rb:372:in `initialize!'
/[REDACTED]/config/environment.rb:48:in `<top (required)>'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/zeitwerk-2.6.7/lib/zeitwerk/kernel.rb:38:in `require'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/zeitwerk-2.6.7/lib/zeitwerk/kernel.rb:38:in `require'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/application.rb:348:in `require_environment!'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/railties-7.0.4.2/lib/rails/application.rb:511:in `block in run_tasks_blocks'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/sentry-ruby-5.8.0/lib/sentry/rake.rb:24:in `execute'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/sprockets-rails-3.4.2/lib/sprockets/rails/task.rb:61:in `block (2 levels) in define'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/sentry-ruby-5.8.0/lib/sentry/rake.rb:24:in `execute'
/[REDACTED]/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/cli/exec.rb:58:in `load'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/cli/exec.rb:58:in `kernel_load'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/cli/exec.rb:23:in `run'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/cli.rb:484:in `exec'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/cli.rb:31:in `dispatch'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/cli.rb:25:in `start'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/exe/bundle:48:in `block in <top (required)>'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/data/ruby/installed/gems/3.1.0/gems/bundler-2.3.7/exe/bundle:36:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => environment
(See full trace by running task with --trace)
-----> Cleaning up the auto deployment dependancies
-----> Deleting the dockerfiles directory
-----> Deploy finished
-----> Building
-----> Moving build to [REDACTED]
-----> Build finished
-----> Launching
-----> Updating the [REDACTED] symlink
-----> Restarting background process: [REDACTED]
Reinitializing monit daemon
[REDACTED]
-----> Done. Deployed version 106
Connection to [REDACTED] closed.
       Elapsed time: 91.16 seconds
-----> install sentry if not installed
       /usr/bin/sentry-cli
       
       changed 13 packages, and audited 14 packages in 6s
       
       found 0 vulnerabilities
-----> make root the owner
-----> propose sentry release version
-----> make sentry release
       Created release [REDACTED]
       Success! Set commits for release [REDACTED]
       Finalized release [REDACTED]
       Elapsed time: 18.96 seconds
Cleaning up project directory and file based variables 00:02
Job succeeded
@reneweteling
Copy link

Hi @PeterVenhuizen,

So as you know I've worked on this issue, for some reason exit codes get ignored when you write multiline commands.

# When writing it like this, it does gets executed but starts ignoring error codes, so the deploy will continue even if something fails
command %(
  if [ -d #{fetch(:deploy_to)}/fake_current ]; then
    echo "-----> Deleting dummy app structure"
    rm -rf #{fetch(:deploy_to)}/fake_current
  fi
  if [ -f ~/.netrc ]; then
    echo "-----> Cleaning up the auto deployment dependancies"
    sed -i.bk '/machine nms-git/d' ~/.netrc
    if [ -f ~/.netrc.bk ]; then
      rm -fr ~/.netrc.bk
    fi
  fi
  if [ -d #{fetch(:current_path)}/dockerfiles ]; then
    echo "-----> Deleting the dockerfiles directory"
rm -fr #{fetch(:current_path)}/dockerfiles
  fi
)
# This respects the error codes, 2 things ive noticed. 
# 1 - make sure you end each line with a ;
# 2 - one if per command, nested seem te be ok :S
command %(
  if [ -d #{fetch(:deploy_to)}/fake_current ]; then
    echo "-----> Deleting dummy app structure";
    rm -rf #{fetch(:deploy_to)}/fake_current;
  fi
)
command %(
  if [ -f ~/.netrc ]; then
    echo "-----> Cleaning up the auto deployment dependancies";
    sed -i.bk '/machine nms-git/d' ~/.netrc;
    if [ -f ~/.netrc.bk ]; then
      rm -fr ~/.netrc.bk;
    fi
  fi
)
command %(
  if [ -d #{fetch(:current_path)}/dockerfiles ]; then
    echo "-----> Deleting the dockerfiles directory";
    rm -fr #{fetch(:current_path)}/dockerfiles;
  fi
)

So I've mittegated this by putting 1 command in the command method. This seems to work. Why and How... no clueue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants