heroku-buildpack-php newrelic segfaults?

[fruitl00p]

Using the latest version of the official Heroku buildpack for PHP and setting the correct environment settings for New Relic (and thus loading the New Relic extension) the resulting app fails to boot / run. And simply dies with a segfault... ?

Since i assume Heroku to still work with New Relic enabled, i suspect it must be something and was redirected here by my colleagues. Any tips?

to reproduce:

# get the heroku PHP example app and store it in the /tmp
git clone https://github.com/heroku/php-getting-started.git /tmp/app
# set the buildpack just to be sure
echo "export BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-php.git#v66" > /tmp/app/.env
# clean up git just to be sure
rm -rfR /tmp/app/.git
# re add a single remote Dokku(alt) host
cd /tmp/app
git remote add dokku dokku@dokku-host:php-examle
# remove the .gitconfig to enable the push of .env file
rm .gitconfig 
# add everything + commit 
git add . && git commit -a -m "initial commit"
# push it to the dokku(alt) host
git push --set-upstream app master:master

This will deploy without newrelic as can be tested. This works so far ;)

now set the correct config to enable the loading of the newrelic .so (as described here)

dokku config:set php-examle NEW_RELIC_APP_NAME="test-php-app" NEW_RELIC_LICENSE_KEY="insert your licensekey here"
dokku rebuild php-example

Now hitting the php-example will report something like:

13:56:55 web.1  | started with pid 13
13:56:56 web.1  | DOCUMENT_ROOT changed to 'web/'
13:56:56 web.1  | 4 processes at 128MB memory limit.
13:56:56 web.1  | Starting php-fpm...
13:56:56        | exited with code 1
13:56:56 system | sending SIGTERM to all processes
13:56:58 web.1  | Going down, terminating child processes...
13:56:58 web.1  | exited with code 0

In other words: crash!

[progrium]

Nothing obvious except maybe in that new relic seems to be described as run
as a separate daemon. Not sure how that interacts with everything else.
Curious if you can find better ways to get debug information from new relic
or the buildpack.

On Mon, Mar 23, 2015 at 9:06 AM, Robin Speekenbrink <
[email protected]> wrote:

Using the latest version of the official Heroku buildpack for PHP and
setting the correct environment settings for New Relic (and thus loading
the New Relic extension) the resulting app fails to boot / run. And simply
dies with a segfault... ?

Since i assume Heroku to still work with New Relic enabled, i suspect it
must be something and was redirected here by my colleagues. Any tips?

to reproduce:

get the heroku PHP example app and store it in the /tmp

git clone https://github.com/heroku/php-getting-started.git /tmp/app# set the buildpack just to be sureecho "export BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-php.git#v66" > /tmp/app/.env# clean up git just to be sure
rm -rfR /tmp/app/.git# re add a single remote Dokku(alt) hostcd /tmp/app
git remote add dokku dokku@dokku-host:php-examle# remove the .gitconfig to enable the push of .env file
rm .gitconfig # add everything + commit
git add . && git commit -a -m "initial commit"# push it to the dokku(alt) host
git push --set-upstream app master:master

This will deploy without newrelic as can be tested. This works so far ;)

now set the correct config to enable the loading of the newrelic .so (as described
here
https://github.com/heroku/heroku-buildpack-php/blob/master/bin/util/newrelic.sh
)

dokku config:set php-examle NEW_RELIC_APP_NAME="test-php-app" NEW_RELIC_LICENSE_KEY="insert your licensekey here"
dokku rebuild php-example

Now hitting the php-example will report something like:

13:56:55 web.1 | started with pid 13
13:56:56 web.1 | DOCUMENT_ROOT changed to 'web/'
13:56:56 web.1 | 4 processes at 128MB memory limit.
13:56:56 web.1 | Starting php-fpm...
13:56:56 | exited with code 1
13:56:56 system | sending SIGTERM to all processes
13:56:58 web.1 | Going down, terminating child processes...
13:56:58 web.1 | exited with code 0

In other words: crash!

—
Reply to this email directly or view it on GitHub
#143.

Jeff Lindsay
http://progrium.com

[fruitl00p]

Jeff,

Haven't been able to get more debug info than this, but you're right that the newrelic agent does launch an extra process which pushes the gathered data to NR itself. But that extra process shouldnt kill the app should it?

(and how would Heroku do this if the buildpack is theirs?)

thanks!

[mattions]

I'm experiencing the same problem.
if NEW_RELIC_LICENSE_KEY is set the buildpack installs the newrelic extension, however it crushes.

This a stack trace I can obtain:

 2015-05-12 17:48:41.782 (97 97) warning: daemon connect(fd=7 uds=/tmp/.newrelic.sock) returned -1 errno=ENOENT. Failed to connect to the newrelic-daemon. Please make sure that there is a properly configured newrelic-daemon running. For additional assistance, please see: https://newrelic.com/docs/php/newrelic-daemon-startup-modes

Any hints on this?

[mattions]

I'm trying to get to the bottom of this, and I've submitted this issue: heroku/heroku-buildpack-php#113 on the heroku-php-buildpack repo.
I don't know if it is buildpack specific, or just dokku specific.

[fruitl00p]

I've given up on dokku+NR... The launching of the extra daemon by NR inside the container (since that extra daemon will push the data to NR periodically) crashes the container i just gave up ;)

[progrium]

Yeah in theory there shouldn't be a problem. I just don't know what it's
doing because I can't get an idea from glancing at it.

On Thu, May 21, 2015 at 8:04 AM, Robin Speekenbrink <
[email protected]> wrote:

I've given up on dokku+NR... The launching of the extra daemon by NR
inside the container (since that extra daemon will push the data to NR
periodically) crashes the container i just gave up ;)

—
Reply to this email directly or view it on GitHub
#143 (comment).

Jeff Lindsay
http://progrium.com

[lenada]

@fruitl00p I ran into similar problems you are describing. I don't think the new-relic daemon segfaults. I can actually start it manually, when attached to the container with:
$ /app/.heroku/php/bin/newrelic-daemon -d verbosedebug -f -l /tmp/heroku.ext-newrelic.newrelic-daemon.5001.log

I am only guessing...

• new relic daemon does not have write permissions for /tmp/heroku.ext-newrelic.newrelic-daemon.5000.log?

[lenada]

seem the logfile gets touched during the build-process as root here? https://github.com/heroku/heroku-buildpack-php/blob/f2b37ffa3aac00ccc6cb11405325298f40904028/bin/util/newrelic.sh#L29

[lenada]

It seems on heroku processes run as root (fpm, as well as the new-relic daemon).
According to their docs (https://devcenter.heroku.com/articles/runtime-principles)

Superficially, it is convenient to think of each of your app’s dynos as managing a single root process

it seems dokku runs as root during build and executes the process as non root later on (which is a good thing 👍 )

@progrium how to ensure the new-relic logfile created during build (root-owned) is writable by the new-relic daemon (which is started trough php-fpm - non-root).

Should I do a PR against heroku/heroku-buildpack-php so the logfile is made world writable during build?

[lenada]

have tried chmod 777 on the logfile, new relic daemon is phoning home just fine with the modified buildpack forked from heroku/heroku-buildpack-php wesrc/heroku-buildpack-php-1@b28a37e. /cc @dzuelke

[mattions]

Has this been added in a pull request?
Can we use the solution proposed by @lenada ?

[fruitl00p]

I haven't seen an updated solution just yet @mattions maybe @progrium has news?

[josegonzalez]

You should be using herokuish now if possible, not buildstep. Herokuish should be a bit more compatible with heroku buildpacks.

[dparcerisa]

I´m using herokuish, so is not a problem only with buildstep, @lenada buildpack works perfect.
I understand that a pr is not possible because is only happening for us using dokku/herokuish/buidstep, and not on heroku.