Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pharos scraper proxy gets OOM killed #1508

Open
jnummelin opened this issue Nov 15, 2019 · 0 comments
Open

Pharos scraper proxy gets OOM killed #1508

jnummelin opened this issue Nov 15, 2019 · 0 comments

Comments

@jnummelin
Copy link
Contributor

What happened:
Pharos scraper proxy (nginx) get OOM killed in some cases.

Nov 15 06:46:39 ip-172-31-23-209 kernel: [65267.037323] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=aba6a64a8753ea5169d3f501d661492696ffeb640da38c613b81198435144dcd,mems_allowed=0-1,oom_memcg=/kubepods/burstable/poddfc7f441-06db-11ea-842a-0227fba247b6,task_memcg=/kubepods/burstable/poddfc7f441-06db-11ea-842a-0227fba247b6/aba6a64a8753ea5169d3f501d661492696ffeb640da38c613b81198435144dcd,task=nginx,pid=22232,uid=0
Nov 15 06:46:39 ip-172-31-23-209 kernel: [65267.037327] Memory cgroup out of memory: Kill process 22232 (nginx) score 1182 or sacrifice child
Nov 15 06:46:39 ip-172-31-23-209 kernel: [65267.042541] Killed process 22964 (nginx) total-vm:14136kB, anon-rss:844kB, file-rss:4kB, shmem-rss:0kB
Nov 15 06:46:39 ip-172-31-23-209 kernel: [65267.047980] oom_reaper: reaped process 22964 (nginx), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Nov 15 06:46:39 ip-172-31-23-209 kernel: [65267.137860] nginx invoked oom-killer: gfp_mask=0x7080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=999
Nov 15 06:46:39 ip-172-31-23-209 kernel: [65267.137862] CPU: 9 PID: 22232 Comm: nginx Not tainted 5.0.0-1021-aws #24~18.04.1-Ubuntu

What you expected to happen:

Not to see OOM kills. :)

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

  • Pharos version (use pharos --version):
  • Cloud provider or hardware configuration: AWS
  • OS (e.g. from /etc/os-release; both for the client and for the used nodes): Ubuntu 18.04 with 5.x kernel
  • Others:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant