Resource temporarily unavailable (os error 35)

[aantn]

Bug Description

Randomly getting this bug:

Proxy error, connectivity issue or a bug.
You may report it to us on https://github.com/metalbear-co/mirrord/issues/new?assignees=&labels=bug&projects=&template=bug_report.yml
io failed: Resource temporarily unavailable (os error 35)

mirrod then exits

Steps to Reproduce

I don't understand how or why this happens, but it happens regularly.

Backtrace

No response

Relevant Logs

No response

Your operating system and version

Mac OS

Local process

robusta

Local process version

No response

Additional Info

No response

[aviramha]

Hi @aantn !
Can you check if the remote pod restarts when this happens?

[aantn]

Hi @aviramha,
It remains running. This is a minute or so after the crash:

~/Projects/holmesgpt → kubectl get pods -A | grep runner
default           robusta-runner-f996cc896-p29xx                                   1/1     Running     5 (101m ago)    29h

[aviramha]

Any chance it happens when you're on a breakpoint for a while?

#2652

[aantn]

Nope!

…

On Fri, Sep 6, 2024, 21:50 Aviram Hassan ***@***.***> wrote: Any chance it happens when you're on a breakpoint for a while? #2652 <#2652> — Reply to this email directly, view it on GitHub <#2742 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADYUBZQUVZXLIUE6EGWOGTZVH2PHAVCNFSM6AAAAABNYYEU5CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZUGYZTINRRHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[aviramha]

Can you please add this to your configuration?

{
  "internal_proxy": {
    "log_destination": "/tmp/mirrord.log",
    "log_level": "mirrord=trace"
  }
}

Run, then attach the output log when the issue occurs? Make sure to retract sensitive information

[aantn]

Done. I've sent you the log privately on linkedin after redaction.

…

On Sat, Sep 7, 2024 at 10:38 AM Aviram Hassan ***@***.***> wrote: Can you please add this to your configuration? { "internal_proxy": { "log_destination": "/tmp/mirrord.log", "log_level": "mirrord=trace" } } Run, then attach the output log when the issue occurs? Make sure to retract sensitive information — Reply to this email directly, view it on GitHub <#2742 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADYUB43ZIO7JXELJTYWUDTZVKUQ3AVCNFSM6AAAAABNYYEU5CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZVGEYDKNJVHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[aviramha]

I believe this is related to this - #1898

[aviramha]

User added that it seems to happens when another container it's connected to (Prometheus) is OOM