kubefwd does not support reserving non loopback (127/8) addresses

[mccaig]

Thanks for building kubefwd, it's been a great help for some of our workflows.

The addition of the ip reservations in #208 is a fantastic feature that I am eager to take advantage of, however I am unable to due the restriction that the IP addresses be limited to the reserved loopback range (127/8).

My use case is that I do my dev loop of a few services locally in docker containers. As 127/8 addresses are always routed to the loopback adapter, I cant run kubefwd on my MacOS host machine, and also have the containers access those same forwarded services - the container based traffic will always hit its own loopback adapter. This means I need to run kubefwd in each individual container, in addition to my host machine as needed.

I can avoid all of this overhead however, if I can configure kubefwd to use a non-loopback ip range, as the containers are able to route to those addresses on the host machine. I hope this illustrates that there is a valid use case for the use of non loopback IP addresses.

There was some discussion about this in #208 (#208 (comment)), where it was noted that:

I do not want to automatically assign non loopback IPs to an interface as this can cause big problems for users unaware of the consequences.

I agree that it could be an issue for users who aren't overly familiar with networking concepts - do you think concern this could be mitigated by printing a warning message when such a configuration is activated?

I would be happy to work on a PR for this feature if you think it would be accepted.

[cjimti]

@mccaig Thanks for the interest in kubefwd. I am not opposed to it in general. I have some use cases for it myself, specifically distributed services that report their internal IP addresses (like Kafka) and make hostnames that resolve to loopbacks useless.

My comment on #208 was more of a concern about auto-incrementing non-loopback IPs. Even savvy users might not expect that behavior and lose access to external network resources because they did not know it would increment.

I, however, need to think through a few other problems regarding bringing up and possibly taking down IPs on the local interface. I have not experimented with it much, but it would have to be a very stable implementation. For example, say a person decides to use 192.168.1.100 unaware that their printer (or router) is using it (same with important public IPs), if kubefwd brings up this interface and for any reason it crashes, the user might not know how to fix it and loose access to that IP.

[mccaig]

For example, say a person decides to use 192.168.1.100 unaware that their printer (or router) is using it (same with important public IPs)

I agree this is a concern - certainly the scenario where the user configures kubefwd with an IP range that matches any of their directly connected subnets is the most dangerous, as they may cut themselves off entirely if their gateway IP is configured on the loopback interface.
Mitigating this may involve checking other interfaces on the device and prohibiting assignment of IP ranges matching their connected networks. For the other scenarios - public IP's, - or perhaps RFC1918 addresses within private/corporate networks, I don't see any obvious mitigation other than putting the feature behind an additional flag?

[cjimti]

I can re-visit this after the new year when I'll have a bit more time. This project does not have many long term contributors which means I'm on the hook for bugs and support. kubefwd has been stable for its central use case and I want to ensure it stays that way. :)

[ndj888]

Introducing a bridge network, I think it is not a good idea. it will make kubefwd more complicated and uncontrollable.
@cjimti

[kikonen]

Any news or workarounds on this. I.e. I just got stuck with problem of needing to run local docker container, which needs to access services in kubernetes. okey, kubefwd nicely forwards them in localhost, but that does not help inside container (and running kubefwd inside every local container causes other setup problems).

Right I'm pondering if I could manage to get some routing done based into "route_localnet" ( as stated in https://serverfault.com/questions/701589/127-8-address-is-not-passed-to-ip-stack).

[kikonen]

Quickly testing this might be "good enough" workaround for my needs for a while.

EDIT
Nope, it does not work, since it plays havoc with docker DNS logic between containers in docker-compose.yml itself

docker-compose.yml

xx-service:
  ...
  networks:
    hostnet: {}
....

networks:
  hostnet:
    external: true
    name: host

Build (separate due due to need for "--ssh" opt)

...
docker build --network host --ssh default .

start containers

docker compose up -d
docker compose exec xx-service bash

In xx-service container:

# access kubefwd proxied yy-service
curl -X POST -H "Content-Type: application/json" "http://yy-service"
.. #got valid response

[kikonen]

Another workaround attempt via using tips from https://stackoverflow.com/questions/17770902/forward-host-port-to-docker-container worked fine.

[a1exus]

i believe this is much better solution:

services:
  kubefwd:
    image: txn2/kubefwd
    command: ...
  app:
    image: bash
    command:
     - sleep
     - inf
    init: true
    network_mode: service:kubefwd

https://stackoverflow.com/a/72671061/91697