Subset of /etc/hosts entries not being cleaned up about kubefwd

[cjimti]

I have started writing a new /etc/hosts parser/manager and should have it complete sometime next week. I am traveling for work so it may be delayed a bit.

[cjimti]

I have tested the new version 1.6.0 https://github.com/txn2/kubefwd/releases/tag/1.6.0 and it seems to have fixed this issue. Is should not cause duplicates and will ignore them if they already exist, just like your system does now. If kubefwd were to crash and leave entries in the /etc/hosts it should no longer cause problems. @miles- please re-open this issue if you experience any regressions with 1.6.0. Thanks!

[cjimti]

Thanks for the bug report. The lib I use to manage the host file looked good at first, but in the long run, it was a bad choice as it is very unstable. Yes, I thought I fixed it in 1.4.10 or at least hacked around the issue but it looks like I did not.

Thanks for the report.

[obiesmans]

Honestly, modifying /etc/hosts seems ironically hard to get right on all platforms, and will also require administrator access anyway. Can you make it so that we don't need it anymore ?

Some domains will resolve all subdomains to 127.0.0.1, or to some configurable IP in xip.io case.

$ dig +short @8.8.8.8 prometheus-server.localtest.me                                                                                                                         
127.0.0.1

$ dig +short @8.8.8.8 prometheus-server.127.0.0.1.xip.io                                                                                                                     
127.0.0.1

Could that help ?

[cjimti]

@obiesmans I agree. I wish I did not have to deal with /etc/hosts. The lib I use is overkill and buggy for my purpose so I still have a plan to write something from scratch.

Your idea looks interesting however I don't think it would solve the main use case of addressing services as though you were in the cluster. As an example, I have a few services, api-access:80, api-provisioning:80 and elasticsearch:9200. The only cross-platform way to get those names to resolve to say 127.27.1.1, 127.27.2.1 and 127.27.3.1 is /etc/hosts.

At the moment kubefwd does work on win/mac/Linux even if a bit buggy with dealing with /etc/hosts at times, I do hate having to require elevated privileges but I just don't see a way around it and not kill the mainline use case.

[obiesmans]

@cjimti : I wasn't sure kubefwd used other IP than 127.0.0.1.
Ok, looks like localtest.me is out the windows then.

xip.io could still be useful, maybe ?

$  dig +short @8.8.8.8 api-access.127.27.2.1.xip.io                                                                                                                          
127.27.2.1
$ dig +short @8.8.8.8 api-provisioning.127.27.2.1.xip.io                                                                                                                    
127.27.2.1
$ dig +short @8.8.8.8 elasticsearch.127.27.3.1.xip.io                                                                                                                        
127.27.3.1

edit : whoops, I understand just now that you'd need the admin rights just to add the IP alias anyways.

[cjimti]

Multiple loopback 127.x.x.x addresses are needed to accommodate duplicate ports on multiple services, for instance, 80 and 8080. Also unless there is another way I believe I need admin on MacOS to bring up these additional addresses as they are not up by default.

Also if I develop an app that needs to first connect to http://api-provisioning:80 then to http://elasticsearch:9200 I would need a host alias for the name api-provisioning to api-provisioning.127.27.2.1.xip.io and elasticsearch to elasticsearch.127.27.3.1.xip.io. I am not familiar with how host aliasing works across platforms or well it is supported.

[miles-]

I have started writing a new /etc/hosts parser/manager and should have it complete sometime next week. I am traveling for work so it may be delayed a bit.

Thank you! Ping me if I can help at all or if you want someone to test changes you're making.

[cjimti]

@miles- I am looking to have a new lib to add sometime next week. I have started to code and am trying to be less intrusive with the /etc/hosts file. I plan to have something to test by early next week.

[cjimti]

@miles- I'll be attempting to refactor kubefwd to use this new lib https://github.com/txn2/txeh I created specifically for this project. txeh attempts to do less "management" of the /etc/hosts file and focus directly on adding and removing host entries without caring as much as possible about the current structure of the file.

I'll create 1.6.x branch for this based on 1.4.10.

[miles-]

@cjimti 👍 sounds good. I'll take a look this weekend and can test it out.

[cjimti]

@miles- ill be working out a few little bugs in the txeh lib but will keep the 1.6.x branch updated with the latest fixes as I do. So far the only bug seems to be adding a blank new line to /etc/hosts each run. I should be able to figure that out quick. Otherwise, it seems to be a much better solution than hostess for kubefwd's needs.

Mac and Linux seem to be good. I'll test on Windows 10 tonight after I fix the newline bug.

[miles-]

@cjimti I'm testing out 1.6.0 now and the /etc/hosts updates / clean-up is working great - thanks!

[cjimti]

@miles- @obiesmans I created a command line util to go alongside the new txeh lib for hosts file management.

If you install txeh you can run a command like sudo txeh remove cidr 127.1.27.0/24 to clean up your /etc/hosts if for some reason kubefwd gits killed or fails before being able to clean itself up. If everything runs smooth you should not have to do this, but if it does happen this little util should save some cleanup time.

If you are interested please take a look at txeh:
https://github.com/txn2/txeh

Thanks