Support for search / highlight

[alexcherka]

Would be really nice if you could filter console lines via search. Or even just highlight a desired key. I know this may not be the intention of this tool but when filtering back in time, would be great to search for specific data if needed!

[amorey]

Thanks for the suggestion! I definitely want to add search/highlight functionality to kubetail. It will take some time to implement but now's a good time to start specing it out. Is there another tool that you think has implemented log search particularly well so far?

[alexcherka]

As far as web k8s logs viewers go, I don't think there is much out there honestly... I know for IDE's I use K8s Lens that when you view a pod's logs it allows for searching via highlight and then jump around to results.

I think this is the best option depending on how you render your front end results. I assume there is some JS packages that allow for search/highlight content on page.

Found something that seems to cover it pretty well.
https://markjs.io/
Specifically the example Search bar with jump to matches

[amorey]

Thanks! I'll take a look at K8S Lens and mark.js.

[nodesocket]

Are you thinking client side search? DataDog Logs (paid solution) does a really good job at filtering and searching.

[amorey]

I'm curious about doing search client side but I have a feeling that performance will be an issue. I think I'll do some experiments with IndexedDB to see if it's feasible.

[nefarius]

I think in this case client-side search and indexing makes sense; no load increase on the backend and the client already has all the searchable data anyway so why not make use of it.

[alexcherka]

I agree with @nefarius, The front end data is loaded in the first place regardless and then filtering can be done client/DOM level and in theory no extra heavy lifting on the backend or really front end either. I think a highlighting approach is best and just have it jump around to matches. Similar to vscode in file search or any other ide as an example

[amorey]

Currently, the frontend only stores new events so the full log files aren't directly accessible locally. We could change this behavior and download the full files in the background and that would work well for small/medium files but it could be problematic for large amounts of data (and egress costs could also be an issue down the road for power users). I'm still curious to test out client side search though because it might be useful for small amounts of data or in other contexts.

[sawan22071995]

Can you tell me how we will get generated token when we use tokenauth.yaml. you didn't mention any commads or steps to get the token value to login in server.

[amorey]

Tokens are a Kubernetes authentication feature that you can use to add user authentication to your cluster. Depending on how you've configured your cluster they can be obtained from the cluster itself (Bootstrap Tokens) or from a 3rd-party authenticator plugin (e.g. aws-iam-authenticator). If you aren't using tokens to authenticate to your cluster you can use the kubetail-clusterauth.yaml manifest file to run Kubetail. If you'd like to start using tokens as part of your normal Kubernetes auth flow here are some places you can start:
https://kubernetes.io/docs/reference/access-authn-authz/authentication/
https://github.com/kubernetes-sigs/aws-iam-authenticator

I'm happy to answer any more questions you have about tokens but this Github issue is about search so please create a new issue if your question isn't search-related.

[amorey]

Hi folks!

I have an update about Kubetail related to search.

As a first step towards enabling search, I set up a new server/agent architecture for Kubetail that runs small agents on each node in the cluster to perform node-specific tasks. This architecture will be useful for searching your log files on-disk in the future.

Although search isn't ready yet, I took advantage of the new architecture and I added a feature that shows you your log sizes and new events in realtime:

Here's a live demo:
https://demo.kubetail.com

Personally, I'm finding the new feature useful because previously I had to ssh into each node to see my log sizes and it was difficult to get a birds eye view of which logs were the most active.

It would be really helpful if you could install the latest version (0.7.0) and let me know if everything is working properly:

helm repo update
helm upgrade kubetail kubetail/kubetail --namespace kubetail

If you're using custom values you'll need to update your values.yaml file to fit the new schema (values.yaml).

Let me know if you need help upgrading and what you think of the new feature!