Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NHeavyHitters transformer #125

Open
martinbomio opened this issue May 25, 2018 · 3 comments
Open

Add NHeavyHitters transformer #125

martinbomio opened this issue May 25, 2018 · 3 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@martinbomio
Copy link
Contributor

Could be useful to have a transformer that allows to apply a global heavy hitters to seq like attribute.

Something like NNHeavyHitters extends Transformer[Seq[A], SketchMap[String, Long], Map[String, (Int, Long)]]

@nevillelyh nevillelyh added the enhancement New feature or request label May 25, 2018
@nevillelyh
Copy link
Contributor

@richwhitjr WDYT?

@richwhitjr
Copy link
Contributor

We already have something close to this. Is the idea around instead of being limited to just a String we could give a list of Strings etc? How would you like use the output? Should the sketchmap be used to return the heavy hitters back into a Seq[(String, (Int,Long))] and remove those that don't make the top N?

@martinbomio
Copy link
Contributor Author

@richwhitjr yeah, that's what I was thinking.
Right now, for my use case, I will like to get a List[Int] representing the indices in the topN for each of the attributes in the input sequence, filtering out those that are not in the topN

@nevillelyh nevillelyh added the help wanted Extra attention is needed label May 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants