Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for iterators #15

Merged
merged 1 commit into from
Jul 14, 2023

Conversation

connormullett
Copy link
Contributor

Adds the ability to iterate over nodes in the ring. This would help users attempting to copy the internal ring for serialization purposes and help with needs for replication where a user would want to access a node that neighbors the node responsible for the key

Additionally, this PR fixes and adds tests related to these changes

Copy link
Owner

@jeromefroe jeromefroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @connormullett! LGTM overall, just had one question on how we want to implement the iterator so as to potentially avoid needing to clone the internal vector.

src/lib.rs Outdated
@@ -215,6 +215,34 @@ impl<T: Hash, S: BuildHasher> HashRing<T, S> {
}
}

pub struct HashRingIterator<T> {
ring: Vec<T>,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be worth using a std::vec::IntoIter here instead of a Vec so we don't have to clone the underlying Vec?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point and a good solution to what I couldn't really figure out. Change is coming shortly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to find a clean implementation that doesn't clone the underlying Vec but also doesn't expose the private Node type. using Map gives a reference which unfortunately means cloning is required. Thoughts?

@connormullett connormullett force-pushed the iterator-support branch 3 times, most recently from 26100a9 to 51343c3 Compare July 12, 2023 07:17
Copy link
Owner

@jeromefroe jeromefroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, just had some minor comments about some of the internal details that I would be curious to get your thoughts on

src/lib.rs Outdated
@@ -215,6 +215,39 @@ impl<T: Hash, S: BuildHasher> HashRing<T, S> {
}
}

pub struct HashRingIterator<T> {
ring: std::vec::IntoIter<T>,
index: usize,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get rid of index now that we're wrapping the underlying vector iterator?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call

src/lib.rs Outdated
.ring
.iter()
.map(|n| n.node.clone())
.collect::<Vec<T>>()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid needing to collect the items from the vector into an iterator here, what do you think about using the internal iterator from the vector directly and mapping Node<T> to T in the hash ring iterator's next method? For example:

pub struct HashRingIterator<T> {
    iter: std::vec::IntoIter<Node<T>>,
}

impl<T> Iterator for HashRingIterator<T> {
    type Item = T;

    fn next(&mut self) -> Option<Self::Item> {
        self.iter.next().map(|n| n.node)
    }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to do it this way as it would expose the internal Node type and expose a layer of abstraction. This is much cleaner way of doing it. Changes coming soon

Copy link
Owner

@jeromefroe jeromefroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me, had just one minor comment on whether we could keep the Node type private.

src/lib.rs Outdated
@@ -100,7 +100,7 @@ impl BuildHasher for DefaultHashBuilder {
// Node is an internal struct used to encapsulate the nodes that will be added and
// removed from `HashRing`
#[derive(Debug)]
struct Node<T> {
pub struct Node<T> {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to make this a public type? If we map Node<T> to T in the next method would that allow us to keep the type private? For example:

pub struct HashRingIterator<T> {
    ring: std::vec::IntoIter<Node<T>>,
}

impl<T> Iterator for HashRingIterator<T> {
    type Item = T;

    fn next(&mut self) -> Option<Self::Item> {
        self.ring.next().map(|node| node.node)
    }
}

impl<T: Clone> IntoIterator for HashRing<T> {
    type Item = T;

    type IntoIter = HashRingIterator<T>;

    fn into_iter(self) -> Self::IntoIter {
        HashRingIterator {
            ring: self.ring.into_iter(),
        }
    }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah this is much simpler and works. This is what I was trying to figure out but was unable to. Changes coming soon. Thanks for the assist

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I missed the Node addition in HashRingIterator. This works great, commit coming soon

Adds the ability to iterate over nodes in the ring. This helps
users attempting to access the internal ring for serialization purposes
and help with needs for replication where a user would want to access
a node that neighbors the node responsible for the key
@jeromefroe jeromefroe merged commit 5f7b583 into jeromefroe:master Jul 14, 2023
3 checks passed
@jeromefroe
Copy link
Owner

Thanks for the contribution @connormullett!

@connormullett
Copy link
Contributor Author

No problem, thanks for taking the time to review and help

@connormullett connormullett deleted the iterator-support branch July 14, 2023 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants