Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement (Maybe bugfix?): Republisher support for OutputStreams in Collections #11

Open
griffinmilsap opened this issue Jan 9, 2023 · 3 comments
Labels
ezmsg-core Relates to the ezmsg.core package

Comments

@griffinmilsap
Copy link
Collaborator

Now that we have a 1:1 correspondence between Publishers and OutputStreams in Units, and the ability to set Publisher specifics from the OutputStream initializer, it is now a little unintuitive and surprising that OutputStreams in Collections do not actually result in the creation of a Publisher that we could subscribe to from another process. In particular, if a user has an OutputStream as part of a Collection that they want to expose on a public port (e.g. by defining the host or port keyword arguments for the collection OutputStream) -- currently, no Publisher will be created which is confusing behavior at best (and maybe a bug at worst). The only time Publishers and Subscribers are created are when there is an OutputStream or InputStream defined as part of a Unit.

Adding a "Republisher" Unit at the collection level for each collection OutputStream could alleviate this confusing behavior, but at the expense of occupying more sockets and performance degradation. A Republisher decorator or hint of some sort might allow a user to explicitly define when an OutputStream is republishing a topic could be a beneficial feature addition, as well as a warning when it appears that a user is attempting to interact directly with Collection level Streams. Currently, this is a bigger issue with OutputStreams because Subscriber objects have no configurable keyword arguments, but that's subject to change (future stream type enforcement/message filtering?) so it may be that republishers are required for all collection streams (performance regressions be damned).

@griffinmilsap griffinmilsap added the ezmsg-core Relates to the ezmsg.core package label Jan 9, 2023
@griffinmilsap
Copy link
Collaborator Author

@pperanich and @hannahgooden; I'd appreciate your thoughts on this when you get a chance.

@griffinmilsap
Copy link
Collaborator Author

lol oops -- reopening

@griffinmilsap
Copy link
Collaborator Author

Another possibility I'm entertaining is that Collections can have InputTopics and OutputTopics in addition to InputStreams and OutputStreams. If Streams are defined, there's a republisher associated with them (and associated functionality throughout), but the current behavior is maintained if there's only a Topic defined. Topics would have no arguments and just act as topics in the graph, with zero performance impact for composition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ezmsg-core Relates to the ezmsg.core package
Projects
None yet
Development

No branches or pull requests

1 participant