Distributed API-X instances

[birkland]

Determine shared state, configuration, and  communication needs between instances of API-X core.  Determine key patterns and use cases (e.g. load balance, failover)

[ajs6f]

If you keep the config (and insofar as possible, the state) in the repository, you can dump some of this work off your shoulders.

[acoburn]

This is also an area where tools like zookeeper (or any of the software built on zk) work well, especially if you want fault-tolerance and/or leader election for api-x core. If you don't care about that, then putting the shared state in a common database (or the Fedora repository) would be really easy.

[DiegoPino]

ZeroMQ (broker less) is also a good option.

[ajs6f]

ZeroMQ is good software in many ways, but it would be better, if possible to design to some kind of interface or contract, not a specific product. ZeroMQ is not going to suit everyone. Nothing will. That is the realization that started us off on the road of specification for the repository itself.

[DiegoPino]

@ajs6f agree. Idem goes for Zookeeper. "design to some kind of interface or contract" == true. Since i'm still trying to get what an API-X compliant service should be (formal framework and/or contract on that side) i will have to dig deeper to get a better sense of what the real needs of topics like "shared state" are here. Conclusion: need to do my homework

[acoburn]

In my POC -- there is an API interface for the registry and binding services. The implementation can be based on any technology (ZK, ZeroMQ, whatever) -- the POC uses a simple in-memory Map -- but a real implementation would use something that persists data somewhere. Either way, the idea is that the (Java) API is completely independent of the implementation.

[ajs6f]

It just might be that @acoburn has captured the type requirements for that component simply by using the Map abstraction. If so, the open question becomes: how fully is it necessary to spec the behavior of that Map to be able to fully spec the complete system? I suggest that the answer is: not much at all.

[DiegoPino]

@ajs6f can you further elaborate those thoughts? Too abstract for my normal Sunday night state of mind.

how fully is it necessary to spec the behavior of that Map to be able to fully spec the complete system?"

I'm probably not getting the goal of a shared state/ inter service communication pattern specs or maybe the how via this implementation. Thanks 😄

[ajs6f]

I'm saying that if a Map did the job for @acoburn, then the component that deals with this problem just might be fully specified for immediate purposes simply by saying, "It can do the things that a Map can do." How does it do them? That's beyond the scope of specification. That's for implementations to decide. Some may decide to use Zookeeper for everything. Some may put some stuff in the repo and some stuff elsewhere. Some may put everything in Hazelcast, or some other kind of distributed structure. Some may decide that clustering is uninteresting and NOOP it. The important issue at this stage of the game is what "an API-X" can do, not how it gets done.

To be honest, I'd rather see API-X ignore this question entirely, at least for now. I don't think it's actually appropriate to specify this kind of behavior, for the same reason that Fedora itself is avoiding the question. But if API-X is going to get into this, then I hope it will be in most abstract and flexible way available.

[acoburn]

@DiegoPino The way I handled this in my POC was to write simple Interfaces for the registry and binding services. Those Interfaces are extremely simple: e.g. https://git.io/vrQjt and https://git.io/vrQjL

These interfaces use Strings at present for the descriptions, when they should, in general, use RDF graphs, but the basic abstraction still holds.

[birkland]

Revisiting this, the present API-X impl is essentially stateless. Any in-memory caches it uses for optimization purposes are initialized from repository contents, and are updated in response to messages from the repository. So I don't think all that much work would be necessary on the API-X side to scale horizontally.

The hard part would be leveraging existing infrastructure to demonstrate it (i.e. track running instances of API-X with Zookeeper or Consul, load balance, etc). I think expanding the demo images would be a plausible way forward