Advertisement
Guest User

Untitled

a guest
May 24th, 2018
79
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 3.39 KB | None | 0 0
  1. Q: What is the technological innovation here? There are plenty of platforms that have employed a microservice architecture. Have you leveraged this in a unique way that to your knowledge could not be achieved by other developers? If yes, what is the uniqueness here? Could you provide us with more technical detail into how the backend works?
  2.  
  3. A: The core aspect is how the data backing the microservice instances is archived. It is standard practice - at least for non-TOP500 companies - to back the core data of the service into a single or multiple relational databases. This makes the microservice approach subtly pointless, since there will always be a single component of the system liable of to bringing it down in case of failure.
  4. Lately, this problem has been somewhat mitigated by the introduction of NoSQL databases which are more fault-tolerant and able to better disperse the dataset across a number of nodes (at the expense of not providing any data consistency guarantee whatsoever). Those are basically glorified key-value datastores, and the gratest part of their architetural choices still have single-node or limited-nodes failure points anyway.
  5. Our system achieves data dispersion without single-point failure points by giving up the differentiation between storage, dataset and application nodes. (They are all identical). Workload is balanced thorought the node swarm and potential network partitioning/outages (i.e: the "split brain" problem of distributed system) is mitigated by a derivative implementation of the Paxos consensus protocol.
  6. This is of course not entirely true to the market, however the specific derivation of the Paxos implementation is of our own inception, and the ability to deploy application logic directly tied to storage nodes is a new with no documented precedents.
  7.  
  8.  
  9. Q: We need more examples of iterations and trial and error. For example, could you provide us with some detail of things/solutions you tried that did not work? What did you change to arrive at the final solution? It does not matter even if the final solution is still not working as expected, we can highlight that the R&D is on-going.
  10. Please note we cannot mention the use of third party solutions or manual processes.
  11.  
  12. An initial prototypal implementation of the service was conducted with the usual approach of multiple-instance relational db backing an in-house grown MVC framework.
  13. This was soon discovered as a route that would prove to be expensive to scale out. Attempts to circumvent the limitations of a relational databases were initially
  14. aimed at removing the parts of the data model that are not crucial for the service (such as timeseries, chat log data, page and content comments, cached metadata) and
  15. moving those over to NoSQL object datastores.
  16. This provided the architecture with a route to achieve better fault tolerance and to partially exploit cheap hardware availability, but the system remained substantially a single-image single-point-of-failure one at its core. True forms of high-availability architecture were out of our reach at this point.
  17. Our efforts were then redirected in creating out own datastore engine capable of being deployed in swarm mode (all nodes are equal). This solution is tailor-made to make it capabile to host application logic directly on what was initially envisioned as a datastore node.
  18. The system consists of basically one layer: all nodes host both application logic and data.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement