Oropo – interesting cluster processing system, but with shortcomings

Following an article in HOWTO Forge (Linked from LinuxToday) I became acquainted with Oropo, a generic system for distributing tasks in a manager/workers computer cluster.

While it does have interesting premise (Having such tools standardized and merged into mainstream distributions), I must say, that being somewhat experienced with such systems, I find Oropo exhibits several shortcomings:

  1. There is no dynamic discovery mechanism for worker hosts, the list  is statically maintained on the central task distribution server, this complicates the setup and routine maintenance on the system.
  2. The central  task distribution server is a single point of failure in the system, there is no mechanism for making it highly available. It also doesn’t seem to have the kind of architecture that makes it easy  to use it in conjunction with existing high-availability solutions. I suspect that if it crashes, it takes the task bookkeeping information down with it, therefore forcing the user to manually figure out what part of the work was already done and restarting it.
  3. There is no mention of the algorithm used to distribute and load-balance tasks between clients, therefore I must assume it is the simplest form of round-robing-single-task-per-host algorithm, that provides no means of utilizing the processing power of stronger hosts which are capable of processing multiple tasks in parallel.
  4. The task result data from the clients (E.g. the standard-output stream of the task process) is copied and stored on the central server, there doesn’t seem to be a way to configure the system otherwise. This turns the central server into an I/O bottleneck for the system as well as a stronger point of failure. I do admit this can be worked around by writing the task process in such a way that it outputs the results to a file on shared storage system, but I do believe such a solution needs to be provided by the cluster management system rather then being baked into the task process.
  5. There is no mention of how the system reacts when a processing host crashes, I can only hope it attempts to start the task on a different host,

All in all I find Oropo to be a rather plain and naive implementation of the manager/workers concept of computing cluster, I do hope that in time it will grow and gain the features that will allow it to be useful in real-world production systems.

Advertisements

2 thoughts on “Oropo – interesting cluster processing system, but with shortcomings

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s