OpenIO has devised its own serverless computing framework to add functionality to its SDS object storage software.
The fruits of its labour can be seen in an extensive upgrade of SDS, v18.10, in which OpenIO used the new framework to build metadata indexing and search.
Keeping the weight down
SDS is open source software that runs on clusters of commodity servers, It is a relatively small piece of code, requiring a minimum 400MB of DRAM and a single ARM core. But adding new features and improving existing ones would bulk up to the code too much.
And so OpenIO made its own serverless computing framework, dubbed ‘Grid for Apps’.
Using the framework, SDS can trigger functions based on events and schedule batch jobs to run when wanted – for instance when the infrastructure is less busy, As Enrico Signoretti, the head of product strategy, explains in a company blog,
“Functions are relatively small pieces of code,” he writes, “that are abstracted from the underlaying infrastructure. They are easy to develop and maintain, giving us a huge advantage, because they do not directly interact with the core.”
Chargeback could be added to OpenIO SDS as a function instead of being added to the core code.
According to Signoretti, the core code already “catches all events that occur in the system, and … can pass them on to Grid for Apps. A function computes all the necessary information and gives you all the metrics you need. It’s asynchronous, scales with the rest of the cluster, and resources to run it are allocated dynamically.”
Grid for Apps is used to create new features and keep the core lightweight and efficient. The functions are, in effect, small applications that run on the object store’s cluster of servers. The object storage infrastructure handles the compute.