Yesterday I had the pleasure of attending the first ZeroVM design summit in San Antonio. The ZeroVM team (recently acquired by Rackspace) is working on building a multi-tenant, distributed compute system built on top of OpenStack Swift. It’s a very interesting project, and I’ll be continuing to keep track of their work.
Yesterday was the second day of their design summit, and we spent a majority of the day focusing on Swift integration work. A couple of key feature additions to Swift were discussed.
First, we spent a lot of time discussing adding object append support into Swift. It’s possible to append to objects today by using large object manifests, but when using that method, all of the object’s constituent parts get stored throughout the cluster. However, if Swift supported adding to an existing object by updating the on-disk data, then a process executor can more efficiently work with the resulting object.
Adding object append support to Swift seems simple at first glance, but it gets rather complicated when considering different edge cases. We had an active discussion on the best way forward. The two basic options are to either add some sort of consistency semantics to this part of Swift or to allow appends to be added out-of-order. Both methods have strengths and weaknesses, and I’m looking forward to seeing what the end design proposal will be. I’m currently supportive of the out-of-order append resolution strategy.
The other major feature discussion was to add triggers or webhooks to Swift. This feature would allow a Swift client to set an endpoint and a filter on a particular container and have that endpoint called when filtered actions happen. For example “send a request to foo when any object is deleted from container A” or “send a request to bar when any object with content-type text/plain is added to container B”.
I’m excited about the idea of adding hooks into Swift. It will add a great piece of functionality that will enable new use cases that are difficult to support today.
Overall, we had some great discussions yesterday at the design summit. ZeroVM has some great potential, but I’ll be really excited when we see some demonstrated production use cases with it. In early 2015, I think it would be great to see some public examples of ZeroVM in use at scale inside of Rackspace.
I hope the discussions we had yesterday continue within the larger Swift community. These big features, along with a couple of smaller ones discussed, allow Swift to support some very interesting new use cases. The future is bright.