Interview: AI-focused storage startup PEAK:AIO talked about its recognition of small-scale datacenter AI processing needs in part one of a briefing. We continued the briefing with a deeper dive into what its technology can achieve.
Blocks & Files: Can I take this off sideways, slightly into RAG, retrieval-augmented generation? My thinking at the moment is that an organization will use RAG once it is already using a trained large language model, and will use RAG to feed proprietary data to that trained large language model. Well, if you aren’t using a large-scale ChatGPT-type model, but you still want to have AI interpreting your proprietary data, then you could train a small language model, an SLM, on that data using PEAK:AIO storage and a couple of GPU servers. Is that logical?
Mark Klarzynski: Yes, and slightly off medical, Solidigm did the big case study on London Zoo. And what’s amazing on this is it’s two DGXs and three petabytes of our storage. Why is it only that? Because that’s all they can power in London.
They just repopulated a bird on an island somewhere in the Philippines, through genetically evaluating this and training on it. That bird has been extinct, apart from a few in captivity, for the last 40 years.
So you start off with these guys that have only got two DGXs, but they’re making such tremendous change. [It’s like] a library and authors. Once you get one author writing a book, and it goes in the library, then you end up with other people being influenced by it, writing more books and more books. And so this library grows on the shelf, because that model and that learning that they they analyzed and they developed has now inspired other people to deal with hedgehogs and tigers in India, and that makes another model.
The obvious response to that, and where we thought we would see a limit, was, why don’t the cloud guys take that market over?
Because if you’re only interested in 16 GPUs on two DGXs, why not go to the cloud? It just turns out that the cost is just too gigantic. And not only that, there’s just an awful lot of proprietary data. And I think there’s a lot of it in the medical world that means just not possible.
Blocks & Files: They don’t want to send it outside.
Mark Klarzynski: Yes … So they’re looking for alternatives, and we’re a wonderful alternative.
Blocks & Files: So you, PEAK:AIO, could scale to serve data to thousands of GPUs if you wished, but there’s no point, because you have behemoths already doing that. And, secondly, every organization that’s using RAG at the moment could train their own small language models using PEAK:AIO and one or two GPU servers. So there’s a whole new underserved market for AI models. Training that’s cross-industry, cross-vertical market, and basically on-prem
Mark Klarzynski: Well, we haven’t actually published this anywhere, but we’ve been waiting for the market, but it’s is coming, and we’re talking about this.
So let’s say, now that you actually have a model that is very good at looking at kidneys, and it can, by the time you have your X-ray, it can determine; is that kidney good, maybe not good, or yes, it is good, or it’s bad? And because we don’t have enough radiographers, if it’s good, then it will go to a radiographer to check it. If it’s either the other two, it will go bypass that radiographer and go straight to the next stage.
So it saves a lot of time, a lot of energy, but the interesting bit is that model is quite small, and it’s not learning anymore. It’s only working on inferencing. So we’ve developed our solution now that what we have is the same storage box, but it’s now able to house a GPU.
We can build those models inside us, PEAK:AIO, so that the MRI scan and PET scan can talk directly to us, inference on us, and then we replicate that data back to the master training set. So it becomes part of that continuous development, and that’s now beginning to happen.
The only thing that’s probably slowing that down in the world is ethics and compliance because it’s a new world.
We’ve worked with many of the people that have created viable models, and we can integrate them into our box. What does that mean? It means you don’t need a big rack connected to an MRI scanner. You just need a desk, a Dell box now with us, and GPUs in it. Then we can inference, store, and transmit.
Blocks & Files: You’ve got something here that’s cross-industry. It’s horizontal, the opportunity is absolutely massive. How are you going to grow your go-to-market? And if you’re doing this, then everybody with a smart, little GPU server like Dell or HPE, Supermicro, Lenovo, Huawei maybe – they’re all possibilities for you.
Roger Cummings: It’s a lot easier to move up and then build infrastructure on top of what you have, but to go down to the point where we can start, and grab those AI workloads so early in their life cycle is really appealing to our partners. Our partners, exactly some of the people that you mentioned, right? We have more and more really great conversations happening on that, as they see our technology as a way to grab those workloads very early, on top of what they already have.
Mark Klarzynski: It turns out that some of the very big HPC guys also want simple, affordable, dense storage as well. The Los Alamos National Lab has one of the biggest supercomputer in the world, many thousands of nodes, is one of our biggest partners now.
Now I’d love to say again that was a great and amazing strategy for us. But it just happened.
So the wonderful thing on this market for somebody at my age who’s been in storage as long as you guys, is this is new, and almost every other week we have something new.
We did an announcement of an archive box not so long ago. Sounds like a backup box that we’ve all seen a thousand times. But there’s a difference on this one. Let’s stay on healthcare. One day they know that model will make a mistake, and so they need to have a different type of archive. What they want is a method of rolling back their model to justify everything.
Our Peak Archive has been working with King’s College London, University College London, Guys, St Thomas, and we’ve worked with them to say: “OK, we’re going to back up your model with each data set.” Link them together, hash them so that each has a globally unique ID, because people take other people’s models and modify them.
Having this reference is really important, and that reference, being able to go back and reboot every single image that made up that model, is becoming important. So it’s a bit like a backup, but more like a different version of backup and a very specific one.
Every week there’s another ask on us, there’s another model that will help us. We’re looking at models to try and learn the workloads, because AI workloads are really weird and complex and everyone’s different. We can take a one AI medical model from one university and another, and the workloads are completely different.
We’re working on adding AI in our box to be able to determine when should we be spinning down these QLC drives to reduce the power, and when are they likely needed really full on, and when should we back up or do audit trails? And when should we slow that audit down?
Blocks & Files: Is storage interesting for you, as a small company?
Mark Klarzynski: Yes, it’s been a while since I could have said that, but it’s almost feels like that, and in no way knocking any other guys.
It’s not as easy for an enterprise, mass-scale product to come down to where all these AI guys start, because very few of them start with $50 million funding. Most of them start with less than a million. And we give them storage at that point and we scale with them.
Large suppliers from a go-to-market perspective, they’re even telling the field, “I don’t want to talk to anybody that’s not over three petabytes, five petabytes.” We’ve just done some work in the UK with a great company called Hypervision Surgica, and they’ve developed in surgery. When we have a camera down the trachea, that’s normally video, but it gathers 16 other sensors’ feeds: oxygen levels, density, tissue density, all these things. It’s a small company, a few $100,000s in funding. They’re going to have one DGX now. In three years’ time they probably get a round for $10 million and grow. It’s amazing being part of their journey. They’re not starting at five petabytes. No, they’re starting at 100 terabytes.
Blocks & Files: Have you got scope here for encouraging the formation of a small professional services type based channel, which would specialize in applying PEAK:AIO systems to specific vertical market niches?
Mark Klarzynski: We’re actually trying to enable this. It would be great, right? And this is actually part of our plan. So a lot of the resellers that we’re working with now in the UK, that’s Scan and Boston, who are the big elite partners. They’re actually building professional services around us.
Everyone sells the same box, the VARs selling the HDX on the same switch. So there’s only really the storage and any professional services to wrap that up for the differentiation.
Roger Cummings: Our partners are a little bit different, [with a] focus on tactical edge. We have partnered with GigaIO. Their Grif [suitcase-size supercomputer] is a great tactical edge appliance that we’re on. We have Keeper Technology out of Virginia helping us with that.
We have folks that focus on computational biology, partners that are helping us with going after the lab researchers and things of that nature. And we’re still talking to the ePluses of the world, the WWTs of the world. Because when a person buys, especially those Nvidia elite partners, when they buy the GPU, that’s when the fun begins. They have to figure out how to build up and support the ingestion of the data. So we are partnering them to meet in the channel.
The Western Digital partner network, and others that we’re talking to are really interested in what we’re doing, because they want to grab those workloads super, super early.
Blocks & Files: Because they can sell more storage drives on the back end.
Roger Cummings: They can sell more. That’s why Solidigm loves us, right? They can sell more storage drives on the back end. Customers can move more funding to the to the GPU, which is their most important thing. And then they can grow linearly with us.
Blocks & Files: Is there scope in boutique financial investment firms for you?
Roger Cummings: You think about it, right? All of the old hedge fund guys, right? They’re all always building new models. They’re breaking down the data, building new models, and the data. It’s a lot of data, but in the grand scheme of things, it’s not a language model. And they’re looking for a economic way to do that.
Mark has done the hard work right of starting a company, building this in UK, and we have some wonderful success stories there. Now we can take these wonderful use cases that we have in Europe and apply them to the United States and build on that. So we’ve got a great foundation to build on.
Mark Klarzynski: We all thought that AI was going to grow like HPC did, but it didn’t. It grows sideways. UCL do an amazing job. King’s College London replicate it in a slightly different field. Guy, St Thomas do another job. It’s sort of hundreds of pockets of little ones, smaller ones, as opposed to one gigantic site … that’s the real life of people that are using AI. They’re not gigantic datacenters … If we take a university, let’s think of UCL, King’s College. They probably have 15 deployments of small systems.
Roger Cummings: A gentleman at [a large US technology institute] says I have 1,100 labs that I have to provision storage for. And, no offense, but I’m not buying VAST and I’m not buying WEKA. I don’t have the budget to do that and I can’t distribute that the way I need to distribute it.
I have some labs that have one or two servers or one or two racks; that’s it. And the minimum configuration for those solutions is an 8-node configuration. I’m talking about computational biologists that can’t spell storage. That’s not their focus. Their focus is in mapping chemistry. This gentleman, he goes, I don’t want features and functions. I don’t want that enterprise story. I just need something simple that’s high performance, cost effective, and dense.
Mark Klarzynski: There’s a lot of small and little datacenters appearing probably for the next two years. That’s the trend we’re seeing. They can’t afford to use 5G, not all the way over to an Amazon cloud somewhere, not to anybody’s cloud. They need it closer.
One of the wonderful things for me, actually, as a guy from the UK, was to see the entire Western Digital booth focused on us at SC24. There we were, the thing that was driving all of their screens and giving them all their performance. And our RAID 6 was faster than anybody’s RAID server. It was amazing for me, a small guy in the UK, to see a big company like Western Digital using us to make their [OpenFlex] product usable, because it’s a great product, but on its own, it’s a JBOF. We take that JBOF and we turn it into a hyper, fast power system that a cluster of GPUs can use.
No one’s going to buy a load of Western Digital’s OpenFlex for one system. It doesn’t make sense. So sticking us a small layer on top of them; they now can talk to a DGX cluster. And that’s what they were demonstrating on their SC24 stand.
Blocks & Files: Would I be right in assuming that you’re making a decent income at the moment, your revenues are good and growing, and you’re not going to need outside funding in the short to medium term?
Roger Cummings: You’re absolutely correct in that our revenues are growing. None of our revenues we’ve done to date really have turned the volume up on our channels. Mark mentioned the Western Digital booth [at SC24]. Within the first couple of days, we have a great pipeline of opportunities with Western Digital and the solution, but the we have no funding in today. We are in a seed round of funding today, but when you’re at a seed round stage, it’s like proving the technology and business model. We’re far past that.
We understand market fit. We’re starting to develop repeatability and velocity. So we need funding to support the channel and to move the channel forward. We have some exciting things planned; Mark has got a couple of bullets in his holster that will work for us as we scale. That’s the funding we’re looking for. We’re having some really good conversations with people in the industry. We’ve both been known in the industry forever, so we’re fortunate in that regard, and hopefully we’ll close this in the relatively near future.
