It’s not just Rubrik; backup supplier Arcserve also has strong reservations about the validity of Gartner’s Data Centre Backup and Recovery MQ and is considering where else it could find reliably independent and objective analysis of its place in the market.
Yesterday Rubrik publicly revealed its dismay and strong objections to what it sees as unfairness, lack of objectivity and rigour in this MQ process and outcome. It also identified a conflict of interest between itself and the lead analyst, as, it claimed, he had pursued a job opportunity at Rubrik, and been rejected.
This MQ has been hotly awaited as the last edition came out in 2017. There have been changes in the suppliers’ organisations, strategy, products and market fortunes since then. So the latest edition, due we understand on Monday, has been enthusiastically awaited.
It has had a somewhat tortuous progression with a history of departed analysts replaced by a new team, altered criteria, strong supplier objections to provisional ratings by this team, and a delay due to a review process following this.
Changed criteria
When the new Garter analyst team started work it updated the criteria for inclusion and evaluation in the Data Centre Backup and Recovery MQ. A third supplier, troubled by this MQ’s process, gave us a look at it. The introduction states;
This 2019 “Magic Quadrant for Data Center Backup and Recovery Solutions” is an update to the “Magic Quadrant for Data Center Backup and Recovery Solutions” that last published in July 2017. As the backup and recovery market is continuously changing, we simplified the market definition.
Because The “Magic Quadrant for Data Center Backup and Recovery Solutions” focuses on upper-midmarket and large enterprise organizations we refined the inclusion / exclusion criteria by increasing focus on international presence and size of the protected environment.
The weight given to certain criteria was changed. For example;
- In Completeness of Vision, the high rating for marketing strategy and sales strategy was changed from high to medium.
- There was no rating in 2017 for “The soundness and logic of a vendor’s underlying business proposition” or its vertical/industry strategy and geographic strategy; they each received a medium weight in the 2019 MQ.
- In the Ability to Execute section, a supplier’s marketing execution was downrated from high to medium.
These changes make comparison with the 2017 MQ rankings harder.
Arcserve points
According to Arcserve;
- The Gartner analysts did not seem to have a good understanding of the market. Like Rubrik it felt that the analysts did not listen to raised objections and submissions, or simply did not understand them.
- The Arcserve team was told that placement in the MQ had more to do with the Gartner analysts’ feelings than the results of a rigorous methodology.
- Its growth was not reflected in its positioning, and growth declines by Veritas and Dell EMC were ignored by the analysts as well. The feeling is that the supplier growth area ratings are inconsistent and unfair.
Arcserve feels that the standing of this MQ is compromised by the poor and inconsistent information gathering process and the unfairness of the results.
Rubrik took its objections to Gartner’s ombudsman, but to little or no no avail. So too did Arcserve, again, it felt, with little benefit.
Arcserve cannot use this MQ and the accompanying critical capabilities document to assess how itself and other suppliers are doing, because of the problems it identified. If the information gathering and assessing process for them was the same as that which it (and Rubrik) experienced then comparisons with the other vendors’ ratings and assessments are simply unreliable.
It thinks that it needs to find a better and more solid source for analysis and reviews of suppliers’ standings in the data centre backup and recovery market.
An alternative
No doubt it is well aware of the Forrester Wave.
Rubrik CEO Bipul Sinha asks; “Could an experienced and objective third party come to a different conclusion [than Gartner]? Naveen Chhabra, an Analyst with 20 years’ experience at IBM and Forrester, recently published the Forrester Wave for Data Resiliency Solutions, placing Rubrik in the Leader Category with the highest possible score for strategy. We now know the answer to that question.”
In this Wave, Rubrik, Cohesity, Veeam and Commvault were all in the leaders, section, while, in the forthcoming Gartner backup and recovery MQ, Rubrik and Cohesity are visionaries.
The stakes are high. CIOs day a lot of attention to MQ position and ratings. A supplier’s growth could be hindered by an adverse placement and Rubrik, with its mega-funding, needs high growth to continue.
Arcserve would just like a fair crack of the whip, and doesn’t think Gartner is supplying that.
Gartner viewpoint
We asked Gartner what it thinks about the points made by Arcserve and it replied: “Please see the Gartner Office of Ombudsman’s blog post titled “Gartner Research Does Not Please Everyone, All the Time” in response to your inquiry. “
That states;
Independence and objectivity are paramount attributes of Gartner research, so even the perception of a conflict of interest requires careful examination by the Office of the Ombudsman. Rubrik absolutely did the right thing to contact us and voice its concerns. In this case, it took a considerable amount of time to thoroughly investigate every single complaint raised by Rubrik, ultimately resulting in the assignment of a new lead analyst for this research. We are fully satisfied that Gartner’s rigorous research methodologies, combined with the actions taken by the Research & Advisory leadership team throughout this process, ensures all the vendors in this market segment are accurately—and fairly—evaluated relative to their competitors in the final Magic Quadrant.
Unfortunately, Rubrik does not agree with Gartner’s point of view expressed in the Magic Quadrant, but we respect the company’s right to voice its opinion. We believe Gartner’s opinion on vendor capabilities in this market is accurately expressed in the Magic Quadrant, a rigorous, independent analysis that helps buyers navigate technology purchase decisions.
