Egnyte shines search light on ‘dark data’ with customisable metadata layer

Egnyte today added an extendable metadata layer with an API to its file storage. This can be applied to a file, to groups of files, to file versions, and to folders.

Vineet Jain, CEO of Egnyte, a venture-backed cloud storage and file sharing vendor, revealed the company’s thinking in a press statement. “Enterprises are dependent on their content. Yet, 80% of all business data is unstructured and 90% of all unstructured data is never analyzed. It’s essentially, ‘dark data’. With the addition of advanced metadata capabilities, content can be automatically given structure which makes it findable and usable.”

With this new feature, metadata intelligence enhancements are added and stored in a layer apart from the file storage software. Users can define custom metadata fields, keys, and use those fields to set values on files, file versions, and folders. A key such as Video-Subject could have values such as vehicles, planes, ships, etc. The metadata keys are grouped together in namespaces, and these can be public or private.

The metadata layer has an API to enable the import of metadata for files from other applications and also to enable admin staff to manage this custom function.

More efficient seach

The overall intent is to build better search mechanisms for file stores holding millions of files. At this level, manual scanning is impractical and basic file attribute information is too broad brush to find specific types of content.

For instance, a search for .jpg files returns a list of all the JPEG image files in a filestore but this is little use in finding images, say, of specific mobile phones or people when there are tens of thousands of results.

Intuitively, users want to find particular types of content and the file application should help them. For example, find all contracts signed by a specific person, all images relating to Mexico, or all pharmaceutical trials ordered by company X.

File and object storage coalescing

Object storage system have traditionally had extendable metadata whereas file storage systems have not. DataCore this month announced vFiLO combined file and object storage, which enables extra metadata to be added to stored data. This is an example of a gradual coalescing of file and object storage capabilities. Object storage is supporting file access protocols and file storage, adding S3 object access protocol support.

The Egnyte and DataCore moves indicate that extendable metadata may become a standard way to make file selection and search more efficient. Hammerspace is already doing this and file sharer Panzura has extended metadata-driven capabilities and says content search is coming.

Egnyte says that this extended metadata capability will provide the foundation for coming file analytics that will be powered by machine learning.