Datameer > Case Studies > Sophos increases security with big data analytics

Sophos increases security with big data analytics

Company Size

1,000+

Country

United Kingdom
United States

Product

Sophos Antivirus
Sophos Encryption Products

Tech Stack

Hadoop
Datameer

Implementation Scale

Enterprise-wide Deployment

Technology Category

Analytics & Modeling - Big Data Analytics

Use Cases

Cybersecurity

Services

Data Science Services

About The Customer

Sophos is a company that has been producing antivirus and encryption products for nearly 30 years. The company helps secure the networks used by 100 million people in 150 countries and 100,000 businesses. As IT networks grow in complexity, Sophos’ mission is to keep IT security simple and reliable. The company's products examine billions of events per day to detect malicious files, with over 300,000 new potentially malicious files reported to SophosLabs daily for analysis. The company's threat research analysts need to analyze these newly suspicious files to determine if they truly are threats. An automated analysis process produces a large set of metadata for each file, generating many millions of records every day.

The Challenge

Sophos, a company that has been producing antivirus and encryption products for nearly 30 years, was facing a challenge with the increasing complexity of IT networks and the sophistication of threats and attacks. The company's products examine billions of events per day to detect malicious files, with over 300,000 new potentially malicious files reported to SophosLabs daily for analysis. The volume and complexity of the data grew to a point where their old analytic infrastructure could not keep pace. Another challenge was the cloud telemetry data consisting of billions of lookups for website and file information. A particular aspect of the analysis – correlating patterns across previous analysis – had become too complex for their SQL-based database and analytic tools to manage. Sophos investigated NoSQL technologies available at the time and selected Hadoop for big data analytics needs related to telemetry and threat correlation. However, out-of-the-box Hadoop was lacking any enterprise-ready tools for creating analytic reports, dashboards, data access controls or mechanisms to easily import or export data in and out of various storage systems.

The Solution

Sophos required an analytic platform that would combine the best of everything they needed – an infrastructure which leveraged Hadoop for power and scalability, while abstracting the technical complexity so their analysts could be productive. They found this in Datameer, which offered a scalable analytic infrastructure built natively on Hadoop, an Excel-like workbook interface that was very familiar to analysts, powerful analytic functions that could be applied to data using point-and-click operations in an easy to use user interface, and easy data connectivity and integration that could combine the variety of data sources and formats they required. With Datameer, the Sophos’ team was able to ramp up quickly without becoming Hadoop experts. Threat researchers and malware analysts use Datameer to analyze data from multiple sources, including its Threat Telemetry (reputation queries), threat feeds (urls, hashes, etc.), product feedback, and other imported datasets. In addition to comparing file samples to the millions of known malware, analysts are canvassing the threat landscape to identify new malware and stop it.

Operational Impact

Sophos no longer needed to filter or aggregate its data, resulting in better insights and faster detection of malware and security breaches.
Datameer is integral to Sophos’ daily malware detection in multiple use cases.
Malware research and analysis. Malware is becoming more evasive and pervasive. Sophos analyzes the characteristics of suspicious files and report the analysis outcome.
Macro trend analysis. Sophos analysts also analyze the data for macro trends of malware movements to better understand and anticipate the direction of the threat landscape.
Measuring detection performance. Analyzing statistics on the performance of malware detection to understand which protection technology is providing us the most value.

Quantitative Benefit