Big Data Systems Get Serious

The use of big data programs such as Hadoop and NoSQL systems is on the rise. Security teams working with the Hadoop teams in their organizations may see an attempt by the latter to convince them that Hadoop security can be handled by built-in mechanisms. These big data staffers may throw a lot of strange names at you and dazzle you with marketing collateral.

Don’t get me wrong: It’s a great thing that Hadoop is beefing up security. But it’s your job to look at the big picture for big data security and compliance. If the organization has standardized a data security and compliance platform, and if that platform supports Hadoop, then the infrastructure is in place to bring Hadoop under your security umbrella.

Register for the tech talk: Guarding against insider threats to Hadoop: What’s new in Guardium

Making the Most of Security Controls

Work with the Hadoop teams to understand how and when native controls fit the bill. Then identify those cases in which compensating controls are required. It may even be that the security solutions complement each other nicely, allowing the organization as a whole to optimize its data security and compliance infrastructure and processes.

These complementary capabilities may include:

• Automated compliance workflow to help automate the solution and to obtain auditable sign-off of reports;
• Policy-based infrastructure that can be used across the data landscape, not just within the Hadoop cluster;
• Low-impact monitoring paired with rich reporting and investigative capabilities;
• Separation of duties in which auditing data is separate from the Hadoop cluster and its privileged admins;
• Use of dynamic data masking and blocking as a compensating control for certain cases; and
• Analytics of user or group behavior to detect outliers that can help filter out the noise from the activity.

For More Technical Information

For the more technical among you, this recorded tech talk provides a deeper dive into the security implications for Hadoop. For the latest and greatest, see this developerWorks article, which includes an overview of the significant enhancements made for security and auditing in Hadoop environments.  This information can help you learn more about how the platform can extend to big data and beyond.