Skip navigation

Control-M and Hadoop

Posted by Jim Gingras May 28, 2015
Share This:

I find myself more and more convinced  of the value and energy around Big Data and, more specifically, how Hadoop is transforming the way we use and derive value from the many forms of information in and around business and IT.  For example, most IT people are aware of how the Hadoop framework can be used to implement workflows that provide specific business information to decision makers to enable them to respond to changes in the market and their individual business strategies.  More recently, IT is embracing this same ability of Hadoop to marshal and search both structured and unstructured data to analyze the billions and billions of events in the application, server, and system logs across their IT enterprises.

 

I always enjoy learning the new names for old capabilities.  The new generation of 'google-driven' terminology is both entertaining and enlightening.  With tool names like Sqoop, Hive, Flume, Soir, BeeHive, Pig, Scalding and Spark, to name a few, it is interesting to watch the new generation of developers and IT experts derive and implement business-focused and relevant solutions. 

 

Even more interesting is to help industrialize and commercialize the production versions of this spark of innovation driven by real-time information analytic capabilities.


With my Enterprise Architect hat on, I am always looking for reusable patterns that promote best practices and standards.  This is where enterprise automation tools come in.  After the innovation is complete and the need to automate the tasks and workflows required are defined, it comes down to marshalling (ETL data and organize into relevant searchable indexes), analyzing (perform sql and sql-like queries in both real-time and batch) and reporting the information derived from all that data in a format that is consumable by business and IT decision-makers.   This also requires the ability to coordinate these activities across the entire IT and business enterprise.

 

This is exactly what Control-M provides through its best-of-breed Enterprise Workload Automation capabilities.

 

I know I am a bit biased, but am eager to hear issues and concerns with this using Control-M to automate the workflows required to implement Hadoop and the business use cases to enable this new age of information management.

Filter Blog

By date: