Since joining the Control-M team in June 2015, I have been immersed in Big Data. Yes, that’s right – perceptions on most things I
the sensors capturing data and sending it back to Chevrolet so they can constantly update me on my vehicle health and when I need to have my tires rotated. My online shopping experiences are now as much focused on what back-end systems and applications the company is using to know that yes, those shoes are just what I want to go with that outfit – and don’t forget the handbag options too!
Data is all around us, being collected all the time. Machine learning is a key contributor driving behavior today – whether you realize it or not. The influence of what’s learned is continually presented to us – just think about how many times you interact with an app on your mobile device in a day. It’s all data driven.
What this view into big data technology really has given me is an appreciation for something quite amazing – Hadoop. And with the release of Control-M for Hadoop 9, BMC continues to enable companies to automate so many aspects of their own Big Data journey.
My favorite feature in this release is the ability to execute workflows defined in Oozie through Control-M or convert Oozie jobs to Control-M using an intuitive, wizard-based interface. What’s better than choosing what works best for you? Either way, all of the
powerful Control-M capabilities including SLA management, easy access to log/output, archiving, workload policies and self-service are extended to your Hadoop workflows.
Another key feature allows you to connect Apache Spark™ with the rest of your eco-system using Control-M’s drag and drop capabilities while running Spark SQL and Spark streaming written in Java, SCALA or Python. You no longer need to create a complicated web of scripts to build and run Spark jobs.
The more I learn about Hadoop, the more I get to see code. Lots and lots of code. I am not an application developer so a lot of it looks “Greek” to me. What I do know is that when that codes gets written over and over again, copied and pasted and then edited for changes, errors will inevitably create hiccups here and there. So it only stands to reason if you can automate
frequently used code in a virtually fail safe mode, why wouldn’t you? Control-M for Hadoop 9 now supports moving data between Hadoop clusters with integrated DistCP (Distributed Copy) commands where you have to provide only the target and source
destinations. This automated approach reduces the time it takes you to code and significantly reduces the potential for errors. And you can copy files to Amazon S3.
And naturally, as new versions of Hadoop are released Control-M will continue to add support. In this release we extended support to include MapR 4.x/5.x and Ubuntu 14.x.
In the end, this really means that Control-M can make it easier for you to drive your Hadoop project to production. It’s when it gets to production that your company will be able to drive answers from all of the data they collect, and likely realize there’s data they can be collecting that they aren’t yet. And of course that drives the need for storage – and Control-M helps there too – offloading data to Hadoop, lowering storage costs and making it easy to search the archived data. Your day job is easier, you saved the company money, and you contributed to uncovering the nugget that drove a competitive advantage skyrocketing the business to the top! And all because you automated with Control-M.
Or, if you want to see it through my eyes, it means that I am more confident that the plane I will be on for the next four hours is going to
safely get me from here to there; that the item my daughter has to have for school will arrive right on time – so says the notifications from the shipping company; and that my go-to music app will indeed point me in the direction of my next new favorite song.
What is your company doing with Hadoop and your data to change the direction you’re headed or the experience your customers are going to have next week, next month and next year? I’d like to hear how data is changing your perceptions.