Big Data is a mega trend that is changing how data centers and businesses operate. It is being influenced by other technology mega trends -- like Cloud and Mobile. Shamoun Murtza, the CTO for financial services at BMC Software, has been researching how big data affects organizations -- and how big data is impacted by other technology megatrends. He recently talked with us about what he is seeing in the marketplace when visiting with companies of all sizes.
This blog shares some of the highlights from the discussion with Shamoun.
Big Data is one of several major technology mega trends that are shaping the world’s data centers and how organizations operate. Whether it’s an insurance company, financial institution or government agency, big data is giving new insights that help businesses to seek new opportunities, enter new markets, protect from security breaches and even reduce cost.
What’s changed? Users are changing and how they use data is changing. You need to rethink that – because mobile is so impactful. An example of this is a story from a bank in Australia. Quarter over quarter, the bank was seeing a 10 percent reduction in Teller transactions. Yet for the same period of time, online transactions increased 400 percent. It’s a completely different usage pattern along with a very different volume of transactions. So what does this do to your backend processing?
Hear more about the nexus of these mega trends in this 3 minute video.
Getting Started with Big Data
Murtza states the first thing you need to do is identify a problem. This can be a new problem or one that you have not been able to solve with traditional solutions. There are three main areas that need to be considered:
- Data acquisition.
Identify the data sources and where the data can be acquired. There can be a lot of considerations for data acquisition – based on the problem you want to solve.
- Data store.
There are choices for data storage, but there is a clear winner emerging in the big data area and that is Hadoop.
There are a lot of start-ups in this area – in particular when it comes to data visualization.
If you strip aware the buzz from big data – what you have is batch processing. This is good news as batch processing is a well-known and established process.
Big data opens a lot of new doors. That is what is getting companies excited. Hear more in this 2+ minute video.
Connect Hadoop to the Enterprise
How do we actually extract data out of Hadoop. You need a method that is resilient and reliable. So how do we do that?
HDFS – is the Hadoop distributed file system where you store the data. MapReduce is how you get data out of HDFS. It is a java-based programming language that works with HDFS. It processes and gives a result set very fast and very efficiently. The reality? It is all batch processing.
How do you manage all of the batch processing in Hadoop? Because of the volume of data and complexity of the analytics, processing in Hadoop can become very complicated very fast when looking at large clusters. The good news – this is a known problem – i.e., managing batch processing. This is what workload automation solutions are designed to do. No need to waste your time finding a new solution for Hadoop. Spend your time on solving the new problems that Hadoop now lets
See how Control-M can help you with your Hadoop batch processing needs. It natively supports Hadoop and various Hadoop projects. So you can now manage Hadoop with the same solution you use for managing all of your other enterprise batch processing. Don’t spend your time re-inventing the wheel.
Many organizations are doing big data to gain a competitive advantage. If you are not doing big data, does that mean that others have a competitive advantage over you?
If you want to listen to the series of thought leadership videos by Shamoun Murtza, here are the links:
Think Forward: Big data and other mega trends
Think Forward: Get started with big data
Think Forward: Control-M connects big data to the enterprise
It is an exciting time to be in the technology industry.