1 2 3 Previous Next

Control-M

162 posts
Share: |


Since joining the Control-M team in June 2015, I have been immersed in Big Data.  Yes, that’s right – perceptions on most things I

encounter are now “data-based”.  I don’t think about driving cars the same way – it’s now2015-chevrolet-suburban-texas-edition-01.jpg about all

the sensors capturing data and sending it back to Chevrolet so they can constantly update me on my vehicle health and when I need to have my tires rotated.  My online shopping experiences are now as much black dress and shoes.pngfocused on what back-end systems and applications the company is using to know that yes, those shoes are just what I want to go with that outfit – and don’t forget the handbag options too!


Data is all around us, being collected all the time. Machine learning is a key contributor driving behavior today – whether you realize it or not.  The influence of what’s learned is continually presented to us – just think about how many times you interact with an app on your mobile device in a day.  It’s all data driven.

 

What this view into big data technology really has given me is an appreciation for something quite amazing – Hadoop.  And with the release of Control-M for Hadoop 9, BMC continues to enable companies to automate so many aspects of their own Big Data journey.

 

My favorite feature in this release is the ability to execute workflows defined in Oozie through Control-M or convert Oozie jobs to Control-M using an intuitive, wizard-based interface. What’s better than choosing what works best for you?  Either way, all of the

Spark 3.png

powerful Control-M capabilities including SLA management, easy access to log/output, archiving, workload policies and self-service are extended to your Hadoop workflows.

 

Another key feature allows you to connect Apache Spark™ with the rest of your eco-system using Control-M’s drag and drop capabilities while running Spark SQL and Spark streaming written in Java, SCALA or Python. You no longer need to create a complicated web of scripts to build and run Spark jobs.

 

The more I learn about Hadoop, the more I get to see code.  Lots and lots of code.  I am not an application developer so a lot of it looks “Greek” to me.  What I do know is that when that codes gets written over and over again, copied and pasted and then edited for changes, errors will inevitably create hiccups here and there.  So it only stands to reason if you can automate

DistCP.png

frequently used code in a virtually fail safe mode, why wouldn’t you?  Control-M for Hadoop 9 now supports moving data between Hadoop clusters with integrated DistCP (Distributed Copy) commands where you have to provide only the target and source
destinations. This automated approach reduces the time it takes you to code and significantly reduces the potential for errors. And you can copy files to Amazon S3.

 

And naturally, as new versions of Hadoop are released Control-M will continue to add support.  In this release we extended support to include MapR 4.x/5.x and Ubuntu 14.x.

 

In the end, this really means that Control-M can make it easier for you to drive your Hadoop project to production. It’s when it gets to production that your company will be able to drive answers from all of the data they collect, and likely realize there’s data they can be collecting that they aren’t yet.  And of course that drives the need for storage – and Control-M helps there too – offloading data to Hadoop, lowering storage costs and making it easy to search the archived data.  Your day job is easier, you saved the company money, and you contributed to uncovering the nugget that drove a competitive advantage skyrocketing the business to the top!  And all because you automated with Control-M.

 

Or, if you want to see it through my eyes, it means that I am more confident that the plane I will be on for the next four hours is going to

CTM Hadoop datasheet image.jpg

safely get me from here to there; that the item my daughter has to have for school will arrive right on time – so says the notifications from the shipping company; and that my go-to music app will indeed point me in the direction of my next new favorite song.

 

What is your company doing with Hadoop and your data to change the direction you’re headed or the experience your customers are going to have next week, next month and next year?  I’d like to hear how data is changing your perceptions.

 

To find out more about Control-M for Hadoop take a look at the web page, datasheet or ask your questions here.

Share: |


In January 2014, I published a workload automation trends prediction for 2014 blog primarily focused on the impact of Gartner’s “Nexus of Forces.” These forces are often referred to as SMAC (Social, Mobile, Analytics and Cloud), widely recognized as the disruptive technologies driving the consumerization and industrialization “mega trends.” In 2016, some of these trends remain strong (with some fine-tuning). As expected there are some new trends emerging as well. The one theme that is reflected across all of my 2016 predictions is the evolution of workload automation to application workflow automation, accommodating the needs of both application developers and IT production control.

Here’s my view on what you should be expecting: Workload Automation: Predicting Trends in 2016 - BMC Blogs

 

wla-blog.png

Share: |


Learn how you can successfully install and configure high availability with Control-M/Server.

 

On Wednesday, January 27th, David Pavoncello will discuss and demonstrate the following:

            • Control-M/Server 9 HA architecture and HA database options (Oracle, PostgreSQL, etc.)

            • Difference between manual and automatic failover

            • Log collection and analysis for HA Control-M/Servers

            • Debugging HA Control-M/Servers

 

There will be a Q&A after the demo. Register Here!

Share: |


Learn how you can successfully deploy upgrades to Control-M Agents.

On Wednesday, October 21st, Octavio Vazquez presented and demonstrated the following:

 

        • Overview of the new Agent Deployment Tool
        • How to setup the Agent Deployment Tool
        • Upgrading existing Agents to V9
        • Apply Fix Packs to V9 Agents (new V9 feature)
        • Maintaining Agent versions and Fix Packs

 

Here is the Q&A for this webinar (Connect With Control-M: Control-M 9 Agent Deployment)

________________________________________________________________
Q:  Is there any maximum limit  for the number of agents that can be handled for a single upgrade activity?

A: No there's not a maximum limit, this number is defined by the deployment_threads Control-M/Server system parameter.
________________________________________________________________
Q:  Hi..Do you have this Agent deployment option is avilable for CM for SAP, CM for Database, CM for AFT and other plugins for upgrade as well?

A: Currently the agent deployment tool is only able to upgrade Control-M/Agent version and deploy fix packs for Control-M/Agents 9.0.
________________________________________________________________
Q:  Can we see the installation log from CCM manage tab?

A: No, you can only see the activity log and the errors, but not the installation log.
________________________________________________________________
Q:  Any special considerations with using root on Unix environments?

A: No, there are no special considerations for agents running in root or non-root mode.

________________________________________________________________
Q:  Is it a "No down time" upgrade/downgrade activity?

A: The upgrade with no downtime for Unix Agents is available since Control-M/Agent 8.0. For windows it's available since Control-M/Agent 8.0.00.200.

________________________________________________________________
Q:  How can we install agent which is running with root user by using agent upgrader

A: There are no special considerations for agents running in root or non-root mode.

________________________________________________________________
Q:  Can you demonstrate how to schedule an agent upgrade?

A: The agent deployment tool doesnt schedule the upgrade action. You can create a transfer package activity to pass it to the agents that you want to upgrade and triger the installation via a regular Control-M job.

________________________________________________________________
Q: Will Control-M 9 Agent Deployment touch existing installed Control Module plug-ins?

A: Currently the agent deployment tool is only able to upgrade Control-M/Agent version and deploy fix packs for Control-M/Agents 9.0. It can be considered for future versions or fix packs.

________________________________________________________________
Q:  Does the upgrade process make sure no activity is happening on the server?

A: No, the agent deployment tool doesnt validate that there are not activities on the agent before upgrading it. You can create a transfer activity to pass the package to the agents that you want to upgrade, schedule a maintenance window and start the upgrade when there's no impact for your environment.

________________________________________________________________
Q:  Is it applicable for the agents running in non-root mode?

A: There are no special considerations for agents running in root or non-root mode.

________________________________________________________________
Q:  Hi on wht scenario's do we downgrade the agents

A: In case that a fix or an enhancement impact your regular operation you can use the downgrade option to rollback the upgrade.

________________________________________________________________
Q:  Is there a free space availability as prerequisite on the agent being upgraded ?

A: Yes, the default is 2048 MB. This size is controlled by the AD_REQUIRED_DISK_SPACE Control-M/Server System Parameter.

________________________________________________________________
Q:  Does the transfer of upgrade package use existing agent ports or does it require other ports to be open?

A: Yes, the transfer use the existing agent ports. There's no need to open other ports.

________________________________________________________________
Q:  Does the zero downtime only adhere to the Agent upgrade or does it include Fix-pack upgrades also?

A: Currently it only adheres to the Control-M/Agent upgrade, fix packs still requires to stop de Control-M/Agent.

________________________________________________________________
Q:  For windows agents does it matter what the agent default install directory is?  Will it just what the existing directory is?

A: No, it doesnt matter. We retrieve the agent install directory from the Control-M/Agent registry keys.

________________________________________________________________
Q:  You've only mentioned agentes update for Windows and Linux, it is the same procedure for OS400?

A: It is the same procedure for OS400 agents.

________________________________________________________________

Share: |


start now.jpg

 

BMC training schedule is posted for January and February.  We want you to be successful in BMC solutions.  We run classes year round and worldwide across the BMC product lines.  Below are the classes listed by class/product name.


Review the below and register today.  Please check BMC Academy for latest availability, BMC reserves the right to cancel/change the schedule.  View our cancellation policy and FAQs.   As always, check back in BMC Academy for the most up to date schedule.


To see all our courses by product/solution, view our training paths.


Also, BMC offers accreditations and certifications across all product lines, learn more.

 

For questions, contact us

Americas - education@bmc.com

EMEA - emea_education@bmc.com

Asia Pacific - ap_education@bmc.com

 

ClassDate/Location
BMC Control-M Workload Automation 8.0: Administering

11 January / Americas / Online

18 January / EMEA / Paris, FR

25 January / EMEA / Online

8 February / Americas / Houston, TX

22 February / Americas / Online

22 February / EMEA / Winnersh, UK

BMC Control-M Workload Automation 8.0: Scheduling

4 January / Americas / Online

11 January / EMEA / Paris, FR

18 January / EMEA / Online

18 January / Americas / Houston, TX

8 February / Asia Pacific / Online

15 February / Americas / Online

15 February / EMEA / Winnersh, UK

BMC Control-M Workload Automation 9.0: Administering

11 January / EMEA / Winnersh, UK

25 January / Americas / Online

8 February / EMEA / Online

15 February / Asia Pacific / Online

BMC Certified Professional: BMC Control-M Workload Automation 8.0 Practical Exam

4 January / Americas / Houston

4 January / EMEA / Winnersh, UK

4 January / Asia Pacific / Pune, IN

4 January / Asia Pacific / Bangalore, IN

1 February / Americas / Houston, TX

1 February / EMEA / Winnersh, UK

1 February / Asia Pacific / Pune, IN

1 February / Americas / Mexico City, MX

1 February / Americas / Sao Paulo, BR

29 February / Americas / Houston, TX

29 February / EMEA / Winnersh, UK

29 February / Asia Pacific / Pune, IN

29 February / Americas / Mexico City, MX

29 February / Americas / Sao Paulo, BR

29 February / Asia Pacific / Bangalore, IN

Share: |


New to Control-M Workload Automation?

 

The Control-M Support team would like to partner with you to give a more cohesive understanding of what to do when needing assistance. We have created a series of informational videos describing how to accomplish common tasks for your Control-M environment.  The goal of these videos is to make your life easier while working with Control-M and with BMC support.

 

Learn how to enhance your experience with BMC Control-M by utilizing the following resources!

 

Resources

How do I register for a BMC Support Login ID?

How do I navigate and use the BMC Support website?

How do I create and manage issues with BMC Support?

How can I download Control-M Products and Fixpacks?

Where do I find documentation on Control-M?

What are the supported Operating Systems, Databases, etc. used by Control-M?

How does Control-M product versioning work and how do I get the latest release?

 

Share: |


Learn how you can install successfully Control-M 9.

 

On Wednesday, September 23rd, Ted Leavitt presented and demonstrated the following:

        • System Requirements and Compatibility
        • Installation Planning
        • Installation modes (Full installation vs Custom installation)
        • Installing Control-M V9 on Windows and UNIX
        • Verifying Installation


Here is the Q&A for this webinar (Connect With Control-M: Installing Control-M 9)

________________________________________________________________
Q: Please, could you repeat the URL to download "check_req" tool?

A: The URL for the check_req tool in Control-M 9 can be found in the installation Guide.  You can also go directly to:
ftp://ftp.bmc.com/pub/control-m/opensystem/PAOST.9.0.00.001/

________________________________________________________________
Q: Do you have Oracle Database?

A: Control-M 9 supports Oracle with UNIX installations.  We use Oracle in both support and the labs for testing and reproducing problems.  In the webinar, installations were performed using the dedicated PostgreSQL database.

________________________________________________________________
Q: How about install of Control-M 9 on cluster server, will it show today? or some other day?

A: We are planning on presenting installation of Control-M 9 on a Windows cluster in the near future.

________________________________________________________________
Q: I did not see an option to use an Oracle database in the Windows installation.  Was that because we chose default options?

A: Available database for Control-M 9 on Windows are MSSQL and PostgreSQL.  I would suggest referring to the SPAC utility or the product release notes for more specific details.

________________________________________________________________
Q: When would you not activate the gateway?
A: When defining a new Control-M/Server, I would not activate the Gateway if I was defining the M/Server for testing purposes or it was not avialable to be connected to.

________________________________________________________________
Q: What are the recommended workstation specs for wa 9 vs 8?

A: The minimum requirements are documented in the release notes.  I believe for both EM 8 and 9 client, the minimum amount of RAM is 4GB.  The EM client will use more memory with the more jobs and alerts are loaded.  If you have hundreds of thousands of jobs, viewpoints showing these jobs will consume more memory and you will have to plan accordingly.

________________________________________________________________
Q: Can Oracle be used for the archive?

A: Control-M Workload Archiving does not currently support Oracle.  There are plans to support Oracle with Workload Archiving with upcoming maintenance, although I do not have a firm date on this.

________________________________________________________________
Q: Which user manages the default agent on unix machine? I see the agent is installed by default and there is a ctm_agent dir under Home dir for ctm server user but, a little confused about who the agent user would be.
A: With the installation of the Control-M/Server on UNIX (performed as the Control-M/Server user), the local Agent is installed as the Control-M user (ctm9 in this case).  The Agent will be started and run as this (non-root) user by default.

________________________________________________________________
Q: Is it possible to install multiple instances of controlm/server 9.0 with a dedicated pgdb on the same physical windows 2012r2 server together with controlm/em server 9.0 also with a dedicated pgdb and what options should be checked/unique/shared ?

A: Only one instance of Control-M/Server or Enterprise Manager can be installed on a Windows host.  They can each have their own dedicated Postgres databases running independantly of one another.  A Windows host can have multiple instances of the Control-M/Agent.

________________________________________________________________
Q: Is there any upcoming Application Integrator session?
A: Most definately.  We are planning on doing this early 2016.  Application Integrator support will be included with the release of the first fixpack for Control-M 9, tentatively scheduled for January 2016.

________________________________________________________________
Q: will there be webinar on failover?
A: Yes, we are planning on hosting a webinar on Control-9 HA in the next couple of months.

________________________________________________________________
Q: Can I use different A->S  & S->A  ports than my control-M_7.0  server/em  so I can have both versions talk to my agent hosts?

A: It is not supported, and you do not want to try, to have multiple Control-M/Servers submitting jobs to the same Control-M/Agent instance.  This can cause a variety of problems with delayed status updates, missing output and missing jobs.  If you are upgrading your Control-M/Server, you can use your old Control-M/Agent with the new Control-M/Server until you have an opportunity to upgrade it.  We will be discussing the Control-M Agent Deployment tool in our October webinar.

________________________________________________________________
Q: Can I leave the control-M7.0_agent software on my windows/linux/solaris boxes & not upgrade the agents ?

A: Yes, please see above.  You will eventually want to upgrade your Agent, as the old versions will eventually become unsupported and new features and functionality will not be available in them.

________________________________________________________________
Q: failover will support SQL Server also means if we use SQL server can we able to one click option to do the changes ?

A: The Control-M failover is managing only the Control-M appplication.  Any failover in MSSQL will be managed outside of the Control-M 9 HA.

________________________________________________________________
Q: Can the V9 Gui coexist with V8 Gui on a machine?

A: The Control-M/Enterprise Manager clients can coexist on the same PC.  You can have version 7, 8, 9, and multiple instances of each on the same PC without issue.

________________________________________________________________
Q: Do you have any documentation on highavailability for control-m 9 anywhere?
A: There is documentation in the Installation and Admin Guides regarding HA.  We are continuously working on improving the documentation, creating knowledge articles, and videos to better describe and demonstrate the functionality.

________________________________________________________________
Q: How we can provide the client distribution to our app users... What is the replacement for weblaunch?

A: The Control-M 9 Client Distirbution is the replacement for weblauch in 9.  We are planning on hosting a Webinar on this topic in the next few months.

________________________________________________________________
Q: Is there an option to point to a MSSQL database in the linux install?
A: No, MSSQL is supported for Windows only.  Linux supports the use of PostgreSQL and Oracle.

________________________________________________________________
Q: I see you installed the EM and Control-m Server on separate databases on Linux, is there advantages or disadvantages installing this way?

A: Installing using different databases gives you the flexibility to manage and maintain these seperately.  Things like backing up and restoring the database are greatly simplified.  Support generally advises that installations of EM and M/Server have their own Postgres in production environments.

________________________________________________________________
Q: If we run more than 10000 jobs (including 500 to 1000 cyclic jobs) then what about the memory utlization of the control-m em?
A: There is a sizing template in the Installation Guide.  We would advise filling this out and contacting support for more accurate estimates of the memory required on the EM Server and Control-M/Server host.

________________________________________________________________
Q: Will the prerequisite script available for agent?
A: Yes, a prerequsite check script is available for the Control-M/Agent 9.

________________________________________________________________
Q: Is there any new features for applying  DST?
A: Time Zones and DST functionality is the same as in verison 8 for the most part.

________________________________________________________________
Q: We have too many videos or may I havn't found the right one , may I get the link to recevie a overall picture of Control M as a software, how things work right from installation and backaend processing. Sorry for a very basic requirement here.
A: There is some higher-level material available on the www.bmc.com site.  We will also add to our webinar topic list an architecure review.  Thank you for the suggestion.

________________________________________________________________
Q: Previously we had to repair Mirroring in case it broke down due to any network issue or any other issue. In this v9 , do we had to do the same in case of a High Availability environment ?

A: If mirring breaks, it will still have to be repaired.  Mirroring is more tolerant in the past, but still requires a low-latency connection between the primary and secondary.

________________________________________________________________
Q: When deploying EM across multiple servers, do you now support multiple naming servers or is this still a single point a failure on the primary host.
A: The Control-M 9 EM failover is designed to work with a single naming service.  The naming service will be running on the primary host.

________________________________________________________________
Q: Can we use controlm 7 agents with controlm 9?

A: Control-M/Agent 7.0.00 will function with Control-M/Server 9, however, we would advise planning upgrades to the new version.  There are a number of new features and improved scalability which you will benefit from.

________________________________________________________________
Q: Control-M server and control-M/EM ver 8,if i install control-M agent v9 is it suport ?

A: The Control-M/Agent 9 should work with the Control-M/Server 8, although this is not extensively tested.  The communication protocol version on the Agent would need to be set to be compatible with the Control-M/Server.

________________________________________________________________
Q: Does it still supports agents from MPE/iX systems ?

A: As I understand HP has discontinued support for MPE/iX, although you should be able to submit jobs to MPE/iX as a remote host if it has a function sshd.

________________________________________________________________
Q: Will v9 have seperate small WA Admin client install package as opposed to v8 having to download the entire 3GB Windows full install package? If not, please consider doing this.
A: The installation media (zip and ISO) are still large and a separate installation has not been prepared on media.  What we do have with Control-M 9 is the new Client Distribution tool, which should allow for simplified client installations.

________________________________________________________________
Q: What changes made in the webserver orientation in V9.0?
A: We are looking to remove Silverlight from our web applications, trainsitioning to HTML5, but this will be coming with future maintenance.  The version of tomcat has been upgraded to 7.0.50.

________________________________________________________________
Q: Does EM server support failover?
A: EM does support failover with MSSQL and Oracle.  We are continuing to work on the HA with Postgres and EM.

________________________________________________________________
Q: Will the BMC migration facility provide any new functionality to allow backout/downgrade to a previous version?
A: There is currently not any specific functionality provided for backing out an upgrade.

________________________________________________________________
Q: In the agent config  can we auth two ctm servers with diffrent versions?

A: The Control-M/Agent should only receive jobs from one Control-M/Server (environment).  Authorizing the Agent to multiple Servers is possible, and would be done, for example, when upgrading, but it Agent should not be receiving jobs from both M/Servers at the same time.

________________________________________________________________
Q: can we change the data size from medium to large and vice versa after few months?
A: With the Postgres database, the database (files) will continue to grow as needed allowing available disk space in the datbase files location.

________________________________________________________________
Q: In current v9 is there a need to enable mirroring in case of High availability environment when it breaks ?
A: The HA for Postgres relies on database mirroring, yes.  When the M/Server fails over to the secondary, mirroring from the secondary to the primary will have to be re-established prior to failing back to the primary.

________________________________________________________________
Q: Can we install control-M 9 server and EM in same machine?
A: Yes, Control-M/Server and EM can be installed on the same host.

________________________________________________________________
Q: So if you are no longer using silverlight with v9, thats that mean Selfservice in v9 will not use sliverlight?

A: With the GA release, silverlight is still in use, but there are plans to move off Silverlight in the near future.

________________________________________________________________
Q: On Windows can we have different versions of Agents?
A: It is not supported to have different versions of Agents installed on the same Windows host.

________________________________________________________________
Q: Can the new Control-M v9 push an agent to install from Enterprise Manager?

A: We cannot install a NEW Agent with the Agent Deployment tool.  We will be talking more about the new Agent Deployment tool in October.  If you were so inclined, you could install a Control-M/Agent using it first as a remote host and then utilizing a silent install script, but this is not something "out-of-the-box".

________________________________________________________________

Share: |


Simply put – the next generation of leading workload automation software is now available.  Don’t delay – download now and see what makes Control-M 9 the best yet.

 

ctm_wallpaper_1920x1080_Updated.pngAnd yes, while I am biased (I’m not afraid to admit it), I am also confident in the statement above.  Being relatively new to the workload automation space (after years with systems and applications performance management, IT asset management and service support solutions), I have the unique position of a fresh set of eyes when it comes to workload automation solutions. The more I see of the solutions available on the market and the more I hear from customers and prospects, the more I realize Control-M is a precious gem.

 

But enough about what I think. It’s really about what you think.  While I can highlight some of the latest features and benefits, it’s how you are able to adopt and leverage the capabilities within Control-M to make your life easier – lower your total cost of ownership, improve the value you deliver to the business and of course deliver applications faster, right? Here are some new features that should do just that:

 

Automated Agent and Client Deployment – Using the Control-M 9 automated agent and client deployment capability, you can reduce manual upgrades and maintenance. However long this has taken you in the past, it will take less time now.  We have customers that will go from months to days and some from weeks to hours.  The agents and clients can be updated in a single step or the update package can be transferred and deployed at a later time increasing the flexibility you have around updates and deployment all of which will help keep your costs down (and we all know if you can help save on costs, that earns brownie points with the boss).

Runtime Estimations.png

Run Time Analytics – Control-M 9 features

advanced out-of-the-box insight into all jobs in an enterprise view at a glance, significantly improving run time estimation with Batch Impact Manager analytics without being defined as a part of a Batch Impact Manager service.


High Availability – Control-M High Availability enables you to maximize production environment uptime, prevent data loss, and ensure business continuity in the event of hardware failure or for maintenance. Both automatic and manual failover capabilities are supported and both are managed from a single point of control.

Promotion.png


Automated Application Workflow Promotion – Using Workload Change Manager, you can now automate application workflow promotion across environments, from test to dev to production, accelerating application change and deployment cycle times.  Because during the promotion process all job definitions are modified and validated according to your Site Standards of your target environment, workflow deployment time can be reduced by up to 80%. This one will go a long way to making your life easier – what’s better than eliminating human errors and manual work?


In addition, Control-M for Databases is also releasing an update. Control-M for Databases offers intuitive, interactive, secure, and seamless integration with database systems. Key features include:

CTM DB 9 - connection profile add.png

Open platform support for JDBC-compliant databases – Simply put, you can now configure connection profiles to any database platform with user-provided JDBC drivers such as SAP Hana, Teradata, MongoDB, and MySQL. 

Support for MSSQL Integration Services Catalog (SSISDB) – Of course you likely already know that SSISDB is a central repository included with MSSQL 2012 (and later) that allows users to manage and monitor SSIS projects with related configuration and parameters. Control-M for Databases 9 supports properties overrides, options to select relevant environments and the ability to browse for SSIS packages in the SSISDB.

 

Support for new database platform releases – out-of-the-box platform support now includes native 64-bit Linux, MSSQL Server 2012 and 2014, Oracle database 12 and IBM DB2 10.

 

Additional enhancements include a secure connection to MSSQL, having a set number of characters in the output columns, leveraging a custom separator in CSV outputs, and the option to keep Oracle comments and hints.


So now you have the highlights – and you want to know more.  Here are some key resources that will help you as you plan to update your environment to Control-M 9.

 

Datasheets: Control-M 9 / Workload Change Manager / Control-M for Databases


Videos detailing new features for Promotion and High Availability


A new whitepaper – Control-M is Free: The Economic Benefits of Workload Automation – I’m including this one because it’s a phenomenal paper – not from BMC but from one of your recognized peers, Brian Anderson at HSN. See, I’m not the only one that thinks Control-M is quite something!  (Brian will also be presenting at Engage if you want to ask him personally about his experience with Control-M. Session 95: Building your Brand: Are you an “Automation Profit Center” or a Faceless Black Hole of IT Spending - Select Keywords and enter "95" for the Session ID.)


If you missed this last week it’s work watching – Alf’s Zoo latest blog focused on an interview with Robby Dick and Tom Geva from BMC discussing Control-M 9.  A great way to spend about nine quick minutes if you didn’t read the details in this post!


Engage 2015.pngIf you’re going to be at Engage be sure to attend Session 314, “What’s New in Control-M 9”. This is an advanced technical session with our experts Joe Goldberg and karol smeczechowski. And don’t forget about the hands-on labs for Control-M 9 – one for the administrators (LL42) and one for the schedulers and operators (LL43).  I love that this release is close to Engage – what a great way to get all the information you will need to know for your update!  For more details on what’s being covered around Workload Automation at Engage take a look here (select track: BMC Control-M Workload Automation). And most importantly – don’t forget to come take a swing at the BMC Control-M: Powering Innovation Power Drive booth (aka golf simulator).  When it’s virtual we can all be Tiger Woods you know!  Or Rory McIlroy…or Jordan Spieth…or Sergio Garcia…you get the idea.


And finally, as if that wasn’t enough – we want to make sure that you have a chance to ask your questions about Control-M and get answers direct from our awesome product management and development teams. Details on the Google Hangout set for mid-October will be coming soon. Look for them here, Twitter (@BMCControlM), LinkedIn and Facebook – yes, Control-M is definitely social!


Can't wait to hear what you think about Control-M 9!


Share: |


Learn how you can benefit from Workload Archiving by storing your critical Job log and output data long term.

 

On Wednesday, August 19, 2015, Joel Brecker presented and demonstrated the following:

 

        •Reducing the size of your Control-M Database
        •Reducing output disk space on the Agent
        •Keeping job log and output for up to a year or longer
        •Using Archiving policies to specify what data to retain
        •Easily viewing Archived data from the Control-M Enterprise Manager Client

 

Here is the Q&A for this webinar (Connect with Control-M Webinar: Managing Job Log and Output with Workload Archiving)

-----------------------------------
Q:  If V9 installation sits on the same machine as Control-M Enterprise Manager, can Workload Archiving be installed with an existing Oracle database?
A:  Workload Archiving version 9 will support the use of an existing Oracle database when Fix Pack 1 has been applied.

-----------------------------------
Q:  What are the supported databases for Workload Archiving?  Oracle, MSSQL, Sybase, PostgreSQL?  Can I use an existing Database install base?
A:  Workload Archiving version 8 supports the included PostgreSQL database.  Fix Pack 2 will allow Oracle database.

-----------------------------------
Q:  How can I backup my Workload Archiving database?
A:  The Workload Archiving Database can be backed up using the same hot backup process used for backing up you Control-M Enterprise Manager Database.  The Workload Archiving Server has the scripts necessary to complete the hot backup.

-----------------------------------
Q:  Will I see Workload Archiving in the CCM?
A:  Workload Archiving 9 will be displayed in your Control-M Configuration Manager 9.  Workload Archiving 8 is managed from the server that it is installed on and is not shown in the Control-M Configuration Manager.

-----------------------------------
Q:  Can I run Control-M reports against the Workload Archiving?
A:  Workload Archiving is only for job logs and output.

-----------------------------------
Q:  What if I have many applications for the same Archive policy?  Can I have multiple entries on the Application field? (separated by comma, for instance)
A:  You can have multiple entries in the “Exceptions” column separated by commas.  The “Criteria” field allows wild cards and single named items.  It is possible to select specific items using both these fields.

-----------------------------------
Q:  Are there any Alerts before the Workload Archiving database fills up?
A:  There are XAlerts when the database fills up.

-----------------------------------
Q:  Workload Archiving uses ARCUSER, can we change and use a different ID than ACRUSER?
A:  The ARCUSER is an internal user created when Workload Archiving is installed.  There is no documented process for changing this user ID.

-----------------------------------
Q:  Once you set up a rule, if you change it later, will that change take effect immediately? or is it necessary to recycle something?
A:  After creating a new rule and saving it, this new rule takes effect immediately.  Any data currently on your Control-M Server and Agents at the time of the rule will be Archived.  If you change a rule it will also take effect right away.  It is not necessary to recycle anything.

-----------------------------------
Q:  In my environment, the archive search is greyed out, does it mean that the Archive add on is not available?
A:  Yes.  To use Archive Search Workload Archiving must be installed.

-----------------------------------
Q:  Should the cleanup start time be set to a different time than our New Day to avoid any performance issue, or it's ok to do it at the same time New Day runs?
A:  Workload Archiving does not have any impact or connection to the New Day process.  You can set your daily Clean up cycle to any time that you wish to have it run.

-----------------------------------
Q:  What about in v8, I see that Workload Archiving is greyed out, what does this mean?
A:  Workload Archiving must be installed prior to its use.

-----------------------------------
Q:  We have version 8 EM. Do I need to install Workload Archive in my existing EM server account in HP UX server, or is it going to be a separate installation like Control-M Server?
A:  Workload Archiving is a separate application and should have its own machine.  It will require a separate installation.

-----------------------------------
Q:  How can more than one item be included in the rule criteria?
A:  The “Criteria” field accepts wild cards, which allows “All” items to be selected, and can be limited by using the “Exceptions” field to enter multiple items separated by commas.

-----------------------------------
Q:  In control-m 9, is Workload Archiving a part of the installation?
A:  Workload Archiving is a separate application with its own installation process.

-----------------------------------
Q:  We can view 4 days logs and output, sometimes when I try to load the history for any environment, I get the error "failed to open an archived viewpoint due to load limitation at server".
A:  Workload Archiving does not handle Archived Viewpoints.  It only Archives job logs and output.

-----------------------------------
Q:  What is the average time for Cleanup? During cleanup can we view the archived job logs and output?
A:  The Clean-up process is relative to the amount of data to be removed and is quick.  You can still view and use the Archive data during cleanup.

-----------------------------------
Q:  How to do you troubleshoot if there is a communication issue between the Control-M Enterprise Manager and Workload Archiving component?
A:  You will use the arc_test_configuration command.

Share: |


The other day I was discussing a particular process that gets performed ad-hoc and how we could turn that workflow into an automated process and then tie it into a self-service request.  This particular process had been done for years by a few folks manually running scripts and submitting data to a few services via web interfaces. It worked relatively well and the particular manager I was talking with was of the mindset that if it worked, let it be, there were other more important things to work on.

 

I then pulled out one of my favorite words to use when talking about Workload Automation - visibility.

 

Wouldn't it be great to have visibility into that process?  To know where we were in the chain, if a failure occurred somewhere, and get an estimate of how much longer we had to go?

 

The manager lit up with excitement, and that "Thanks, but no" turned into a "Yes! Yes! Yes!"  In less than two days we had the process automated, setup so that end users could access it via a self-service mechanism and fully visible.

 

Visibility is one of the great benefits of Workload Automation.  Often we tend to talk about the IT benefits of getting process flows automated or put into a Workload Automation system and we tend to forget that many of the workflows we support are IT-driven, but are supporting business units on the other side.   Managers often don't care how something gets done, just that it gets done.  They want information and insight into the process, whatever it may be.  Give them more than they had now - such as visibility - and you will always win them over.

 

Batch scripts litter network drives everywhere.  CRON and Scheduled tasks hide on servers across data centers around the world.  Those scripts are great for what they do - and indeed they are a first step in the automation cycle - but they provide zero visibility into what is happening and how it impacts the rest of the business.  What happens when the person that runs that batch script on their desktop leaves for vacation for two weeks?  How about maternity leave for 6 months?  The work stops happening often times without anyone away until weeks or months later when someone notices data missing.

 

We talk about agility in IT operations, but how agile is our business processes if they are still depending on people and processes hidden away on servers?  Agility depends on visibility.  A set of tasks that is only known to one or two people stops agility in it's tracks when those people suddenly aren't available, or something goes wrong and only one person can diagnose it while entire groups of people are scrambling to figure out why a process that has worked for years has suddenly stopped without any warning.

 

The next time you get the age-old question of "Why should I put this into Control-M?" ask them if visibility matters.  You may find you turn the entire conversation around, and give people a new way of thinking about the benefits of Workload Automation.

Share: |


Learn how to maximize managing Control-M jobs with Viewpoints and Job Versioning.

 

On Wednesday, July 22, 2015, James Pendergrast presented and demonstrated the following:

        • Using Viewpoints to manage your current jobs and active environment
        • Advanced features available in the Monitoring Domain
        • Using Archived Viewpoints to view job flows from previous days
        • Using Job Versioning to audit job changes
        • Using Job Versioning to recover jobs

 

Here is the Q&A for this webinar (Connect With Control-M: Job Viewpoints, History and Versioning)

________________________________________________________________

Q: For the parameters MaxOldDay and MaxOldTotal, are these set for each defined gateway or they apply in general for all the gateways?
A: These parameters apply to all the Gateways. This is per data center, so if there are two data centers, and 7 days, there will be a total of 14 days worth of data. 
________________________________________________________________

Q: What needs to be done if you change the MaxOldDay and MaxOldTotal?
A: If these parameters are changed, then the Gateway needs to be restarted.
________________________________________________________________

Q: Any data how to predict table space increase needed when increasing MaxOldDay and MaxOldTotal parameters?
A: The space needed to store depends on several factors including how many jobs run. A good way to estimate is to check the size of the current database tables IOALOG_*, and divide by the current number of days being kept. This will give an approximation for one days worth of logs. Then multiply this value by the number of days MaxOldDay and MaxOldTotal will be set to.
________________________________________________________________

Q: When I create a viewpoint is it available for all users or just for me?
A: Viewpoints are stored in the database, so they are accessible for all users.
________________________________________________________________

Q: There is a filter that can be used with ALLJOBS, called NOTOK.  In fact, that filter can be used on any Viewpoint.  Why the need to create a new one--as Viewpoints all need to be loaded into Memory of the EM Server when started?
A: This is true, viewpoint ALLJOBS is loaded, then a filter can be applied for Job Status.
Another method is to create a Viewpoint for users who may not know how to use the filter. 
________________________________________________________________

Q: If I have 5 of the same viewpoints open, does it take 5 times as much memory?
A: If there are five different viewpoints open, then memory for each will be used. If five of the same Viewpoint is opened, then only one will need memory.
________________________________________________________________

Q: when I do playback, what time is it starting and ending from?
A: The Start and End times of the Playback is based on the start and end time of the Gateway download. Normally this is the NewDay time.
________________________________________________________________

Q: If I want to keep more versions of the job, is it necessary to recycle a component if I change "VMVersionsNumberToKeep" parameter?
A: The GUI Server will need to be cycled if the "VMVersionsNumberToKeep" parameter is changed.
________________________________________________________________

Q: Will History ever allow for viewing more than one datacenter? If I have two Control-Ms, I can only view one at a time?
A: This may be changed in a future release, but for the current versions, the History is stored based on the DataCenter/Gateway.
________________________________________________________________

Q: When you restored the deleted job did it go back to the folder it was deleted from or did it create its own folder?
A: The job is restored to the Folder/Table that it is deleted from. 
________________________________________________________________

Q: I can see neighbors but I cannot see the neighborhood feature as you showed when selecting the neighborhood?  Is this a setting? 
A: This is a feature that is in Control-M EM version 8, fixpack 6.
________________________________________________________________

Q: How can we create a view point for the jobs that are running longer than 5 days?
A: If you use the AllJobs Viewpoint, and set a filter to ODATE, then you can see jobs that are currently running that are older than that date.
________________________________________________________________

Q: Is there a way to find out when a job last ran without going thru archived viewpoints?
A: If the Job is visible in the EM GUI, then you can check the Statistics. If the job is not in the GUI, then the only way would be to look in the Archived Viewpoints, or run a Report on the Job Statistics. The Server utility ctmruninf can be used.
________________________________________________________________

Q: How to find who did the changes in the active job environment? All I can see in the logs were job is Hold by x user and Free by x user...
A: The Server logs show what actions were taken by what user.
________________________________________________________________

Q: what is the max value you can put for VMVersionsNumberToKeep paramater?
A: There is no known limit to this value.
________________________________________________________________

Q: In versioning, can a complete Folder be restored?
A: Versioning only restores a Job. Restoring the Job, will also restore the Folder if it doesn't exist. All the jobs can be selected and restored, but the Folder itself can't be restored.
________________________________________________________________

Q: Is there a way to expand the tree in the monitoring view with one command?
A: The Tree View does not have a shortcut to expand. Each "plus" sign will need to be selected.
________________________________________________________________

Q: Can we create viewpoint that display long running jobs (eg. running for more than 6 hours)?
A: Any of the Job Attributes, including StartTime can be used. The time will be static from when the Viewpoint or Filter is opened, so it will need to be refreshed.
________________________________________________________________

Q: Could you please explain the need of Control resource in Control-M?
A: The Control resource is used to control access to a resource. For example, you may have a job that needs exclusive control of the database while the job is running.
________________________________________________________________

Q: Are job statistics not longer available for the last several runs?
A: Job statistics are available for the last runs of the job. If the job is in the AJF, and visible in the GUI, then statistics will show when the job ran last. If the job is not in the AJF, and not visible, there isn't a way to get to the statistics.
________________________________________________________________

Q: Anyway to do the playback feature on the current active?
A: The Playback feature is only available for History/Archived Viewpoints
________________________________________________________________

Q: Can you identify who has an open history view, and close the history view after it has been opened a set time to prevent people holding some of the quota all day?
A: This question has been answered verbally.
________________________________________________________________

Q: Can we compare 2 versions of a folder and get a summary?
A: This functionality is not built into Control-M, but could be easily done by exporting the folders to XML and then comparing the files.
________________________________________________________________

Q: Will increasing the stored number of days affect database performance?
A: Increasing the stored # of days will cause more database storage but it won't affect the performance (speed). 
________________________________________________________________

Q: General Question -- Is there a timeout on WLA Client?  Self Service and WCM both have timeouts.  Is there a setting to do this in the client? Thick Client WLA.  Timeout to lock it such as it does in Self Service and WCM.
A: The Client does have a timeout, and will attempt to re-connect 3 times. There isn't a method to lock the Client after a specified time. The Client can be Kicked off from the CCM, by manually selecting the GuiServer, and using the Control Shell.
________________________________________________________________

Share: |


If you are running multiple Control-M servers, including Mainframe and Distributed environments, you can control your workload flow between the environments using global conditions. For example, after a job completes on your Mainframe Control-M/Server, it can trigger a job on your Distributed Control-M/Server and vice versa.

 

On Wednesday, June 17, 2015, Froilan Reyes and Richard Talbert will discuss the following about Using Global Conditions:

 

        • The Global Condition Server(GCS) process overview
        • GCS configuration
        • Passing conditions between Control-M/Servers
        • Demonstrate adding, using and testing global conditions between mainframe and distributed environments

 

________________________________________________________________
Q:  Are we able to pass conditional (maybe) conditions?
A: Maybe conditions are normal conditions that get added with the ctmldnrs. Once this conditions are added, if it is a global condition it will be passed to the specified CTM Servers.

________________________________________________________________

Q: What is the length of a condition?
A: For the DS side it can be 256, for the Mainframe it is 39
________________________________________________________________

Q: If passing conditions between cyclic jobs what is the best way handle them if they run closely together?
A:  There can be a two minute limitation for the same global condition to be updated. To be safe make the updates at least two minutes apart.
  
________________________________________________________________

Q: What happens if the GCS is down?
A: The global conditions won't be sent between Control-M/Servers.
________________________________________________________________

Q:  Does timestamp on global conditions come from the originating datacenter?
A: Condition time stamps are when they get posted to the ctm server. GCS is only concerned when a condition that has not been processed, comes in with in 2 mins. So if a datacenter was down for several hours, GCS would treat all of the updates as if they were just posted. There are some limits on how far back (ODATE) GCS will look at conditions

________________________________________________________________

Q:  Are any kind of characters or symbols allowed in the name of a condition?
A:  Special charters are allowed, but this is limited to what is specified in the Parameter Guide for conditions name.
________________________________________________________________

Q:  In system configuration, system parameter VARIABLE_INC_SEC is set to LOCAL where does the Global Condition get saved if it doesn't go to the agent?
A: VARIABLE_INC_SEC is used with global variables not global conditions  
________________________________________________________________

Q: A condition can be posted into two or more Data Centers at the time?
A: If the global condition is setup to go to more than one ctm server, it can post to how ever many it is setup to post too.
________________________________________________________________

Q: Can a global condition performs a force job also from a Folder?
A: Global conditions don't force or order jobs. I merely creates conditons that allow jobs or folders to be executed.
________________________________________________________________

Q:  Is it advisable to use bypass option when we have global conditions ?
A: At that point, for the job, it is just a condition that has not been posted. You then get to choose to bypass it, add it (be careful), or hold the job and delete the condition from it. Bypass does not add the condition.
   ________________________________________________________________

Q:  On those global maybe conditions is there documentation that tells how to configure ADDMNCND and CTMLDNRS to check for them on another server?
A: We do not have the ability to do global maybe conditions, out of the box.
________________________________________________________________

Q: Why Global Condition doesn't show up in the listing ?
A: If the prefix is not on the listing then most likely that prefix will no longer be used.
________________________________________________________________

Q:  Say for instance - If the GUI is down, Is there a way to add GC from command line or any other source?
A: For now there is no way to create the Global Condition Prefix, other than the client Workload Automation
________________________________________________________________

Share: |


Learn how you can benefit from Workload Archiving by storing your critical Job log and output data long term.

 

On Wednesday, August 19, 2015, Joel Brecker will discuss and demonstrate the following:

        •Reducing the size of your Control-M Database
        •Reducing output disk space on the Agent
        •Keeping job log and output for up to a year or longer
        •Using Archiving policies to specify what data to retain
        •Easily viewing Archived data from the Control-M Enterprise Manager Client


Don’t miss a live demo of these features. There will be a Q&A after. Register Here!

Share: |


On Wednesday May 20th 2015, Pilar Soria will discuss how to install and configure Control-M for Hadoop and demonstrate creating jobs to automate Big Data. The webinar will include:


• What is Control-M for Hadoop?
• Installing, configuring and setup of Control-M for Hadoop
• Creating Map Reduce and Sqoop jobs
• Hadoop job Conversion tool
• Demo configuring Control-M for Hadoop with Kerberos

 

------------------------------------------
Q: Do we have developer studio to practice?
A: Control-M for Hadoop jobs can be defined from Workload Automation Client.

------------------------------------------
Q: Does my kerberos ticket lifetime in the confirmation management have to match my ticket lifetime in my kerberos configuration?
A: Equal or less than what is configured

------------------------------------------
Q: Do I need to run kinit by hand to run Hadoop jobs from Control-M?
A: No, Control-M for Hadoop will run kinit command automatically

------------------------------------------
Q: Do I have to run the agent as root to do impersonation?  Do I have to run as root without kerberos?
A: For impersonation (Kerberos impersonation) Control-M Agent should start as Control-M Agent owner. The impersonation is done at Kerberos level with “ksu” utility.
   Without kerberos, Control-M Agent has to be started as root user in order to submit jobs with other users (Run as)

------------------------------------------
Q: Does the CM for Hadoop support Windows?
A: Currently, Control-M for Hadoop only supports Linux 64 bit machine

------------------------------------------
Q: Are you planning on supporting spark with the CM for Hadoop?
A: Control-M for Hadoop will support spark in next patch that is tentatively planned to be released in July.

------------------------------------------
Q: what is the minimum space required to install hadoop?
A: To install Hadoop you will need about 1.2GB.

------------------------------------------
Q: Can I run commands that are not HDFS commands as pre/post?
A: No, in Pre/Post commands only Hadoop fs commands can be submitted.

------------------------------------------
Q: Where can we use Control M¬?
A: Not sure I completely understand the question. Control-M Supports a variety of Operating Systems, including MainFrame, Windows and Unix.

------------------------------------------
Q: Does CM can handle various jobs at a time by automation?
A: Control-M for Hadoop can submit and monitor jobs concurrently.

------------------------------------------
Q: does DB schema and procedure can be run through Hadoop module?
A: Not sure I understand completely the question but if you meant manipulate Database schema and run procedure with a Database we have Control-M for Database the supports that.

------------------------------------------
Q: How can I get access to the conversion utility?
A: Control-M for Conversion Tool can be downloaded from EPD (http://www.bmc.com/available/epd.html)

------------------------------------------
Q: do we have any conversion tool to move jobs from hadoop to control M.
A: Yes, with BMC Control-M Conversion Tool we have the capability to read the Oozie scheduling metadata and build the scheduling definitions for you in Control-M.

------------------------------------------
Q: Can you provide a brief ppt on what exactly control M does?
A: I might suggest you take a look at the overview @ http://www.bmc.com/it-solutions/control-m.html which discusses Control-M's functionality

Share: |


 

I have the fortune, and sometimes misfortune, of travelling around much of the world for my job. Every time I am in Tel-Aviv I make it a point to visit what is quite possibly my favorite restaurant in the whole wide world.  It is an unassuming spot on the ground level of an office building.  It is a Tripolitanian place that has magnificent meats, vibrant vegetables, great grains, and super spices. I would guess they also have delicious desserts, but the starters and main courses are so amazing I never make it that far!

A typical spread looks like this:

In all honesty I am not 100% sure on the exact details of what is in every dish.  I would also go so far to say that individually each dish is good, maybe even great, but when combined together the whole experience and combination of tastes is simply amazing. 


As I span the globe talking to customers and prospects, the subject of applications and how to manage their workloads is often the topic of choice. It occurred to me that much like my favorite restaurant and its wide variety of dishes, the same holds true for all those applications.  Individually they are good, maybe even great, but when combined together they deliver the business services that ultimately are not just part of your business, they ARE your business.  From billing to payroll, logistics to shipping, online to back office, there is no business service you provide that is not backed by a number of applications working together.

 

For a longtime I have been convinced the best way to integrate all those applications together is through workload automation.  And with the June 2015 release of Control-M Application Integrator there are more ways than ever to integrate most any application into your Control-M Workload Automation environment.  Application Integrator is a self-service web design tool that allows you and your application developers to create your own Control-M job types for almost any purchased, open-source or homegrown application.  Once created you can then manage the workloads for that application inside of Control-M the exact same way you do for the rest of your mission critical workloads.  You don't have to rely on insecure and hard to maintain scripts.

I have 4 kids that aren’t crazy about Tripolitanian food, but they do love pizza.  Once in a while they will ask for a frozen pizza and my response is always the same….”Life is too short to eat bad pizza”.  Living in the Chicago area means I have many amazing pizza places nearby and I think it is nearly a sin to waste time on lesser pizza.  Life is also too short to manage workloads in an inefficient and risky way. If you agree, check out any of the following resources to see how Control-M Application Integrator can help you manage ALL your workloads in a consistent and comprehensive way.

Control-M Application Integrator Web Page

Community Sourced Repository for Application Integrator Job Types

Webinar Covering Application Integrator

 

Comments are welcome and I would specifically like to know 2 things:  What food combinations are better together than separate and what applications are most important to the delivery of your business services?

Filter Blog

By date:
By tag: