Skip navigation
1 2 3 Previous Next

Control-M

174 posts
Share:|

Need to integrate an application with Control-M? Interested in making your own Control-M job types? Want to ensure that operators understand how to schedule custom jobs that integrate with your applications? If so, register for the April Connect with Control-M webinar!

 

On Wednesday, April 27th, Joel Brecker demonstrated, step-by-step, how you can successfully create your own job types, and integrate your custom applications with Control-M. During this Connect with Control-M webinar, Joel:

 

 

    •   Provided an overview of the Control-M Application Integrator

 

 

    •   Provided examples of the different types of applications you can integrate with (SOAP, REST, command line)

 

 

    •   Had a demo about the creation of new job types, and showed how to schedule them

 

 

    •   Provided basic troubleshooting

 

 

Here is the Q&A for this webinar (Connect With Control-M: How to Integrate Applications with Control-M Application Integrator)

 

_____________________________________________________________

 

Q: How many execution steps can you have?

A: There is no documented limit for the number of execution steps allowed for an Application Integrator Job Type.

_____________________________________________________________

 

Q: When do we need to make a connection Profile?

A: You only need to make a connection profile if your integration Job Type is command line.

________________________________________________________________

 

Q: Is there some kind of version management for custom job types created using Application Integrator?

A: Version Management is planned for a future version of Application Integrator.

________________________________________________________________

 

Q: Has Application Integrator been given special considerations for an automated upgrade, for example moving from Control-M 8 to Control-M 9?

A: Application Integrator is an add-on to the Enterprise Manager and is upgraded when the module itself is upgraded.

________________________________________________________________

 

Q: What is the Application HUB?¬

A: Yes, the Application Hub is a community page where you can share Application Integrations with other members! You're free to take and add as much as you'd like!

________________________________________________________________

 

Q: Does running Application Integrator on a Linux machine require the use of XWindows or a GUI to launch it?¬

A: The Application Integrator uses a Web based designer for creating the Job Types and does require a supported Web Browser:

- Chrome 20 or later

- FireFox 25 or later

- Internet Explorer 10 or later (compatibility mode OFF)

________________________________________________________________

 

Q: Is there a user authorization setting for read/write permission in the Application Integrator Designer?¬

A: This functionality was added in Application Integrator Fix Pack 2.

________________________________________________________________

 

Q: When creating connection profiles for Application Integrator is an account file also created on the Control-M Agent side which contains these settings?¬

A: Yes there is a cm_accounts.xml file created in the /cm/AI folder on the Control-M Agent which holds all the connection profile settings.

________________________________________________________________

 

Q: Question - Can you make a copy of built-in job type (or other custom job type) and use it as a starting point? Is it done via EXPORT/IMPORT?¬

A: The Application Integrator will only export/import files of the type .ctmai and the built-in job types of Control-M are exported as XML.

________________________________________________________________

 

Q: Do you recommend maintaining non-production and production Control-M environments and promoting jobs from non-prod to prod?

A: It is good practice to test any new job type thoroughly before placing it into a production environment.

________________________________________________________________

 

Q: How do you move a job type from one environment to another?

A: To move a Job Type from one environment to the next you will need to export it from the current environment and import it to the new environment.  Keep in mind that if the job already exists in the destination environment that it will first be undeployed before it is re-deployed.  This will remove any connection profiles and associated files for that AI Job Type.  It is recommended to make a backup of the connection profiles before the undeploy.

________________________________________________________________

 

Q: How do you import a job type from the Application Integrator Hub?

A: To import a Job Type from the Application Hub you can click on the "Import from Application Hub" button in the AI Designer and choose the Job Type you would like to import, so long as you have access to the internet from your Control-M Enterprise Manager Server.  The alternative to this is to visit the Application Hub from a machine with internet access and download the Job Type and Import it using the AI Designer.  The application Hub can be found here:

https://communities.bmc.com/groups/bmc-control-m-application-hub

________________________________________________________________

 

Q: How many PRE and POST execution steps can you have?

A: You can only have 1 PRE and 1 POST command per Job Type.

________________________________________________________________

 

Q: Where do I download application types?

A: From the Application Hub located here: https://communities.bmc.com/groups/bmc-control-m-application-hub

 

________________________________________________________________

 

Q: How are these job types published to new Agents after creating the new job type, is it Automatic or a manual task?

A: If the Job Type already exists when you add a new Control-M Agent then you would need to log into the AI Designer and deploy the job to that new Agent.

________________________________________________________________

 

Q: Can an executing process return status codes directly to Control-M, or is parsing text the only way to read status information?

A: Standard out error messages and status codes are sent based on the return code handling you put into place while designing the new job type.  You can add additional error handling by parsing for specific information that should cause a job to fail.

________________________________________________________________

 

Q: In the example, password entered into connection profile is a parameter in the command. Is the command with password exposed in SYSOUT?

A: The password is not exposed as long as you are using a Masked textbox.  The "password" textbox is a Masked textbox and can only be placed into the connection profile section.

________________________________________________________________

 

Q: We have an application for which we kick off cubes within the database. Could the tracking of that cube within the DB be tracked with AI?

A: This will depend on how the "Cube" reports its progress or what indicates it has finished.  An example would be if the "Cube" creates output based on its progress, you could take this output and return specific progress reports.

________________________________________________________________

 

Q: Do we have full control on AI from CCM? (Like enabling debug..etc)

A: Application Integrator is controlled from the AI Designer.  You have full control of debug by logging into the AI Designer.  The designer is a Web Application and will be up as long as the Web Server is running.

________________________________________________________________

 

Q: If you have a job with multiple execution steps and 1 fails, can you rerun the job from the step that failed?

A: This functionality is coming in a future release of Application Integrator.

________________________________________________________________

 

Q: Are there any AI integration prebuilt for Oracle Data Integrator?

A: Please check the Application Hub for the current listing of Job Types shared within the community.

https://communities.bmc.com/groups/bmc-control-m-application-hub

Share:|

Did you know that by integrating your Control-M/Agent with sudo you can enhance your auditing capabilities, and add more flexibility to your jobs’ user authentication? Interested in seeing Control-M/Agent run jobs as a root user? Want to ensure your Control-M administrators learn how to take advantage of, and understand the different Control-M/Agent 9 modes in your environment? If so, register for the May Connect with Control-M webinar!

 

On Wednesday, May 25th, Neil Blandford will demonstrate, step-by-step, how you can successfully configure Control-M/Agent 9 to run in three different modes (root, non-root, and sudo). During this Connect with Control-M webinar, Neil will:

 

  • Provide an overview of the different modes in Control-M/Agent 9, and their considerations

 

  • Explain the new functionality in Control-M/Agent fix pack 100

 

  • Demonstrate jobs running in the different Control-M/Agent modes (non-root, root, and sudo)

 

  • Provide basic troubleshooting

 

Don’t miss a live demo of these capabilities. There will be a Q&A after the demo. Register now!

Share:|

If I were to answer that question, based upon emotional connection, it would be Magnum.  Ruggedly handsome man, awesome moustache, crazy cool shirt and best of all – that Ferrari!  Throw in 2 best friends, one that flies helicopters and the other a bar owner, and a stuffy but interesting in a funny sort of way “guardian” in Higgins, and what’s not to love?!

80magnumpi_05.jpg

But that answer is based upon the proverbial “good old days”.  While I had fun watching Magnum PI as a kid, it really has no impact on my life today.  But SAP does.   SAP is often regarded as the most used commercial software application in the world.  It is used by hundreds of
thousands of companies, many of which I interact with on a daily basis.  From the car I drive (which is nothing like a firetruck red Ferrari 308), to the work travels I routinely take (in commercial airliners and not cool helicopters with neat paint jobs), to the shirts I wear (which are exclusively not vibrant Hawaiian shirts) – and most everywhere in between - SAP software is behind the scenes.

 

Back in the early 2000s the first application Control-M provided support for, in the form of an Option, was SAP.  Over the years many  enhancements and additions have been made to Control-M in support of SAP (Business Warehouse, Data Archiving, Business Objects, etc), with the latest being the addition of support for SAP Process Integration (SAP PI).

 

SAP Process Integration is part of the SAP Process Orchestration bundle.  It is a middleware component of the SAP NetWeaver product group that is used to facilitate the exchange of information among SAP ERP internal software and systems and those of external parties.   SAP PI is the SAP recommended way to integrate between SAP HANA Cloud and SAP NetWeaver – which are part of SAP’s strategic direction.

 

Control-M for SAP Process Integration enables the management and automation of SAP PI channels by scheduling ‘channel start / stop’ events and monitoring the channel status, allowing improved utilization of the SAP PI channels which in many cases are a limited resource.

job1.jpg

To get to the bottom of some mystery in your personal life you can find someone like Magnum PI in a city near you, they just won’t be as amazing or as cool as Magnum was.  To solve the mystery of how to best manage workloads for your SAP environment, check out the Control-M for SAP page where you can find information on managing workloads for SAP PI along with many other SAP related components.

 

And that is both amazing and cool!!

Share:|

Need for High Availability? Interested in reducing downtime? Want to ensure application continuity?

 

On Wednesday, March 30th, Martin deCastongrene demonstrated, step-by-step, how you can successfully implement high availability for your Control-M/Enterprise Manager 9. During this Connect with Control-M webinar, Martin presented and demonstrated the following:

 

Provide an overview of the new Control-M/Enterprise Manager 9 High Availability
Give step-by-step installation guidance for the Control-M/Enterprise Manager 9 High Availability failover node
Show how to configure Control-M/Enterprise Manager 9 for High Availability

 

Here is the Q&A for this webinar (Connect With Control-M: High Availability with Control-M/Enterprise Manager 9)

 

________________________________________________________________

 

Q: When will this capability be available for PostgreSQL?

A: The integration of PostgreSQL for Control-M/Enterprise Manager is being looked at and will be available in a future Control-M/Enterprise Manager Version or FixPack.

________________________________________________________________

 

Q: How are extra items such as Forecast, BIM, Self Service etc. handled?  Do they need to be installed on the HA server also?

A: The additional installations such as BIM, Forecast and Self Service are activated when installed on the Primary Control-M EM  platform.  As the Secondary installation points to the same database as on the Primary, no additional action is needed on the Secondary.

________________________________________________________________

 

Q: We would like to have the failover instance in a remote datacenter. Are there any latency limitations? What if the database did not failover?

A: DNS and network latency will need to be responsive.  Testing should show if it is successful.

   The Secondary Node will failover to the same Database the Primary Node originally uses.

   If the failover does not complete it will automatically fall back to the Primary node.

________________________________________________________________

 

Q: Can a previous standalone Control-M/EM installation be converted to a secondary HA failover node?

A: For a standalone to be converted, it will need to be uninstalled and reinstalled as a High Availability Secondary Node.

________________________________________________________________

 

Q: Are the Failover Dialog items logged in a file somewhere that could be viewed at another time?

A: The EM CHA (Coordination for High Availability) process can be set to a higher debug level (MiniumDbgLvl 5 5) in order to log failover events to the cha*.log file located under the Control-M EM installation "log" directory

  ________________________________________________________________

 

Q: How long should the failover process normally take?

A: Anywhere in the 3-5 minute range would be considered normal.

________________________________________________________________

 

Q: Why isn't PostgreSQL database supported for HA in EM?

A: Development is continuing work on integrating a PostgreSQL solution for HA.

________________________________________________________________

 

Q: We are planning to set HA on our DR datacenter, can we install database on both sites (primary and DR)?

A: As there is only one Database, it is recommended to be on a third machine, which can be located in either the network where the Primary is or in DR.  The Database just needs to have good network response time to and from the Primary Node and the Secondary Node.

________________________________________________________________

 

Q: The switch over looks great out in demo, what are the common issues which we may come across or based on your experience any comments on the same...

A: DNS and network latency should both be performing well for the Failover and Fallback to complete.

________________________________________________________________

 

Q: Can you point the HA instance to an alternate Oracle DB that is kept in sync with the primary via Oracle Golden gate?

A: The Secondary HA instance is designed to point to the Primary database. We would ask to open a support ticket to discuss specific requirements.

________________________________________________________________

 

Q: Notice webinar did not actually go through HA secondary install or show HA status of primary/secondary after install. I found that the Secondary EM and CTM config agent is not auto started either after install or secondary reboot.

A: The Control-M Install Guide has a section on what to include in startup scripts for Enterprise Manager on the Primary to start.

   Only the start_config_agent script section will be needed for the Secondary startup.

________________________________________________________________

 

Q: How can I leverage Control-M failover to have 24x7 availability when applying maintenance?

A: You can Failover from the Primary to the Secondary and run on the Secondary while applying OS level maintenance or reboot.

   After Fallback to the Primary the same can be done on the Secondary.

________________________________________________________________

 

Q: In what order do I have to apply fixpacks (primary first, then secondary)?

A: EM Fixpacks are to be applied first to the Primary and then to the Secondary HA platform.

________________________________________________________________

 

Q: When the failover occurs, will the root_menu on the primary be able to manage the secondary also?

A: Once the Failover occurs to the Secondary platform, the root_menu on the Secondary must be used to manage itself.

________________________________________________________________

 

Q: You have to manually start the EM and CTM config agent and put startup scripts in place to start after reboot of secondary. This needs to be documented in Admin and/or Install guides.

A: The Control-M Install Guide has a section on what to include in startup scripts for Enterprise Manager on the Primary to start.

   Only the start_config_agent script section will be needed for the Secondary startup.

________________________________________________________________

 

Q: Can you use the command line to initiate a failover, rather than using the ccm interface?

A: You can use the stop_config_agent script to shutdown the Configuration Agent and the CHA and that will trigger a Failover.

________________________________________________________________

 

Q: Where does Oracle db reside on primary or secondary ?

A: Our recommendation is that a third machine be used for the Oracle Database. 

________________________________________________________________

 

Q: Any plans to create a similar interface in the CCM for failing over Control-M Server?

A: The Control-M Server HA is also supported with the High Availability Tab in the Control-M Configuration Manager.

________________________________________________________________

 

Q: When you say that Fix Packs need to be applied to both nodes and then restarted before Failover and Failback can be accomplished, what (exactly) are the steps for applying a fix pack to both nodes?

A: You will want to bring both Primary and Secondary nodes down and EM Fixpacks are to be applied first to the Primary and then to the Secondary HA platform.

________________________________________________________________

 

Q: Will there be Failover/High Availability for the database as in version 7.0

A: The Control-M Server still can enable Mirroring in version 9.

_______________________________________________________________

 

Q: I have noticed in version 8 if I get disconnected I have to logout and back on to get a solid connection. Will the failover process have the same issue?

A: After Failover or Fallback you may need to press the retry button on the connection Dialog.

________________________________________________________________

 

Q: How long does it take automatic failover to happen?

A: Anywhere in the 4-6 minute range would be considered normal.

________________________________________________________________

 

Q: The presentation says: Fixpacks will need to be applied on both nodes... Can we failover to the secondary, apply FP2, and then fail back to primary, and apply FP2 on the secondary?

A: You will want to bring both Primary and Secondary nodes down and EM Fixpacks are to be applied first to the Primary and then to the Secondary HA platform.

________________________________________________________________

 

Q: Is there an additional license charge for HA?

A: You will want to discuss license issues with your Account Manager.

________________________________________________________________

 

Q: In this scenario, you assume 2 em on 2 different hostnames and 1 database in 1 some different hostname. What if the database is down or fails? is there some scenario of ha that includes the database?

A: Please open a support ticket and we can review the various options available with high Availability and disaster recovery as it relates to the database.

________________________________________________________________

 

Q: We have set HA with Postgres, should we enable archive mode for the PostgreSQL database?

A: You are welcome to enable the archive mode for the PostgreSQL.

________________________________________________________________

 

Q: If the alternate primary option is activated for a secondary node, is it possible to rollback the primary for the primary node without reinstall the first node?

A: Yes, you can just select the Fallback to Primary option and it will return to the original configuration; running on the Primary.

________________________________________________________________

 

Q: I found that the secondary EM and CTM config agent is not auto started either after install or secondary reboot. You must manually restart and create scripts for reboot.

A: The Control-M Install Guide has a section on what to include in startup scripts for Enterprise Manager on the Primary to start.

   Only the start_config_agent script section will be needed for the Secondary startup.

________________________________________________________________

 

Q: Do we need a shared disk in cluster for the installation?

A: The Enterprise Manager Database and Primary Enterprise Manager and Secondary Enterprise Manager can be on a standard disk with no cluster needed.

________________________________________________________________

 

Q: When it's necessary to debug this feature, should we enable debug on both sides, primary and secondary?

A: You will want to enable debug on both sides, Primary and Secondary.

________________________________________________________________

 

Q: Is there a way to configure the EM API web service to HA?

A: Currently High Availability is being supported for the Control-M/Enterprise Manager and Control-M/Server only. 

________________________________________________________________

 

Q: Can you define "same database"? SQL Listener is a single point of contact for Always On multi node setup. Will EM work point to the non-changing listener?

A: Please open a support ticket and we will gladly review the Oracle scenario in your environment.

________________________________________________________________

 

Q: If you weren't logged in the CCM, and it failed over, will the banner be displayed once I log in? 

A: The Failover banner will be displayed the next time someone does login to the Control-M Configuration Manager

________________________________________________________________

 

Q: When the HOST from primary is DOWN, the CLIENT connection remains asking for the primary node name for connections on the Secondary with the HA Secondary ACTIVATED?

A: This question has been answered verbally. 

________________________________________________________________

 

Q: If I have 2 environments setup with HA and then mirror the DB to the 2nd environment  if I shut down the 1st environment and bring up the 2nd can I expect that to work?

A: Please open a support ticket and we will gladly review your environment.

________________________________________________________________

 

Q: What will be the status of the executing jobs in the monitor? (during the failover process)

A: During the failover the Monitoring will be suspended until the GUI Server and Gateway have started on the Secondary node and the WLA Client is reconnected.

________________________________________________________________

 

Q: Does Control_M record the reason for failover and log the same ?

A: The EM CHA (Coordinator for High Availability) process can be set to a higher debug level (MiniumDbgLvl 5 5) in order to log failover events to the cha*.log file located under the Control-M EM installation "log" directory

________________________________________________________________

 

Q: If a FP contains DB changes then you can't failover correct?  You will have to shut down both EMs?

A: Correct.  You will want to bring both Primary and Secondary nodes down and EM Fixpacks are to be applied first to the Primary and then to the Secondary HA platform.

________________________________________________________________

 

Q: When failing over could there be any issues getting Alerts or Global conditions during the fail over process?

A: During the actual Failover Alerts and Global Conditions will be suspended until Failover completes.  Then any queued up Alerts or Global conditions will be propagated.

________________________________________________________________

 

Q: How would the account management settings change for Control-M Web Services API ?

A: This question has been answered verbally. 

________________________________________________________________

 

Q: For Control-M environments with Mainframe and Distributed are there any mainframe parameters that have to be configured for the IOAGATE for HA?

A: The Gateways for the Control-M datacenters, whether Mainframe or Distributed are defined and activated initially on the Primary platform and require no special handling when moved to the Secondary.

________________________________________________________________

 

Q: Can I automate, or schedule when to execute a failover from primary to secondary?

A: Using the stop_config_agent script with the password file option it could be possible to automate the failover.

________________________________________________________________

 

Q: Will HA work if the primary and secondary are at different fixpacks?

A: The Primary and Secondary Nodes need to be at the same FixPack level.

________________________________________________________________

 

Q: The install of the HA is part of the product install, I assume. Or is it a separate install?

A: The HA functionality is part of the Base install for Control-M EM 9.0.00, there isn't a separate download. We strongly recommend applying the latest fixpack to ensure you have the most recent fixes on the system.

________________________________________________________________

 

Q: Clarification for the FP install. If the FP we are installing changed the database, will the secondary (still not updated) know that it needs to operate with a possibly different schema?

A: Correct.  The FixPack install detects that it is being installed on the Secondary and installs only what is needed on the Secondary.

_______________________________________________________________

 

Q: So, in order to KNOW why it failed over DEBUG has to be turned on in order for that to be logged.  Is there not a performance impact to constantly running with DEBUG on? 

A: Leaving the EM CHA process in debug may have a slight resource overhead on the system as the activity is written to the log file.

________________________________________________________________

 

Q: What processes answer for a connection from a CLIENT calling for the primary NODE when it was jumped for the secondary node? If the HOST (machine) is offline, how the connections calling for the first node (that is offline) arrives for the secondary node?

A: Please open a support ticket and we will gladly review specifics of the Failover architecture.

________________________________________________________________

 

Q: What will be the status of the executing jobs in the monitor? (during the failover process)

A: The status of executing jobs will stay in executing state until Failover is complete and the Client reconnects and the status is updated. 

________________________________________________________________

 

Q: What are all the conditions need to checked before by control-m before switching to secondary EM?

A: The Primary CHA checks the Secondary CHA process in up, communicating and communicating with the Database.

________________________________________________________________

 

Q: Are the primary and secondary using the same CCM?

A: Yes.  A Control-M Configuration Manager connected to the Primary before Failover will be reconnected to the Secondary after Failover.

________________________________________________________________

 

Q: About the FPs: Then, the HA is not for applying CTM FPs. All components must be down in primary and secondary when applying FPs.

A: That is correct, when applying fixpacks, Control-M must be down. 

________________________________________________________________

Share:|

Need to integrate an application with Control-M? Interested in making your own Control-M job types? Want to ensure that operators understand how to schedule custom jobs that integrate with your applications? If so, register for the April Connect with Control-M webinar!

 

On Wednesday, April 27th, Joel Brecker will demonstrate, step-by-step, how you can successfully create your own job types, and integrate your custom applications with Control-M. During this Connect with Control-M webinar, Joel will:

 

•   Provide an overview of the Control-M Application Integrator

 

•   Provide examples of the different types of applications you can integrate with (SOAP, REST, command line)

 

•   Demo the creation of new job types, and show how to schedule them

 

•   Provide basic troubleshooting

 

There will be a Q&A after the demo. Register now!

Tom Geva

Batch is Back!

Posted by Tom Geva Apr 4, 2016
Share:|

Check out our multi-talented but anonymous (at his request) performer who came up with this amazing April Fool’s day song!

(mp3 attached below...)

 

I was justified when compute was new

To manage jobs, I typed commands a few

Times are changing, now the kids just tap But the fever's gonna catch you when the batch gets back

 

Post a picture or just check in

Even write an email, that’s not a sin

I can batch the jobs at your hack-a-thon I get high running workflows, it’s a lot of fun

 

It's the batch, It’s the batch

Oh the batch is back

Stone cold sober as a matter of fact

I can batch, I can batch

`Cause I'm faster than you

It's the way I compute

Algorithms I do

 

I entertain by running jobs

Process Big Data, even DB BLOBs

Scheduling jobs, my God, what's that

Oh it's nifty doing work when the batch gets back

 

Batch, Batch, the Batch is Back.

 

Now it's your turn! Record your favorite version of Amy Winehouse's "Back to Batch", AC/DC's "Back in Batch" or "Batch, Batch...Baby" (a little '80's ice here...), MC Hammer's "U Can't Batch This", or any other song that related to batch, scheduling or workload automation - and share it!

Share:|

blog1.jpgIsn't documentation a wonderful thing? Knowing exactly how "Tab A" fits in "Slot B" or how to connect your shiny new electric car to the charger or how long to bake those brownies is a good thing, right? When "winging it" is not an option, where do you turn?  “Read The Free Manual”?  That’s like stopping to ask for directions… “I know where I’m going!!!”  Why should you take time to read the documentation or the instructions or the recipe?  If only I followed my own advice… The “do what I say, not what I do” mentality works, right?

 

When I started at BMC five years ago, I did a lot of trial and error.  Lots of cool new stuff to learn. BMC, as everyone who uses and loves Control-M knows, does not rest on its laurels.  Control-M is continually being enhanced and improved with new features, functions and innovations.  New applications and platforms come down the pike every day.  From ETL tools to ERP tools to Big Data applications to Managed File Transfer the frequency and volatility of change is increasing.  And that level of change will only accelerate.

 

So how do you keep up? How do you handle this change and all these new tools and applications when you’re asked to schedule jobs blog2.jpgfor them? I decided to take a step back and look at this from a comfortable place; I put my scheduler hat back on and looked at this from a “how to” perspective. I figured out how to build jobs in Control-M and then created a “cookbook” document for it.  A Cookbook, as you know, is made up of many recipes on “how to make something”.  This revolved mostly around the Control Modules, each of which all have unique requirements and connection profiles to enable us to schedule that work.

 

Creating these “recipes” in v7 and updating them for v8 and v9 and then added new documents as new integrations came out took quite a bit of time. As the library grew it became clear a more “formal” process needed to be put in place to keep this information up-to-date.   To that end, the Control-M cookbook documents are in the process of being transitioned into the Control-M documentation. The “how to” documents will be melded with existing and new documentation as new versions and releases are introduced. And as new features and functions are added, this “cookbook” or “recipe” approach will continue to be incorporated into the documentation.

 

Where do I go? Here is a basic list of the contents of the “Cookbook Library”. The topic headings should be self-explanatory:

blog3.png

•    Control-M Best Practices and ITIL

•    Control-M CM & Agent How-to Examples

•    Control-M Non-standard Integrations

•    Control-M Other Procedures

 

As this transition proceeds you should see this “cookbook/how-do-I-do-that” approach be folded into the documentation. Until that transition is fully realized, I will keep updating and “publishing” cookbook articles. You may always ping me to get PDF versions of these documents until they are fully incorporated into the Control-M documentation. I have added the docs to Dropbox so you can grab the ones you need. I don’t pretend to have created a document for everything; I’ve tried to hit the high spots/most frequently used CMs and some of these docs are still v7 and v8. But once you delve into these documents, hopefully you will see some value and will be able to short-cut your learning curve. There are also quite a few documents other BMC SCs have contributed for “other processes and procedures”. I have added those documents to this Dropbox as well.

 

Use as you see fit and hit me up if you have questions. Happy scheduling!!

 

blog4.jpg

Share:|

 

  

I remember back in the day working in a datacenter batch environment.  I was responsible for working on failed processes, doing the research, figuring out what happened and why. 

 

This was on the Mainframe and we had sysout storing software that made it easier to look at previous job outputs or search to find errors in other jobs.  Mean time to repair was shortened because we had this capability.

 

These days nearly all of our customers have a distributed environment and the tools either do not exist or are costly to purchase and maintain to store the data.  The output files from jobs can be archived, but accessing them is not easy. Depending on the number and size of the outputs, some customers try to keep them on a server for as long as possible but that requires a sizable disk drive.

 

Workload Archive

 

As a Principal Software Consultant for Control-M, I was pleased when BMC introduced the Workload Archive component.  Since it integrates to Control-M, it can retrieve and store output, Control-M log and job runtime information in a separate database.  Since it can store this data for extended periods of time you now have a way to see previous runs of a job or search outputs and logs for specific information.

BMC Control-M Workload Archive

 

Simple to Use

 

Perhaps what I like most about Workload Archive is how easy it is to retrieve the data.  Access it from either Self Service or the WA GUI by simple performing a right click on the job in question.  In the WA GUI it takes you to the History domain and in Self Service it brings up a window with the previous job runs.

 

 

                    Self Service Right Click on Job                                           Self Service List of Jobs

 

                           

 

Easy to Search

 

Did you ever see a job failure and think: “Where have I seen that error before?”  Think of what you would have to do to find a job that had the identical error or message. It would take a lot of work.  With Workload Archive you can simply copy the output or log statement you want to search for, open a new Archive Search window, select * for jobname and paste the statement into either the output or log field.  You can look back over a specific time period or through every job in the Archive.

                   Highlight Statement in Output                                     Paste Highlighted Statement in Search

                                                                                  Results of Archive Search

 

 

 

 

 

In this example I actually found that another job was doing a file transfer at the same time my production job was attempting to send the file.  This is what caused the error.  Because Workload Archive allowed me to compare the output files from two independent jobs, I was able to determine why it received the error.

 

 

                                                                                                                                                                                                                                                                                                                                                                                                            Result of Archived Compare

 

 

Finding Changes

 

 

How many times have you seen a job fail and wonder “What changed?” Workload Archive allows you to pull up the different runs of that job and compare a successful run to the failed run. This is a simple as a right click on thejob, select More Analysis then Archive Search. 

 

 

The search window will open in the History domain showing the previous runs of the job.  You then select the two jobs you want to compare and also whether you want to compare the job log or the output.  The bottom of the screens show what each line looks like as it is clicked.

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Select Archive Search

    

                                      Result of Archive Search

 

                                                                                                                                              

 

Define Archive Retention Period

 

 

How long archived data is kept depends on factors such as auditing or how far back you want to keep it.  Within the Control-M Configuration Manager is an option to define how long to keep data.  You can keep SAP jobs for 2 years and test jobs for 2 weeks.  Multiple policy definitions can be made for different job types or platforms.  Workload Archive can store both distributed and Mainframe jobs. 

                                                                           Workload Archive Policy Definitions

 

MTTR Made Quicker

 

While things weren’t bad in the good old days, Workload Archive has definitely decreased our customers Mean Time to Repair.  Not only does it help the IT side but because it can be seen just as easy in Self Service, the end users have the same accessibility to the outputs and logs. 

 

Share:|

Learn how you can successfully implement high availability for your Control-M/Enterprise Manager Server 9 step-by-step to reduce downtimes, and keep your business up and running 24/7.

 

On Wednesday March 30th, Martin deCastongrene will discuss and demonstrate the following:

 

• Overview of the new Control-M 9 Enterprise Manager High Availability

 

 

• Configuring the Enterprise Manager High Availability

 

 

• Installing Control-M 9 High Availability failover node

 

 

There will be a Q&A after the demo.

 

Register Now

Share:|

Learn how you can successfully deploy Control-M 9 clients across your user base with an easy to use, centrally managed distribution tool.

 

On Wednesday, February 24th, José Luna presented and demonstrated the following:

        •        Overview of the new Control-M 9 Client Distribution Tool

        •        Configuring the Client Distribution Tool

        •        Distributing and installing Control-M 9 clients

        •        Applying Fix Packs to Control-M 9 clients with the Client Distribution Tool

Here is the Q&A for this webinar (Connect With Control-M: Control-M 9 Client Distribution)

________________________________________________________________

 

Q:  If I have multiple environments, must I install clients for both environments or can I install just one and use it to connect to either one?

 

  A: When you finish to installed a client with    Client Distribution Tool you can use the same client to define and connect to other environments as well although you will not receive updates from the original server. Connecting back to the original EM environment will get the updates.

________________________________________________________________

Q:  For clients that are remotely located (i.e in the US, while the EM Server is in the HK) – is there a way to deploy the client from a repository located in the US instead of HK to minimise the impact on WAN traffic?

 

A: It's not possible, the packages are placed automatically in the EM server    box, so when the installer is executed on client machine there isn't any way to select from  where the packages will be gathered.

_______________________________________________________________

Q:  How can I update plugins and forms on clients? Is it possible with Client Distribution?

 

A: Plugins are part of a fix pack/patch and will be updated once new fix pack or patch will be enabled. Client Deploy is not meant to update specific files, only complete packages with installation procedure.

________________________________________________________________

Q: How about a change in security policy (change from non-SSL to SSL)?

 

A: When deploying a client, the SSL certificates are automatically distributed and installed.

_______________________________________________________________

Q:  Is there a way to limit the number of clients that can be updated / deployed in parallel?

 

A: No there isn't.  

________________________________________________________________

Q: Can I configure the client from the install to work with another EM environment?

 

A: Yes, you can. Although you will not receive updates from the original server. Connecting back to the original EM environment will get the updates.

________________________________________________________________

Q: How can client disribution push AI (Application Integrator) fixpack 2?

 

A: Fix Packs from any Control-M module are not considered by Client Distribution Tool, it only considers    clients, Fix Packs, patches and SSL certificates   .

________________________________________________________________

Q:  Can updates to clients be downloaded in the background to the end user?

 

A: It's not possible

_______________________________________________________________

Q: It's the webserver included with controlm 9 or is it a seperate purchase?

 

A:    For Control-M 9 BMC provides Apache-Tomcat version 7.0.50.

________________________________________________________________

Q:  Can an alternate web server be used (i.e websphere) ?

 

A: Control-M does not officially support the use of other Web Servers in the orginization other then the embedded version of Tomcat.   

_______________________________________________________________

Q: So is this like the old weblaunch and follow up how is this different or better than a sccm push? - in our current v8 env we use sccm push to get it out to everyone 

 

A:    No, it isn't. Client Distribution tool is more flexible and easy to use.

________________________________________________________________

Q: I want to know can a client without control-m em server support just the control-m server?

 

A: In order to use a EM client you must have a Control-M/EM server which you'll connect to.

_______________________________________________________________

Q:  How can I uninstall client FP?

 

A: You need to do it manually in the same way is stated in the Release Notes   .

________________________________________________________________

Q:  Does the end-user need admin rights on their machine in order to install?

 

A:    No need for administrator privileges.

_______________________________________________________________

Q: Is .NET required?

 

A: In order to install the EM client on a machine, it's necessary to install .NET Framework 4.5.2 before on that machine otherwise the EM client won't be installed   .

________________________________________________________________

Q: Can we use any browser?

 

A: Yes you can.  

________________________________________________________________

Q:  Will this work on citrix or regular clients if the user account logged in is not the same as the user who originally installed the base version of the client?

 

A: We don’t see any advantage in using the Client Distribution on Citrix since only one client installation is performed. Anyway, updates will be installed on a client that was originally installed by the same user.

________________________________________________________________

Q:  Can we use this tool even if we have EM server installed on an AIX machine?

 

A: Yes you can, you can have EM server installed on any Unix flavour or Windows and deploy the clients, Fix Packs, patches and SSL certificates.  

_______________________________________________________________

Q:  Is the timezone.dat / SSL config and certificates included in the deployment?

 

A: Just the SSL certificates are automatically distributed and installed   .

_______________________________________________________________

Q:  Can clients that were manually installed be updated via client deployment?

 

A: Initially No they can't, update notifications    will work just for clients installed through Client Distrubution Tool. From EM Fix Pack 3 it's considered that notifications will work for any kind of EM client no matter how was installed.

________________________________________________________________

Q:  Does the same user that installed the base product also have to be signed on to get the fix pack push?  

 

A: Correct, updates will be installed just with same user that installed the base client   .

________________________________________________________________

Q:  Does the applying of the fix pack automatically remove the fix pack installation file after the install has completed?

 

A: No it doens't. The packages are downloaded to path %programdata%\BMC Software\Client_Updates and kept in that location on the client machine.

________________________________________________________________

Q:  Can multiple clients be installed on the same client machine from separate EM Servers (e.g. test / dev / prod systems) ?

 

A: Yes you can have multiple instance of EM clients on the same client machine to connect to separate EM servers, but you also use one single client to connect to different EM servers.

________________________________________________________________

Q:  File TimeZone.dat can be also distributed?

 

A: It's not possible

________________________________________________________________

Q:  Do you still need separate Control-M Configuration Manager installations?

 

A: You can use same Control-M Configuration Manager to connect to a different EM server, just need to change the connection settings.  

________________________________________________________________

Q:  Is this distribution tool included with the base EM package or as an additional addon? 

 

A: It's include with the Control-M package.

________________________________________________________________

Q: Is the webserver required for fixpacks?

 

A: Yes it is. If Web Server component is down, the update notifications won't be distributed to the EM clients and furthermore EM client won't be able to perform any update installation.

________________________________________________________________

Q:  Fix pack deployment for client is user base or machine? Mean to say here if we are using one client server but it used by multiple user so every user need to install fix pack?

 

A: The client installation is for the user. The installation user is written to the “installed-versions.txt” of the client installation.

DRNFT.9.0.00             Windows        Jun-11-2015    Feb-17-2016    9.0.00.000     INSTALLATION   EM Client installed by Admin user (akarmon)

 

Updates will be installed on a client that was originally installed by the same user.

________________________________________________________________

Q: What if they dont have the .net required?

 

A: EM client installation will fail due to that software is not installed on the client machine. If you don't have that software at hand, you can get it from the DVD BMC provides or from the Control-M package you get from our EPD site.

________________________________________________________________

Q: Will Report be available via the V9 method?

 

A: Yes, when deploying a client installation with Client Distribution Tool, Reporting Facility is also installed.

________________________________________________________________

Q: What if we have multiple versions of EM clients, e.g (6.4 and 9) will it detect the version 9 EM client and install the fp?

 

A: The update is sent through    the same EM server and EM client communication, so since the EM client 9 is connected to Control-M 9 environment it will send the update and package properly.

________________________________________________________________

Q:  What if there is a scenario where we would need to roll-back a fix-pack to a previous one?  Can that be done through the push or does it have to be locally per PC?

 

A: You need to do it manually per PC in the same way is stated in the Release Notes   .

Share:|

You work hard all day, every day, morning to night - and for many of you likely overnight sometimes too. Don't let all that hard work go unnoticed - by everyone outside your organization that is.  Now's your chance to share your best Control-M stories with your peers.  Submit for a session at BMC Engage 2016 now!


Everyone I meet with that works with Control-M has an incredible story to tell - it continues to amaze me how many daily activities we all take for granted just because Control-M works for companies all around the globe. And you're the key reason why! Here's your chance to tell everyone how and why Control-M matters to your business while sharing with others what you've learned as you've worked with Control-M - regardless if it's been years or only a few months. All of your stories matter!

 

The Call for Papers is now open - but not for much longer...you have until March 7, 2016 to get your submission in and have it considered for the conference. Be one of the selected speakers and receive a complimentary full conference pass.

Engage 2016.png

 

September 6-9, 2016

Aria Resort and Casino, Las Vegas

 

Pre-Register to secure the early bird rate - When registrations open will receive a code guaranteeing you the lowest possible rate for the conference - share with your co-workers and colleagues!

 

Don't miss this chance to collaborate, network and learn from your peers and the experts at BMC - see you in Vegas!

 

Any questions about submitting a session or Engage in general - don't hesitate to contact me.

Bill Sager

Control-M Paradigm Shift

Posted by Bill Sager Feb 19, 2016
Share:|

Since the beginning of data center time, Workload Automation or Job Scheduling has been an integral part of IT. The consumers of Workload Automation fall into two disparate camps.  You are either the IT centric team that manages and controls it or you might be the end user who has limited access and relies on the IT team for your information.  For the IT team you go by the name of Schedulers, Operations, NOC, Production Control or Production Support.  You end users can be Support, Application Developers or even Business Users.  If you are the IT user the burden has been on you to monitor, schedule and control the batch processes, notify end users of issues, calling them, taking calls and being responsible for how it ends.  At the other end of the spectrum end users assume no news is good news.  You check your email, wait for a phone call or an open ticket to see if there is a problem.

 

DIGITAL ENTERPRISE MANAGEMENT PARADIGM SHIFT


The advent of the Digital Enterprise is causing businesses to create content that is consumed by both internal and external users.  BMC Control-M enhances the Digital Enterprise approach by creating a paradigm shift, allowing consumers of batch processing to be more innovative, providing visibility and accessibility to schedules and active processes as well as access to archived logs and outputs.  With a methodology to control your own workloads and ease the burden previously placed on the IT teams, Control-M delivers speed, agility and an intuitive interface to you, the end user.

 

HOW DO THEY DO THIS

The three modules that help you turn the light on with this Digital Enterprise paradigm shift are Workload Change Manager, Self Service and Workload Archive.  The key to the shift is the presentation of data and information via web link or a mobile app with little or no installation required.  The web-based and mobile views are based on the authorizations defined in BMC Control-M.  

 

Workload Change Manager

  • Provides the capability to review, create and update schedules
  • Allows the Scheduling team the ability to create Standards for end users to follow
  • Reduces overhead on updating and creating spreadsheets and forms which are typically used to get changes to the Schedulers
  • Gives teams the capability to promote flows between environments such as test and prod

 

Self Service

  • Provides end users a view of critical applications and business services using browsers on a computer, or a native mobile device app
  • Presents pre-authorized  process views including access flows, statistics, output and logs
  • Depending on access level, allows end users to rerun, hold, release and take other actions on jobs

 

 

Workload Archive

  • Supports both auditing and reducing Mean Time to Repair
  • Captures the output files (STDERR/STDOUT), Control-M log files and job data for every occurrence of a job run
  • Supports flexible policies determining how long to keep the output file data.  This data can be based on a type of job.  For example, Informatica or SAP jobs can be kept for 3 years for audit compliance. Test jobs can be kept 1 month. 
  • Provides search capabilities when trouble shooting, along with the capability to compare logs and schedules from previous runs

 

WHERE WILL THIS TAKE US

 

While the traditional IT centric control team will never be eliminated in most shops, the overhead and responsibilities placed on you will be moved to your end users.  The resources required to maintain your IT group will decrease.  By becoming part of the Digital Transformation and innovation paradigm shift with easy to use interfaces, everyone will now have control and visibility into their environment.  You end users are no longer dependent on having to call or receive a call.  You can make updates or follow critical batch services. You will not be left in the dark again.

Share:|

Since joining the Control-M team in June 2015, I have been immersed in Big Data.  Yes, that’s right – perceptions on most things I

encounter are now “data-based”.  I don’t think about driving cars the same way – it’s now2015-chevrolet-suburban-texas-edition-01.jpg about all

the sensors capturing data and sending it back to Chevrolet so they can constantly update me on my vehicle health and when I need to have my tires rotated.  My online shopping experiences are now as much black dress and shoes.pngfocused on what back-end systems and applications the company is using to know that yes, those shoes are just what I want to go with that outfit – and don’t forget the handbag options too!


Data is all around us, being collected all the time. Machine learning is a key contributor driving behavior today – whether you realize it or not.  The influence of what’s learned is continually presented to us – just think about how many times you interact with an app on your mobile device in a day.  It’s all data driven.

 

What this view into big data technology really has given me is an appreciation for something quite amazing – Hadoop.  And with the release of Control-M for Hadoop 9, BMC continues to enable companies to automate so many aspects of their own Big Data journey.

 

My favorite feature in this release is the ability to execute workflows defined in Oozie through Control-M or convert Oozie jobs to Control-M using an intuitive, wizard-based interface. What’s better than choosing what works best for you?  Either way, all of the

Spark 3.png

powerful Control-M capabilities including SLA management, easy access to log/output, archiving, workload policies and self-service are extended to your Hadoop workflows.

 

Another key feature allows you to connect Apache Spark™ with the rest of your eco-system using Control-M’s drag and drop capabilities while running Spark SQL and Spark streaming written in Java, SCALA or Python. You no longer need to create a complicated web of scripts to build and run Spark jobs.

 

The more I learn about Hadoop, the more I get to see code.  Lots and lots of code.  I am not an application developer so a lot of it looks “Greek” to me.  What I do know is that when that codes gets written over and over again, copied and pasted and then edited for changes, errors will inevitably create hiccups here and there.  So it only stands to reason if you can automate

DistCP.png

frequently used code in a virtually fail safe mode, why wouldn’t you?  Control-M for Hadoop 9 now supports moving data between Hadoop clusters with integrated DistCP (Distributed Copy) commands where you have to provide only the target and source
destinations. This automated approach reduces the time it takes you to code and significantly reduces the potential for errors. And you can copy files to Amazon S3.

 

And naturally, as new versions of Hadoop are released Control-M will continue to add support.  In this release we extended support to include MapR 4.x/5.x and Ubuntu 14.x.

 

In the end, this really means that Control-M can make it easier for you to drive your Hadoop project to production. It’s when it gets to production that your company will be able to drive answers from all of the data they collect, and likely realize there’s data they can be collecting that they aren’t yet.  And of course that drives the need for storage – and Control-M helps there too – offloading data to Hadoop, lowering storage costs and making it easy to search the archived data.  Your day job is easier, you saved the company money, and you contributed to uncovering the nugget that drove a competitive advantage skyrocketing the business to the top!  And all because you automated with Control-M.

 

Or, if you want to see it through my eyes, it means that I am more confident that the plane I will be on for the next four hours is going to

CTM Hadoop datasheet image.jpg

safely get me from here to there; that the item my daughter has to have for school will arrive right on time – so says the notifications from the shipping company; and that my go-to music app will indeed point me in the direction of my next new favorite song.

 

What is your company doing with Hadoop and your data to change the direction you’re headed or the experience your customers are going to have next week, next month and next year?  I’d like to hear how data is changing your perceptions.

 

To find out more about Control-M for Hadoop take a look at the web page, datasheet or ask your questions here.

Share:|

In January 2014, I published a workload automation trends prediction for 2014 blog primarily focused on the impact of Gartner’s “Nexus of Forces.” These forces are often referred to as SMAC (Social, Mobile, Analytics and Cloud), widely recognized as the disruptive technologies driving the consumerization and industrialization “mega trends.” In 2016, some of these trends remain strong (with some fine-tuning). As expected there are some new trends emerging as well. The one theme that is reflected across all of my 2016 predictions is the evolution of workload automation to application workflow automation, accommodating the needs of both application developers and IT production control.

Here’s my view on what you should be expecting: Workload Automation: Predicting Trends in 2016 - BMC Blogs

 

wla-blog.png

Share:|

Learn how you can successfully install and configure high availability with Control-M/Server.

 

On Wednesday, January 27th, David Pavoncello will discuss and demonstrate the following:

            • Control-M/Server 9 HA architecture and HA database options (Oracle, PostgreSQL, etc.)

            • Difference between manual and automatic failover

            • Log collection and analysis for HA Control-M/Servers

            • Debugging HA Control-M/Servers

 

There will be a Q&A after the demo. Register Here!

Filter Blog

By date:
By tag: