1 2 3 Previous Next

BMC Control-M

153 posts
Share: |


The other day I was discussing a particular process that gets performed ad-hoc and how we could turn that workflow into an automated process and then tie it into a self-service request.  This particular process had been done for years by a few folks manually running scripts and submitting data to a few services via web interfaces. It worked relatively well and the particular manager I was talking with was of the mindset that if it worked, let it be, there were other more important things to work on.

 

I then pulled out one of my favorite words to use when talking about Workload Automation - visibility.

 

Wouldn't it be great to have visibility into that process?  To know where we were in the chain, if a failure occurred somewhere, and get an estimate of how much longer we had to go?

 

The manager lit up with excitement, and that "Thanks, but no" turned into a "Yes! Yes! Yes!"  In less than two days we had the process automated, setup so that end users could access it via a self-service mechanism and fully visible.

 

Visibility is one of the great benefits of Workload Automation.  Often we tend to talk about the IT benefits of getting process flows automated or put into a Workload Automation system and we tend to forget that many of the workflows we support are IT-driven, but are supporting business units on the other side.   Managers often don't care how something gets done, just that it gets done.  They want information and insight into the process, whatever it may be.  Give them more than they had now - such as visibility - and you will always win them over.

 

Batch scripts litter network drives everywhere.  CRON and Scheduled tasks hide on servers across data centers around the world.  Those scripts are great for what they do - and indeed they are a first step in the automation cycle - but they provide zero visibility into what is happening and how it impacts the rest of the business.  What happens when the person that runs that batch script on their desktop leaves for vacation for two weeks?  How about maternity leave for 6 months?  The work stops happening often times without anyone away until weeks or months later when someone notices data missing.

 

We talk about agility in IT operations, but how agile is our business processes if they are still depending on people and processes hidden away on servers?  Agility depends on visibility.  A set of tasks that is only known to one or two people stops agility in it's tracks when those people suddenly aren't available, or something goes wrong and only one person can diagnose it while entire groups of people are scrambling to figure out why a process that has worked for years has suddenly stopped without any warning.

 

The next time you get the age-old question of "Why should I put this into Control-M?" ask them if visibility matters.  You may find you turn the entire conversation around, and give people a new way of thinking about the benefits of Workload Automation.

Share: |


Learn how to maximize managing Control-M jobs with Viewpoints and Job Versioning.

 

On Wednesday, July 22, 2015, James Pendergrast presented and demonstrated the following:

        • Using Viewpoints to manage your current jobs and active environment
        • Advanced features available in the Monitoring Domain
        • Using Archived Viewpoints to view job flows from previous days
        • Using Job Versioning to audit job changes
        • Using Job Versioning to recover jobs

 

Here is the Q&A for this webinar (Connect With Control-M: Job Viewpoints, History and Versioning)

________________________________________________________________

Q: For the parameters MaxOldDay and MaxOldTotal, are these set for each defined gateway or they apply in general for all the gateways?
A: These parameters apply to all the Gateways. This is per data center, so if there are two data centers, and 7 days, there will be a total of 14 days worth of data. 
________________________________________________________________

Q: What needs to be done if you change the MaxOldDay and MaxOldTotal?
A: If these parameters are changed, then the Gateway needs to be restarted.
________________________________________________________________

Q: Any data how to predict table space increase needed when increasing MaxOldDay and MaxOldTotal parameters?
A: The space needed to store depends on several factors including how many jobs run. A good way to estimate is to check the size of the current database tables IOALOG_*, and divide by the current number of days being kept. This will give an approximation for one days worth of logs. Then multiply this value by the number of days MaxOldDay and MaxOldTotal will be set to.
________________________________________________________________

Q: When I create a viewpoint is it available for all users or just for me?
A: Viewpoints are stored in the database, so they are accessible for all users.
________________________________________________________________

Q: There is a filter that can be used with ALLJOBS, called NOTOK.  In fact, that filter can be used on any Viewpoint.  Why the need to create a new one--as Viewpoints all need to be loaded into Memory of the EM Server when started?
A: This is true, viewpoint ALLJOBS is loaded, then a filter can be applied for Job Status.
Another method is to create a Viewpoint for users who may not know how to use the filter. 
________________________________________________________________

Q: If I have 5 of the same viewpoints open, does it take 5 times as much memory?
A: If there are five different viewpoints open, then memory for each will be used. If five of the same Viewpoint is opened, then only one will need memory.
________________________________________________________________

Q: when I do playback, what time is it starting and ending from?
A: The Start and End times of the Playback is based on the start and end time of the Gateway download. Normally this is the NewDay time.
________________________________________________________________

Q: If I want to keep more versions of the job, is it necessary to recycle a component if I change "VMVersionsNumberToKeep" parameter?
A: The GUI Server will need to be cycled if the "VMVersionsNumberToKeep" parameter is changed.
________________________________________________________________

Q: Will History ever allow for viewing more than one datacenter? If I have two Control-Ms, I can only view one at a time?
A: This may be changed in a future release, but for the current versions, the History is stored based on the DataCenter/Gateway.
________________________________________________________________

Q: When you restored the deleted job did it go back to the folder it was deleted from or did it create its own folder?
A: The job is restored to the Folder/Table that it is deleted from. 
________________________________________________________________

Q: I can see neighbors but I cannot see the neighborhood feature as you showed when selecting the neighborhood?  Is this a setting? 
A: This is a feature that is in Control-M EM version 8, fixpack 6.
________________________________________________________________

Q: How can we create a view point for the jobs that are running longer than 5 days?
A: If you use the AllJobs Viewpoint, and set a filter to ODATE, then you can see jobs that are currently running that are older than that date.
________________________________________________________________

Q: Is there a way to find out when a job last ran without going thru archived viewpoints?
A: If the Job is visible in the EM GUI, then you can check the Statistics. If the job is not in the GUI, then the only way would be to look in the Archived Viewpoints, or run a Report on the Job Statistics. The Server utility ctmruninf can be used.
________________________________________________________________

Q: How to find who did the changes in the active job environment? All I can see in the logs were job is Hold by x user and Free by x user...
A: The Server logs show what actions were taken by what user.
________________________________________________________________

Q: what is the max value you can put for VMVersionsNumberToKeep paramater?
A: There is no known limit to this value.
________________________________________________________________

Q: In versioning, can a complete Folder be restored?
A: Versioning only restores a Job. Restoring the Job, will also restore the Folder if it doesn't exist. All the jobs can be selected and restored, but the Folder itself can't be restored.
________________________________________________________________

Q: Is there a way to expand the tree in the monitoring view with one command?
A: The Tree View does not have a shortcut to expand. Each "plus" sign will need to be selected.
________________________________________________________________

Q: Can we create viewpoint that display long running jobs (eg. running for more than 6 hours)?
A: Any of the Job Attributes, including StartTime can be used. The time will be static from when the Viewpoint or Filter is opened, so it will need to be refreshed.
________________________________________________________________

Q: Could you please explain the need of Control resource in Control-M?
A: The Control resource is used to control access to a resource. For example, you may have a job that needs exclusive control of the database while the job is running.
________________________________________________________________

Q: Are job statistics not longer available for the last several runs?
A: Job statistics are available for the last runs of the job. If the job is in the AJF, and visible in the GUI, then statistics will show when the job ran last. If the job is not in the AJF, and not visible, there isn't a way to get to the statistics.
________________________________________________________________

Q: Anyway to do the playback feature on the current active?
A: The Playback feature is only available for History/Archived Viewpoints
________________________________________________________________

Q: Can you identify who has an open history view, and close the history view after it has been opened a set time to prevent people holding some of the quota all day?
A: This question has been answered verbally.
________________________________________________________________

Q: Can we compare 2 versions of a folder and get a summary?
A: This functionality is not built into Control-M, but could be easily done by exporting the folders to XML and then comparing the files.
________________________________________________________________

Q: Will increasing the stored number of days affect database performance?
A: Increasing the stored # of days will cause more database storage but it won't affect the performance (speed). 
________________________________________________________________

Q: General Question -- Is there a timeout on WLA Client?  Self Service and WCM both have timeouts.  Is there a setting to do this in the client? Thick Client WLA.  Timeout to lock it such as it does in Self Service and WCM.
A: The Client does have a timeout, and will attempt to re-connect 3 times. There isn't a method to lock the Client after a specified time. The Client can be Kicked off from the CCM, by manually selecting the GuiServer, and using the Control Shell.
________________________________________________________________

Share: |


If you are running multiple Control-M servers, including Mainframe and Distributed environments, you can control your workload flow between the environments using global conditions. For example, after a job completes on your Mainframe Control-M/Server, it can trigger a job on your Distributed Control-M/Server and vice versa.

 

On Wednesday, June 17, 2015, Froilan Reyes and Richard Talbert will discuss the following about Using Global Conditions:

 

        • The Global Condition Server(GCS) process overview
        • GCS configuration
        • Passing conditions between Control-M/Servers
        • Demonstrate adding, using and testing global conditions between mainframe and distributed environments

 

________________________________________________________________
Q:  Are we able to pass conditional (maybe) conditions?
A: Maybe conditions are normal conditions that get added with the ctmldnrs. Once this conditions are added, if it is a global condition it will be passed to the specified CTM Servers.

________________________________________________________________

Q: What is the length of a condition?
A: For the DS side it can be 256, for the Mainframe it is 39
________________________________________________________________

Q: If passing conditions between cyclic jobs what is the best way handle them if they run closely together?
A:  There can be a two minute limitation for the same global condition to be updated. To be safe make the updates at least two minutes apart.
  
________________________________________________________________

Q: What happens if the GCS is down?
A: The global conditions won't be sent between Control-M/Servers.
________________________________________________________________

Q:  Does timestamp on global conditions come from the originating datacenter?
A: Condition time stamps are when they get posted to the ctm server. GCS is only concerned when a condition that has not been processed, comes in with in 2 mins. So if a datacenter was down for several hours, GCS would treat all of the updates as if they were just posted. There are some limits on how far back (ODATE) GCS will look at conditions

________________________________________________________________

Q:  Are any kind of characters or symbols allowed in the name of a condition?
A:  Special charters are allowed, but this is limited to what is specified in the Parameter Guide for conditions name.
________________________________________________________________

Q:  In system configuration, system parameter VARIABLE_INC_SEC is set to LOCAL where does the Global Condition get saved if it doesn't go to the agent?
A: VARIABLE_INC_SEC is used with global variables not global conditions  
________________________________________________________________

Q: A condition can be posted into two or more Data Centers at the time?
A: If the global condition is setup to go to more than one ctm server, it can post to how ever many it is setup to post too.
________________________________________________________________

Q: Can a global condition performs a force job also from a Folder?
A: Global conditions don't force or order jobs. I merely creates conditons that allow jobs or folders to be executed.
________________________________________________________________

Q:  Is it advisable to use bypass option when we have global conditions ?
A: At that point, for the job, it is just a condition that has not been posted. You then get to choose to bypass it, add it (be careful), or hold the job and delete the condition from it. Bypass does not add the condition.
   ________________________________________________________________

Q:  On those global maybe conditions is there documentation that tells how to configure ADDMNCND and CTMLDNRS to check for them on another server?
A: We do not have the ability to do global maybe conditions, out of the box.
________________________________________________________________

Q: Why Global Condition doesn't show up in the listing ?
A: If the prefix is not on the listing then most likely that prefix will no longer be used.
________________________________________________________________

Q:  Say for instance - If the GUI is down, Is there a way to add GC from command line or any other source?
A: For now there is no way to create the Global Condition Prefix, other than the client Workload Automation
________________________________________________________________

Share: |


Learn how you can benefit from Workload Archiving by storing your critical Job log and output data long term.

 

On Wednesday, August 19, 2015, Joel Brecker will discuss and demonstrate the following:

        •Reducing the size of your Control-M Database
        •Reducing output disk space on the Agent
        •Keeping job log and output for up to a year or longer
        •Using Archiving policies to specify what data to retain
        •Easily viewing Archived data from the Control-M Enterprise Manager Client


Don’t miss a live demo of these features. There will be a Q&A after. Register Here!

Share: |


On Wednesday May 20th 2015, Pilar Soria will discuss how to install and configure Control-M for Hadoop and demonstrate creating jobs to automate Big Data. The webinar will include:


• What is Control-M for Hadoop?
• Installing, configuring and setup of Control-M for Hadoop
• Creating Map Reduce and Sqoop jobs
• Hadoop job Conversion tool
• Demo configuring Control-M for Hadoop with Kerberos

 

------------------------------------------
Q: Do we have developer studio to practice?
A: Control-M for Hadoop jobs can be defined from Workload Automation Client.

------------------------------------------
Q: Does my kerberos ticket lifetime in the confirmation management have to match my ticket lifetime in my kerberos configuration?
A: Equal or less than what is configured

------------------------------------------
Q: Do I need to run kinit by hand to run Hadoop jobs from Control-M?
A: No, Control-M for Hadoop will run kinit command automatically

------------------------------------------
Q: Do I have to run the agent as root to do impersonation?  Do I have to run as root without kerberos?
A: For impersonation (Kerberos impersonation) Control-M Agent should start as Control-M Agent owner. The impersonation is done at Kerberos level with “ksu” utility.
   Without kerberos, Control-M Agent has to be started as root user in order to submit jobs with other users (Run as)

------------------------------------------
Q: Does the CM for Hadoop support Windows?
A: Currently, Control-M for Hadoop only supports Linux 64 bit machine

------------------------------------------
Q: Are you planning on supporting spark with the CM for Hadoop?
A: Control-M for Hadoop will support spark in next patch that is tentatively planned to be released in July.

------------------------------------------
Q: what is the minimum space required to install hadoop?
A: To install Hadoop you will need about 1.2GB.

------------------------------------------
Q: Can I run commands that are not HDFS commands as pre/post?
A: No, in Pre/Post commands only Hadoop fs commands can be submitted.

------------------------------------------
Q: Where can we use Control M¬?
A: Not sure I completely understand the question. Control-M Supports a variety of Operating Systems, including MainFrame, Windows and Unix.

------------------------------------------
Q: Does CM can handle various jobs at a time by automation?
A: Control-M for Hadoop can submit and monitor jobs concurrently.

------------------------------------------
Q: does DB schema and procedure can be run through Hadoop module?
A: Not sure I understand completely the question but if you meant manipulate Database schema and run procedure with a Database we have Control-M for Database the supports that.

------------------------------------------
Q: How can I get access to the conversion utility?
A: Control-M for Conversion Tool can be downloaded from EPD (http://www.bmc.com/available/epd.html)

------------------------------------------
Q: do we have any conversion tool to move jobs from hadoop to control M.
A: Yes, with BMC Control-M Conversion Tool we have the capability to read the Oozie scheduling metadata and build the scheduling definitions for you in Control-M.

------------------------------------------
Q: Can you provide a brief ppt on what exactly control M does?
A: I might suggest you take a look at the overview @ http://www.bmc.com/it-solutions/control-m.html which discusses Control-M's functionality

Share: |


 

I have the fortune, and sometimes misfortune, of travelling around much of the world for my job. Every time I am in Tel-Aviv I make it a point to visit what is quite possibly my favorite restaurant in the whole wide world.  It is an unassuming spot on the ground level of an office building.  It is a Tripolitanian place that has magnificent meats, vibrant vegetables, great grains, and super spices. I would guess they also have delicious desserts, but the starters and main courses are so amazing I never make it that far!

A typical spread looks like this:

In all honesty I am not 100% sure on the exact details of what is in every dish.  I would also go so far to say that individually each dish is good, maybe even great, but when combined together the whole experience and combination of tastes is simply amazing. 


As I span the globe talking to customers and prospects, the subject of applications and how to manage their workloads is often the topic of choice. It occurred to me that much like my favorite restaurant and its wide variety of dishes, the same holds true for all those applications.  Individually they are good, maybe even great, but when combined together they deliver the business services that ultimately are not just part of your business, they ARE your business.  From billing to payroll, logistics to shipping, online to back office, there is no business service you provide that is not backed by a number of applications working together.

 

For a longtime I have been convinced the best way to integrate all those applications together is through workload automation.  And with the June 2015 release of Control-M Application Integrator there are more ways than ever to integrate most any application into your Control-M Workload Automation environment.  Application Integrator is a self-service web design tool that allows you and your application developers to create your own Control-M job types for almost any purchased, open-source or homegrown application.  Once created you can then manage the workloads for that application inside of Control-M the exact same way you do for the rest of your mission critical workloads.  You don't have to rely on insecure and hard to maintain scripts.

I have 4 kids that aren’t crazy about Tripolitanian food, but they do love pizza.  Once in a while they will ask for a frozen pizza and my response is always the same….”Life is too short to eat bad pizza”.  Living in the Chicago area means I have many amazing pizza places nearby and I think it is nearly a sin to waste time on lesser pizza.  Life is also too short to manage workloads in an inefficient and risky way. If you agree, check out any of the following resources to see how Control-M Application Integrator can help you manage ALL your workloads in a consistent and comprehensive way.

Control-M Application Integrator Web Page

Community Sourced Repository for Application Integrator Job Types

Webinar Covering Application Integrator

 

Comments are welcome and I would specifically like to know 2 things:  What food combinations are better together than separate and what applications are most important to the delivery of your business services?

Share: |


BigData_2267x1146_white.png

Who needs information

When you're living in constant fear

Just give me confirmation

There's some way out of here

 

According to the Radio K.A.O.S page on Wikipedia, the event that inspired Roger Waters to write the lyrics of the song “Who Needs Information” was the 1985 miners' strike in Britain where a striking worker threw a concrete block off a motorway bridge, killing a taxi driver who was taking a working miner to his job. This was an example for how far people will go to pursue their monetary goals.

 

You can blame me for using Roger Waters, Pink Floyd, Tom Petty, Queen, or even The Matrix movie for monetary reasons as well, being an employee of a commercial software vendor, but if you are reading this blog there is good chance you as well work for a commercial company with stakeholders, owners or someone that cares about revenue and profitability. You are expected to help your organization be successful and to generate revenue.

 

Being successful means to have an advantage over competition, and the way to get that advantage these days, is information. The more you know about your market, about your competitors, and most important – your customers, the greater the chances you’ll be successful.

But there’s a catch. The more information you have, the greater the challenge it is to analyze it. Traditional technologies such as relational databases and data warehouses can no longer process the amount of data generated by social channels, or gathered from online websites and other machine generated data, at least not in the timeframes that allows you to take advantage of the processed results to drive relevant business decisions.

 

Technologies such as Hadoop and in-memory databases are becoming more and more popular these days. Big Data is no longer just a buzzword. If you search for use cases on the websites of all the major Hadoop distributers (for example Hortonworks, Cloudera and MapR), you will find plenty of stories describing how companies are taking advantage of Big Data technologies and Hadoop specifically to become or remain competitive.

 

But Hadoop is not an island, and it will not replace the traditional database platforms which all your business applications connect to. Commonly large amounts of data are processed by Hadoop and the results are then sent to a legacy data warehouse or back to a mainframe if Hadoop is used to reduce mainframe license costs for example. ERP systems such as SAP, Oracle E-Business Suite, PeopleSoft or others might be involved in the process as well, along with file transfers, direct access to databases, ETL or data integration activities and eventually business intelligence or analytics tools that are used to expose the information to business users.

 

So how do you make sure that all these systems are in sync?

How do you monitor the process from beginning to end and ensure that you are meeting your deadlines?

How do you manage changes across all these systems, making sure everything is audited and compliant with your company standards and policies?

How do you provide business users self-service access to their parts of the workflow?

 

You need a solution that that allow you to manage everything from a single point of control, to configure automatic recovery from failures so the amount of manual corrective actions is as minimal as possible (including recovery from point of failure to reduce downtime in case of a problem) and to define proactive SLA/deadline notification so potential delays or failures in critical services are identified as early as possible when you still have the time to fix the problem before you miss the deadline.

 

And how do you get all these capabilities and more? You select the leading workload automation solution in the market, the only one that really can handle all the applications and platforms that you need. It even has an online open community that allow users to collaborate and share custom integration they created. Specifically for Hadoop, it offers a native interface that can eliminate any scripting, homegrown integration workarounds, or the use of Hadoop-only limited schedulers such as Oozie.

 

If you want to learn more about Hadoop workflow automation or about centralized approach for automation of any application or platform, simply click on the links above or below...

 

It’s also the time now to register to BMC Engage, the largest BMC conference of the year, which will have a dedicated track for workload automation - while you can still get the early bird pricing

Make sure to attend Joe Goldberg's session on Hadoop and the "Elephant in Your Computer Room" (Session #386)

 

If you are planning to attend Strata+Hadoop World in New York City this year, make sure to stop by the BMC booth and see a live demo from one of our experts.

 

There is plenty information for you online (who needs it anyway?), but if you have any question or need more information, post it here as a comment.

 

Share: |


Evaluation.pngOf all the projects I have worked on since joining BMC 14 years ago, Control-M Application Integrator is definitely one of the most innovative. For those of you who missed the press release announcement, it is a self-service web design tool that allows “teaching” Control-M how to integrate with any application, easily and quickly: how to submit the jobs to the application, how to monitor the jobs, how to determine if the jobs completed successfully or with errors, and how to abort the jobs if they need to be stopped. This is a first-of-its kind self-service tool that allow designing new job types without any development skills.

In addition, the team also created an online open community, The Control-M Application Hub, where customers and partners can find, share and submit their job types. This is also an industry first community that can allow users to reuse job types created by other community members and collaborate for the greater good.

 

Throughout the design and development of Control-M Application Integrator, we worked closely with customers and partners, many of whom evaluated the PreGA releases of the product and provided feedback. As I reflected on the release and how we were able to deliver this awesome technology, (and yes, I am a bit biased…) I found myself focusing on four things that customers and partners were able to get out of participating in this process:

 

1. Your opinion matters

Every product or feature we develop is aimed to solve a business problem and to provide value, and we need to hear from you what challenges you have and how we can help you solve them. When you share this information with us, we can make sure that the product covers your specific use cases and that it includes the features you need the most. It doesn’t guarantee that every enhancement requested will make it to the first release of the product but when we hear consistent feedback from customers we take that into consideration when we build the product roadmap.
With Control-M Application Integrator we have made various usability changes based on feedback from customers, and we added features which were originally considered for future releases. These include for example enhancements to WebService security and the output handling functionality.

 

2. Your platform of choice

Our quality control team is extremely focused on testing products by use cases provided by customers, and on the platforms that are most commonly used by customers. But when you install, configure and run a PreGA software build in your environment, you increase the chances that defects that are unique to your specific platform or use cases are identified and fixed before the product is released to the market, making it much more stable. With Control-M Application Integrator we were able to address a number of these defects identified by customers, some of those identified in very early stages of the development process.

 

3. Your application

Control-M Application Integrator allow integrating Control-M with any application, commercial or homegrown. Customers that participated in PreGA evaluation created job types for applications such as SAP Business Objects Data Services, Tibco, Oracle Human Capital Management (HCM), MicroFocus, Lawson, SalesForce.com, GlobalScape, EMC Networker, Teradata, QlikView, and other homegrown banking and telco applications. We expect many of those new job types to be shared with the community in the near future on the Control-M Application Hub.
What applications do you need to integrate Control-M with? To make sure you aren’t recreating the wheel, be sure to check the Control-M Application Hub – and if you create a job type that you don’t see on the hub – share it!

 

4. Your time to value

Learning how to use a new product in parallel to its development, and testing it on your own specific environment, can dramatically shorten the time it will take you to start using it in production after it is generally available. Many of the customers that participated in the Control-M Application Integrator PreGA program are now in final stages of production deployment, some of them are among the largest global banks and telco companies in the world, and others are leading companies from the retail, healthcare and utilities industries. They all understand that in order for them to remain competitive and grow, they need tools that will allow them to innovate and automate, and that’s exactly what Control-M Application Integrator offers them. Don’t wait to get started – it’s there for you too!

 

 

The Control-M product team regularly travels to visit customers around the world in an effort to understand our customer’s ever changing business challenges and needs, and to share news on the latest product releases as well as what’s to come. So next time we meet, ask us about the Control-M PreGA evaluation programs and how you can join.  Help us help you! Get it while you can!

 

For more information about Control-M Application Integrator, visit www.bmc.com/integrate and www.bmc.com/hub.

 

 

 

Share: |


Learn how to maximize managing Control-M jobs with Viewpoints and Job Versioning.

 

On Wednesday, July 22, 2015, James Pendergrast will  discuss and demonstrate the following:

    • Using Viewpoints to manage your current jobs and active environment
    • Advanced features available in the Monitoring Domain
    • Using Archived Viewpoints to view job flows from previous days
    • Using Job Versioning to audit job changes
    • Using Job Versioning to recover jobs

Don’t miss a live demo of these features.  There will be a Q&A after.
Registration is now open here!

Share: |


It's that time again - and there's not much time left before you miss it!  Chicago and Tampa Control-M User Groups will be meeting before the month is out.  Be sure to mark your calendar and RSVP for the meeting in your area.  It's a great way to meet and network with others in your area working with Control-M - compare your stories and successes and build your local "I have a question, can anyone answer this one?" network!

 

Here are details for both - again, don't delay - not much time left to get your name on this exclusive list!

 

Florida User GroupChicago User Group

 

When:

Friday, June 19th

8:30am-3:00pm

 

Where:

BMC

2202 N. Westshore Blvd., 6th floor

Tampa, FL 33607

Map & Directions

*Free Visitor Parking

 

Register Here


Agenda at a Glance:

8:30 am Continental Breakfast, Meet-N-Greet

9:15 am President's Welcome & Opening Remarks

9:30 am Control-M Conversion/Migration

10:30 am Morning Break

10:45 am Group Discussion / Q&A Session (Part I)

11:15 am Application Integrator & Archival Tool

12:00 pm Hosted Lunch *Sponsored by GSS

12:30 pm Industry Insights - GSS

1:00 pm User Group Member Roundtable:

2:00 pm Control-M Version 9 Update

2:30 pm Group Discussion / Q&A Session (Part II)

3:00 pm Meeting Adjourns

GSS Infotech logo.png

Special thanks to Steve McCormick and GSS Infotech for sponsoring the user group and providing lunch for everyone!                                           

 

For questions please contact Jim Gingras (jim_gingras@bmc.com) or Loren Gross (loren_gross@bmc.com).

 

FL CTM User Group Image.png

Click here to join the Florida Control-M User Group Community.

 

When:

Tuesday, June 23rd

9:00am-2:00pm

 

Where:

Accenture

161 North Clark Street

Chicago, IL 60601

*There will be discounted parking available for those that plan to drive and park near Accenture.

 

RSVP

 

To ensure you have a seat at this exclusive event email Robby Dick (robby_dick@bmc.com) or Jeff Sanderson (Jeffery.s.sanderson@Accenture.com).

 

Agenda at a Glance:

  9:00 am   Continental Breakfast, Meet-N-Greet

  9:30 am   Welcome & Opening Remarks, Accenture & BMC

10:00 am   User Presentation with Jeff Sanderson, Accenture

10:30 am   Morning Break

10:45 am   Group Discussion / Q&A Session Part I

11:15 am   Application Integrator Deep Dive w/Dave Leigh, BMC

12:00 pm   Lunch

12:45 pm   Group Discussion / Q&A Session Part II

  1:15 pm   Control-M 9 Update / Roadmap w/Robby Dick, BMC

  1:45 pm   Discussion on User Group Moving Forward

  2:00 pm   Meeting Adjourns

 

Special thanks to Jeff Sanderson and Accenture for hosting this user group meeting!

Share: |


-by Joe Goldberg, Solutions Marketing Manager, Control-M Solutions Marketing, BMC Software Inc.

 

You’re implementing a new application and you have either downloaded a new Application Integrator job type from the Application Hub or built a job type for it. How is your life better than if you had not done that?2me.png

 

First

You can build a Connection Profile which contains all the general information about your application such as which server it’s on, which ports or libraries it uses and any credentials that may be required to run its jobs.

 

This means none of your jobs have to specify this detailed information and if it ever changes, you just update the connection profile instead of tons of jobs or scripts.

 

Your auditors and ultimately your management love you because you don’t fail audits and you don’t expose potentially sensitive and thus dangerous information.

 

Second

When you build jobs, it’s IDENTICAL to the way you build jobs today. Grab the job type from the job palette and drop it into the flow wherever you need it. The forms ensure you specify correct application characteristics because you can select from a list that is retrieved from the application in real time. And if you ever need help with a particular application, the information as to which person or group for you to contact for support is part of the job type. If you use Control-M Workload Change Manager, you can even specify site standards so that these application jobs are built correctly and in adherence to your operational requirements.

 

Third

When you are running these jobs and someone in the business tells you to hold all jobs for “application x”? NO SWEAT! Find, filter or create a viewpoint (your choice) that shows you all the jobs in the active environment for that application and hold them or delete them or whatever you need to do. Because Application Integrator created a new “job type” you can search for it easily. And if that same person in the business told you to change something in a bunch of those jobs? You can use Find and Update to do that quickly.

 

Fourth

What about looking at output when something fails? Well, if your developers wrote their own scripts, they probably put the output somewhere where they can get it. The problem is the person analyzing the current problem may not know where that is or worse not have access to it. And each script could be different and your developers spent a lot of time writing that code. Tell them they don’t have to bother because Control-M will take care of it for them when flows are built using Control-M job types instead of scripts that run as “black boxes”.

 

Finally, considering you are probably seeing more and more new applications and they’re coming at a faster and faster rate, all the above benefits may sound better and better.

 

So why not visit the Application Hub today and grab a new job type for a new application you are implementing. If there isn’t one there, perhaps you can build it and help the next person who may need that very same job type. Remember, you have the technology!

 

The postings in this blog are my own and do not necessarily represent the opinions or positions of BMC
Share: |


-by Joe Goldberg, Control-M Solutions Marketing, BMC Software Inc.

 

MapReduce, Pig, Hive and Sqoop are “legacy” Big Data applications that require workflow management with Spark, Flume, Kafka and dozens others arriving on a regular basis.  And to make matters more interesting, embedded workload solutions like Oozie and Sparrow don’t know how to handle anything outside their own world.   So how do you cope with complex workflows that include many of these technologies, especially since new ones keep arranging on the scene?

 

Control-M with Application Integrator is the BMC answer. Control-M provides core application integration out of the box for MapReduce, Pig, Hive, Sqoop and HDFS operations together with every major commercial platform, technology and application from the traditional enterprise computing world. And using Control-M Application Integrator, all other applications not included in the previous lists can be easily supported by building a job type with a simple web-based designer tool.  Or even better, operations and developers can look for job types that may have already been built by other users and shared via the unique Control-M Application Hub. This approach enables Control-M to deliver a complete workflow management platform that meets all your current and future requirements.

 

BMC Control-M has strongly committed to the Big Data Market and is continuing its investment by:Snap1.png

  1. Platinum sponsorship at Hadoop Summit San Jose 2015
  2. Releasing Application Integrator
  3. Joining the Open Data Platform consortium

 

Stop by booth P9 at Hadoop Summit and learn more about the most comprehensive workflow solution in the Big Data and Enterprise market - BMC Control-M.

 

 

The postings in this blog are my own and do not necessarily represent the opinions or positions of BMC Software

Share: |


If you are running multiple Control-M servers, including Mainframe and Distributed environments, you can control your workload flow between the environments using global conditions. For example, after a job completes on your Mainframe Control-M/Server, it can trigger a job on your Distributed Control-M/Server and vice versa.

 

On Wednesday, June 17, 2015, Froilan Reyes and Richard Talbert will discuss the following about Using Global Conditions:

 

• The Global Condition Server(GCS) process overview
• GCS configuration
• Passing conditions between Control-M/Servers
• Demonstrate adding, using and testing global conditions between mainframe and distributed environments


Join us as we go over creating and testing global conditions and don’t miss the live demo. There will also be a Q&A.
Registration is now open here.

Share: |


On Wednesday April 22nd 2015, Neil Blandford will demonstrate how to use Agentless technology to access unsupported platforms and reduce agent maintenance. 
There will be a live demo of Agentless setup for both SSH and WMI.  There will also be a Q&A after the demo.
__________________________________________________
Q: In my shop WMI is typically locked down due to security reasons - before we could use agentless on windows we would have to address security concerns first - can you share any known ways to sure the agentless/WMI setup.

A: WMI should be accessible by the Agent service user that is configured. In this respect you should treat WMI as you would with any other monitoring/management system.
If you are unable to configure WMI for the remote host, you can always install an Agent.

__________________________________________________
Q: What user rights are need it for the agent id to prevent security issues with agentless processes?

A: User rights are covered in KA395027
https://kb.bmc.com/infocenter/index?page=content&id=S:KA406217

__________________________________________________
Q: Is BMC_ERROR_LEVEL a special environment variable?

A:This is used to pass the return value back to output file (.dat) on the Remote Host.

__________________________________________________
Q: Is the passphrase anything we want to make it?

A:Yes you can specify whatever you like. Typically the longer the better.

__________________________________________________
Q: Can you explain the problems with "impersonation level" on remote hosts from windows machines?
For example, when running a script located on central repository server, having a batch program accessing a
shared folder or printing the job’s output to a network printer.
When the remote host is configured for WMI communication mode, such network resource

A: Agentless is restricted by the Windows “double hop” security mechanism. In this case the OS won’t (by default) pass credentials to a third machine from a remote connection. The machine will attempt an Anonymous connection, which will usually fail.
It is possible to create a trust between the machines to prevent this (you Windows Administrator can do this), however it’s preferable to use local resources in the command/script.

__________________________________________________
Q: Is there a way to setup the public key on the remote host via control-m or do we have to set the public key directly on the remote host by connecting to it like an admin would and manually setup the public key

A: Yes, you can initially configure the run as user to use a password, you can then create and run a job to place and configure the public key. Once you have this you can then re-configure the run as user.

__________________________________________________
Q: Are there any additional considerations to set agentless process to a cluster environment?

A: As long as the Agent looks at the VIP or Virtual Hostname correctly it will connect.

__________________________________________________
Q: If I have the need to run a system command on "thousands" of machines, is there any limitations to using agentless technology vs. agent?

A: The scalability of your Agentless solution depends mainly on the Agent. The Agent will need to have sufficient resources available to handle 1000’s of connections in this case. This is also the case with an Agent installation however. The best option here is to gradually increase the load to test.

__________________________________________________
Q: Does Agentless support file watcher jobs?

A: Filewatcher depends on many libraries to function, at this time it’s not practical to copy them to the Remote Host to run. So Filewatcher isn’t available.

__________________________________________________
Q: When we have issue on Control-M Agent we run ctma_data_collector, do we have any similar in RH?

A: The Agent data collector will gather information Related to the Remote Host. When ruuning the Data Collector, be sure to run it on the correct Agent – the Job log indicates which Agent the job was run from.

__________________________________________________
Q: Is there a limit on the number of jobs that can be run on a Remote Host?

A: There is no defined limit in Control-M, other that resource usage, connections for example.

__________________________________________________
Q: Can SSL be used with agentless?

A: Agentless uses SSH, WMI, SMB, and SFT depending the configured options.

__________________________________________________
Q: Is it possible to use the SSH option with Windows?

A: Yes, you will need to configure an SSH Server to connect to.

__________________________________________________
Q: Where are the SSH keys stored?

A: They keys are stored encrypted in the Control-M Server database.

__________________________________________________
Q: We have a PRDO and Failover servers. Only one Control-m\EM is available at the time. but agents are running on both servers. Remote host showing the only prod agent in the list. When we transfer to Failover Control-m server, we have to switch?

A: You can do this, alternatively you can use an Agent that has both Control-M configured.

__________________________________________________
Q: Any recommendations on the best agents to use as the host  i;e the control-m server agent?

A: You can use either depending on your needs and environment.

__________________________________________________
Q: Is any way we can point remote host to Host name instead of agent directly?

A: The Agent manages communication with the Remote Host and provides the Control-M infrastructure to allow jobs to run. Jobs have to still be submitted to an Agent.

__________________________________________________
Q: Can a Remote Host be switched to an Agent and vice versa?

A: Yes, an Agent can be converted to a Remote Host via the Configuration Manager.
To convert a Remote Host to an Agent, remove the remote host and install an Agent on the machine.

__________________________________________________
Q: What are a few of the things you would NOT be able to do with remote host that an actual agent would provide?

A: Control Modules require the Agent installation to be able to run.
The Agent provides the option of a persistent Connection in firewall environments.

__________________________________________________
Q: Any option can triger notification when Remote Host is unavailable?

A: When a Remote Host becomes unavailable it will be marked as such in the Configuration Manager.

Share: |


On Wednesday May 20th 2015, Pilar Soria will discuss how to install and configure Control-M for Hadoop and demonstrate creating jobs to automate Big Data. The webinar will include:


• What is Control-M for Hadoop?
• Installing, configuring and setup of Control-M for Hadoop
• Creating Map Reduce and Sqoop jobs
• Hadoop job Conversion tool
• Demo configuring Control-M for Hadoop with Kerberos

 
There will be a Q&A after the demo. Register Here!

community_185x185.jpg

Filter Blog

By date:
By tag: