Gilles Robert just posted a really neat blog about an under reported feature of MVI 6.1. It is the JOBSTAT view which provides an SDSF-like interface for information about jobs, TSO users and STC's in the JES queues. If you are not already following Gilles in MV Communities, check out his post at; https://communities.bmc.com/people/grobert/blog/2013/07/17/a-golden-nugget-of-the-mainview-infrastructure-v61
My colleague, Trish Nolan, reminded me that there is a really great resource available from BMC that sometimes goes under-utilized. It's called the BMC Documentation Center.
Now, most folks have known for many years that, from our Support webpage (http://www.bmc.com/support/product-documentation/), you can select the Supported Product A-Z List link and navigate to each MainView product. From there, you can find links to the full set of documentation related to that product in PDF format. However, what many seem to be unware of is that the same Documentation webpage contains a link to the Documentation Center and Quick Course Library for Mainframe Products. This is a relatively new documentation link that consolidates docs for many BMC mainframe products, including MainView.
As the overview says it provides many useful functions, including a search facility that spans all documents in the library. You can create a simple or complex query using wildcards, AND,OR, and NOT relationships, as well as exact phrase searches. For example, you could find all references to XCF in any BMC product doc and then limit the scope of the search to only MainView products. This is much more powerful than the links in the A-to-Z list and faster, too.
In the Documentation Center, there is also a link to the BMC Quick Course Demo Library. This is an extensive set of recorded 'explainers' that cover selected product concepts, tasks and features. In the MainView section there are nearly 100 Quick Courses on topics ranging from product overviews to usage tips such as 'Creating a Predictor Model' in Capacity Management for Mainframes (CMM) to 'Setting Priorities for DFHSM Recalls' in MainView for SRM. Most Quick Courses are only 5 to 10 minutes in length, with the longest less than 30 minutes.
So check it out and let us know what you think. Oh... and if you think of a topic for a Quick Course that is not in the library, let us know that, too. BMC experts will be happy to record another one.
Happy New year to all...
I was recently included in an email string between one of our field SC's and one of our CMF/MVzOS R&D engineers. While it was actually a discussion about where overhead is incurred in CMF/MVzOS, it contained some discussion about the difference between SAMPLERS and COLLECTORS within the architecture of of the MVSPAS address space. I have had this conversation with many customers over the years and I thought I would share it here (with a few annotations).
My thanks to Fred Novakov (Principal SW Consultant - MSM) for the inspiration and Keith Moe (Lead MSM Technical Supp Analyst - CMF/MVzOS) for the text.
First a terminology clarification.
CMF MONITOR consists of a set of SAMPLERS. The PRIMARY intent of the Samplers is to produce 7x SMF records. This function is frequently referred to as the CMF Extractor (as opposed to the CMF Analyzer).
MainView for z/OS is (with many other pieces) a set of DATA COLLECTORS. Their PRIMARY intent is to produce interval and history records.
Wayne's Note: Historically, CMF started back in the late 1960's with just the SAMPLERS. It was only when the MAINVIEW architechture (then called BBI-3) was built to provide online visbility to CMF data, that COLLECTORS were added on top of the core CMF product. There is also sometimes confusion between SMF 7x records and Interval /History records. Generation of SMF 7x is done by the original CMF Monitor code
CMF MONITOR Online is a SUBSET of MainView for z/OS with fewer Data Collectors (and corresponding views, etc.).
If a customer has both CMF MONITOR and MainView for z/OS, they have all of the Data Collectors and can run ALL of the CMF Samplers. If they have only MainView for z/OS, they have all of the Data Collectors and can only run the CMF Samplers necessary for the Data Collectors (and cannot produce 7x SMF records). If they have CMF MONITOR only (which includes CMF MONITOR Online), they have all of the Data Collectors except JSTM (Job Step), MSCP (MVScope), and RWRN (SYSPROG AEW), plus no real time views . Of course, MVUSS has a additional set of Data Collectors not included in either.
For minimum CMF Online functionality, the Data Collectors REQUIRE 6 CMF Samplers to be running in the CPM or IPM CMF "session" the Data Collectors are “connected” to: ASMDATA, CACHE, CPU, DEVICE class="DASD", DEVICE class="TAPE", and PAGING.
The Short-term Data Collector and Long-term history records are available to both products. These are controllable by UBBPARM members.
Unlike RMF, where Monitor I (7x records) and Monitor III (online and history data and SOME 7x records) run in separate address spaces, the CMF Samplers and the MainView for z/OS (CMF MONITOR Online) Data Collectors run in the SAME address space. This allows the Data Collectors to make use of SOME of the data gathered by the CMF Samplers (without using a lot of CSA to do so).
So you now see that SAMPLERS and COLLECTORS, for the most part, perform two different functions within the CMF/MVzOS address space. These are (1) to obtain the basic z/OS performance data that is written to SMF 7x, and (2) to make it availabe for online perusal.
BTW - The other part of Fred/Keith conversation had to do with overhead. I'll try to blog that info later...
On Dec 4, BMC R&D announced BETA availability of MV/CICS v6.5. This new version includes support for CTSv5.1 & CTGv9.0, and there are many new views and features that have been introduced in this version. For complete documentation of this new BETA version, see the attached MV/CICS v6.5 Beta Release Notes and Resource Security Updates PDF docs.
I've been pretty busy since I returned from System z U last week, but I wanted to publish this writeup on zAware that Curt forwarded to me. Thanks, Curt!
"Back at the z Conference again, one of the other major buzz topics was the new zAware facility for the EC12. IBM positions zAware as "Smart Monitoring of z/OS" so naturally as a MainView Software Consultant I was curious whether this was a competitor to MainView. In a word: No.
While zAware is an interesting concept I would not call it a monitor as I think most of us understand the term. To me a monitor is a program that extracts realtime or near realtime data from the monitored system via control block peeking, APIs, etc. and displays the data, creates alarm events from the data and can take action against the monitored objects (such a cancelling a looping job). This is not what zAware does at all.
So what does it do and why do I think it is interesting? Well, the first thing to understand is the zAware architecture and functionality. IBM refers to zAware as "out of band" monitoring. What this means is that zAware is not an address space that resides on z/OS; rather it is a completely separate and self-contained LPAR that performs it functions based on data it receives from the z/OS LPARs. This new type of LPAR can only be created on an EC12 machine. When I say it is self-contained everything in the LPAR is delivered in firmware, there is no maintenance via the traditional SMPe route. IBM made an analogy to a coupling facility LPAR although emphasizing that the functions in a zAware LPAR were by no means similar to a CF LPAR but it is just there and it does it thing, no help from you required.
So what is in this new LPAR? Essentially it is an expert system/pattern recognition engine which analyzes data is receives for anomalies based on an historical database. It receives its data from the monitored LPARs over TCPIP; this means that z/OS images on other hardware such as a z196 or z114 can be supported as long as you have an EC12 to support the zAware LPAR.
And what data is it that it analyzes? Simply the message stream going to your console. The good news is that zAware is "agnostic": it does not care whether the message comes from IBM, an ISV product, your application or wherever. The bad news is if there isn't a console message, zAware is not aware of the situation.
Another issue with zAware is the length of time it takes to analyze and display the data and how the results are delivered. According to the presentation I attended it "samples" every 2 minutes (which I take to mean it analyzes the incoming data) and reports every 10 minutes. This is an eternity on an active z/OS system. Realtime monitors' analysis cycles are typically every 30-60 seconds (users often want them lower) and the results are available immediately. Then there is the delivery of the results themselves. At present, as far as I can tell, the only way to see the results is literally to "see" them: Sit at a GUI screen and watch for it to turn orange or red. And it only changes once every 10 minutes. I think we have all learned by now that this no longer happens in a modern data center; there needs to be some kind of alarm mechanism that notifies someone when an anomaly is detected. To be fair, the results are also written to an XML file but currently there is not an application such as MainView or Omegamon to process that file into useful events that can be fed into existing notification channels. I expect that will be coming sooner rather than later as happened for Health Checker data.
So why is this interesting? Because it is the beginning of an ability to spot problems based on individual system behavior, not static rules. And, according to IBM, zAware will learn more about the systems it is monitoring based on feedback whether the anomalies it detects are actual problems or not (sort of like the now famous Watson incident on Jeopardy involving the placement of Toronto as a US city -- that will never happen again). And as software such as CICS and DB2 become "smarter" about themselves and put out more useful messages, well, you can see where this is going.
I'm sure there are some major IBM customers who will be putting this to use soon, I will be interested to hear how it works for them. Maybe we'll hear more at the next z Conference."
Curt just forwarded the following regarding zTechU sessions he attended at zTechU. The following are his comments;
"Yesterday afternoon I attended 2 sessions on the new Flash Express memory feature. Flash Express is PCIe card-based solid state memory which looks like a very high speed I/O device to the operating system. It will be used initially by the ASM for paging; speeds in the neighborhood of 5-20 microseconds are claimed. So while this is much slower than cache or main memory access it is literally hundreds of times faster than conventional DASD paging, even from SSD. For those of you around in the 80s, and that would probably be most people reading this blog, it should remind you of the expanded storage that was available for a time when memory was pretty expensive. I asked one of the IBM folks if this feature was conceptually similar to ESTOR; he replied emphatically not. In the second presentation they mentioned this was supported by an enhanced version of the original ADM code that supported ESTOR. Oops! If it walks like a duck ... I rest my case.
So who would be interested in this? Well first you have to buy an EC12 and run z/OS 1.13 plus some PTFs to get the support (btw, the initial set of PTFs is due 12/14/2012, additional feature support in 1Q/2013). If you have done all this you can buy Flash Express memory in 1.6 TB (that's right, terabyte) chunks up to 6.4 TB per CPC. The additional "memory" can be attached so much per LPAR up to the amount you have purchased. And this is one of the best things about Flash Express: it does look like a new kind of memory to the system and is configured through the HMC; no change to the IOGEN is necessary, the CE just plugs the card in and it's "there".
IBM is clear however that Flash Express is not a substitute for main memory (it's waaay to slow for that and doesn't work that way anyway), it is intended to provide a much faster paging mechanism. They are also clear that it benefits workloads that can tolerate some paging but will not benefit those which cannot. So if you are paging a significant amount and especially if you have implemented storage isolation to protected "loved ones" (which simply pushes off the paging to less favored workloads and may even make it worse) Flash Express is for you! Also, Flash Express can help any customer who experiences paging "spikes" with operations such as start-of-day processing and system dumps.
And Flash Express has a couple of new features that may also help that paging problem now and in the future. The primary one coming out in December is that the new "large" 1MB pages will be pageable to Flash Express (not DASD). Currently they are all page-fixed in main memory so if you are using them to a significant degree this could be a big benefit. And there is a Statement of Direction from both DB2 and JAVA that they will be using pageable large pages in the future so the utilization of Flash Express for this function could go up dramatically in the future.
Another feature of Flash Express in this area is the "speculative page-in" where multiple pages are brought into memory on a single read based on locality to the initial page fault page. Similar to the DB2 prefetch function, this is intended to avoid additional fetches for pages that are judged "likely" to cause a page fault in the near future. IBM is working on several algorithms to better identify which pages they bring in.
In the 1Q 2013 update IBM will allow you to completely eliminate the Common and PLPA datasets and send everything to Flash. Obviously these datasets are good candidates for multiple access and putting them on a much faster "device" is probably a good idea. Another feature coming in at this time is dynamic reconfiguration of the Flash Express storage although in most cases you could put the equivalent of all of your LPARs' paging subsystems to on a single Flash Express card and then some so I'm not sure how often this would be required.
I hope this has given you some idea of the new Flash Express features and whether they would be of use at your site. As always contact IBM for official information."
Lots of good information exchange going on System z Tech U this week...
BMC has Booth 42 on the Expo floor and we are constantly busy. I did a vendor presentation on Threshold Advisor and MVI61 yesterday morning to a full room. That recent enhancement has become the big buzz at the booth and Mark Rascoe & I have been doing non-stop demo's. One customer said that his company has been entertaining replacement offers to MainView from competitors, but that this will likely stop that discussion in it's tracks. An IBM Professional Services tech for Omegamon saw a MV demo with ThrAdv and walked away saying 'Awesome!'. I kinda like that.
We are also represented by several other BMC folk who are attending sessions and gathering information on the latest news. I will try to post additional blogs on interesting bits gleaned in upcoming blogs...
I just attended an interesting presentation by Sridhar Gangavarapu of BMC's MVzOS/CMF R&D group on how to add a link to run MVE in IBM's z/OSMF.
For those unaware, z/OSMF is IBM's "Web-browser based management console for z/OS." which according to the z/OS 1.13 docs is intended to support such launch points as;
- Incident Log
- Configuration Assistant for z/OS Communication Server
- Workload Manager
- Resource Monitoring and System Status (RMF)
- Software Deployment
- Capacity Provisioning
- Classic ISPF Task to launch to ISPF functions directly
Sridhar & Alan Whitman (MVE developer) figured out how to imbed MV/Explorer into z/OSMF for our customers who are using it.
Here is a sample screen shot that I captured during the demo.
Sridhar is going to provide me with a copy of his PPT. When I get it, I will post it into the documents section for this community.
...and I am a Moderator for this group. Wow. New stuff for me.
It may seem odd for folks such as we who make our living in IT, but I am a self-described luddite who does not have a Twitter account or a Facebook page.
So I will be learning the idiosynchrasies of this Forum right alongside you. So, please feel free to open discussions, post documents or create blogs on your own. I will help as much as I can and I have some great mentors to rely on when I too hit the wall.
By the way, I am thinking about some useful documewnts to post here as an add-on to many of the other repositories of useful information related to MainView. An immediate thought is the many MainView Quick Course recordings that the MainView SC's have labored so hard to create these last 6 months or so. (BTY - for those who do not know, they can be found at the BMC Support websaite at https://webapps.bmc.com/infocenter/index.jsp. Select BMC Demo Library, then BMC MainView Products.
Any other ideas you have for making this a useful Forum are welcomed. Either post your comments in the discussions section or email me.
Wayne (the Luddite) Wilson
- Alice Ma
- Brenda Hooper
- Brian Hunt
- Bronna Shapiro
- Dani Scherer
- Fred Novakov
- Jay Lipovich
- Gilles Robert
- Ken Ziolkowski
- Martin Beasley
- Paul Spicer
- Susan Rice
- Tom Nurse
- Wayne Wilson