Skip navigation

TrueSight Orchestration

4 Posts authored by: Wookie Houle
Share This:

The support issue came in as "Schedules Running On Wrong Days".  After some back and forth, the problem was that workflows were converting dates to determine if they should run or not, and were running on the wrong days.

 

The customer had a collection of jobs that needed to run on specific days of the month and/or year.  The pattern was both beyond the reoccurance capabilities of BAO's internal scheduler, and frequent enough that it would have taken defining 20+ schedules in the module and that wasn't going to scale.

 

They solved this by creating one daily schedule for each job and having the job check to see if it should continue running that day.  They created a module configuration item with XML of the form:

 

<dates>
  <day>0101</day>
  <day>0401</day>
  <day>0701</day>
  <day>1001</day>
</dates>


 

(Theirs was way bigger, but you get the idea.)

 

The process called by the scheduler started with a utility process that converted "now" from the Utility Activity into a string of the form "mmdd".  The main process then did an Xpath transform to compare the "now" string to the values in the XML in the config item, and if a match was found, it would return "true".  When "true", a subsequent Switch activity would allow the process to actually run, otherwise it would just stop immediately.  They didn't need to even return the <day> element of the XML, because just the fact a match was found was enough.

 

The date comparison was done via an Xpath transform in the form of:

 

contains(., '$[now]')




 

This worked perfectly in BAO Development Studio, but once activated on the grid it sometimes ran on days it wasn't supposed to.  The job had run when it should have on January 1st, but had run incorrectly again on January 4th.  They opened a support ticket with BMC Customer Support and we dived in to figure out what the problem was.

 

The root cause of this issue was the use of the contains() Xpath function.

 

Xpath functions generally operate on an "input document" that may be all or part of a XML document.  The "." is used to signify "here".  Where "here" is, depends on the context.  When used, like in this instance, the "." turns into the whole XML that was fed in.  But the contains function operates against values only, so all the XML elements were stripped out, leaving just the string "0101040107011001" to compare against.  Doing the replacement on the contains function above, it actually looked like this to BAO:

 

contains('0101040107011001', '$[now]')




 

This is going to return true on the days expected, but also on "0104", "0107", "0110" and "1010".

 

The solution is to change the Xpath to look at lines first, and search within them second.  This can be done by doing a search for the day elements, and then picking only the one that has the date in it that matches $[now].  We still use the contains function, but we use it in the ordinal to only choose the line with the right day in it.  That transform looks like:

 

//day[contains(., '$[now]')]/text()





When $[now] is "0101", this will return:

 

<value>0101</value>




 

When $[now] is "0104", it will return

 

<value />




 

You can sexy the transform up some, to come up with a true/false result:

 

string-length(//day[contains(., '$[now]')]) > 0



 

There are other transforms that can be used, but I'll leave them to the Teeming Millions.

Share This:

BMC Atrium Orchestrator 7.8 adds new functionality that is not backward compatible.  BAO now has a 'While' activity in Development Studio that can be used in processes.  BAO users that have multiple installed environments, such as production, staqing, dev, and test, maybe not be able to upgrade all environments in a timely manner.  It may happen that development environments and developers are on BAO 7.8, while production environments are still previous versions, and development and deployment of new modules to production can't be halted in the interim.

 

We've been asked more than once how to handle this.

 

BMC will not be providing support for, or backporting, the new While activity available to BAO 7.8 to versions of BAO prior to that.  Modules created and/or edited in BAO 7.8 dev studio or later, may still be usable on grids running BAO 7.6.03 or 7.7.xx.

 

BMC Atrium Orchestrator QA has tested modules created in BAO 7.8 Development Studio on a BAO 7.6.03 grid.

 

  • If a BAO 7.8 module contains no processes with the new While activity, jobs will execute as expected.
  • If a BAO 7.8 module containing processes with While activities is activated on a 7.6.03 grid, the module will activate, but any job that executes a process containing a While activity will fail without compensation.
  • Modules created in BAO 7.8 containing While activities cannot be imported into previous versions of Dev Studio.  It will throw errors until the offending module is removed.

BMC Documentation Survey

Posted by Wookie Houle Sep 29, 2014
Share This:

Documentation is never good enough, it seems.  I suspect there is a Law Of Software Documentation that states this must always be so.  We slave over hot document processors day and night, and yet somehow there is always something missing, something confusing, or something that would be well documented if said documentation wasn't scattered over 8 separate sections.

 

We'd like to ask everyone to take a survey to help us prioritize problem areas.  Click the link below, and follow the link there to take the survey.

 

Efforts to Improve Product Documentation

 

We thank you for taking the time.

 

Wook

Share This:

All data in BAO is XML.  Even data that doesn't look like XML when you enter it gets wrapped in <value> tags so it is valid XML. Entering a string in an assignment activity input panel gets wrapped in <value> tags.  Most of the time, this is a transparent process and developers have no issues with this.

 

The secret is that BAO treats any XML that is only one level deep as a string.  When it outputs XML like this, it will automatically remove the enclosing tags so it looks like a non-XML string again.  It will even do when the enclosing tags are not named <value>, it will remove any element tag when it prints a "string".

 

<crazy-element-name>This is a string!</crazy-element-name>














 

<value>
  <string>This is not a string. This is XML</string>
</value>














 

BAO does this everywhere.  For example:

 

  • Printing to logs
  • Tokens in transform editors
  • Input text boxes that contain context item objects
  • OCP dialogs
  • Maybe others, you can post below if you find one

 

In the last month or so I've had a couple of support cases where developers ran into problems having to do with how BAO handles what it considers strings.  The problem was the "Parameter type" drop-down in the input selection of an assign activity set to "String" when it should have been "XML".  BAO does not treat an input parameter type of "String" the same as it does "XML".  Entering XML in the input box when the drop-down is set to "String" will destroy the XML.

 

Try this experiment:  In a new process, create an assign activity that puts a string in a context on the first assign row.

stringXMLinputvalue.png

In a second assign row, feed that context into this XSLT transform:

stringXMLtransformvalue.png

You can't do this on a single assign activity row because Studio won't allow you to create a transform for a string value.

 

When run, if your string is "I am a string!", you will see the following:

 

<new-element>
  <value>I am a string!</value>
</new-element>














 

The "Copy-of" node will copy the entire input XML so we can see the <value> tags that get added.


Something similar happens when you put XML into the input panel with the "Parameter type" set to "String".  In addition to wrapping the "string" in an XML <value> element, BAO also escapes the illegal XML characters in the "string".  The characters <, >, &, and others are illegal in XML values, because they are XML markup characters.  This has the side effect of destroying any XML entered.  The problem often isn't immediately noticeable because BAO always converts escaped text back to the normal form when printing.

 

In another workflow, put some XML in an assign activity input panel with Parameter type set to 'String":

 

stringXMLbadxml.png

The contents of "string context" are now:

 

<value>&lt;list&gt;&lt;value&gt;this is a string, NOT!&lt;/value&gt;&lt;/list&gt;</value>














 

But is will always print in Dev Studio and in the processes.log file as what it looked like in the input box.  If this was instead XML that was going to end up in an adapter request, that adapter call probably won't work.

 

To see this, we'll write the "string context" context item above to a file with the file adapter. Add another row to the assign activity like so:

 

broken-xml-request.png

 

We feed the context item "request" into a Call Adapter activity, pick a file adapter, and set up some logging for the request and the response.  When we run the resulting workflow, the processes logging in the Console Window even falsely shows everything is going to be okay:

 

[request=
<request-data>
  <fileRequest>
  <filename>C:\\broken-xml.txt</filename>
  <character-set>cp437</character-set>
  <filetype>XML</filetype>
  <lines>
    <line><list>
  <value>this is a string, NOT!</value>
</list></line>
    </lines>
  </fileRequest>
</request-data>]












 

But the actual contents of "C:\broken-xml.txt" are:

 

<?xml version="1.0" encoding="UTF-8"?>
<lines>
  <line>&lt;list&gt;
  &lt;value&gt;this is a string, NOT!&lt;/value&gt;
&lt;/list&gt;</line>
</lines>












 

Which is most likely not what you were expecting to write into this file, if this was a real attempt to write data into a file.

 

You'll notice in the broken-xml.txt file, if you decode all the escaped characters, there are no <value> elements surrounding the destroyed XML.  That is the last part of how strings are treated by the assign activity input panel.

 

To see what happened, take the first workflow above that showed us the <value> tags added to the string, and replace the assign activity row with the XSLT transform above with something like the following:

 

string-test-2.png

 

The results of this, when run, are:

 

[some XML=I am a string!]











 

No XML tags, even tho we might have expected some.  What happened is this:

 

When placing a context item with a "string" in it in the input panel as I've done above, the string is unwrapped from its XML tags, so that those tags don't show up in the resulting XML.  This is a feature.  It would otherwise be impossible use the input panel as I did to create a valid adapter request.

 

What is in the "some XML" context item is actually:

 

<new-element>I am a string!</new-element>











 

If you used the XSLT transform above, you could verify this.  When that is printed into the Console window, what gets printed is printed without the <new-element> tags, because those tags are only one level deep, meaning this will be printed as a string.

 

The Studio console window and processes.log will show the un-escaped text, which will often look perfectly normal.  When you are dealing with adapter requests and actual data, it can be difficult to find out that some or all of the XML is actually escaped.  Usually, the first clue that this is the issue comes from DEBUG logging in grid.log once you've opened a BMC Support case, and we've asked you for adapter debug logging.  The output in grid.log from debug logging doesn't usually escape the XML going into adapters.

 

Let's go back to the XML test case where we write the broken XML into a file.  If I change the Parameter type to "String", with the same request XML in it, like so:

 

broken-request.png

 

The workflow will compensate with an error, or maybe it won't but you won't get any data back.  There are lots of actual errors you can get, depending on just how the adapter request is borked.  When using the AutoPilot wrapper modules, where the whole adapter request is formed out of parts, where maybe only some of the parts are XML broken by being escaped, the error might not really tell you what is wrong.  Adapter debug logging is usually where you'll first notice the escaped strings.

 

When we run the above, the compensation we get is:

 

An error occurred that triggered compensation:
  Summary: Execution failed.

Caused by:
   Summary: XML being processed is invalid.
   Detail: The XML is considered to be in an invalid format due to the
           following reason "The adapter request data was not found in
           the XML", and processing cannot proceed.
           The XML being processed was: <value><request-data>
  <fileRequest>
    <filename>C:\\broken-xml.txt</filename>
    <character-set>cp437</character-set>
    <filetype>XML</filetype>
    <lines>
      <line><list>
  <value>this is a string, NOT!</value>
</list></line>
    </lines>
  </fileRequest>
</request-data></value>











 

Note that the actual XML looks okay.  It is all pretty, and indented, which is ALWAYS how BAO prints XML.  The two clues here we should notice are the <value> tags wrapped around the whole thing, and that there is no line break between the <value><request-data> tags indicating BAO doesn't consider one of them, the inner one, to be XML.  The other common clue is that the XML prints all on one line, something BAO doesn't do when printing what it considers to be XML.

 

Depending on exactly how the adapter request is broken, the results might not have these clues, and might look completely correct.  You'll need to verify in the grid.log what is going on.

 

I've not gone into examples around tokens and the OCP.  I'll leave those as an exercise for the teeming millions out there.

Filter Blog

By date:
By tag: