Hi Keshav,

We are facing exactly that same problem. Did you get a solution?

Thanks in advance. Best regards,
Carlos

Is it an error you see when (for example) accessing Atrium Core console with a flex popup saying "RPC error" or something?

If so we have the problem too though working with support at the moment.

What version of ITSM are you using in 7.6.04? Are you in server group with load balancer?

Hi Laurent,

It's exactly as you say. A message appears in Atrium Core Console with the message "RPC Call Failed".

We are on version 7.6.03 patch 001. Our configuration is a servere group with a hardware load balancer in front.

I'm managing the issue with support as well. We have seen other similar errors, all of them regarding connectivity with the ar system servers, specially with the atrium console.

Please, share with us any progress on this issue. I'll do the same.

Regards.

I see, we are currently trying a "debug version" .jar which looks promising (no errors in two days), customer will try it and "validate".

We are currently in 7.6.04.

Though it seems there are in fact two possible issues:

Customer <-----> Load Balancer 1 <------> Mid Tier <-----> Load Balancer 2 <------> ARS server

Do you have several mid-tiers and several ARS Servers, so two Load Balancers?

It seems there is a known issue when you got a Load Balancer "in front of" Mid-Tiers and they already have a "aui-resources.zip" file for that.

Our issue was because of the "Load Balancer 2" in front of the ARS servers.

The configuratioin is exactly the same (maybe your Load Balancer 2 is not configured with sticky sessions as ours, since from version 7.6.04 it is not required).

I'll wait for an answer form support, but, what is the .wrong jar file ? In the other hand, you mean that there is a new version of the aui-resources.zip file? Did you have to set any specific configuration on the Load balancer 2?

(Sorry for the bulk of questions)

Well LB2 doesn't have sticky since it "must" be off for 7.6.04, apart from that it's pretty standard I guess.... It's a BIgIp F5.

In the beginning support gave me an "aui-resources.zip" telling me that it happened in 7.6.03 and it solved this issue for some customers, in our case it didn't.

The .jar I'm talking about is "/arsys/ThirdPartyJars/cmdbapi7604.jar"

I guess you should start with the aui-resources.zip first, I guess that's what the support will give you

We have the exact same kind of architecture. It used to happen to us too, while opening Atrium Console first time RPC failed message used to popup. But I usually hit refresh / F5 or reload the console and the error would go away.

As Laurent pointed out it should be because of the load balancer in front of the mid tier servers and network delays.

Why sticky bit should be off on LB2?

thanks.

Hi Amey,

From 7.6.04 AR Servers support real load balancing without the use of the sticky bit. It's documented like that (you have to configure some parameters on the mid-tiers).

Indeed Carlos, it's supposed to be like this though "not really" in fact

In the documentation:

If you aren't in 7.6.04 (so 7.6.03):

-> set sticky bit to "on",

If you are in 7.6.04:

-> put LB sticky bit to "off" for the LB in front of the ARS Servers.

-> check "Enable lifespan" into the "connection settings" tab of the mid-tiers,

That's the theory. It seems that if you have CMDB 7.6.04 (no sp), support "might" suggest (if you have some errors) or to upgrade to CMDB Sp2 or to set sticky bit to "on" and so disable it into the mid-tier.

RPC failure issues, sometimes gets resolve by restarting Midtier/Tomcat server.

Yes, but usually in this peculiar case it works ok for 20-30 minutes and then RPC errors spawn again.

Are there any errors generated from plugins?  IME, RPC errors usually stem from some type of network or connection error...  MidTier busy, plugin not starting properly, etc

From my Android phone on T-Mobile. The first nationwide 4G network.

Check the cmdb_eng_debug_log, and arerror.log

Like I said it's two known errors

You'll see that in client side in logs from "Flash debug" logs and Mid-tier "flex" logs.

Server side you'll see nothing special if you don't have a debug dll.