12 Replies Latest reply: Aug 14, 2012 11:29 AM by InfoJester RSS

RT clouds. How big, how many, how?

InfoJester

Hello esteemed fellows and keepers of the Patrol knowledge.

 

I've been struggling with my RT cloud, it's very distributed with a few pairs of RT servers in several locations all connecting up to a pair of RT servers that don't do anything but console server and BII4Patrol connections.  Although I've increased the max connections to 600 it is no higher than that anywhere in the cloud.  In total there are now 983 patrol agents, 3 console servers, and 12 BII4P adaptors. These are spread over 8 feeding RT servers each in pairs and the 2 top level RT servers.  Many of the patrol agent are on one of the pairs which has 544 agents.

 

The problems I get are two fold.

  • Firstly agents seem to drop off the cloud,  although generally these loss of connections are fairly short in nature and I've learned to put up with them,  but I suspect they are a symptom of something a miss.
  • Second when a server reboots/crashes, and I suspect the patrol agent gets killed rather un-gracefully the console server seems to think the agent is still there for ages.  I've watched for 10 minutes after a server has been rebooted and the patrol agent has remained 'connected' in the console.  Eventualy the cServer shows the agent and disconnected and then reconnects it.

 

I suppose I'm wondering if people think that I have too many agents?   I know that there is a limit of 700 connections for an RT server but I didn't think that this limited how many connections you can have in the cloud?

 

If I don't have too many agents, I suspect that I either have too many RT servers or too few?  I used to have the connection timeouts much lower and this helped with closing connections when a server rebooted but caused more agents to loose connection here and there.  Perhaps with more RT servers I can afford to decrease the timeouts again?

 

I'm afraid the RT servers are a mystery to me, it seems to come down to setting values and trying to figure out over time if that's made any difference.  When I talk to support about the RT servers it seams to me they're doing the same thing.  However the consequences of getting this stuff wrong in the cloud can be hideous.

 

Many thanks.

  • 1. Re: RT clouds. How big, how many, how?
    Jonathan Coop

    Why do you not have separate clouds which console server has supported for several years. Jon

     

    Sent from my iPhone

  • 2. Re: RT clouds. How big, how many, how?
    InfoJester

    I don't much like that idea as you can't have an agent for each cloud in one cServer profile.   It seems restrictive and I'd rather avoid it if I can.

  • 3. Re: RT clouds. How big, how many, how?
    Jonathan Coop

    Not true, you can have multiple clouds in one profile. Always have been able to do this since multiple clouds supported. Jon

     

    Sent from my iPhone

  • 4. Re: RT clouds. How big, how many, how?
    InfoJester

    Oh right!   I don't know where I got that from then.

     

    Sounds like you suspect my problems are down to having too many agents in the cloud then?

  • 5. Re: RT clouds. How big, how many, how?
    Jonathan Coop

    Definitely, depends on whether your doing reporting, personally I'd suggest no more than 300 per rtcloud. Plus a cloud for your console servers. Also no more than two RTservers per cloud, and have one of the pair point to the other but not the other way round as you can get race conditions, that cause stale data to be propagated. Jon

     

    Sent from my iPhone

  • 6. Re: RT clouds. How big, how many, how?
    InfoJester

    How can you have a separate cloud for the console servers,  surely they have to be aware of all the clouds to be able to see the agents?

  • 7. Re: RT clouds. How big, how many, how?
    Oleg Protokolov

    Hi,

     

    You can add these settings (see below) into configuration file of PCS, if you want users to connect to the PCS only through a specific RT-cloud. By another words, it means that the "rtcloud1-for-cservers-bii4p-pco-etc" RT-cloud is allocated only for PCS needs (user consoles, bii4p services, etc...)

     

    instance of Acfg_8_0_0_RTCloudConnection
    {
      cosPath="AgentSetup/communication/rtcloud1-for-cservers-bii4p-pco-etc";
      rtServerLocators="tcp:rt1:2059,tcp:rt2:2059";
    };

    instance of Acfg_8_0_0_RTCloudConnection
    {
      cosPath="AgentSetup/communication/rtcloud2-patrolagents";
      rtServerLocators="tcp:rt3:2059,tcp:rt4:2059";
      serviceName="none"; 
    };

    instance of Acfg_8_0_0_RTCloudConnection
    {
      cosPath="AgentSetup/communication/rtcloud3-patrolagents";
      rtServerLocators="tcp:rt5:2059,tcp:rt6:2059";
      serviceName="none"; 
    };

    instance of Acfg_8_0_0_RTCloudConnection
    {
      cosPath="AgentSetup/communication/rtcloud4-patrolagents";
      rtServerLocators="tcp:rt7:2059,tcp:rt8:2059";
      serviceName="none";
    };

    instance of Acfg_8_0_0_RTCloudConnection
    {
      cosPath="AgentSetup/communication/rtcloud5-patrolagents";
      rtServerLocators="tcp:rt9:2059,tcp:rt10:2059";
      serviceName="none"; 
    };

     

    Now, about your initial post...

    Questions:

     

    "... I've watched for 10 minutes after a server has been rebooted and the patrol agent has remained 'connected' in the console..."

     

    1. Such behavior is found for all RT-clouds or only for the largest (with 500+ PA)?

    2. Such behavior appears in the same way, if you see this PATROL Agent from a different PCS?

     

    --

    Regards,

    Oleg

  • 8. Re: RT clouds. How big, how many, how?
    Jonathan Coop

    You configure them in console server config, full details in manual. Jon

     

    Sent from my iPhone

  • 9. Re: RT clouds. How big, how many, how?
    InfoJester

    Yeah sorry,  I was being dense.   I realize you mean a separate cloud for the users (and BII4P) to connect to the cServer on right?

     

    Then the users only know about one cloud, the cServers know about all the clouds and the agents only know about their own clouds and everyone is happy. 

  • 10. Re: RT clouds. How big, how many, how?
    Jonathan Coop

    Yes. Jon

     

    Sent from my iPhone

  • 11. Re: RT clouds. How big, how many, how?
    InfoJester

    Thanks OProtokolov it's becoming clear now.

     

    1. Such behavior is found for all RT-clouds or only for the largest (with 500+ PA)?

    Well I only really have one big cloud, however they are split into these different sites where the agents only know about the RTservers on those sites and the RTservers there only know about the top RTservers.   I'm pretty sure it is quite a bit worse on the larger sites.

     

    2. Such behavior appears in the same way, if you see this PATROL Agent from a different PCS?

    Yes it happens exactly the same on all the cServers.  Mind all my cServers are connected to the top RTservers so I guess I'd expect that.

  • 12. Re: RT clouds. How big, how many, how?
    InfoJester

    Hi Guys,   I just wanted to say a big thank you for you're help.   I've reconfigured my Patrol 7 environment completely and have split it into 5 RT clouds each of no more than 2 RT servers and up to 500 agents.   I'll work on getting some more RT clouds up and bring the agent count per cloud down.

     

    It's still early days but I can already see that everything works a lot better, it feels much more like it used to with agents reconnecting to the cloud as quick as they should.

     

    Finally may I say that it's great to have had you guys answer my question.  When I see answers from names like yours I know that I'm in good hands.  I've often found your replies on other users posts and they have solved me so many problems.

     

    Thank you!