1 2 Previous Next 17 Replies Latest reply on Oct 5, 2018 10:53 AM by Jason Miller

    Server Table Field Chunk Size

    Edison Pioneer

      Hi folks

       

      The Server Table Field Chunk Size is set 2000 in production and we would like to increase that to 5000.

       

      When I had run a query using company field it failed to return all expected results.

       

      When I changed to 5000 in QA and then re-ran the search, I was able to get the expected results.

       

      We have around 4k companies and need to get them all in the visibility group form.

       

      Would you cause memory performance issue?

       

      Any feedback would be highly appreciated.

       

      Thanks in advance

        • 1. Re: Server Table Field Chunk Size
          LJ LongWing

          Edison,

          The higher the value, the more memory will be used in a single chunk, so yes, potentially a memory issue, but if you do valid testing, you should be good.

          • 2. Re: Server Table Field Chunk Size
            Edison Pioneer

            Hi LJ,

             

            In our case, since we have 4000 companies, should we set it to 4000 or 5000? What would be better in your opinion?

            We have 5 app servers and 5 MT servers and 24 gigs of RAM distributed across all 10 servers.

             

            Thanks.

             

             

            • 3. Re: Server Table Field Chunk Size
              LJ LongWing

              I think 5000 should be an ok value....I've used values as high 100k in here for various reasons....5k isn't likely to crash your server.

              1 of 1 people found this helpful
              • 4. Re: Server Table Field Chunk Size
                Edison Pioneer

                Hi LJ,

                 

                Thanks a lot for responding.

                 

                May I ask what was your environment configuration?

                 

                We have around 150 minimum daily users on an average, lets consider all of them login simultaneously and work, and a few of them love running complex searches or pull out heavy reports. We have 10 servers - 5 MT and 5 AR  and 24GB RAM among them.

                 

                Whats your opinion?

                 

                Thanks in advance

                • 5. Re: Server Table Field Chunk Size
                  LJ LongWing

                  it's not how many users are connected, it's how many users are doing things via filters that use the chunk size.  my configuration was two servers (8.1 realm) and about 150 concurrent users....I think you'll be ok, but you'll want to watch your memory usage before you make the change (jvisualvm) and after, ensure that your heap size isn't growing too large.

                  1 of 1 people found this helpful
                  • 6. Re: Server Table Field Chunk Size
                    Edison Pioneer

                    Million thanks LJ

                    The JVisualVM suggestion is way too awesome.

                    • 7. Re: Server Table Field Chunk Size
                      Edison Pioneer

                      Hi LJ,

                       

                      Apologies for asking you this question so late after you gave me the original idea, but the lightning struck just now.

                       

                      I need to ask you, if the end purpose is to monitor the memory performance, why should I go for JVisualVM?

                      I can just see the memory performance using Task Manager.

                       

                      My point here is, what extra benefits will I be reaping if I decide to use JVisualVM?

                       

                      Also, how do I set up JVisualVM such that an email goes to administrator when memory usage is perilously high?

                       

                      Thanks in advance

                      • 8. Re: Server Table Field Chunk Size
                        LJ LongWing

                        Edison,

                        You didn't specify the version of Remedy that you are using, and I assumed 9.x...if I'm wrong...you can just use Task Managager...but if you are using any iteration of 9.x, Remedy is now a Java process, which means that the memory usage of the Java process is defined by the min/max memory in the config files, but doesn't properly represent the actual usage of Remedy inside the JVM...to see that you must attach to the JVM with jvisualvm to look at the heap usage INSIDE the VM.....relating to how to send email when it reaches high value....I'm not sure.

                        1 of 1 people found this helpful
                        • 9. Re: Server Table Field Chunk Size
                          Jason Miller

                          There is a lot of processing going on inside the JVM. A JVM is a "Java Virtual Machine". The concept is kind of similar to VMware/VirtualBox/MS Hyper-V virtual machines (I will just use VMware / Windows as an example from now on for simplicity). A Virtual Machine (VM), no matter if it is a full host/guest setup like VMware or a "J"ava "VM", the VM handles all the specifics of how the environment is operating inside. In the VMware VM example you would not use the host system's Task Manager to manage all of the guest VM OS's running processes and memory usage. The host hands out a chunk of memory and each guest manages that chunk to perform tasks (run processes). This is the same with a Java VM. The OS (your Windows system) gives the JVM in which AR System is running a chunk of memory and from there the JVM handles its internal operations.

                           

                          Typically you set a max/min memory allocation for your JVM and you will see this memory being used in the host system's Task Manager but in reality very little of that memory may be used by the JVM. An example is you may setup your AR System's JVM with a min/max of 16 GB of memory. When your JVM starts that 16 GB is used right away (more or less) but within that JVM it may be only using 2 GB when the system first starts. Over time AR Server (and plugins) will use more of the memory within the JVM but the host OS is (roughly) staying at 16 GB used for the JVM (don't forgot other Windows process need memory too so don't use all of the host system's memory for your JVM).

                           

                          A situation where issues are often seen is the JVM needs more memory than the host system can/will give. This is where you start seeing heap errors, etc. From the example above say your min/max is set to only give the JVM 2 GB, AR System is going to use all most all of that when it starts (this is just an example, every system is different). The system will run for a little while but if it is a busy system with 100's of users or integrations all of those processes are going to start to need more than 2 GB and you will likely end up with a heap error.

                           

                          JVisualVM is like Task Manager for your JVM. After awhile you get familiar with how a system's JVM patterns look and you can tell just by seeing a pattern change that there is an issue (or different process load than normal). For example I can tell by watching a Tomcat instance (with Mid Tier) start I know if MT pre-load kicked off and completed. There have been times our server team has rebooted our systems out of order and pre-load did not complete. MT was usable but I knew by looking at the JVM heap usage I our users were going to have MT cache corruption issues if I did not address the problem.

                           

                          Here is some info on JVM memory (I am sure there are a ton other pages with info too)

                          https://betsol.com/2017/06/java-memory-management-for-java-virtual-machine-jvm/

                          4 of 4 people found this helpful
                          • 10. Re: Server Table Field Chunk Size
                            Edison Pioneer

                            Hi Jason,

                             

                            Cannot thank you enough for writing such a painstakingly elaborate explanation.

                            The concept went right into my pea sized brain without any hassles.

                             

                            Before coming to your answer , I had Googled "VisualVM vs Task Manager" and had tried reading up on a few sites. One of them was this - Garbage Collection in Java (Part 1) - DZone Performance  and List of performance analysis tools - Wikipedia . So, there's heap memory, non heap memory and some native memory.

                             

                            Thanks again

                            • 11. Re: Server Table Field Chunk Size
                              Edison Pioneer

                              Many thanks, LJ.

                               

                              You have patiently answered all my nagging queries

                               

                              Merci beaucoup

                              • 12. Re: Server Table Field Chunk Size
                                Jason Miller

                                I am happy I was able to help

                                 

                                That page about CG is good and definitely something to be aware of but there are more basic concepts around a JVM that need to be understood before diving into GC.

                                • 13. Re: Server Table Field Chunk Size
                                  Edison Pioneer

                                  Hi LJ,

                                   

                                  Here I come with another question

                                  I was discussing about this with one of my colleagues when he commented that setting server table chunk size to 5000 may cause perilous memory usage since there are four default server users (I think he was talking about aruser, arescalator, remedy app service et al) which also perform searches at server side, and they may end up eating a lot of memory.

                                   

                                  How true is that?

                                   

                                  Do I need to worry about those server side users?

                                   

                                  Thanks

                                  • 14. Re: Server Table Field Chunk Size
                                    Edison Pioneer

                                    Hi Jason

                                     

                                    Never have I explored Java concepts as much as I have done working on Remedy. Even in university I just did some superficial studies.

                                     

                                    I have one more question which I have asked LJ above. Would like to hear your opinion as well.

                                     

                                    Thanks. Regards

                                    1 2 Previous Next