10 Replies Latest reply on Sep 18, 2019 12:03 PM by Bill Robinson

    blcli get only returns 1000 results

    Sae Park
      Share This:

      I'm trying to get a list of components in a group.

      Using bquery, I was able to get all the components using a browser, using rest.

      However, using blcli get command, I only get 1000 results.

       

      I thought I saw a post about this, but I can't seem to find it.

      Is there a limitation on how many results I can get back with blcli get command?

      I am using maxResults=2000 in the command, but no luck.

       

      Thanks!

        • 1. Re: blcli get only returns 1000 results
          Bill Robinson

          support is probably going to send this too, but iirc you need to loop.

           

          so like:

          START=0

          BATCH=1000

          While returnedRows –lt ${BATCH}

          blcli_execute get "/query?firstResult=$START&maxresults=$BATCH&BQUERY=SELECT name, FROM \"SystemObject/Server\""

          START=$START+$BATCH+1

          blcli_storeenv returnedRows

          # do something w/ the returnedRows value

          # count how many rows returned and exit the loop when you don't get 1000 rows back

          Done

           

          the maxResults does work - you can limit to something < 1000.

          1 of 1 people found this helpful
          • 3. Re: blcli get only returns 1000 results
            Greinger Longbotton

            This is the code that worked for me:

             

            #!/bin/nsh


            FILE_FINAL="/tmp/stage/SERVERS.xml"

            FILE_TEMP="/tmp/stage/SERVERS_TEMP.xml"

            RESULT=0

            START=0

            BATCH=900

            touch $FILE_FINAL

            touch $FILE_TEMP


            while [ $RESULT -eq 0 ]; do

                echo $START

                blcli get "/query?firstResult=$START&maxresults=$BATCH&BQUERY=SELECT NAME FROM \"SystemObject/Server/\""  > $FILE_TEMP

                 #The XML has the attribute "value" if the BLQUERY retrieved something

                cat $FILE_TEMP | grep -i "value" > /dev/null

                RESULT=$?

                if [ $RESULT -eq 0 ]; then

                    cat $FILE_TEMP >> $FILE_FINAL

                    START=$((START + BATCH + 1))

                fi

            done


            rm $FILE_TEMP

            2 of 2 people found this helpful
            • 4. Re: blcli get only returns 1000 results
              Bill Robinson

              this:

                  cat $FILE_TEMP | grep -i "value" > /dev/null

                  RESULT=$?

                  if [ $RESULT -eq 0 ]; then

                      cat $FILE_TEMP >> $FILE_FINAL

                      START=$((START + BATCH + 1))

              fi

              can be re-written as:

               

              if ( grep -q -i "value" "${FILE_TEMP}" )

              then

                cp "${FILE_TEMP}" "${FILE_FINAL}"

                START=$((START + BATCH + 1 ))

              fi

              because cat is useless

              • 5. Re: blcli get only returns 1000 results
                Greinger Longbotton

                If we apply the changes you proposed, some more changes must be done.

                1. We are using the variable $RESULT in the main "while", so we cannot get rid of it.
                2. We want to concat the $FILE_TEMP content into $FILE_FINAL, that is why I use "cat >>"
                • 6. Re: blcli get only returns 1000 results
                  German Coll

                  Hello B. Robinson,

                   

                  I don't understand this:

                  BATCH=1000

                  While returnedRows –lt ${BATCH}

                       blcli_execute get "/query?firstResult=$START&maxresults=$BATCH&BQUERY=SELECT name, FROM \"SystemObject/Server\""

                  ...

                       blcli_storeenv returnedRows

                  the var $returnedRows does not have a numeric value but an XML, so how do you compare it with $BATCH which has a numeric value?

                   

                  Thanks

                  1 of 1 people found this helpful
                  • 7. Re: blcli get only returns 1000 results
                    Matthew Ragland

                    maxresults doesn't need to be set, it will automatically fetch the max results each time as you increase the firstResult value. Our appservers are running Windows, and therefore I am able to use Powershell to parse the XML output to a file, without further ado, this is what I do:

                    ROWNUM=0
                    RETURNEDROWS=1000
                    
                    until [ "${RETURNEDROWS}" -lt 1000 ]; do
                         blcli_execute get "/query?firstResult=${ROWNUM}&BQUERY=SELECT Name,IP_ADDRESS,OS FROM \"SystemObject/Server\"" > NUL
                         blcli_storeenv BQUERY
                    
                         SERVERDATA=$(echo "${BQUERY}" | powershell -c "([xml]\$input).RESTXMLResponse.PropertySetClassChildrenResponse.PropertySetClassChildren.PropertySetInstances.PropertySetInstance | Select-Object \
                         @{N='NAME';E={\$_.name}},\
                         @{N='IP_ADDRESS';E={\$_.PropertyValues.PropertyValue.value[1]}},\
                         @{N='OS';E={\$_.PropertyValues.PropertyValue.value[2]}} | ConvertTo-Csv -NoTypeInformation | Select -skip 1" | tee -a /C/Temp/BSAData.csv)
                    
                         RETURNEDROWS=$(echo "${SERVERDATA}" | wc -l)
                         (( ROWNUM = ${ROWNUM} + ${RETURNEDROWS} ))
                    
                    done
                    
                    2 of 2 people found this helpful
                    • 8. Re: blcli get only returns 1000 results
                      Greinger Longbotton

                      Our appservers are running on RHEL, so I cannot use powershell but I love your idea

                      • 9. Re: blcli get only returns 1000 results
                        Matthew Ragland

                        You could use the same method, but use xmllint instead to do the xml parsing

                        2 of 3 people found this helpful
                        • 10. Re: blcli get only returns 1000 results
                          Bill Robinson

                          i'm not sure - it was two years ago.  probably a bad copy/paste.