Howto: RabbitMQ Shovel configuration in Bladelogic

Version 1

    So we have custom shovel configurations for our rabbit servers, here's a quick and "dirty" way (might be a cleaner way, but this was the cleanest I found) to do a custom rabbit configuration.  Note, I'd love some feedback to see what people think.


    FIRST.... we have Server properties that we pull from so each server can define it's settings for things like the hosts.  Then we have a set of Server properties that are specific to various data center sites, i.e. "Site X" gets these settings, Site Y gets a slightly different set of the same settings.  This is all done by creating separate instances of our configuration settings, then setting those settings onto the Server object.


    So the commands:
    BLPackage Depot object,

    1) custom command to turn on the plugin:

         rabbitmq-plugins enable rabbitmq_shovel.....

    2) upload a and config.template.erl file.  See end of post for source.

    3) Create a custom command to sh /var/tmp/rabbitmq/ "??SHOVELS??" ??TARGET.RABBIT.RABBIT_ENTERPRISE_HOST?? ??........ etc.

    -- SHOVELS is a local property, the TARGET.RABBIT... properties are all set on the server configuration.


    Then you can dynamically call this script, pass as many shovels to the file as you want, and it'll build the shovel configuration, all based entirely off of server properties.  We do this since each server can have completely different configuration and the config files are pretty complex to generate, without a simple grammer syntax, and it's a lot easier to modify a template file like below than try and get grammar scripts to work.  Obviously this is just a start and designed for Linux systems.



    Further, you can create these files and upload them as depot objects so they're modifiable and viewable in Bladelogic. I'm getting more and more to the point of using such a methodology for building configuration objects, as it seems the built in configuration dictionary is too... simple?  Not flexible enough?  to be really usable.  Further, you can do the above without having to have an initial rabbit system to be built first.  Last, you can do IF statements and more complicated statements in such a script using reusable open source components that are no longer bladelogic specific.


    Next, you can use this to conceptually build other configuration files, like tomcat.








    for RABBIT_QUEUE_NAME in $1 ; do










    ' /var/tmp/rabbitmq/single_shovel.template.erl





    echo "[{rabbitmq_shovel,

              [{shovels, [





    ------------------------------------------------- template.erl

         { ${RABBIT_QUEUE_NAME}, [
                {sources,      [{broker, "amqp://${RABBIT_LOCAL_ADMIN}:${RABBIT_LOCAL_PASSWORD}@"}] }
              , {queue, <<"${RABBIT_QUEUE_NAME}">>}
              , {prefetch_count, 0}
              , {publish_properties, [ 
                   {delivery_mode, 2}
                   , {headers, [{<<"source">>,longstr,<<"${HOSTNAME}">>}] } 
              , {reconnect_delay, 5}
         ] }


    you might want to create a 'document' out of this post since it's informational vs asking a question...


    This document was generated from the following discussion: Howto: RabbitMQ Shovel configuration in Bladelogic