Azure blob storage job

Version 4
    Share This:

    Short description:

    Version History:

    Changed 10 december 2018:

     

    Change log:

    • Added option Delete container
    • Renamed labels to avoid duplicate label names (this gives better support for jobs-as-code). Label updates:
      • Action List:
        • Container renamed to Container (List):
      • Action Upload and Download:
        • Container renamed to Container (Up/Download)
        • Blob name renamed to Blob name (Up/Download)
      • Action Delete
        • Container renamed to Container (Delete)
        • Blob name renamed to Blob name (Delete)
      • Action Create Container
        • Section label renamed from ====== Specify for action Create a container ====== to ====== Specify for action Create/Delete a container ======
        • Container name renamed to Container (Create/Delete)

     

    Short description:

    Job type to transfer files from and to Azure Blob storage

     

    Detailed description:

    Azure Blob storage is the storage service for unstructured data. This job type supports file transfer and management capabilities. The operations that are available are:

     

    • Download: Downloads a blob into a local file
    • Upload: Uploads a file into a container
    • List: List all blobs (files) in a container
    • Copy: Copies a blob from one container into another
    • Delete: Deletes a blob
    • Create container: Creates a container
    • Delete container: Deletes a container

     

    Usage:

    To use this job type, you would need to create a connection profile with the Azure storage account details:

     

    Once the connection profile is configured, you can define your first job. The first section of fields are general parameters required for every action. Then specify the fields in the applicable section for the action. E.g. action list, you would need to specify the Container field in the ====== Specify for list action ======.

     

    Below table holds a description for each field

     

    ====== General parameters  ======

    • Action: Choose the action to perform (see above)
    • Output: Choose your desired output. If you need to process the output in an automated way, json might be your best option. If you just use the job output for monitoring purposes, table might give you the best human readable format.
    • Additional parameters: For some advanced scenarios, you might want to pass an extra parameter to the azure cli. For all other scenarios you can leave this field empty.

     

    ====== Specify for list action ======.

    • Container (List): Select the container of which you want to list the content

     

    ====== Specify for actions upload or download ======

    • Container (Up/Download): Select the container where the blob need to be uploaded to or downloaded from
    • Blob name (Up/Download): The name of the blob. For download, you can select an existing blob via the load button. For upload you can just specify the name
    • File path: This is the path to the file on the agent's local file system. E.g. /home/ctmag/1GB.zip

     

    ====== Specify for action delete ======

    • Container (Delete): Select the container where the blob need to be deleted from
    • Blob name (Delete): The name of the blob to be deleted.

     

    ====== Specify for action copy======

    • Source container: Select the source container
    • Source blob name: Select the source blob name
    • Source uri: Select the source uri. The load button will populate this based on the source container and source blob fields
    • Destination container: Select the destination container
    • Destination blob name: The name of the blob to be deleted

     

    ====== Specify for action Create container ======

    • Container (Create/Delete): The name of the new container
    • Public Access: Determines if the container is accessible for anonymous users or not. The default value is Off. Possible values are:
      • Container: Full public read access. All container and blob data can be read by anonymous request. Clients can enumerate blobs within the container by anonymous request, but cannot enumerate containers within the storage account.
      • Blob: Public read access for blobs only: Blobs within the container can be read by anonymous request, but container data is not available. Anonymous clients cannot enumerate the blobs within the container.
      • Off: No public read access. The container and its blobs can be accessed only by the storage account owner. This is the default for all new containers.

     

         Please check the actual Azure documentation on the behavior of these values.

     

    Prerequisites and installation notes:

    This job type has the following prerequisites:

     

    • Control-M Agent with the Application Integrator CM
    • Azure CLI v2.0 installed
    • Azure CLI authenticated with the “az login” command

     

    Note: This job types supports multi-factor authentication by using Azure's az login command as prerequisites. This job type can be modified to include the azure authentication parameters as part of the connection profile in scenarios where a 2 factor authentication is used. This would eliminate the prerequisites to authenticate the Azure CLI.

     

    Installation steps:

    - Install the Azure CLI on the agent where you want to run the jobs on.

    - Authenticate the CLI using az login

    - Deploy the job type to the agent. Make sure the Application Integrator is installed on that agent.

    - Configure a connection profile. You can create and receive the storage account details trough the Azure portal.

    - Create your first job

     

    Compatibility:

    • Application version: This job type is tested against Azure CLI v2.0
    • Platforms: This job type follows the Azure CLI compatibility and was tested on both Linux and Windows.
    • Control-M version: Tested on Control V9 and V9.0.18 and V9.0.18 FP2
    • Application Integrator version: Tested with Application Integrator 8 and 9