1 2 Previous Next 16 Replies Latest reply: Aug 21, 2008 2:38 AM by Alfi RSS

PemnBPutFile problem

Geert De Peuter

Original email:

 

Hi,
It seems there is a problem with PemnBPutFile. There seems to be a limitation of 8k and it doesn't seem to work for binary files.

 

Any workarounds available ?

 

Reply:

 

I have to warn you when sending large files !! (and you should warn your customers as well, if you will expose this feature)

 

PemnBPutFile will send files using the PEMAPI.  That means that the file fragments will be handled by the same mainloop as events or other things in PATROL. So you should prevent sending big files over the line or you should understand the consequence ! As a result you may see the agent's CPU go high, or the agent might even disconnect !

 

Anyway, that's not an excuse... so I produced this workaround.

 

Cheers,
-- Geert

 

#include <stdio.h>
#include <string.h>
#include <pemapi.h>
#include <malloc.h>
#include <ctype.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>

 

#define STREAM_MAX    8096

 

int g_timeout=2000;

 

char *
FileSend(PemnCommHandle commhandle,char *localfilename, char *remotefilename)
{
    FILE*     fp;
    struct stat buf;
    int iStreamSize;
    char *pcStreamData;
    int i,n;
    int TransferOK=1;
    char *ret;
         
    if ( (fp= fopen(localfilename, "rb")) == NULL )
        return ("ERROR: cannot open source file\n");
 
    if (fstat (fileno(fp), &buf) != 0)
    {
        fclose(fp);
        return("ERROR : cannot obtain file stat on sourcefile\n");
    }
 
    iStreamSize=(int) buf.st_size;
 
    if ( iStreamSize < STREAM_MAX){
        pcStreamData= (char*)malloc(iStreamSize);
    } else {
        pcStreamData= (char*)malloc(STREAM_MAX);
    }
         
    n =  (iStreamSize/STREAM_MAX) +1;
         
    i=1;
    while ((i <= n) && TransferOK) {
        size_t iSize = fread(pcStreamData, 1, STREAM_MAX, fp);
        ret=PemnBPutStream(
            commhandle,
            iSize,
            pcStreamData,
            NULL,
            NULL,
            0,
            remotefilename,
            (i==1)?PEMN_WRITE:PEMN_APPEND,
            "",
            "511",    /*permissions (decimal !!)*/
            g_timeout
            );
             
        TransferOK=(strcmp(ret,"OK") == 0) ? 1 : 0;
     
        i++;
    }
    free(pcStreamData);
    fclose (fp);

 

    if (TransferOK)
        return "OK";
    else
        return "ERROR : Error in filetransfer\n";
}

  • 1. Re:
    patrick

    Hi Geert,  Thank you for producing this code.

    One question though.

    Should the line:   TransferOK=(strcmp(ret,"OK") == 0) ? 1 : 0;

    not read:  TransferOK=(strcmp(ret,PEMN_OK) == 0) ? 1 : 0; 

     

    Otherwise the strcmp seems to always fail, as ret never returns the string OK.  I think if it is evaluated this way, what is actually returned is:  OK  Am I understanding this correctly?

  • 2.
    Geert De Peuter

    Patrick,

    What you say sounds right. However, it used to work.
    Thanks for making the suggestions to make it work on current releases

    -- Geert

  • 3.
    svanmaren

    I came across those limitations with PemnPutFile too. I ended up using remote_file_send(). It required a Patrol agent on the source end, but the remote_file_send() appears to not have that tight file size limitation (I sent some pretty huge files) and I had success sending binaries too.

  • 4. RE: PemnBPutFile problem
    harrisop

    Hi

    Thanks for the code snippet Gert. the file size would have caught me out later on.

    The code you sent works fine when I'm sending a text file from windows to windows but I get a ^M character at the end of each line when sending files to unix boxes.

    Is there a solution to this?

    Regards
    Pete

  • 5. RE: PemnBPutFile problem
    Vladimir Tsichevski

    "... when I'm sending a text file from windows to windows but I get a ^M character at the end of each line when sending files to unix boxes.
    "

    Files do not change while they are being transferred. You never see any ^M characters on Windows because  on Windows (aka MS-DOS, aka CP/M) that characters are part of the "end-of-line" sequence, and no windows text tool ever displays them.

  • 6. Re: RE: PemnBPutFile problem
    Geert De Peuter

    Probably the easiest solution for you is to replace

     

    if ( (fp= fopen(localfilename, "rb")) == NULL )

     

    with

     

    if ( (fp= fopen(localfilename, "rt")) == NULL )

     

    Of course this will only work if you are transferring text files. Like mentioned before, with a "b" the file will not change (but probably you want it to change). See manpages on fopen() for more information.

     

    Hope this helps,

    Geert

  • 7. RE: PemnBPutFile problem
    harrisop

    It all makes sense now!

    Looks like I have a few options. Either send the file as text and lose the ability to send binary files (something I may need to do in the future). Try to find out if the source file is text or binary before opening with the apropriate parameter or using the "dos2unix" tool on the unix boxes.

    Thanks both for the information and suggestions.

    Pete

  • 8. RE: PemnBPutFile problem
    harrisop

    I've gone for the option of finding out if the file to send is binary or text. 

    The only way I could find out the file type in windows is by reading through the file and looking for null strings.

    Anyway here's my code for information and comment.

    // Test to see if the file is binary or text #######
    if ( (fp= fopen(localfilename, "rb")) == NULL ){
    LogError("FileSend: cannot open source file","ERROR");
    return (false);
    }

    fseek(fp,0,SEEK_END);


    iStreamSize=(int) ftell(fp);
    rewind(fp);


        pcStreamData= (char)malloc(iStreamSize); <br />   <br />     fread(pcStreamData, 1, iStreamSize, fp);<br /><br />     for (i=0;i&lt;=iStreamSize;i+){<br />          if (pcStreamData==0){<br />               AmountOfNulls++;<br />          }<br />     }<br /><br />     free(pcStreamData);<br />   fclose (fp); <br />   if ( ((float)((float)AmountOfNulls/(float)iStreamSize))100 &gt;2.5 ) {
    strcpy(OpenType,"rb");
    }else{
    strcpy(OpenType,"rt");
    }
       // ##################################################

  • 9. RE: PemnBPutFile problem
    harrisop

    One last question on this (I promise).

    I'm trying to create a function to retrieve a file in the same way as this "FileSend" function does.

    Do you have any examples of this as I'm having some difficulty creating the function.

    Regards
    Pete

  • 10. RE: PemnBPutFile problem
    LynneLawrence

    Does anyone know if the 8K limit on PemnBPutFile still exists in the newer Patrol agents (v3.6)?  The original post on this topic is old and I thought I had read somewhere that this was no longer an issue.

    Anyone know?

    Thanks,

    Lynne Lawrence

  • 11. RE: PemnBPutFile problem
    Vladimir Tsichevski

    "Does anyone know if the 8K limit on PemnBPutFile still exists in the newer Patrol agents (v3.6)?  The original post on this topic is old and I thought I had read somewhere that this was no longer an issue.

    Anyone know?

    Thanks,

    Lynne Lawrence"

    No the maximum size of data which can be transferred in one packet is limited by PEMAPI, which uses statically pre-allocated buffers. Agents have no size limits AFAIK.

  • 12. RE: PemnBPutFile problem
    harrisop

    Hi, (been on holiday and just seen this reply).

    Are you saying that I no longer need the function kindly shared by Geert?

    That I can now use PemnBPutFile instead of PemnBPutStream and the supporting code?

    Pete

  • 13. RE: PemnBPutFile problem
    Vladimir Tsichevski

    "Hi, (been on holiday and just seen this reply).

    Are you saying that I no longer need the function kindly shared by Geert?

    That I can now use PemnBPutFile instead of PemnBPutStream and the supporting code?

    Pete"

    No, I am saying that the problem is on the client, not on the agent side.

    PEMAPI uses statically allocated fixed size buffers for all functions, so no matter which PEMAPI function will you use (PemnBPutFile or PemnBPutStream), the 8K limit will remain.

  • 14. RE: PemnBPutFile problem
    harrisop

    Hi,

    I still can't get my head around trying to bring a file back using PemnBGetStream and iterating round until I have it all. I've tried to turn Geerts code round to do this but I can't figure out all the parameters that PemnBGetStream requires and it seems to be missing in the documentation.

    Any help or a code snippet to give me a leg up would be great.

    Pete

1 2 Previous Next