fwrite function

[suni]

Hi,

In my application in a thread,i am processing data and writing to a file with a certain format. In order to craete the format i am using fwrite function 5 times. i am writing a structure and few variables etc.... uisng the fwrite. i am wondering if this method is efficeint or if there is more effieciant method? any pinters on this will be greatly appreciated.

[RobertoBozzolo]

If you are in search for efficiency, generally speaking writing formatted text to ASCII files is not the best you can use. Writing unformatted data to  binary files is generally faster but you will need to code the appropriate function to read back data from the file and translate it in human-readable form.

Since your data is already organized in a struct, you can write down the whole struct in a single pass by using this code:

typedef struct {
     // field definition
} myStruct;


int SaveFunction (myStruct x)
{
  FILE	*fH = NULL;

  errno = 0;
  fH = fopen (fsts, "rb+");
  if (errno) goto Error;

  fwrite ((char *)&x, sizeof(myStruct), 1, fH);
  if (errno) goto Error;

Error:
  fclose (fH);
  return (errno > 0);
}

A symmetric function with the appropriate fread can be used to read back data from the file.

[OVR_CZ]

hi,

    i also recommend to reduce number of fwrite call.Prepare data to memory buffer and then write it at once(or when memory buffer is full).

    Even for text data, you can get realy big performance boost if you write one megabyte at once,instead of 10000 small writes.

   Alternatively you can use ansi function setvbuf (FILE *stream, buffer,bufferingMode,bufferSize).But from my tests, using your own memory buffer improve speed better.

[suni]

hi,

Thanks for your reply.

By setting my own memory buffer you mean using malloc function and writing to the buffer using memset function or some thing like that and then to use fwrite finally. Am i correct?

if i use this setvbuf (FILE *stream, buffer,bufferingMode,bufferSize) then i will use this fuction in the loop multiple times. after i am done writing to the buffer how do i flush this buffer to the file?

Thanks for all your help.

[OVR_CZ]

function setvbuf  is called once after file is open, this simply change buffer size for file handle. There is no need to do anything special with this buffer, just use fwrite,fread,fclose like before. (buffer is typically flushed when it is full, when file is closed, or eventualy when fflush function is called)

for example setvbuf(fo,0,_IOFBF,1024*1024);

If your own buffer is small it can be array like  char buffer[10*1024];  but for bigger one it is better to alocate it dynamicaly.

there is litle sample what i mean for writebuffer

//this is just for ilustration, i do not test this code.

//structure to hold information about buffer
typedef struct{
  FILE *fo;    //output file
  char *buffer;//memory buffer pointer
  int   buffer_maxlen;//size of buffer
  int   buffer_len;//number of used characters
}TOutBuff;

TOutBuff OutBuff;

//file and buffer initialization
OutBuff.buffer_maxlen=1024*1024;
OutBuff.buffer=malloc(OutBuff.buffer_maxlen);
if (OutBuff.buffer==0)return -1;//error
OutBuff.buffer_len=0
OutBuff.fo=fopen(...);
if (OutBuff.fo==0)return-1;//error

//buffer usage (see below for BuferWrite function)
char newdata[32]="sometextordatatowrite";
r=BuferWrite(&OutBuff, newdata,strlen(newdata));//instead of fwrite, we use buffered write
...
r=BuferWrite(&OutBuff, newdata,strlen(newdata));//instead of fwrite, we use buffered write

//flush data in buffer to file
if (OutBuff.buffer_len>0)//if there is any data left
{
 fwrite(OutBuff.buffer,1,OutBuff.OutBuff.buffer_len,OutBuff.fo);
 OutBuff.buffer_len=0;
}
//free memory and close file
free(OutBuff.buffer);OutBuff.buffer=0;
fclose(OutBuff.fo);OutBuff.fo=0;




//write function
int BuferWrite(TOutBuff *OutBuff, char *newdata,int newdatalen)
{
 if (newdatalen>OutBuff->buffer_maxlen)
 {
   //buffer is too small for this newdatalen
   //(instead of error you can upgrade code to first flush buffer and then write data directly
   return -1;
 }
 if (newdatalen+OutBuff.buffer_len>OutBuff->buffer_maxlen)//check if there is enought space for data in buffer
 {
   //if there is no space flush buffer to file
   if (fwrite(OutBuff.buffer,1,OutBuff.OutBuff.buffer_len,OutBuff.fo)!=OutBuff.OutBuff.buffer_len)return -1;//error
   OutBuff.buffer_len=0;
 }
 //add data to buffer
 memcpy(&OutBuff.buffer[OutBuff.buffer_len],newdata,newdatalen);
 OutBuff.buffer_len+=newdatalen;
}

note:I recommend to not mix both approach(own buffer and setvbuf). From my measures it does not improve speed, only memory consumption is bigger.