Periodic and Substantial Increase in the Time Required for File Output

[DieselPower]

I am logging data using continuous asynchronous acquisition and writing this data (~100 columns) to a file in ASCII format at 1Hz (10Hz will eventally be needed).  I believe that the harddrive in this PXI RT system is a 4GB solid state drive.  In my loop I am logging the number of microseconds required to carry out certain operations, and file output shows some odd behavior.  A (not perfectly) periodic occurence is happening where the write takes almost 100ms when a normal write is about 2-4ms.  What could be causing this issue and how can I remedy it?  NOTE: A chart is attached.

The line of code is just a fputs(msg, DataFile);

[Danceman]

Hello,

   Looking at that graph it seems like every 100 seconds or so there is some lag in recording the data.  With 4 GB of Solid State storage the only thing I can think causing this bottleneck is something in either the programming or O/S.  Therefore I have some questions for you.
     Where are you marking the time between portion of the code?  I would try the following timing measurement to see where the delay is.
      For (i = 0; i < iterations; i++)  

         Tic();  // Start Timer
         //   Gather the data
         Gatherdata_Time[i] = Toc();  // Get time since Tic() 

         // Write Data to String format
         ProcessingString[i] = Toc();  // Get time since Tic() 

         // Write data to HDD
         Writetodisk_Time[i] = Toc();  // Get time since Tic()  
      End For loop 

Daniel de Gaston 

[DieselPower]

My devbox won't boot right now, but I can tell you that I have 4 timers using the GetMicroseconds (psuedonym) function.  I get the us JUST before and JUST after the file write call.  I also get the entire loop time, the time to build the string, and the time to send the string over TCP (though this time is not correct because TCP is threaded)

[Danceman]

Is the graph I see in your first post the micro-seconds for just the write call?   How much data are you writing?

[DieselPower]

Yes. About 600 characters.

[Danceman]

Could you try using the following lines of code.

      fprintf(File_ID, "%s" , msg);

      Determine timing.

      printf("%s" , msg);

      Determine timing. 

The point of this test is we can then determine if the following delay is found just with writing to the file or hard disk drive.   If the long delay is only found on the first fprintf then I would say we look into hard drive issues.  If it is found on both print statements,  I think the Operating system or system resources are being tied up by something else at the moment.

[DieselPower]

Will printf work in real time systems?

[DieselPower]

See the file below.  I'll run it longer, but I was just able to bounce out of an all day training event for lunch.

[DieselPower]

The columns for time are at the end and they are | Build Msg Time | FileIO Timer | PrintF Timer | 1Hz Loop Time

I used my method of getting the time GetTimeUS.  I hope that was OK.

[Danceman]

Hello,

    I feel like we should use the Real-Time execution trace in determining what is taking the most time.  Whether this is a speed issue with cache data or data transfer the following links should help us determine this.

                https://forums.ni.com/t5/SignalExpress/Flush-log-buffer-to-disk-while-running/m-p/701887 

Daniel