real-time text file read out

[swiftmoun]

Hi,

I'm trying to figure out if it is possible to read out (pull out) data from a text file that is constantly being updated.

Basically this text is a temperature log that is written by a computer. This computer updates this text file every 2 seconds with a new set of temperature measurements.

I want my VI to pull constantly pull out the last set of mesurements (it is the last line of the text file) so it can be used to calculate other parameters using other measurements coming out of my DAQ instruments.

Any idea how I could achieve this?

Thanks

[BrentSchenk]

can you just pull out the first time, do what you need to do, then keep reading constantly until the two are not equal to each other and once that occurs do what you need to do then start reading the file again until they are not equal.  the only time this wouldnt work is when it takes more than 2 seconds to do what you need to do and read the file in, in which case your program will get behind.

otherwise, if you have any control over the other program i'd tell it to send a message somehow to your labview program (via TCP or something) that the file was updated start performing operations. 

[Ray.R]

I would recommend keeping things simple.

From what I understood, you really don't need to read the file again to update the latest data.

Your program already has access to the data since it is writing it to file.  So why not use the same data (memory) that is being written and that is being analyzed.  This may require a change in architecture, but it will be a lot more efficient code.

An easy way to avoid changing the architectureand have access to the data without having to read it back is to use an ActionEngine (or more to the point, a Functional Global Variable).  You would write the latest data to the ActionEngine.  You could even set a flag that says new data is available.  After reading the data to process it, you can reset the flag and clear the contents.

My preferred architecture would be something like a producer consumer, where the consumer loop would both process the data and write to file.  A queue could be used to store new data every 2 seconds.   Easy to do, clean & efficient. 🙂

EDIT PORTION:

Oops..  I think I misread...  Another program is writing data to a file every 2 seconds...  Is that correct?

Humm...  Well... if that is the case, you could have a loop monitoring for a file size change and triggering an event or changing a state (ie TRUE for a Case Statement).  The loop would also keep track of the last location where you read your file and store it in a Shift Register.  When new data is available from the file, your program would read the file starting from the last offset; thus, reading only the new data that was written.  The data would then be processed..

Message Edited by Ray.R on 09-24-2009 03:49 PM

[swiftmoun]

Thanks for the answers.

Yes, it is a separate program that does write data into a text file.

I can't change this program, it's very limitating but that's all I can use.

I wouldn't mind my VI to loop on reading the last line of that TXT file. The 2-second timer is arbitrary, I could change it to 1s or 5s if I wanted to.

To put things in a simpler way, let's say this program is basically acting as a separate DAQ and writing data into this TXT file. What is of interest to me is to compute this data with the rest of the measurments my NI DAQ is reading. I usually do my readings on the DAQ every 1 or 2 seconds.

I could simplify this to the following example:

what my TXT file writes is:

2009/09/24,09:02:04,24.05

2009/09/24,09:02:06,24.12

2009/09/24,09:02:08,24.23

2009/09/24,09:02:10,24.31

2009/09/24,09:02:12,24.33

The fifth (last) line is the last measurement this separate program has taken.

My existing VI is reading a NI9213 (thermocouple in) every 1 second:

I basically want my VI to pull the "24.33" (or the last data written to file) from the TXT and do the usual DAQ job that pulls temperatures from my 9213. Then with those 2 numbers available I can do whatever I need with them (calculate a temperature delta for example).

this really boils down to reading the last line of a TXT file, pulling one or more numbers and then using those numbers to calculate parameters. when it's done, it starts all over again.

[Rob_F1]

I thought there was a function to pull the last row from a file Because I know I have done something similar in the past. Maybe that only works if you set the position.

Try this vi example LV 8.6.1. Just keep in mind if the file is open when you try to write to it youll get an error.

[Ray.R]

Hi swift,

I unfortunately do not have LV installed on this PC, so I cannot look at the VI Rob has submitted.  So I may actually be proposing the same thing as he has implemented..  😉

If I understand correctly, you only want to retrieve the very last entry which is comprised of 5 characters (ie 23.45).  You might want to get a 6th character just in case.

I wish I have LV to test out what I'm about to propose in order not to send you on a wild goose chase..

Since the interval is fixed, you could use a timed loop which first looks at the file properties and verifies if the filesize has increased since the last read.  Then read the last 5 or 6 bytes of the file (or the bytes matching the comma to the end of the data).  The only thingI do not like about this approach is if you "beat" the file by a few milliseconds, you could end up skipping a reading every once in a while.  So you could check the file properties at every 1 sec (or less, such as 500 ms) and read the last bytes if the file size changed since the last time you read it.  You can store the last filesize in a shift register.  Reading the last bytes which may include the comma will make life easier to parse the data.  Then convert the data string to numeric and plot it, etc.

R

[Rob_F1]

Here is a pic

[Coq_Rouge]

If every line has the same number of characters it sould be easy to calculate the offset for the last line and read it.

[Ray.R]

Thanks Rob,

Your solution could be implemented right after the read file section. 

In that case, to play it "safe", simply read the last 14 bytes of the file and use Rob's solution.  🙂

[Coq_Rouge]

Here is something i whipped together. It is crude but, the principals should be there.