Continuously append data from an open file to an existing file

[rpscott02]

Ok here's the situation:

- Each time a test is run a new directory is created and the data is saved to a file in this directory.

- I need to give the user the option to append this data to an existing file or just proceed with the new file.

- If the user chooses to append, the new file is continuously being written to and I need for this data to be continuously copied to the data file to which it is appended. Basically, I need to be able to open the appended file during the test and see the latest data from the new file appended therein.

I've attached my best attempt and I think I'm close but I'm not even sure this is possible. The test I've setup is using .txt files and I'm expecting to see this happen: I add a new directory to my desktop and open the new file along with the existing file, when I type something into the new file I'm expecting to see the old file update itself. Right now this only happens if I save the new file first.

Am I testing this correctly? If not, should the program do what I think it should? If so, what can I do to make sure the data from the new file is being appended to the existing file right away?

Your help is greatly appreciated

[smercurio_fc]

I'm a little confused as to what you're trying to do, to be honest. You keep mentioning "existing file". Who/what defines this? Also, this statement:

If the user chooses to append, the new file is continuously being written to and I need for this data to be continuously copied to the data file to which it is appended

Huh? Under what circumstances are you dealing with a new file as opposed to appending, and why do you have to deal with a new file if you're appending?

I add a new directory to my desktop and open the new file along with the existing file, when I type something into the new file I'm expecting to see the old file update itself.

Type where?

It would help if you could provide a better explanation of what you're doing. 

[rpscott02]

Thanks for the reply. I'll try to clear things up.

When I say "new test", I'm talking about a simulation the engineers here are running that may or may not be using some other Labview program. This test is totally independent of the  program I'm running. All I care about is the file it creates. So when a new test is run, a new directory with a new data file is created. This is what I'm referring to as the "new file". If the user chooses to append, he can choose to append to a file thats already been created. This is what I'm referring to as the "existing file". This was created before the test was run.

Hopefully  this adds more clarity to the first statement you quoted. The "new file" is continuously being written to as long as the test is being run so I need to also have this newly written data appended to the "existing file" as well. In the VI I attached, you see a series of shift registers I've used to try to accomplish this. I'm attempting to open the "new file", copy any new data since the last iteration, append said data to the "existing file", then do it all over again until the program is stopped.

Reading that second statement of mine, I understand why one could be confused...I was trying to explain how I was testing the VI. I would copy and paste a new folder with a dummy .txt file inside it. The VI then recognizes the new directory, opens the file therein, appends this data to the "existing file" I've chosen for it. What I meant by "type" is that I'd open the file in the newly added directory and begin typing random characters. These characters only show up in the appended (existing) file after I save the "new file". My question is - should I be able, as I view the screen with both the existing and new files open, to type characters in the "new file" and watch these characters appear on the "existing file" as the VI appends them?

After all this, I'm pretty sure I confused things more than cleared it up....hopefully I didn't. Really appreciate your help

[nathand]

I think you've answered your own question here: new characters are written to the new file only when it is saved to disk.  Until then they're not part of the file - they're just stored in memory somewhere - and there's no way to copy them into your existing file.  So, whether or not this approach is going to work for you will probably depend on how your test program writes out its results.  If it continuously writes data to disk then your approach will probably work, but if it only writes out its results once at the end of the test then you'll have to wait for the test to complete before updating your existing file.

Also, I have to ask - how are you opening your files for your testing?  If you're just opening them in notepad there's no way you'll ever get the results you expect.  I'm pretty sure notepad only reads a file once, when it first opens the file, so even if you update the file after that you won't see the changes.  Finally, it may not be a good idea to try to read and write a file simultaneously... and if you're going to do so, make sure you open the file for reading as read-only.

[smercurio_fc]

I agree. The key statement is: "new characters are written to the new file only when it is saved to disk". In order for you to "see" the characters as they're typed you would essentially need a key logger, and I doubt you really need to go down that road. 

That said, under Windows (assuming that's the OS you're using - you didn't indicate that), you can set up a program to act as a file system watcher. I recently wrote a small little program in C# that I converted into a service that uses the FileSystemWatcher class to detect changes in a directory. Basically, it sets up a hook in Windows so that if a new file is created in a specific directory it will automatically email the file. You could try something similar. 

[rpscott02]

Yea, that makes sense. I'm writing this program for another group of test engineers so I'll need to find out how their program writes out the results. As far as testing, I was opening in notepad and your explanation there makes sens also. I agree on the simultaneous read/write to a file...don't think I'm doing that though. I'm reading from the "new file" and writng to the "existing file" in each iteration

Thanks for the help. I think I'll need to gather some more info from my guys here before I can proceed