write/read to file and remove duplicates

[km76107]

Hi:

Here is my situation... and forgive me if this has already been discussed in the forum.. i looked and couldn't find a solution..

I am using TCP/IP VIs to read data continuously.  My goal is to store this data in an excel file and update the excel file if new data comes in.  I don't want any duplicates in the excel file.  I think I have solved the problem...but I am unable to get rid of the duplicates...

I have attached my VI (LV 7.0).  Any advise is appreciated.

Thank you.

[tbd]

Hi kmoorthy,

      Nice work - it was a pleasure going over this code - wiring left-to-right, little wasted space and great comments.

I don't think the code would have done what you wanted, though: it looks like "equal" was intended  to search for duplicates, but it would produce a cell-for-cell comparison between arrays (comparing the first elements of both arrays, then comparing the second element of both arrays, etc)  the long-and-short of it is, there could never be a false comparison.

There was quite a bit of file IO.  Here's a VI that keeps an array of Device ID's and only writes a record if the ID is unique - no need to read-back all the data!  There's still room for improvement though.  If I knew how you wanted the file formatted, I'd have used the low-level File VIs to open a reference once, and just use a Write File on each iteration of the loop.

Hope it helps!

[CoastalMaineBird]

Your attachment is missing some VIs, so it's hard to tell exactly what you're trying to do.

Some general advice: It's always better to avoid doing something, rather than doing it and then undoing it.

Can you hold all the data in memory at one time? If you can, then search for duplicates IN MEMORY, and avoid putting duplicates IN MEMORY as they come in. Write the current data out periodically, if needed.

If you can't hold all the data, can you hold all the DEVICE IDs? If you can, then just search your internal list for a duplicate device ID and avoid writing the data if it's a dupe.

HTH,

[km76107]

Hi tbd:

Thanks for the VI.  It works fine. I appreciate your help.