read delimited spreadsheet with ~10e6 rows

[EngrStudent]

@Ben - would you mind sharing the code or toy code for some of that?

[Ben]

---

@EngrStudent wrote:

@Ben - would you mind sharing the code or toy code for some of that?

---

I would love to but my hands are tied.

Spoiler

or would be tied and locked up if I shared it outside of the barbed wire fences (smiley-wink)

But I can outline the process to modify the Read From Spreadsheet.

Open the read from spreadsheet VI and do a "save As..." so that you will have a copy outside of VI.lib

Change the input to a I64

Find the sub-VIs inside your new Read From Spreadsheet

Do a "Save as" on them and update the caller.

Lather rinse repeat all of the way down.

Be careful not to have the VIs from VI.lib open when you proceed. You do not want to foul your stuff in VI.lib

Use the "VI Hierarchy" screen with option to "view full path in file label" to make sure you are working with your version and not screwing up vi.lib.

If you are uncomfortable with the above, back-up the stuff in VI.lib before you start.

Have fun and be patient!

Ben

[EngrStudent]

@Ben - I just say "I have a non-disclosure agreement, and violation would be 'career-limiting'".  I understand.

I was more interested in the producer-consumer for reading/transforming a stack of files asynchronously. 

[Ben]

---

@EngrStudent wrote:

@Ben - I just say "I have a non-disclosure agreement, and violation would be 'career-limiting'".  I understand.

I was more interested in the producer-consumer for reading/transforming a stack of files asynchronously. 

---

On that note...

Normal producer/consumer but use fixed size queues between them. If the processing loops backup, the producer loops will stall waiting for room in the queues.

Beyond that, I would have to look at the code and not just that fuzzy image in my heads.

Ben