Problem processing multiple files with dll

[aricks]

I'm using a function in a dll that turns a binary data file into a TSV file with actual numbers.  The function takes three inputs, a pointer to a string for the input file path, a pointer to a string for the file to write and an integer for "integrity check" which as far as I can tell does nothing.  I have the dll in a subvi, pass it a string corresponding to the input file path and let it convert the file.  This works but will only convert the first 254 files then errors out.   The only way I can get it to work again is to close labview completely and reopen it.  My experiment requires the conversion of literally thousands of files so closing and reopening labview gets old real quick.  I have tried creating a reference to the subvi and closing it for each file but this still does not work, I have also tried on windows xp and windows 7.  When the new files are created they have a size of 0 kB and are empty until I close labview then they assume a reasonable size (few kB) and have data in them.  Additionally if I try to delete one of the newly created files while labview is open I get an error saying the file is open in labview.  I can delete the files when labview is closed.  I have also tried opening each file and closing it after the conversion but that does not work either.

I believe this is some sort of file open limitation but I don't know how to get around it, I'm almost to the point of writing one of those mouse move/button click macros nerds use for games to convert the files because this is driving me insane.

[myle00]

The reason why the size is zero until you close LabVIEW, I believe, is that the dll probably doesn't flush to disk so the text  in the file doesn't actually get written until you close LabVIEW which flushes everything. Can you run the program on Win7 and at the same time open task manager->performance->Resource monitor->Memory tab and look at the RAM usage of the program as it runs until after you close LabVIEW. Does the RAM usage increases a lot?

Also, you might want to run process monitor: http://technet.microsoft.com/en-us/sysinternals/bb896645 and filter so that it only displayes LabVIEW events and look and post exactly what the dll is doing.

[aricks]

I managed to get the c code for the dll (it was somewhere on the computer).  I can only pass an integer to validity otherwise it errors, is there an unsighed char in labview?

_declspec (dllexport) int converting(char *filetoRead, char *filetoWrite, unsighed char validity)
{
	FILE *fpin, *fpout;
	int result;
	int records=0;
	int valid=0;
	unsigned long MTOFL=0;
	unsigned long MT;

	struct {unsigned ADC:12;
			unsigned INVALID
			unsigned MTOV:1;
			unsigned GAP:1;	
			unsigned ZERO:1;
			unsigned MTHIGH:
			unsigned R:8;
			unsigned MTLOW:
			} DataRecord;

	fpin=fopen(filetoRead,"rb");
	fpout=fopen(filetoWrite,"w");

	 while(1)
	 {
		result = fread( &DataRecord,
		if (result!= 6)
			 return(0);

		records++;
		if(DataRecord.MTOV)
			MTOFL += (unsign
  
		if(validity==1)
		{
			if(DataRecord.INVA
			continue; // don't sav
		}

		valid++;
		MT = (((unsigned long)Data

		fprintf(fpout,"%11lu",MTOF
		fprintf(fpout,"%5u",DataRec
		fprintf(fpout,"%4u",DataRec
		fprintf(fpout,"%6u",DataRec
		fprintf(fpout,"\n");
	 }

	 fclose(fpin);
	 fclose(fpout);

	 return 0;

}

[myle00]

Is this the whole function? Either way, as I suspected the file is opened but never closed or flushed. So as each file is read and written it all stays in memory and since the file is still open you can also not open the file in notepad because LabVIEW has exclusive access to it. I suspect that your RAM and file system resource is being eaten up (can you check your RAM usage as I said above?) so after a couple hundred files too many files are open so there's not enough memory/system resources to make new ones.

Can you recompile the dll and fix it so that the files are properly closed? Alternatively, with some work you can skip the whole dll and do the conversion within LabVIEW itself using the file IO VIs. You do have enough disk space?

[aricks]

sorry the code got chopped off

_declspec (dllexport) int converting(char *filetoRead, char *filetoWrite, unsigned char validity)
{
	FILE *fpin, *fpout;
	int result;
	int records=0;
	int valid=0;
	unsigned long MTOFL=0;
	unsigned long MT;

	struct {unsigned ADC:12;
			unsigned INVALID:1;
			unsigned MTOV:1;
			unsigned GAP:1;	
			unsigned ZERO:1;
			unsigned MTHIGH:8;
			unsigned R:8;
			unsigned MTLOW:16;
			} DataRecord;

	fpin=fopen(filetoRead,"rb");
	fpout=fopen(filetoWrite,"w");

	 while(1)
	 {
		result = fread( &DataRecord, 1, 6 ,fpin);
		if (result!= 6)
			 return(0);

		records++;
		if(DataRecord.MTOV)
			MTOFL += (unsigned long)MAXMT;
  
		if(validity==1)
		{
			if(DataRecord.INVALID) 
			continue; // don't save this record
		}

		valid++;
		MT = (((unsigned long)DataRecord.MTHIGH)<<16) + DataRecord.MTLOW;

		fprintf(fpout,"%11lu",MTOFL+MT);
		fprintf(fpout,"%5u",DataRecord.R);
		fprintf(fpout,"%4u",DataRecord.GAP);
		fprintf(fpout,"%6u",DataRecord.ADC);
		fprintf(fpout,"\n");
	 }

	 fclose(fpin);
	 fclose(fpout);

	 return 0;

}

[myle00]

The fclose call at the end (which I figured might have been cut off) can never actually be reached - so it's still true that the file doesn't get closed. The only way to exit out of the loop is the return 0 statement after result!= 6 conditional. That happens when the end of file is reached. At the point instead of saying break so that we can jump out of the loop and close the file the code returns which skips the fclose calls completely. So, to fix it I'd replace "return(0)" with "break".

[aricks]

sorry, I tried to edit the post but ran out of time.  Ram and disk usage are not a problem because all the files are small (>20kB).  I will figure out how to compile the dll with your recommended fixes tomorrow and report back

Thanks very much for your help

[rolfk]

To be a bit more detailed to what myle said, only replace the first return(0); with a break;. The second is required.

And an unsigned char is an unsigned 8 bit integer in LabVIEW.

[aricks]

That fixed it thanks guys for your help.