RS232/USB Xon/Xoff reading only 2200/2046 bytes

[eclx]

hi everyone,

im working on a project that requires me to receive large amounts of data (16 megabytes) as fast as i can from a RS232/USB serial communications interface.  I currently have the port set at 8 bits, no parity and 1 stop bit.  I succeeded in taking data at baud rates of 115200, 230400 and 460800, although I start running into problems at 921600.

I get an overrun error at 921600 baud without flow control.  When I review the data, I see where the data was lost (at seemingly random places), and I only end up with about 14 megabytes out of the expected 16777216.  Knowing that I am not reading the data as fast as it is coming in, I reduced the size of the receive buffer in the Device Manager --> [port device] --> port settings --> Advanced... but that didnt correct the problem.  I then turned the flow control to Xon/Xoff.

I started running into problems again.  With flow control on, I get no loss of any data, however, the port just stops receiving when it reaches a consistent 2200 or 2456 bytes (coincidentally 256 bytes different from each other).  I recorded the digital waveform and I see where the computer told the device to stop and start using the ctrl-s and ctrl-q commands, yet it just stops transmitting once it reaches 2200 or 2456 bytes. 

Has anyone else run into this problem?  Does the device need troubleshooting or is the problem on the receive side?

[Pnt]

"I reduced the size of the receive buffer "

Why did you reduced it ? I would make it as big as possible.

 How do you read the data ? You must have no delay, between two serial reads, and a big number at it's input.

Also have you tried increasing the VISA buffer ?

[eclx]

Thanks for the quick reply.  I forgot to mention that I am fairly new to LabVIEW

1. I reduced my receive buffer size under device manager.  After I opened the port and under the port settings tab, I clicked on advanced... then reduced the buffer.  Reducing the receive buffer size on the port is better for troubleshooting so that the computer can quickly detect the overflow and then interrupt the external device to either "stop sending" or "start sending."

2. I have increased the VISA buffer size to 32megabytes so it definitely wont overflow.

I attached the vi to this post.  I basically modified the Basic Serial Write and Read so that I increased the VISA buffer from the default 4096, and so that I may output a text file. 

I am also unsatisfied with this VI because the amount of data read solely depends on the wait time I put in.  The wait time actually waits for the VISA buffer to fill, and when the wait time is done, it begins reading immediately.

Is there a way to just read whatever was coming out of the device without any time restraints?

Is there any way to monitor the hardware buffer so that I can tell the external device to stop transmitting data when the buffer starts to get full?

[RavensFan]

First, the basic serial write/read is not a good basis for what you want to do.  It just executes once.  It opens the port, writes, reads, then closes the port.

It sounds like you want to do continuous reading, so you should have a while loop around the write/read part of the program.

[eclx]

I basically have my device to wait for an "S" then just start basically memory dumping to the port.  At the 921600 baud might I add.

So all I did in the basic serial write and read was to:

1. Write the "S" so that

2. I may Read all the 16megabytes of ascii just pouring into my port. 

Note 1: another "S" will queue the device to send the 16 megabytes of data again.  I found this out by accidentally typing "S" twice in hyperterminal  Therefore, If I do a continuous loop, I would probably need to exclude writing that "S" portion repeatedly.

Thought 2: The delay between looping it again will probably make me lose some data since the device is mindlessly pouring data out into the port.  I would have to either tell the device to "stop sending" at the end of the loop and "start sending" at the start of the loop.  I thought about that but I also thought that it would be more eloquent to just monitor the hardware buffer so that I tell the device to "start" and "start" only when needed to.

I believe I get the overrun simply because reading the data from the hardware buffer is slower than data coming INTO the hardware buffer.  One method I tried to prevent that was to turn flow control on, which, apparently doesnt work.

the two most important factors I need to keep are: 1. 921600 baud rate and 2. no data loss whatsoever

Is there a LabVIEW function that monitors the port buffer in any way?  Do I need the Real-Time LabVIEW thing?

The learning curve is a bit too large for my "inexperience with LabVIEW" buffer

Jin

LabVIEW noob

[rgambin]

You could use the Bytes At Serial Port property node to read all of the data avaiable and then loop to do it again until you reach 1677216. 

What kind of device are you talking to? You said by typing "S" it sends data, do you have any other documentation for the device. 16MB of data at one shot seems to be a lot to process 

What are you trying to do with the data? 

[eclx]

The Bytes At Serial Port property node only reads the number of bytes in the port buffer.  In the Basic Serial Write and Read,  Im assuming this is what happens:

I set up the properties of the port and open it... 

1. I set the buffer to 32 mb at the VISA Set I/O Buffer Size

2. The VISA Write sends an "S" to the T.I. 320F2812GHHA DSP Chip.

3. The DSP Chip starts dumping out all the memory into the VISA buffer (size of that buffer set at the VISA Set I/O Buffer Size: 32mb)

4. We wait for the Wait Time function to finish. (During this time, the DSP chip is still dumping memory into the buffer at a 921600 baud rate)

ex: set at 500 ms

5. After the Wait Time is done, it will then count the bytes in the buffer at exactly the time the wait time finishes

 ** The DSP chip could probably be still transmitting data after this, where it will simply be ignored and neglected **

ex: continues to VISA Read when 500ms is up.. it will probably only read about 80kb of data... and Bytes Read At Port will display only 80kb

6. Those bytes that have been counted (at the end of that specified wait time) will then be read and written to an output text file.

7. Port is closed using VISA close

I usually set the wait time to about 5 minutes (300,000ms) to read all of the 16 megabytes of data.  It should take about 3.3 minutes to complete so there is plenty of time (and extra room) for the DSP chip to finish; however, I get about 14 megabytes when the DSP chip is done transmitting everything.  I have lost chunks of data all over the place due to the overrun problem after everything is done.

Note: The RS232/USB adapter I use is the high speed TUSB3410 from Texas Instruments.

- Jin

[Ben]

I would like to suggest using the following approach to read all of that data.

1) Configure the buffers to as large a seeting as possible. THis will help compensate for the indeterminism of Windows.

2) Use flow control on both machines. This is (ideally) handled by the hardware and will signal the trnasmitter to stop transmitting until there is room for more data.

3) Develp a Produced Consumer architecture where the "producer" will

a) Use a property node to query how many bytes are waiting at the port.

b) Read the number of bytes that are waiting.

c) Queue up the results for use by the consumer.

d) Wait a short period of time (10 ms?)

e) Loop back to "a'

IN the consumer periodically read the queue from the producer and buffer these in a shift register.

NOTE:

NI Spy MAY be helpful in logging what is happening at the port. I say MAY because the serial device is not NI so...

Just trying to help,

Ben

[rgambin]

How is this wired? From what I can find both of these part numbers are for IC's only. Are you trying to integrate the DSP to the serial to usb converter? Are you using the ADC function of DSP and who created that code?

Have you tried doing a serial loopback with the converter? I would try this to make sure when you write the "S" you actually receive "S".

Maybe if you describe what you are trying to do we could offer more help.

Robin