LabVIEW interface to CMUcam5 PIXY camera

The_ROG · ‎12-23-2016

I know this camera has a Lego Mindstorm block for it. I am trying to use it with LabVIEW but cannot find any interface for it. I am trying to write my own serial interface and the data seems a little off. Is the LabVIEW code for this available from the Lego development? I will be connecting it to a roboRIO for the FRC competition SteamWorks. For now I am using a computer with serial UART interface. With RIO will probably use SPI bus.

Bob_Schor · ‎12-25-2016

Did you look here, the CMUCam entry "Hooking_up_Pixy_to_LEGO_Mindstorms"? Do you have the Lego-Mindstorm Pixy with the Special Cable, or have you made your own Special Cable?

Bob Schor

The_ROG · ‎01-09-2017

I have made my on cable for UART communications to my PC. I am using a USB/COM port adapter that works for my other UART devices. My goal is to connect the camera via SPI to the roboRIO in our FIRST Robot. Using UART to work offline on data decoding. We will be using LabVIEW so I cannot use the mindstorm blocks. So far i have not been able to view the code in the blocks to try and write the LabVIEW code. So far I have used the VISA serial tools and can get data from the camera and it is almost correct. Looking at the data as hex I get some FFs i don't expect and the header is xFF55 xABAA. Seems to be off by a bit or so. this is an example of 1 block:

FF00 FF55 ABAA ABDE FAFD FF89 FD09 FFEF FFED FF00 FF55 ABAA AB16 FBFD FF99 FDD1 FFCD FF53 FF00

BoKnows · ‎01-10-2017

If the Mindstorms blocks have the functionality you desire, have you tried adding the module to your PC? http://sine.ni.com/nips/cds/view/p/lang/en/nid/212785

I'm not entirely sure this will work for you as I believe you'd want these blocks running on the roboRIO rather than your PC.

Is there something this camera offers that a more typical USB camera isn't doing for you?

The_ROG · ‎01-10-2017

I added the Lego module and looked at the pallettes last week. This module is a way to build your own mindstorm blocks with labview that can be used with mindstorm. i did not see any pallette tools for the camera i have, neither did i see a way to pull in an existing mindstorm block into LabVIEW. Do you have any experience with that?

The camera i have is pretty cool it is easy to teach colored objects. They have some good videos on the wedsite. The camera system has an on board processor to do the heavy lifting and can just give you object data - size, color and location. Pretty slick. We were thinking of using it for an FRC First competion robot just may not have time to build a complete interface unless i can get it running soon.

http://www.cmucam.org/projects/cmucam5

BoKnows · ‎01-11-2017

That looks rather interesting.

If I understand it correctly, it's built with a processor that you'd use something like C++ to handle on board processing and then ship out the data via one of the communication buses to your roboRIO. On the roboRIO, you'd take some action based on that.

It looks like you've got a decent amount of progress and are using the serial option. It looks like your header is coming after a few bytes rather than a single bit or two. Is everything other than the FF00 something you'd expect? I'm noticing this comes just before the header both times. Is it possible the FF00 is a termination character? If so, I'd expect the next thing we'd see after the string you gave us to be FF55 ABAA again followed by the appropriate data. Beyond that, what data would you be expecting?

The_ROG · ‎01-11-2017

I don't expect any extra header data. according to the info from the camera manufacturer about the data it should be straight forward.

Do you know if the LabVIEW serial tools take into account the 10 bits in each UART. I think i read somewhere that UART always starts with a LOW (Start bit) then 8 bits of data then Stop Bit. it seems my data may be shifted that 1 start bit.

https://en.wikipedia.org/wiki/Universal_asynchronous_receiver/transmitter - UART definition.

http://www.cmucam.org/projects/cmucam5/wiki/Porting_Guide

Copied from link:

UART

The UART interface is 8 data bits, 1 stop bit, no parity, no handshaking
The baudrate can be configured in the "Interface" tab of the Configure dialog in PixyMon
RX signal (pin 1) is 5V tolerant input
TX signal (pin 4) is 0 to 3.3V signal output
Baudrates up to 230 kbaud supported.

The serial protocol

Whether you're using SPI, I2C or UART serial, the protocol is exactly the same.

The protocol is data-efficient binary.
The objects in each frame are sorted by size, with the largest objects sent first.
You can configure the maximum number of objects sent per image frame ("Max blocks" parameter).
SPI and I2C operate in "slave mode" and rely on polling to receive updates.
When there are no detected objects (no data) Pixy sends zeros if the interface is SPI or I2C (since Pixy is a slave, it has to send something).
Each object is sent in an "object block" (see below).
All values in the object block are 16-bit words, sent least-signifcant byte first (little endian). So, for example, when sending the sync word 0xaa55, Pixy sends 0x55 (first byte) then 0xaa (second byte).

Object block format

Bytes    16-bit word    Description
----------------------------------------------------------------
0, 1     y              sync: 0xaa55=normal object, 0xaa56=color code object
2, 3     y              checksum (sum of all 16-bit words 2-6, that is, bytes 4-13)
4, 5     y              signature number
6, 7     y              x center of object
8, 9     y              y center of object
10, 11   y              width of object
12, 13   y              height of object

To mark between frames an extra sync word (0xaa55) is inserted. This means that a new image frame is indicated by either:

Two sync words sent back-to-back (0xaa55, 0xaa55), or
a normal sync followed by a color-code sync (0xaa55, 0xaa56).

So, a typical way to parse the serial stream is to wait for two sync words and then start parsing the object block, using the sync words to indicate the start of the next object block, and so on.

BoKnows · ‎01-12-2017

I don't have personal knowledge of the serial implementation. However, any time I've ever had discussions about implementation for any of our drivers with R&D, the answer has always been something along the lines of "we develop to the spec." With the UART being a defined spec, I'd expect the driver to account for every element of that spec, in this case the 10 bits.

It looks like we're getting mostly decent data. We just have some extra header information which is a bit strange. I'm also a bit curious as to why we are seeing the FF55 instead of the ABAA twice as called out in the product documentation. Are you able to test this with something else outside of LabVIEW or via a wireshark to see if you're getting the right data across the line and it's being handled awkwardly within LabVIEW? I'm curious to see what the raw data is to see if the data is actually coming in this way or if it's being interpreted slightly off of what we'd expect.

The_ROG · ‎01-13-2017

I have used other software to read the data and it is the same as what LabVIEW sees. I have started to believe the data is what's being sent. I will have to go to the vendor i guess to make sure the camera is sending what it is supposed to.

I'm really hiting a wall as far as pulling in Mindstorm blocks into labview. My main question here was that. Thanks for all your help. i will try and remember to re-post with my solution. Since it is build season now i may not have time for this camera. BTW- a spell checker would helllp me a lot in this post/editor. See what i mean.

pcurls23 · ‎01-20-2018

Hello,

Im curious to know if you had success with this project? I am trying to make an autonomous RC car that will follow strips of colored tape for a school project. I was planning on using the PIXY camera to teach it the tape to follow, but was wondering if I would be able to interface with LabView and the MyRIO? Anything advice helps.

Thanks

LabVIEW