Applying a texture to a 3D mesh in 3D picture control

[AsianSensation]

Hello,

I am working on 3D scene reconstruction using a flash LIDAR and a visual camera.  I would like to rebuild a 3D scene using the LIDAR then apply a texture based on a picture taken with the camera.

I am currently having difficulties applying the texture to the camera, and thus I would like some advanced help with 3D picture control.

My steps are as follows:

1) Collect LIDAR and camera data.  This is saved in the form of a point cloud and a jpeg.

2) Take point cloud and triangulate it with Point Cloud Library (I dump the lidar data to a directory, then execute a console command to run a triangulation algorithm)

3) Read in triagulated data 

4) Transform point cloud to camera coordinate frame

5) Project point cloud onto camera uv frame using camera equation

6) Build mesh based on vertex data, triangle data, and uv-coordinates

I have a triangulated environment, but I cannot get a texture to display properly.

Attached is the front panel output from the scene, the rendering loop, and the rendered scene.

I've also attached the parsing VI that takes a .vtk file and outputs my vertex, index, and texture coordinate array; as well as a sample scan and picture.  The math for the texture coordinate array is what can be found here: http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html

[Daniel-E]

Hey AsianSensation,

We've got a few examples on our website and in LabVIEW that might help you. Here's a page from the LabVIEW 2011 help file: http://zone.ni.com/reference/en-XX/help/371361H-01/lvhowto/3d_apply_texture/ 

One thing that sticks out to me here is that the provided code has a SceneObject property node that sets Specials.Texturing to On. My research indicates that texturing will not work without this property node. It looks like you might not have it in your code, so if not, I would suggest adding it. Also, I notice that most of the examples have the Scene Object property "Set Drawable" before the Texture.SetTexture property. So you might try switching that around in your code as well.

I'll admit that I'm not too familiar with 3D texturing in LabVIEW, so I'm working on finding more resources on this functionality, but I think those steps would be a good place to start. We also have an example in LabVIEW that you might find useful. If you go to Help > Find Examples, it's under Building User Interfaces > Displaying Data > Pictures > solarsystem.vi. That example seems to use a lot of the same concepts that you have in your code. 

I hope this helps, and please let us know if it works! This seems like a really cool application and I'm interested in seeing the result.

Thanks,

[AsianSensation]

Hello Daniel,

Thank you very much for this reply. I am not sure how I didn't notice what I was missing but, I found that your two suggestions fixed my code and I have a working texturized environment now.  Thank you so much.