Find Nearest Index in 3D (LUT) Array

[Eerl_Spec19]

Hi,

I am new to LabVIEW, but trying to develop a search routine involving a 3D lookup table (LUT) with four columns (eg. attached .txt file with 4608x4 entries). 

Given three specified inputs (ie. the 0th, 1st, and 3rd column values), I want to determine and return the row index with the nearest (or matching) corresponding values from the LUT array (and missing second column entry).

The LUT itself contains 'all' possible combinations of the first three columns over a desired range and resolution, and the resulting output as the final column. The desire is to minimize the amount of searching the VI does as it encounters repeating entries - is there a better way to accomplish this than nest for-loops and magnitude comparisons?

[Kevin_Price]

There will be better ways than what you've envisioned so far, but some more info will be needed to help steer toward an appropriate method.

Tell us some more background about intended usage of this LUT.  Where does column 3 "absorption" come from -- is it a measurement result or a theoretical calculation? 

I would recommend you visuallize this like a 3D surface plot, where columns 0 and 1 (pressure and temperature) are coordinates in a horizontal plane and column 3 "absorption" is the surface elevation at that point.  If the surface has a lot of undulation to it, then a given elevation might occur in multiple regions of the surface.  So one possible lookup method might use absorption as the key measure to try to match while using pressure and temperature to greatly narrow down which region to examine.

-Kevin P

[Eerl_Spec19]

Hi Kevin,

Apologies! Yes, the third column column of the LUT is based on the theoretically calculated absorption values depending on the first three parameters (temperature, pressure, concentration).

The idea is that the LUT will be used as a reference for comparing empirical data measured through experiment. In this case however, pressure, temperature, and absorption are known and the desire is to find the nearest corresponding concentration index at those specified conditions.

My crack at it was to narrow down to an initial index based on the absorption, then gradually use the temperature and pressure to refine the LUT indexes such that the index of the nearest bounding matches could be obtained for linear interpolation (ie. (P00, T00), (P01,T00), ... etc.). I've laid out my thoughts of the logic, though its definitely not pretty - perhaps others or yourself might have suggestion building off from it? 

[Eerl_Spec19]

With respect to your comment, that's is indeed the case. The absorption is mainly driven by concentration (which can be seen as the large steps in amplitude below). Between each step you can see the general trend resulting from increasing the pressure (increased abs.) and incrementing the temperature (lowers abs.). Were a finer step choice used for the concentrations, there would be multiple possible matches for absorption regions (in addition to the overlap trend already seen at higher concentrations) - something that has been complicating the envisioned look up process and that I'm not sure how to best handle.    

[Kevin_Price]

I'm interested, but I'm certain there are plenty of folks with better background & insight about this kind of thing.  So I only have some crude & rudimentary initial thoughts, but I'll offer them up in the meantime until better suggestions come along.

1.  The *use* of this LUT is to take 3 measured values (pressure, temperature, absorption) & use them to estimate concentration.

1.1. there may be some uncertainty in your measured values that you should try to account for.  You may need to consider a wider range of combos around your measured value.  This complicates things.

1.2. but you also have the great advantage of knowing a theoretical relationship you can use to guide your algorithm.   

    Looking at that graph, it appears that a given absorption will imply no more than 2 possible concentrations (possibly increasing to 3 if your absorption uncertainty is large enough) to consider from the LUT.  It'll either be near the high end of temperature at the higher concentration or the low end of temperature at the lower concentration (and possibly mid-mid if you had 3 concentrations to consider).  So now your measured temperature can probably tell you which of the LUT concentrations is most likely the nearest one.  That quickly narrow things down quite a bit.

From there you can further use knowledge to consider the extremes: 1st suppose that your measurement uncertainties combine in the particular way that gives you the lowest estimated concentration.  Then suppose the opposite.    This leads you to high and low bounds for the estimated concentration.

But *all* of this work begs the question: why not just use algebra?  Whatever formula you use to generate the LUT, why not use algebra to solve for concentration as a function of pressure, temperature, and absorption?  Then you can just plug your measured values (and perhaps also their values at the uncertainty extremes) into the equation to calculate concentration.

-Kevin P

[Eerl_Spec19]

Hi Kevin,

In reading your reply I realized a mistake in my phrasing.   

The lookup table is based on empirical data and experimentation - there is a theoretical relationship tied to it known as the Beer-Lambert-Bouguer law (for light attenuation through a specified absorbing medium - I don't know if this context helps), however this relationship itself contains additional unknown parameters which can complicate and make direct calculation difficult (and computationally intensive) - hence the use of the pre-processed LUT.

The lookup table is generated based on specified Pressure, Temperature, and Concentration - with the resulting Absorption from empirical testing outputted for those known parameters.

In the case of VI, the absorption, pressure, and temperature are known and the desire is to determine the corresponding concentration. As you mention, for varying conditions its possible to have multiple concentrations - narrowing this down would be desirable, but requires more knowledge of the relationship (which relates back to the computational intensive note).

The earlier VI uses absorption to narrow down an initial search region - if it were possible to identify all the alternative areas and to obtain their similar indexes for interpolation (similar to what the current VI does for a single match) then I think it would be possible to do a final interpolation for concentration based on that same indexes corresponding absorption (if that makes sense). 

I'm not sure how to implement the code, but in its most general application if there were an way to iterate through the entire LUT itself across lower and upper bounds for the 0th and 1st columns (and corresponding pairs for the 2nd and 3rd columns), then interpolation at the measured conditions would allow you to narrow down to a finalized region without needing to consider the trend of the data itself. 

Just thinking aloud incase it helps anyone's eureka moment!