Which is the best way to Log result in Database?

[V_T_S]

I have a test sequence which produce half a million results and takes approx 30 hours. Results are mainly like Pass/Fail/Error type. In case of failure or Error, I generate the report text which also needs to be logged. This text sometimes can be very large.

The test sequence has all the custom step types build in Labview and I have On the Fly reporting enabled HTML type –my report size is > 5M (that means in database during each execution I will be logging that much of data)

My options are:

1. Enable the On the Fly Database Logging and let TestStand log the result in DB or
2. Include the SQL to Log results in all the custom step types.

Which one is best suitable? I am using TestStand 4.0

All suggestion/ideas are greatly appreciated.

Thank you,
Vidula

[leek]

I can't imagine a runtime of 30 hrs.  If you can manage to partition your tests down, it should improve your runtime.  Also disable On the Fly logging will be a big help.  TestStand will keep track of test results and save or print when all tests are done.

leek

[V_T_S]

If On the Fly is disabled, then Test stand will keep the result in memory and the memory usage will increase.

I noticed that when I was generating report at the end my test execution, the virtual memory usage for test stand on my system was hitting the 1 GB mark; then I started generating the report on the fly which drastically reduced the memory usage of test stand.

So I suspect the same will happen if ‘On the fly logging is disabled’

-Vidula

[leek]

If you have no problem with 1GB of memory (meaning your system doesn't crash consistently.), with On The Fly Report disabled, you total runtime should reduce considerably.  With On The Fly enabled, TestStand pauses at each step to collect data to generate partial report and save it in the memory.  It is true that you use less memory this way.  Because each step generates small portion of report.  But this is extremely time consuming, particulary when you have lots of tests (30 hrs runtime).  When all tests are done, the final report is print regardless of which option you choose.

Memory is very inexpensive these days, relative to 30 hrs of runtime.

My suggestion is to try both ways.  And see which way you can live with.  However, each try will take about 30 hrs before you can decide on the solution.  Also adding memory to your system will help.

leek

[Norbert_B]

---

leek wrote:

I can't imagine a runtime of 30 hrs. [...]

---

Well, there are test systems in this world which do tests on a single device under test for more than a week.... So this isn't really new to TestStand. On the Fly reporting is possibly needed here in order to prevent memory shortages. This requires the OTFR to be configured to conserve memory and the process model has to discard results if not needed anymore....

I think that both checkboxes are marked.

You can/should use OTFR with DB logging. The only thing you might want to change is the schema since this defines which data is logged in what way to your database (so it defines the minimum content of the DB in regard to tables and columns).

hope this helps,

Norbert 

[leek]

I have been a test engineer for more than 24 years, in addition to other engineering jobs prior to this position.  The 1st rule of thumb is to minimize the total runtime and still cover all tests needed to test features and capabilites of the UUT.  If this means ITA or cable must be redesigned or Test Set specification need to revise, so be it.  I have had my boss screaming at me when my runtime exceeds 1.25 or 1.5 hours.

If 30 hours runtime is required for this UUT, it is justifyable to me.  The design engineer knows best.

If this runtime is what you have to live with, and On The Fly logging is prefered, you may want to consider adding more memory to you PC.

leek

[V_T_S]

---

Norbert B wrote:

Well, there are test systems in this world which do tests on a single device under test for more than a week.... So this isn't really new to TestStand. On the Fly reporting is possibly needed here in order to prevent memory shortages. This requires the OTFR to be configured to conserve memory and the process model has to discard results if not needed anymore....

---

Norbert,

Thank you for the suggestions.

I have configured the ‘on the fly reporting’ + ‘Conserve memory and only display latest results’ + ‘Discard Results or Disable Results When Not Required by Model’; all these option together gave a pretty good result for reporting.

Is there a way I can stop each and every step report writing, but club the report writing part for say 10 steps. (to reduce the File I/O processing)?

---

Norbert B wrote:

You can/should use OTFR with DB logging. The only thing you might want to change is the schema since this defines which data is logged in what way to your database (so it defines the minimum content of the DB in regard to tables and columns).

---

I have updated the TestStand to 4.2 since it has 'Additional Results' option.

I am not sure if this is possible to do or not or is it a good option, but I am exploring these options :-

Since my number of results are more than half a million and there will be multiple test systems running alltogether, I am planning is to create the result tables on the fly depending using some unique id something like <station_name>_UUT_RESULTS.

The schema need to modify on the fly for the table names to log result.

Is it possible to modify on the fly, the OTFR for DB logging?

Thank and Regards,

Vidula

[V_T_S]

---

leek wrote:

you may want to consider adding more memory to you PC.

---

Leek,

This test application will be deployed on "N" number of PCs, so adding more memory to each and evey PC is not a ideal solution.

[Eric_B.]

Vidula,

Rather than modifying your schema to write to a different table for each station, you should add a column to your results table to keep track of the station name that way.  Then if you want all the results for a given station, you can run a query on that table with STATION_NAME ="<whatever station name>" to get all of those results.  The size of a database table is only limited by the disk space available, so you can have millions of entries in one table without a problem.

As far as writing to the file every ten steps, you can not do that.  You can only switch between on-the-fly reporting or the end-of-test reporting.

[V_T_S]

Eric,

Thank you for the reply.

With On the fly report writing enabled - Is there a way I can stop each and every step report writing, but club the report writing part for say 10 steps. (to reduce the File I/O processing)?

-Vidula