NI TestStand Idea Exchange

Community Browser
Top Authors
cancel
Showing results for 
Search instead for 
Did you mean: 
Post an idea

 

something Kekin proposed with the digital signature made me think, that it'd be nice if there was a  means to suppliment/append to header/footer of reports in general without having to do it with raw statement steps...  (and if this already exists I apologize for not checking first).

 

I think it'd be neat if there was an approach similar to the 'Additional Results' logic per step that's available.  In the 'ModifyReportHeader' callback, or in the report options somewhere, a user would be able to supply the data within a certain list of available options/allowable types, and the report generator/DLL would handle the formatting/whitespace such that it works for txt/html/xml headers without the user having to do the formatting themselves.  From my experience, most items added to headers consist of (1) supplimental string data, (2) supplimental numeric data (3) small images/logos(in the case of txt, maybe the alt text can be used, or just the path put into the file...?) 

 

22449i9B8276AA25BFE2CA

 

even if the new info was just tucked into the list/table before the failure chain block, it would be a nice time-saver for users who want a generic report but just have one more UUT detail that needs to show up in the header.

 

TestStand File Diff and Merge Utility is not very useful for code reviews on its own.  It seems adequate for notifying the user that a sequence was added, however from the tool itself the user cannot actually review the newly added or removed sequence's contents.  Why is there no + on the item tree to go deeper.

 

Alternatively:

If I have to right-click a sequence and select "Go to location" then why bother with the separate tool to begin with?  Why isn't the diff utility integrated into TestStand's sequence editor itself?  Seems like a side-by-side comparison within Sequence Editor would allow a reviewer to poke and prod around all the hidden settings that are often missed using the existing utility.

 

Hello,

 

It would be nice if there were an easy means of programmatically loading a database schema for result processing.

 

Here's my context for asking:

We have a custom database schema that must be loaded on all of our stations and we'd prefer that it "just works" when our customized database result processing plugin is loaded.

 

I managed to accomplish this by studying the format of TestStandDatabaseSchemas.ini and writing a parser to insert our schema as necessary. This was a bit painful, though, and the possibility of the format changing without notice has crossed my mind.

 

Thanks as usual to the R&D team.  (Assuming this idea gets some traction!  Smiley Wink)

 

Kind regards,

 

Mr. Jim

 

 

The suggestion is to support logging a datetime with millisecond accuracy to an sql database with an NI_DataOperation step.

I tried to implement this, but the milliseconds are not written to the database.

It would be very usefull to support this feature.

 

The datetime field in the sql database is defined as follows:

CREATE TABLE

dbo.UUT_RESULTS (

UUT_STARTTIME datetime NOT NULL

)

 

The datetime is logged to the database, but only up to the seconds, although the format string is defined as "mm-dd-yyyy hh:ii:ss.sss" and the string does contain the milliseconds e.g. "2-10-2016 11:49:15.505".

Software Details: TestStand  version 2010 SP1, Operating System:  Windows 7
 
 

We in our company are done now several test systems where the best process model is Batch Model. Customers however does not want other reports than UUT report from each test socket. Simple skipping of batch report generation is not possible at the moment particularly now when plug-in structure has arrive for joy of us all. For me this wanted feature is late now because I manage to find out solution, with a great support of local NI team. Finally this feature of easy batch report disabling would serve future generations and today newbies. They will save hundres of hours when they don't have to use try&fail -method to skip those few meaningful steps but just make one selection in report options.

Because a customer asked me about it... And in looking around on the forums I'm surprised there's no opensource 3rd party solution yet to fill this gap Smiley Happy

 

TS has the ability via that little .ocx component to 'ploty' via the graphcontrol tool in an HTML or XML report.  But after many years, there is no 'plotxy'.   As this seems to be a common need for customers, maybe this could be rolled in with other changes? (if it isn't already?)

I think it'd be a good idea to have a simple (checkbox?) option to change graph scaling in TestStand Reports to logarithmic. There doesn't appear to be a simple way to do this right now.

I would like a new option in the Step Properties -> Run Options -> Results Recording Option.  The new option would appear for Sequence Call steps only and would allow the step to disable results recording for all steps in the sequence being called.  I would like this to recursive and apply even to subsequences within the sequence being called.Disable Recording of Steps in Sequence Call.jpg

Using the stock reportgen_xml.seq file, the text value of the XML node shouldn't contain the characters < or >:

 

http://www.w3.org/TR/xml/#syntax

 

When using LabVIEW VI's to parse this, you (rightly) get errors, so it's incredibly difficult to just search and replace the offending characters with their XML escapes. 

 

Example node contents from the XML report:

 

<Prop Name='ReportText' Type='String' Flags='0x400000'>
                            <Value><![CDATA[{0} Locals.i = 0; Locals.i < 2; Locals.i += 1]]></Value>
                        </Prop>

TS does not allow the ability to rename steps that are of the Flow Control type (i.e. If, Else, For, While, etc.). In a report, these generic names are then used, making it very difficult to know what condition the step was evaluating to determine whether to enter or continue within the flow control block. A step name would normally give the reader of a report some indication as to what is being tested or reported on. This is not the case for these step types.

 

For example, an "If" step that is determining whether a condition is met (and whether to enter into the subsequent steps within the flow control block) simply shows up in the report as:

Step Status Measurement Units Limits
Low Limit High Limit Comparison Type
If {True}  Done        

 

There is no indication of what it evaluated to determine it was True, meaning someone reading the report has no idea what the purpose of the step is. As designer of the test, I can look through the sequence to determine exactly what it was doing, but a report is a useful tool to be able to show to someone not familiar with the test and allow them to be able to see what happened. A report gives the reader the history of the test. Without knowing what a step is for, a lot of the "story" being told by the report is either lost or becomes much harder to follow.

 

I would assume there is a reason as to why these step types are not able to be renamed, but I'm not familiar with what that would be. It would be nice to be able to modify, or at least add to, the step name so that the report gives some indication of what it was that it evaluated. If that is not possible, there should be a simple way to include the parameter/expression being evaluated with the step name in the report. This can be somewhat accomplished using other methods, such as the Additional Results of the Properties tab, but this is not intuitive or clean and I would think this behavior should actually be default, not something the user has to manually create and enable.

 

Regards,

The Additional Results Step property is a great way to add customized results to the TestStand report for a particular step without much effort required.  However, there should be an equally simple way to add additional results too all of the steps in a sequence.  The current options are all lacking:

 

  1. Create a SequenceFilePostStep engine callback to manually append results to the result list - this feels hacky
  2. Add additional results to each step - this is tedious
  3. Create custom step types - this is also tedious, as a new step type would be required for each step

I think there should be an additional results option available in either the report options or in the sequence file properties window or the report options window.  Configuring the results here would be identical to adding the additional result to every step in the sequence.

 

Potential implementation of button on report options window:

tsgar.png

 

Additional results button launches a dialog:

ardialog.png

It'd be neat, in the scenario when a parallel thread is called, if there was some smooth way of gathering the results back into the main thread of the execution for the sake of the report. 

 

I've handled this historically by passing a parameter into the subsequence/thread with the ResultList-index of the parent-step, and then doing some notification based handshaking during cleanup of both threads to quickly copy the Result container out of the sub and back to the parent via API calls. But I bet the R&D types could do this far slicker and reduce the whole thing to a checkbox... 😉

 

21413i267B71BB01EB46E5

 

When I create reports they usually look like:

21417iA1EFA6BABE8471F4

 

I just think it'd be a simple feature to add for users who might be uncomfortable with the API  & ResultList? (Naturally if the sequence doesn't wait at the end it may never be safe to 'pass back' results. But if you're already killing time on the back stretch, why not copy the block up?

 

Cheers,

 

Elaine

When user opens the Offline Processing Utility and at the same time starts to type on the keyboard the user can accidently rename the profile.

(Attached a screen-recording to visualize)

It would be great if there was no selected row or column when starting/opening ORPU. The renaming of the profile disturbs the production since the database logging will not work as expected.

 

When we start ORPU we already have the '/tray' enabled but somehow its still possible to accidently rename the Profile.

Recently I tried to add in custom error handling to the NI_DatabaseLogger.seq, but because Plugin Sequence files don't support Engine Callbacks, this was difficult.

 

This idea is to allow model plugins to access Engine Callbacks, so that we can override their behaviour with customised sequences.

Allow for Additional Results to be selectively added to the reports of your choosing instead of all-or-none.

 

Currently:

1 - Step settings.PNG 

 

 

 

Proposed:

A - Compiled Step Settings.png

 

B - Compiled Report Select.png

Hello,

 

It would be nice to be abble to define the report file name "using tags". Smiley Happy

 

For the moment the Report name is created automatically. You can't easily change the order of the filed you want to see in the filename.Smiley Sad

You can only modify the date/time position.

 

I would like to see something like this :

 

Reportname = [%T<YYYY_mm_DD HH_MM_SS>]_[TestSocket]_[UUT].txt

 

ReportOptions.png

 

Thanks.

 

PS : I am using TestStand 2012 ... perhaps this already exists in TS 2013 ?

 

Manu.

Since reports are to certify the test results, it would be nice to have DIGITAL SIGNATURE option to give it FORMAL touch.

It will also help in organizations to authenticate the result.

 

It will be also nice to add some PPT or PHOTOS to report to describe the test setup & UUT.

Even though we already have an About box for TestStand, it would also be nice to have one for the active TestSequence file as well. The Help menu would have a new item called "About MySequenceFile ...".

 

MySequenceFile About box splash screen:

19679i37712516107E8A41

 

The information in this screen should be loaded directly from a single FileGlobal string whenever a sequence file is made active. This can probably be done already by those that know how, but it should be already part of TestStand by default.

 

 

Eugene

Hi there.

 

This may seem like a radical idea, and probably my most blunt idea I've ever posted but please bear with me:

 

NI folks working on TestStand as software, I don't mean to be offensive. I understand being in the position of having to make incremental changes, eventually outgrowing the original design of the software. I'm just saying what I think needs to be said and is long overdue.

 

Anyone who has had to customize the NI Report Generator will know that it is a complete hodgepodge of incremental changes from over the years.

 

As a high level summary:

  • Within NI_ReportGenerator.seq, all of the report types (ATML, XML, HTML, text, etc.) are handled in the same sequences
  • Depending on the report type, different steps are executed via preconditions, Post Actions (essentially GOTOs ☠️) and somewhat rare if-then flow logic.
  • This tight coupling of the report types makes it really difficult to follow the flow of steps that execute, and it's also just bad software design.
  • Further, there are sequence calls set by expression depending on the report type, making it even harder to follow. I'm guessing that this was supposed to be extensible by adding more ReportGen_<type>.seq files, but we're long, long past that being realistic.
  • Nomenclature is often inconsistent
  • Naming of steps often doesn't match what's really happening.
  • Preconditions and post action conditions are very often implicit.

 

AllOf( This, that, AnyOf( Sometimes the other, or another still), ! The other options that might apply)

&& !( Other implicit conditions because post actions in prior steps precluded execution of this step )

 


Highly technical details follow:

First off, in the TestReport callback, if report generation is disabled, the message to display is stored in Locals.ReportSection.AsReportSection.Body. Tricky, tricky, you didn't fool me! 😄 Yet, why not use an appropriately named variable?

 

Also, in the configuration dialog, the report style is referred to as the "Report Format", but the report format in the code is independent of the report style. The nomenclature isn't consistent. Along this theme, in the c code, there is a struct for ReportOptions, but it's not the report options from the TestStand sequence, and it's gradually populated over several consecutive function calls.

 

Taking ATML, for example, depending on which ATML standard is selected, the execution flow varies substantially, is inconsistent, counterintuitive, and seemingly arbitrary:


ATML 2.02

  • In the TestReport callback, if and only if the ATML standard is 2.02, the step "Get Report based on ReportStyle" is called.
    • ...but it's only for one style, and not even a style, which would be "xml"
    • This Get<report type>Report (GetATMLReport) sequence only exists for ATML (no other formats) and is only called for ATML 2.02, thereby completely removing any justification for a sequence call by expression.
  • GetAtmlReport then calls one of two functions in ATMLSupport.dll, depending on whether the caller has initialized a report section 
  • The report section is always initialized two calls up in "Model Plugin - UUT Done", so there is one case with a DLL call that never executes and is dead code.
  • Ultimately, the Get_Atml_ReportSection function is called in ATMLSupport.dll.
  • In ATML_Report.c, this calls Get_Atml_Report_Impl, which is where most of the business of appending report strings takes place

ATML 5.00

 

In the TestReport callback:

  • Locals.ReportGenSequenceFileName is determined by looking for the first four characters of FileGlobals.ATML202StandardReportStyle. (i.e. ATML) This is regardless of the ATML version, even though there are respective constants for each ATML version. You might incorrectly assume that it only applies to ATML 2.02
  • This time, by precondition we skip execution of "Get Report based on ReportStyle", which is a misnomer and exclusively for ATML 2.02
  • Counterintuitively, we also skip "Get Report Body (DLL)", even though we've configured body generation to be via DLL. By precondition, this step is implicitly for any format other than "xml", which I guess means ASCII and HTML? But they're not explicitly named.
  • No, we execute "Get Report Body (Sequence)" instead, even thought we're configured to use a DLL. Via various preconditions and post actions, this step is implicitly: Not for ATML 2.02, not for ASCII, and not for HTML. So I guess that means ATML 5.00, ATML 6.01 and XML? 
  • "Get Report Body (Sequence)" is yet another call by expression to an external reportgen_<format>.seq
  • "Get Report Body (Sequence)", when called for these XML based report styles, does not use a sequence. In all of those cases, the AddReportBody sequence it calls calls a DLL and only has half a dozen steps, so it's not a sequence, really.
  • Within AddReportBody, there are two possible DLL calls based on whether Parameters.ReportSection is null, but it's never null because the calling sequence always initializes Parameters.ReportSection. (More dead code)

I thought about talking about ATML 6.01, and I could go on, and on... and on. I think you all get the idea.

This is an absolute nightmare to navigate and very messy to customize without an extreme makeover crying edition.

Surely we can do better.

 

Thank you for reading my missive.

 

Mr. Jim

When running teststand deployment licence there is no Teststand GUI available to set or configure Results Processing.

I think it should be easy to export the active configuration to file. And then have a property loader type step to load the settings back into the sequence at runtime on the deployed machine. Note in my case I want to run different sequences with different reporting requirements on the same deployment PC.


I have found https://decibel.ni.com/content/docs/DOC-32076
and after more stuffing around than was justified I have got it working with TestStand 2014.

 

Export.png

 

 

Also it should be much easier to specify a relative UDL file. 

UDL.png

 

 

I resolved this by adding an expression after ReadEx that was:

Parameters.ModelPlugin.PluginSpecific.Options.ConnectionString="\"FILE NAME="+Left(RunState.SequenceFile.Path, Find(RunState.SequenceFile.Path, "\\", 0, True, True)) +"\\60467Database.udl\""

 

This means I can select an absolute path to the UDL file in the development directory on my laptop and then use a relative to sequence path in the deployment. 

 

I feel there should be an more obvious way or am I missing something?