Jeff,
TestStand offers many options to randomly execute tests. I made a
couple quick
and dirty examples.
You can download RandomlySelectTests.zip from ftp://ftp.natinst.com
These examples demonstrate how to randomly execute different tests.
RandomSelectionInClientSequenceFile demonstrates one technique to
randomly
execute different tests.
The step, Pick Next Test, uses this expression: RunState.NextStepIndex =
Random(2,11)
to pick the next test to execute.
The PreExpression increments a loop count.
The PreCondition is set so that only 10 loops will execute.
Finally, all other tests use a Post Action to return execution to Pick
Next
Test.
RandomSelectionInProcessModel uses a specific process model.
This is set under Edit>>Seq File Properties under the Advanced tab.
The MainSequence Callback in the process model is modified. It is no
longer a
callback but a normal
sequence call step type. Since it is not a callback, it will not call
the
MainSequence sequence
of a client sequence file.
MainSequence Callback is configured to loop 10 times and randomly call a
different sequence
in RandomSelectionInProcessModel.seq.
RandomSelectionInProcessModel.seq has 10
new sequences,
Test0 - Test9.
These examples were made with TestStand 1.0.1. If you have 1.0, you
should be able to open and run them (I think). There are several other
possibilities to execute random tests and it shouldn't take much time
to tailor the process model to your specific needs.
I'm sure you'll have more questions if you continue to investigate
TestStand. Feel free to ask questions as they arise.
Paul Mueller
National Instruments
In article <7ritrd$jan$1@nnrp1.deja.com>,
winkjj@my-deja.com wrote:
> In article <7rbm02$qng$1@nnrp1.deja.com>,
> paulmueller@my-deja.com wrote:
> > Jeff,
> >
> > I find it interesting that you want to execute the steps in a random
> > order. Why would you want to do this? What is wrong with a
> sequential
> > execution? Are you trying to perform tasks in parallel? If so, we
> > have a solution for parallel executions. You can download an
> > application note from http://www.natinst.com/teststand/ts_news.htm
> >
> > You could use the process model to directly call the steps.
However,
> > this would tie your process model to a specific client sequence
file.
> > So, you would have a specific process model for each sequence file.
> >
> > You can use the NewExecution method to start a new interactive
> > execution of a particular step. Use a random number generator to
pick
> > which step to execute. You also need an algorithm to ensure that
you
> > execute all the steps (if that is what you want).
> >
> > The biggest problem will be to get the results back so the report
can
> > be generated. Or you could generate a separate report for each step
> > that executes.
> >
> > There is an example, Reports for Interactive Executions, that shows
> how
> > to generate a report for interactive executions at
> > http://digital.natinst.com/explprog.nsf/web%2Fswgrp?
> > OpenView&Start=1&Count=500&Expand=7.1#7.1
> >
> > If you wanted to get the results back from the interactive
execution,
> > you could use the GetPropertyObject method on the execution to get
the
> > ResultList. Then you could get these results and append these to
the
> > ResultList for the process model.
> >
> > This would require some work which brings me to my first question,
why
> > do you want to execute tests in random order?
> >
> > Paul Mueller
> > National Instruments
> >
> > In article <7rb8p2$g81$1@nnrp1.deja.com>,
> > winkjj@my-deja.com wrote:
> > > I would like to create a sequence model that executes the steps
of a
> > > sequence in random order. Does anyone have any suggestions? In
> > > studying the general model that comes with TestStand, it has a
> > > MainSequence call back that executes the entire sequence. This is
> not
> > > my desired behavior.
> > >
> > > I thought about having the model call the steps of a sequence
> > directly,
> > > not via a callback, but I am not sure if this is the right choice.
> > >
> > > Any suggestions would be appreicated.
> > >
> > > Thanks,
> > >
> > > Jeff
> > >
> > > Sent via Deja.com http://www.deja.com/
> > > Share what you know. Learn what you don't.
> > >
> >
> > Sent via Deja.com http://www.deja.com/
> > Share what you know. Learn what you don't.
> >
>
> Hi Paul,
>
> We are testing a product that is similar to a telephone switch. The
> tests that we are performing are functional pass/fail tests, and
> ideally, we want to run these tests in a random order. The reason for
> this is because the tests simulate the actual operation of the switch
> environment, therefore, the tests that we have identified could occur
> in any order.
>
> To truly simulate the environment that the switch will see, actions
> performed on the switch have to appear to be in a random order. If we
> don't test this way, then we could possibly be masking some behavior
> or sequence dependent problem that would not be caught when the same
> tests are run in the same order every time.
>
> In this case, the more testing we do, the better coverage we have
> because we have testing our product in a larger number of scenarios,
> which are being randomly generated.
>
> This type of testing is very popular in software testing where you are
> trying to simulate as series of random events.
>
> Either way, it sounds like this will be more difficult than I had
hoped
> with TestStand. It appears that TestStand's model is heavily
dependent
> on a sequential concept.
>
> If you want to see an example of a test executive that can do this,
> take a look at http://tetworks.opengroup.org/. We would have like to
> have used this, but the Win32 cost is 10k, and it lacks a professional
> user interface, as well as some of the features of TestStand.
>
> Anyway, let me know if you come across any more ideas.
>
> Sent via Deja.com http://www.deja.com/
> Share what you know. Learn what you don't.
>
Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.
