How to keep track of changes in sequence files?

[Atoosa]

I wonder if a user with admin access level changes certain parameters in an existing sequence file and saves it with same name, or a different one, can Teststand create a a report of all parameters changes?

Or similarly, can Teststand create and report differences in between any two sequence files?

It is important for us to keep track of changes in sequence files in production line.

[~jiggawax~]

I seriously wish there was a SequenceFileSave engine callback for this purpose right here.  That way you could write to a log file or do whatever you wanted.

The bigger question here is why are they allowed to edit a deployed sequence file?  That seems like a bad idea.

Generally you use a source code control tool for development and then you have tracked all changes. 

You can use the diff/merge tool to generate a report for the differences between 2 sequence files.  I think it can even be executed through the command line silently.  However, you would have to somehow trigger that.  If you had a custom editor/UI then you could trigger that when they save the file.

This is not native to any out of the box stuff though.

[Elaine_R.]

TestStand has a Diff tool (Edit>> Diff sequence file against) which can be interacted with manually, you can invoke the exe directly via "C:\Program Files (x86)\National Instruments\TestStand 2012\Bin\FileDiffer.exe" or something of that sort, but I don't know off hand of whether there's an API.

I'm assuming that users logging in as admin is unusual, so you're trying to cover an odd-ball-security case? but realistly if you can't trust your admins... you're giving one or more of them too many privileges? perhaps?

I'm inclined to agree with Jiggawax, how is it that these files are editable still? if you're using an older TS install, and the *.seq source is still a text file, you'll want to secure them via OS level settings for read only access to prevent meddling outside of TS if possible.  If you're worried purely about edits within TS by super users, you could implement a md5 checksum style check as part of the test execution to log the file version run against  (or do a pass/fail test by comparing Checksum against a known-good copy of the file on a server, (or just lean on the TS API to catch the save-counts on the file).   I know of 2 manufacturing sites that leverage TS's ability to do FTP within a test sequence in order to have TS automatically pull a fresh copy of it's test code from the server every run if it detects a difference...

so yes. there are several manual tools you can leverage, if you're looking for something more rugged you'll have to decide for yourself just how much 'version control' you wish to implement in your system.

-Elaine R.

[Scott_Richardson]

To ISofi and jiggawax -

1) We have heard of some customers wanting to prevent a deployment from running if any edit to a file is made. They specifically wanted a built-in checksum (MD5) validation feature that could verify that files on disk are the original files that were deployed.

a) Is this something that would solve your specific issue?

b) Would you want someone to have to approve the sequence file change in some way before allowing the file to be used, or are you just interested in a log of the sequence file changes that were made?

2) You talk about monitoring changes to sequence files

a) Are there other non-TestStand files that you would want to also monitor?

b) Are non-sequence files less important than sequence files?

3) Can you give details on the operations that you need to perform in a save callback?

a) Would you need the ability to prevent the save?

b) Would you need to present user interaction before committing the save?

c) Would you need to edit the file before the save?

4) Would a synced event on ApplicationMgr by the user interface be sufficient instead of an engine callback; if not why?

[Elaine_R.]

I only saw this response now, Scott, so I hope it's not too late to reply -

.

I can't speak for the others but if TS did build in some sort of md5 check step / passfail test / API call that I could use, I'd definitely use it!

.

Maybe a combination of

(1) some sort of tool menu utility that would capture the MD5 values of all files within a folder heirarchy... and store them in some non-casually-editable way to a path of my choosing.

(2) an API / step that I can invoke to 'check' a single file path or recursive folder against that repository of values. that i could then incorporate into my process model etc.

.

The short answer to the rest of your questions is 'I'll want to control any/everything, and I'll want to do it myself' 🙂

.

When I've written MD5 checkers for myself I make them filetype blind, because sometimes I want to 'control' portions of the public folder too. Sometimes I want to control source code, sometimes I want to control limits, and it really depends on the end user / system setup.  I'd rather not have TS do any of the controlling of files / file saving/loading 'for me' because of this as it may end up being more grief to turn off the safeties that are assumed on my behalf than it is to implement new ones.

.

I'd be very happy just having the easy ability to tap into the checks in my custom models without having to write/maintain it myself.  Currently I'm leaning heavily on P4 to do these sorts of version checks. But even here, we don't stop the tests from running in my case, we just query &  log to the report that the test was not 'the latest approved version' and what our best-guess as to the version was, for traceability.

.

Hope this helps,

Elaine