General purpose error logger in highly parallel application?

[ijustlovemath]

Hi all,

I'd like to create a general purpose error logger the functions much like syslog() in the C/Linux world. Here's what I'd like for it to have:

1. Error Source (enum) - Provide some context to where the error occurred

2. Error Message (string) - User provided string to append to the message

3. Error Cluster Code (int) - self explanatory

4. Error Cluster Message (string) - self explanatory

5. Log Level (enum) - sets severity of the error, allows you to filter for critical errors/set compile time flags to ignore, for example, LOG_DEBUG errors.

6. Rate Limit (int) - Represents the number of milliseconds between appends for a particular message. Only works in reentrant versions of the logger.

I was initially thinking of using an action engine, but being nonreentrant, this would become a bottleneck in my application.

Then I thought of making a queued message handler, which would be nice from the perspective of fire-and-forget, but to prevent everyone from needing a handle to this queue in their private data, I'd need to make the queue named, or a global variable of some sort.

Another option is to make the action engine reentrant, and simply store the log path in a configuration file which all the clones could access before appending. This would be a little heavier on disk I/O, but given that I'm writing an error logger in the first place, that's not really a concern.

There could also be a hybrid QMH/action engine, where the AE simply stores the queue reference and a separate consumer actually processes them.

What do you all think?

[Ben · Accepted solution]

I have been using something that I started in LV 6 and has evolved.

A background thread is a QMH and does all of the writing to file.

An AE hold the queue to which the messages are sent.

Re-entrant VIs are used in error wires and on first call they cache the queue ref read from the AE. From that point forward they no longer access the AE and have there own copy.

Ben

[ijustlovemath]

A tribrid! Nice. How do you start/stop the background thread? Using VI server/wait on async? Or just a special message type in the queue?

[Ben]

---

@ijustlovemath wrote:

A tribrid! Nice. How do you start/stop the background thread? Using VI server/wait on async? Or just a special message type in the queue?

---

Code I wrote way back then and I have not touched in decades.

I think the string field of the error cluster was "exit" ?????

Ben

[aputman]

Is this a project that you want to do just for the sake of it? or are you actually making something for a larger project?  If you are looking to reinvent the wheel, then by all means, have at it.

If you aren't in the business of reinvention, have you looked at the Structured Error Handler toolkit found on VIPM?  I've used it once before it has a configurable dialog box to look for specific error codes and allows you to specify an action for it.  And then a separate process handles logging the errors and other actions (safe state, shutdown, etc). 

[ijustlovemath]

I finished building it a few hours ago; it's actually really straightforward! This is meant to be hooked into a desktop app that my startup is building; it's there for us to debug. I'll take a look at that though! I'd like have something that can be configured behind the scenes, as our end user base is not very technologically literate. 

[PhillipBrooks]

I recall playing with this example many years ago.

You did say syslog, correct?

LabVIEW Syslog Protocol Reference Library

http://forums.ni.com/t5/Components/Syslog-Component/m-p/746501

 

[Gregory]

Most of my code is separated into modules that often look like queued state machines / message handlers. So I like to include the module where the error took place (which it looks like you'll capture with "error source (enum)" as well as the previous state. In my code, when an error happens the module goes to the error handling state instead of processing the next message, so the "previous state" is where the error occurred, and helps me track down the source of the errors very quickly.

[drjdpowell]

My similar “QMH in the background” logger is the Cyth SQLite Logger.  I actually used a private Global variable to hold the queue reference.  Logger shutdown is automatic based on the queue dying when the main VI stops.