Calling schedule task on server

[jyang72211]

I have a question about schedule task on server and semaphore operation on server.  

For example, I have 10 lab computers and 1 server.  All the lab computers will do the same thing at different/same time through a LabVIEW exe.  They would generate some data, put them in the database, and access the html that the server has generated with the most recent data in the database.  I want to do the following:

1. At the end of the program on the lab computer, I want it to trigger the scheduled task on the server, so that the html will get updated with the newest data.  How do I do that?  

2. Since multiple lab computers can run at the same time, I want to create a semaphore that will allow only one lab computer access the scheduled task on server at a given time, and the rest of the computers have to wait.  It is a semaphore condition, but I am not sure how to do this in a server context.

Thanks!

[nathand]

---

jyang72211 wrote:

1. At the end of the program on the lab computer, I want it to trigger the scheduled task on the server, so that the html will get updated with the newest data.  How do I do that?  

2. Since multiple lab computers can run at the same time, I want to create a semaphore that will allow only one lab computer access the scheduled task on server at a given time, and the rest of the computers have to wait.  It is a semaphore condition, but I am not sure how to do this in a server context.

---

Have you already decided how the computers will talk to the server?  I assume they're all connected by an ethernet network.  Do they share a network drive?  If so, one very simple solution would be for all of the computers to write to a file on the network that includes some identifier unique to that computer (to avoid conflicts) and a timestamp.  Then your server just scans the network directory periodically and looks for the file with the most recent timestamp.

If you use VI Server, you could have a functional global variable hosted on the server.  Each of the clients would obtain a reference to that VI and run it to update it with their data.  LabVIEW will handle the locking for you - only one copy of that VI can run at a time.  That VI should also contain some signaling mechanism to indicate to the server that there is new data - or the server could just run it at regular intervals, pulling the latest data.  For example, the VI might obtain a reference to a named notifier, and the main VI would wait for a new notification, indicating that new data is available.

[jyang72211]

I have not decided on how the computers will talk to the server.  The lab computers need to tell the server to update an html.  All the lab computers and server are connected by a Ethernet network and share a network drive.  

For the timestamp approach, I can write a LabVIEW app to do that, but the app on the server will have to read the file very frequently, and the server will also need to create another file to record when the html was updated, which needs to be polled by the VI on the lab computers.  It seems to be a lot of polling.    

The VI server approach sounds great.  The server will contain a VI that wait for a notification from a lab computer, and the server VI will run when it receives a notification.  Then, the server VI will send another notification back when the update is competed.  I can send a server/lab computer specified message within the notified to tell the notification apart.  I noticed that when multiple lab computers are sending notification to server at approximately the same time, some of the notification is dropped.  I think a notifier alone may not work for my application.  

I think I need a semaphore to restrict access, since the lab computer will wait if a semaphore is taken; and a notifier from the server when the server is done with the task.  Did I over complicate things?  What do you think?

Please point me to some reference.  I am not familiar with VI server.  Thanks!

[nathand]

---

jyang72211 wrote:

The VI server approach sounds great.  The server will contain a VI that wait for a notification from a lab computer, and the server VI will run when it receives a notification.  Then, the server VI will send another notification back when the update is competed.  I can send a server/lab computer specified message within the notified to tell the notification apart.  I noticed that when multiple lab computers are sending notification to server at approximately the same time, some of the notification is dropped.  I think a notifier alone may not work for my application.  

I think I need a semaphore to restrict access, since the lab computer will wait if a semaphore is taken; and a notifier from the server when the server is done with the task.  Did I over complicate things?  What do you think?

---

If you haven't yet determined the communication mechanism, then how were you able to notice that some notifications are dropped?

Using VI Server there's no need for a semaphore; LabVIEW will handle it for you.  VI Server documentation and examples are included with LabVIEW.  My suggested approach is:

In the server program, create an action-engine type VI that stores a reference to a queue.  In your main server VI, create the queue, then store the reference to it in the action engine.  The action engine should have a second action to enqueue data.  Each of the client computers will obtain a reference to that subVI.  When the client completes its task, it calls that subVI - remotely, by reference using VI server, and passing the data to enqueue.  The server VI waits for new data in that queue and updates the HTML file as new data arrives.  If you only need the latest data from the most recent client connection, you could use a notifier instead of a queue.  LabVIEW will prevent more than one client from running the VI at the same time, the same way it does in a normal LabVIEW program that tries to run the same non-reentrant VI in multiple places concurrently.  This happens quietly, without generating an error, and you don't have to manage it at all.

[jyang72211]

When I said that some notification was dropped, I was only working with a local VI (see attaached).  For a funcitonal global on a server, the problem that appeared on the local vi probably won't happend according to your explaination.  

If can call a vi (functional global) on a server remotely through VI server, does the serves has to have LabVIEW development suite or run time engine?  Let me explain my applicaition a little bit better.

1. Lab computer X finishes testing and exports data into the database

2. Lab computer X notifies the server that new data is ready and is located in the database

3. Server queries the database and updates the HTML

4. Lab computer X display the new HTML.  

Notes:

- During 1-4, all other lab computers will not locked out and can't do the same thing

- No data is passed between server and lab computer directly.  All data transaction is through a database.  Database acts as a medium between lab computers and server.

[nathand]

Your test VI doesn't show much, other than that notifiers behave exactly as documented - they only hold their most recent value.  If for some reason you want a structure like that and you need to receive every value, use a queue.  The structure I suggested is not like that at all.

VIs within executables can be made accessible through VI server; the LabVIEW development environment does not need to be installed.

Why do you need to prevent concurrent access to the database?  If your database can't handle multiple simultaneous connections, you might want to choose a different database.  Is all data stored in the database, or only the data from the most recent client connection?  If so, why use a database?

If you can get around the requirement to avoid simultaneous database access, this is much simpler.  The lab computer uploads information to the database, perhaps including some unique identifier.  It then calls a VI on the server, passing in that unique identifier.  That VI uses a queue (or notifier) to trigger the server to get the data and do the processing.  Meanwhile the VI that was called remotely waits on a notifier, which the server will trigger when it finishes generating the new HTML.  When the notifier triggers, the VI finishes, and the lab computer retrieves the new HTML.

[jyang72211]

My system consists of lab computers, server, and database. 

All lab computers are exporting to the same database, and the database contains data for the past months and the data point just generated.  Each time a lab computer generates a data point, it is appended to the database.  For example, when lab compouter A finished appending the data point to the database, lab computer A will call the server to update the HTML, which contains a control chart with time as x-axis and with the latest point.  If another lab computer can access the database before lab computer A displayes the plot, the plot will be corrupted by an extra point that is out of sequence.  

[nathand]

---

jyang72211 wrote:

All lab computers are exporting to the same database, and the database contains data for the past months and the data point just generated.  Each time a lab computer generates a data point, it is appended to the database.  For example, when lab compouter A finished appending the data point to the database, lab computer A will call the server to update the HTML, which contains a control chart with time as x-axis and with the latest point.  If another lab computer can access the database before lab computer A displayes the plot, the plot will be corrupted by an extra point that is out of sequence.  

---

So the steps you listed are not quite enough; there should be additional steps specifying that the lab computer waits until the database is available, obtains a lock on the database, and releases the lock after it has read the updated HTML.  Is there any chance the data is tagged with some identifier - a timestamp, or the lab computer that supplied the data - that the server could use to exclude any data that might have arrived after the current request?

Using the LabVIEW program to lock the database is going to be difficult to do reliably, and you'll probably have to run some VI at regular intervals to determine if the database is available.  You'll need at least one additional VI exported through VI server that acts as the lock; it might be as simple as a lock/unlock functional global with a boolean indicating whether or not the database is available.  When a client wanted to connect to the database it would first run that VI (remotely) in an attempt to lock it until the VI returned an indication that the database was locked succesfully (could be as simple as a boolean, or some more complex solution such as a non-zero random number that would act as a "token" which would be needed to unlock it).  To do this truly right you should also handle timeouts in case the VI server connection dies.

[jyang72211]

I think you gave me some great ideas.  Let me recap the idea that I think is the best for my application and see did I miss anything.

Server will contain a VI that the clients (multiple lab computers) call.  The VI will have the lock/unlock funcction.  If the VI is unlock, a client can lock it and receive a random number as key to unlock the VI later.

1. When the functional global of the VI has a unlock value, a client can set it into a lock value and receive a random number as key for unlocking later.  After the client lock the VI, the server will run with data from that client and all other clients would have to wait.  

2. When the functional global of the VI has the lock value, all clients who want to lock it will poll it every 10s until they can lock it.  Since it is a function global VI on server, only one client can access it at a given time.  

You mentioned that I need to have a timeout case in case the VI server connection dies.  You meant the clients that poll the VI on the server should only poll for a little bit and give up, right?  When you said timeout case, you didn't mean using a event structure, right?  How often would the connection die?  How reliable is it?  By the way, what kind of additional benefit would the random number key gives me?  

Would the approach that I described works?  It is from your idea, but I just want to make sure that I didn't misunderstand you.  Thanks!

[nathand]

---

@jyang72211 wrote:
 

You mentioned that I need to have a timeout case in case the VI server connection dies.  You meant the clients that poll the VI on the server should only poll for a little bit and give up, right?  When you said timeout case, you didn't mean using a event structure, right?  How often would the connection die?  How reliable is it?  By the way, what kind of additional benefit would the random number key gives me?

---

By timeout, I meant the server should reset itself back to the unlocked state if the client doesn't finish its task within a certain period of time, so that one crashed client doesn't prevent all the others from connecting indefinitely.  The clients can poll forever, you don't need a timeout there (although perhaps you would want one at least to alert the user that the software is having difficult connecting to the server).  The connection won't die frequently - perhaps it will never happen - but it's worth planning for that case where a user accidentally bumps the power switch, or Windows crashes unexpectedly while your software is running.

The random number prevents the server from accepting data from a client when it should not.  You can do without it.  Your clients will poll the server to see if its unlocked, and if it is, the first one to request it will lock it.  The random number then prevents the server from accepting data from any other client.  Clients that are running properly won't need that lock, but if you have a timing bug in your code, or someone else is writing a client that doesn't follow your rules, then you might want it.  Also, if you really need to limit writes to the database, the best solution may be to have your server handle all the database connections, don't let the clients connect to it directly (at least for writes).

The theory of your design sounds fine.