Lookout

cancel
Showing results for 
Search instead for 
Did you mean: 

In Lookout 6.5, new Windows 7 Computer running server process file stops allowing clients to connect over time

Ryan,

 

Unfortunately, a few hours after my last post my issue returned. However, unlike before the Lookout services are not crashing - in fact, my problem is similar to the same problem I had approximately 7 years ago >> http://forums.ni.com/t5/Lookout/Lookout-5-1-quits-serving-data-to-clients/td-p/102593

 

I did not want to post on the forum until I could pinpoint exactly what happens and make sure the same thing is happening over and over again. I already reinstalled Lookout 6.5 on my server, reactivated Lookout 6.5 on my server, recompiled my server file, and wiped out my database on my server. Windows firewall is disabled on my server and my clients. I can telnet to port 2343 before and after the problem occurs from the server to the client and vice versa.

 

Here is exactly what is happening: (this can happen 15 minutes or up to 45 hrs. after my computer running my server process has been rebooted)

 

1.) Lose Trends on clients (nicitdl.exe still shows running in task manager and resource monitor), but Lookout is still updating between client and server and I still can access server from client running development through symbolic link and network in object explorer. And, the client file running on the same computer as the server file behaves normally. At this point, I can reopen Lookout server file after stopping nictdl5 and everything works fine again.

 

2.) After some time without trends, I am unable to access server from client running development through symbolic link and network in object explorer. No red X's appear on the client machine that was running when trends were lost. However, if I open client process on a machine that wasn't running when the problem occurred I will see red x's everywhere. Also, on the server file I can still see clients listed in the Client Connection Monitor. And, the client file running on the same computer as the server file behaves normally. At this point, I can reopen Lookout server file after stopping nicitdl5.exe and everything works fine again.

 

3.) After some time without trends and being unable to access server from client running development through symbolic link and network in object explorer, my client connection monitor list goes blank. I then get red x's on all my clients. And, the client file running on the same computer as the server file makes lookout hang if I try to access any screen. At this point, if I reopen Lookout server file without stopping lkads.exe, Lookout will momentarily hang at splash screen then show "loading file" screen, and then "process registration failed" box will pop up. I have to close Lookout, stop lkads.exe and nicitdl5.exe, then reopen Lookout and everything works fine until the problem occurs again.

 

Through this entire process I do not see any evidence of the lkads or nicitdl5 service crashing in event viewer on the server computer or clients, I can telnet into port 2343 as I mentioned before, and the computer running my server process is still receiving data from the plcs in the field.

 

I have uploaded my server process and one of my client process files to the ftp site in the incoming folder under JPSCADA_jphillips folder. There is also cbx file you will need to open the process files.

 

The computer I am running my server process file on is a Dell Precision T3500 CMT, 32-bit Intel Xeon Six Core Processor,X5650,2.67GHz, 4Mb RAM.

 

Please see if you can duplicate this problem as soon as possible. If you need me to ship you my computer as was done in the past to solve the problem, please let me know.

 

Thank you for a quick response.

 

 

Jason Phillips
0 Kudos
Message 11 of 21
(3,409 Views)

Jason,

 

I see two monitor objects. You have two server, primary1 and primary2. The symbolic Link switch between two servers according to the monitor's status.

Are there two server processes running at the same time? Can you check the value of monitor object when you see the problem? I'm thinking if it's possible that the symbolic link switches to the other server which may not have the trend and data.

 

I also suggest you to use tool, such as wireshark, to capture the network data on client.

Ryan Shi
National Instruments
0 Kudos
Message 12 of 21
(3,404 Views)

Ryan,

 

I have a server (PRIMARY1) and a backup server (PRIMARY2). PRIMARY2 runs a standby file, and loads the server file if the Lookout panel it's looking at on PRIMARY1 is inactive for more than 90 seconds. However, since this problem has been occuring, I am not running the standby file on PRIMARY2 - so basically, I do not have a backup server right now. And no, two server processes are not running at the same time.

 

When I see the problem described below (taken from my earlier post), my clientstandbymonitor object goes from "off" to "on" as it should. Also, attached is the wireshark file after the problem below occurs:

 

3.) After some time without trends and being unable to access server from client running development through symbolic link and network in object explorer, my client connection monitor list goes blank. I then get red x's on all my clients. And, the client file running on the same computer as the server file makes lookout hang if I try to access any screen. At this point, if I reopen Lookout server file without stopping lkads.exe, Lookout will momentarily hang at splash screen then show "loading file" screen, and then "process registration failed" box will pop up. I have to close Lookout, stop lkads.exe and nicitdl5.exe, then reopen Lookout and everything works fine until the problem occurs again.

 

For issues 1 & 2 from my previous post, my clientstandbymonitor object stays "off" as it should because at that point I still have connections showing in my client connection manager. Clientstanbymonitor2 stays on all the time as it should because I don't have the standby file running.

 

In case I wasn't clear in my previous post, problems 1, 2, & 3 occur sequentially - NOT separately.

Jason Phillips
0 Kudos
Message 13 of 21
(3,395 Views)

I'm trying to read the wireshark file to figure out anything unexpected.

What's the ip for client and servers?

 

As you don't have the standby server running, can you set the static link for the symbolic link object intead of dynamic expression? I don't want it to connect to a non-existing server when the problem occurs.

Ryan Shi
National Instruments
0 Kudos
Message 14 of 21
(3,391 Views)

The server is 10.4.38.10. Some clients in the wireshark file I sent you would be

 

10.4.4.170

10.4.4.137 (this is my computer in my office which I used to connect to 10.4.38.10 remotely to generate the wireshark file)

10.4.51.30

10.4.60.22

10.4.169.21

10.4.171.21

 

I have anywhere from 40-50 clients running all the time, and a total of ~150 client machines total, so changing that symbolic link to static and updating every client would take some time. Besides, when everything was working fine with my old XP server machine, I wasn't running the standby file on the secondary machine for some time and never had any issue like this.

Jason Phillips
0 Kudos
Message 15 of 21
(3,388 Views)

Do you see this problem on most of clients or just one or two client?

 

By the way, do you use the logos workaround now?

Ryan Shi
National Instruments
0 Kudos
Message 16 of 21
(3,385 Views)

If you can set up a client for me to logon, it will be helpful.

Ryan Shi
National Instruments
0 Kudos
Message 17 of 21
(3,383 Views)
This happens on all of the clients as far as I know. I am relying on my client users to call me when this happens, so I get calls from different users whenever it occurs.

As for the logos workaround, are you referring to disabling logosxt? If so, yes. Already done.
Jason Phillips
0 Kudos
Message 18 of 21
(3,380 Views)
Ryan, can you email me at jphillips@jeffparish.net to let me know what you need to log in to one of my clients?
Jason Phillips
0 Kudos
Message 19 of 21
(3,379 Views)
Is
Jason Phillips
0 Kudos
Message 20 of 21
(3,327 Views)