Router Web Page Authentication

[ChristopherS]

Sorry to be bringing this up again, it looks like it's been addressed a few times in the last couple years, but I've read these multiple times and attempted to apply the lessons they teach and have not had success.

http://forums.ni.com/t5/LabVIEW/HTTP-Connection-with-Password/m-p/1146346#M504441

http://forums.ni.com/t5/LabVIEW/Read-data-of-embedded-web-page/m-p/1243792#M525335

http://forums.ni.com/t5/LabVIEW/Use-LabVIEW-to-login-username-and-password-into-device-IP/m-p/651832...

and of course http://en.wikipedia.org/wiki/Basic_access_authentication

What I'm trying to achieve here is use LabVIEW 2010 to rip text from multiple pages of a local router via WiFi, parse the text, and make decisions based on the values retrieved.  Unfortunately some of the information I need requires a password be entered (just a password, "admin") before being granted access to to the pages I want.  Pretty typical for routers though I think.

Ripping the text and parsing the strings is not a problem (though it can be a drag), I'm having difficulty with logging in to the router with LabVIEW.  As I said the password is "admin" and unfortunately I have to assume that the username is also "admin". 

This is the page I'm trying to get to after logging in: http://192.168.0.1/user/advanset.html?sid=824643 (I'm not trying to post a link for anyone to click on, just showing the URL)

So using MikeS81's WebPageAuthentication_LV85.vi (found here: http://forums.ni.com/t5/LabVIEW/Read-data-of-embedded-web-page/m-p/1251826#M527035) I can get the source html, but not the info from the page.

Therefore I'm trying to use MikeS81's WebPageAuthentication_LV85.vi in conjunction with the Rip Web Text.vi (sorry, can't seem to find a link to it right now, but I did get it form the NI LabVIEW forums).  When Rip Web Text is incorporated the browser returns

"The website declined to show this webpage
 HTTP 403
  Most likely causes:
This website requires you to log in."

I don't know a whole lot about web pages or TCPIP or HTML but I'm assuming that "sid=" in the URL means I'm being granted temporary access via some kind unique ID that expires, a cookie or something.  The VI is attached to this post, please let me know if you have any thoughts on it.  I've changed the indicators to use the output I received as the default values (so you should be able to see what I've gotten back in the VI).  It seems to me that I should be trying to rip the text before closing the TCP connection opened for authentication, if this is incorrect please let me know.

I know it may seem that I'm Frankensteining some serious crap together here, hope this doesn't end up in the Rube Goldberg thread... but I'll understand if it does. 

Thanks, and have a great weekend.

[Z'atari]

Hey Christopher,

I tried opening your VI but it has some kind of conversion subVI that I couldn't find.

Regards,

A. Zaatari

[ChristopherS]

Thank you for responding.

I'm sorry, it is missing the convert to base 64 utility found here: http://decibel.ni.com/content/docs/DOC-5544.  I haven't quite gotten the hang of how to include the sub VIs when uploading.

I've continued to work on this and have not had any luck.  I think I may have to use the HTTP or TCP pallete, which I've been trying to find an excuse to learn anyway.  Any thoughts or ideas would be greatly appreciated.

Thanks.

[nathand]

Could you save your code to 2009?  I'd be happy to look at it, but I don't have 2010.  It does sound like you have two separate pieces of code both establishing separate connections to the server - one that can authenticate, the other that can't authenticate but can get you the data you need - and there may not be a way to share the connection between them, in which case you'd need to take the relevant parts of one and add them to the other.  I do not think you want to use the TCP palette as that will involve you writing code to handle all the authentication yourself, and that may not be simple.

[charris]

Christopher:

I ran across this page once, and if you have Firefox, it might be useful. It uses an extension to intercept the POST data and headers when submitting a form (which I assume is happening when you submit the password on the router).

Here's a link to the article (please note that this is on a non-NI site, and this is in no way officially supported   )

http://ssscripting.wordpress.com/2009/03/17/how-to-submit-a-form-programmatically/

Hope that helps out a little...

[charris]

Just a sidenote on Tamper Data: you can use it to compare the GET and POST headers to see if they differ at all. I suspect you can just feed the GET header back into the POST function. For formatting, just use whatever formatting the GET function returns for the header in LabVIEW.

The tricky part will be figuring out the POST data. If you post up a screen shot or capture from Tamper Data, hopefully we can figure out how to format the data input.

Overall, I think the process will just be a matter of replicating the GET and POST steps that you capture and translating them to requests in LabVIEW.

[ChristopherS]

Attached is the tamper data output we talked about, there is a LOT of information there.  XML and RTF versions are attached here.

The first few entries are from google.com, I then navigated to the router's web page, then logged in, then logged out.

I need to look closer at this.

[ChristopherS]

This should be the response from me logging in, the cookie is populated

11:17:13.097[113ms][total 113ms] Status: 200[OK]

GET http://192.168.0.1/images/adminloggedinbig.png Load Flags[LOAD_NORMAL] Content Size[-1] Mime Type[image/png]

   Request Headers:

      Host[192.168.0.1]

      User-Agent[Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13]

      Accept[image/png,image/*;q=0.8,*/*;q=0.5]

      Accept-Language[en-us,en;q=0.5]

      Accept-Encoding[gzip,deflate]

      Accept-Charset[ISO-8859-1,utf-8;q=0.7,*;q=0.7]

      Keep-Alive[115]

      Connection[keep-alive]

      Referer[http://192.168.0.1/style/style.css]

      Cookie[ddddd=493002; dddddddd=493002; lct_remember_me=; mac_remember=; lct_psw=; checked=]

   Response Headers:

      Content-Type[image/png]

      Date[Tue, 17 Mar 2009 15:38:22 GMT]

      Connection[close]

So is this the header information we want?

      Accept[image/png,image/*;q=0.8,*/*;q=0.5] 

      Accept-Language[en-us,en;q=0.5]

      Accept-Encoding[gzip,deflate]

      Accept-Charset[ISO-8859-1,utf-8;q=0.7,*;q=0.7]

      Keep-Alive[115]

      Connection[keep-alive]

      Referer[http://192.168.0.1/style/style.css]

      Cookie[ddddd=493002; dddddddd=493002; lct_remember_me=; mac_remember=; lct_psw=; checked=]

[ChristopherS]

Using HTTP GET VI the header is:

HTTP/1.0 200 OK
Content-type: text/html
Date: Wed, 18 Mar 2009 13:45:53 GMT
Connection: close
Pragma: no-cache
Cache-Control: no-cache, max-age=0
Expires: -1

 The next step would be to retrieve the POST header, so I opened Wireshark and found the POST entry from logging in:

POST /ajax_request ?Endpassword=admin HTTP/1.1

And another from logging out:

POST /ajax_session ?sid=logout HTTP/1.1

So does this mean that the POST header in it's entirety is "HTTP/1.1"?  I also picked through the source code for the page and found that the username is in fact "“adminâ€" and it doesn't look there is any base64 conversion necessary.

When I use these lines with the HTTP GET VI though the body returned is:

<HEAD><TITLE>501 Not Implemented</TITLE></HEAD>
<BODY><H1>501 Not Implemented</H1>
The requested method is not recognized by this server.
</BODY>

Which tells me something about the way I submitting the POST is incorrect.  I've attached the VI I'm working with now and really wish there were some LabVIEW examples for the HTTP pallette.

If it seems like I'm learning this as I go... it's because I am.  😜

[charris]

Christopher:

A couple of thoughts regarding your POST VI:

The 501 error makes it sound like (and I'm just guessing here) we have a header parameter off. LabVIEW is probably just assuming a set of defaults or not even specifying the header, whereas your browser is probably turning around and re-using the headers it got from the GET requests.

I think it might be worth manually inserting the header data (you can get it from the tamperdata capture) line by line into the request (see screen shot).

Here's what looks like an/the important POST:

POST http://192.168.0.1/ajax_request?Endpassword=admin Load Flags[LOAD_BYPASS_CACHE  LOAD_BACKGROUND  ] Content Size[-1] Mime Type[text/html]
   Request Headers:
      Host[192.168.0.1]
      User-Agent[Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13]
      Accept[*/*]
      Accept-Language[en-us,en;q=0.5]
      Accept-Encoding[gzip,deflate]
      Accept-Charset[ISO-8859-1,utf-8;q=0.7,*;q=0.7]
      Keep-Alive[115]
      Connection[keep-alive]
      X-Requested-With[XMLHttpRequest]
      Referer[http://192.168.0.1/]
      Cookie[ddddd=; dddddddd=; lct_remember_me=; mac_remember=; lct_psw=; checked=]
      Pragma[no-cache]
      Cache-Control[no-cache]
      Content-Length[0]
   Response Headers:
      Content-Type[text/html]
      Date[Tue, 17 Mar 2009 15:38:22 GMT]
      Connection[close]

Here's the beginning of how we'd put in the header data:

Also, in your VI, it looks like there's a space in the request data: /ajax_request ?Endpassword=admin.

Try changing it to /ajax_request?Endpassword=admin.

The issue may just be that space, so try that first...