Reading HTML Code From a Web Site with login

[NTampelloni]

Hi, I need your help to build a VI that reads the HTML content of a web page.

The web page you want to read is protected by username and password ,
so first you must to log with your username and password and then read the HTML content .

I tried with DataSocket , but I do not understand where and when to enter your login credentials
to login and read the content of the page .

Can you help me??

THANK YOU

[NTampelloni]

SOS

[tyk007]

Have you tried using the HTTPClient VIs instead of DataSocket? This may be a better fit.

Otherwise attaching you current attempt might be helpful.

[NTampelloni]

I 've tried to also use HTTPClient VIs , no results. 😞
Below my VIs , where am I wrong ?
THANK YOU

[Sam_Sharp]

It won't be easy, but here are your options:

1) You can try to 'spoof' the login by POSTing the login form details to the servers login handler script - most websites try to protect against this sort of thing. You need to maintain the session between calls to keep the session alive.

Also, you might want to try turning off 'verify server' on the configure SSL VI - the certificate details you provide should be your own and unless you update your local certificate bundle, the verify server option will fail and it isn't really needed unless you want to be ultra-secure.

2) Use the web browser control and use the activeX scripting to try and login 'as a user would'

3) Find out if they have an API you can access (e.g. JSON / XML) to retrieve the information you need.

Option 3) is the only reliable solution as websites try to prevent against automatic data scraping from unknown sources and the form could change at any time and your code would break.

Also - you say 'no result' - but what do you mean? Do you get an Error on your Error Out? Do you receive anything in the HTTP Headers/Body?

[NTampelloni]

Ok, thanks for the reply.

For now we reason on point 1.

The problem is that I do not know how to pass credentials with the HTTP POST to login .

can you help me?

This is the html code of your site .

[Sam_Sharp]

So the login box is loaded into an iframe - you can see the long URL about half-way down your screenshot. That should be your request URL (of course, it looks like they include some other parameters which might change on every login - like the random number).

You then need to find all of the form inputs and replicate them in your POST data you send from LV.

Also - if their login uses AJAX/Javascript to POST the form data - you'll need to delve into the javascript to see what/how (e.g. URL, fields) the information is POSTed.

[NTampelloni]

so does not fit the root URL that I used as input in the VI HTTP POST ?
I use the URL found after " src=" the " iframe id " tag ?

Tanks

[Sam_Sharp]

You would normally use the URL/path specified in the <form> tag - but since it is missing, it submits the form to itself (which is the html default) - which is the URL loaded into the <iframe> tag.

[NTampelloni]

Ok , I entered as the URL of the iframe src tag but still does not work .
To set your login credentials , I put the string in the buffer of the " VI " HTTPClient : POST

loginId=myusername&password=mypassword

it's correct?
Thanks for your patience 😉