trying to read an html file into visual c++ with username & password?

Question

I am trying to read the html contents of a web page into my program (using visual c++). Normally what I do is something like...

CString inputLine;
CInternetSession intSess;
CStdioFile *myInFile = intSess.OpenURL("http://www.xyz.com/");
while(myInFile->ReadString(inputLine) {
// process each line of the html file that is read into inputLine
}
myInFile->Close();

that works okay. But, now what I want to do is access something where I have a username and password. For instance, lets say I want to read the contents of http://answers.yahoo.com/my-activity with my program. That page provides different html code based on your yahoo username & password.

I found CHttpConnect which looks like I can specify a username and password. I tried calling that between constructing the CInternetSession and doing the OpenURL. With it looking like
CHttpConnect myHttp = intSess.GetHttpConnection("www.yahoo.com", 80, "username@yahoo.com", "password");
But that didn't seem to work. I suspect I might need to use the CHttpConnect method OpenRequest, but I can't make heads or tails out of what any of the parameters mean. Can someone who's more knowledgeable about internet programming please provide a snippet of code that does the same thing as my little snippet except works with usernames and passwords?

[as a side question, do these usernames and passwords get encrypted or do they go out on the net as clear text?]

SteffenL · Accepted Answer

I never actually worked much with MFC, but I'll try to help a bit with what I can.
If you don't *have* to use MFC, you may want to look at other frameworks (like Qt).

If I get you right, you want to access content on pages restricted by a username and password. It's not a HTTP basic authentication, so you cannot specify a username/password that way.

In order to sign in, you actually need to form the correct HTTP request to the server and to the right URL.

You have to mimic a real web browser going to the sign-in page and submitting the form.

Made this sample code for you. It's not complete, but hopefully enough to get you started:
http://pastebin.com/t2d1mHUn

Please note that I use regular expressions to parse the page. While it's not recommended, I did it because it's the simplest way I know of without a HTML/DOM parser.

Edit:
Oh yes.. And to answer your side question... It is usually sent as plain text unless the connection is secure (HTTPS). Yahoo Answers also uses HTTPS on pages where sensitive information is exchanged between the client/server, such as on the the sign-in page.

trying to read an html file into visual c++ with username & password?

1 Answer