Maybe you should consider using 'requests'. It's a python library, I use it and it worked very well where I was earlier unsuccessful with urllib and urllib2.
http://docs.python-requests.org/en/latest/
Check this out. We can then continue the discussion, I have provided you with a simple sample below and you could then just try to logon and capture the pages starting from a small python script (before fixing the plugin).
Basically, what I did, I installed the library with the setup. Then I just copied the subdirectory 'requests' and include that as a subdirectory to my plugin. It might be possible to install this one time in EG somehow but for the time, it will be included in the plugin distro.
Below is the sample python script that you could elaborate with. For convenience I have also attached the 'request' sub that you can start with if you want. To use with python scripts in EG, just unzip it in the main EG folder (to use with a plugin, unzip it in the actual plugin folder as a sub).
Best regards, Walter
Code: Select all
from requests import session
username = 'myself'
password = 'passw'
theUrl = (
'https://mypages.xxx.com/xxx_security_check?locale=xz_YW&j_username='
+username
+'&j_password='
+password
+'&xxx-security-redirect=%2Fse%2Fstart.html'
)
s = session()
#to logon
r = s.post(theUrl)
print r.text
#if you are looking for a specific page section
start = r.text.find('start keyword')
end = r.text.find('end keyword',start)
txt_section = r.text[start:end]
print txt_section
#other good methods
#get a page
#thePage = s.get('https://mypages.xxx.com/news')
#some status infos and other useful stuff
#print thePage.info()
#print thePage.text
#print thePage.headers['content-type']
#print thePage.headers
#print thePage.status_code
#print thePage.encoding