Python Urllib2.httperror Http Error 403 Forbidden
Contents |
here for a quick overview of the site Help Center Detailed answers to any questions you python requests 403 forbidden might have Meta Discuss the workings and policies of this site python requests 403 error About Us Learn more about Stack Overflow the company Business Learn more about hiring developers or yolk urllib2.httperror: http error 403: must access using https instead of http posting ads with us Stack Overflow Questions Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of
Raise Httperror(req.full_url, Code, Msg, Hdrs, Fp) Urllib.error.httperror: Http Error 403: Forbidden
6.2 million programmers, just like you, helping each other. Join them; it only takes a minute: Sign up urllib2.HTTPError: HTTP Error 403: Forbidden up vote 41 down vote favorite 25 I am trying to automate download of historic stock data using python. The URL I am trying to open responds with a CSV file, urllib2 user agent but I am unable to open using urllib2. I have tried changing user agent as specified in few questions earlier, I even tried to accept response cookies, with no luck. Can you please help. Note: The same method works for yahoo Finance. Code: import urllib2,cookielib site= "http://www.nseindia.com/live_market/dynaContent/live_watch/get_quote/getHistoricalData.jsp?symbol=JPASSOCIAT&fromDate=1-JAN-2012&toDate=1-AUG-2012&datePeriod=unselected&hiddDwnld=true" hdr = {'User-Agent':'Mozilla/5.0'} req = urllib2.Request(site,headers=hdr) page = urllib2.urlopen(req) Error File "C:\Python27\lib\urllib2.py", line 527, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 403: Forbidden Thanks for your assistance python http urllib share|improve this question edited Nov 9 '12 at 7:14 Sudar 5,407124790 asked Nov 9 '12 at 6:51 kumar 8491810 Are you use windows as platform ? –Denis Nov 9 '12 at 7:08 add a comment| 2 Answers 2 active oldest votes up vote 77 down vote accepted By adding a few more headers I was able to get the data: import urllib2,cookielib site= "http://www.nseindia.com/live_market/dynaContent/live_watch/get_quote/getHistoricalData.jsp?symbol=JPASSOCIAT&fromDate=1-JAN-2012&toDate=1-AUG-2012&datePeriod=unselected&hiddDwnld=true" hdr = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Sa
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the python urllib headers workings and policies of this site About Us Learn more about Stack
Python User-agent
Overflow the company Business Learn more about hiring developers or posting ads with us Stack Overflow Questions
Python Requests User Agent
Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of 6.2 million programmers, just like you, helping each other. Join http://stackoverflow.com/questions/13303449/urllib2-httperror-http-error-403-forbidden them; it only takes a minute: Sign up HTTPError: HTTP Error 403: Forbidden up vote 7 down vote favorite 4 I making a python script for personal use but it's not working for wikipedia... This work: import urllib2, sys from bs4 import BeautifulSoup site = "http://youtube.com" page = urllib2.urlopen(site) soup = BeautifulSoup(page) print soup This not work: import urllib2, http://stackoverflow.com/questions/13055208/httperror-http-error-403-forbidden sys from bs4 import BeautifulSoup site= "http://en.wikipedia.org/wiki/StackOverflow" page = urllib2.urlopen(site) soup = BeautifulSoup(page) print soup This is the error: Traceback (most recent call last): File "C:\Python27\wiki.py", line 5, in
the following standard way. import urllib2 page=urllib2.urlopen(url) content=page.read() And i was into a http://www.myudayippuapp.appspot.com/?p=12003 small hobby project, it is all about fetching images from comic websites. My absolute favorite when it comes to comics is C&H. The above method worked for most of the websites, all except www.gocomics.com/calvinandhobbes.I got an error like this File "/usr/lib/python2.7/urllib2.py", line 518, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: http error HTTP Error 403: Forbidden The problem was that some servers may block requests from programs like mine, considering it to be coming from a bot with a malicious intentions. The only way to fix this will be to fool the server that the request is coming from a webbrowser. After all http error 403 even browsers are merely sending GET and POST requests, so the key to solving this problem is that extra parameters need to be added along with get requests. And this is how i did it eventually. Solution: import urllib2 url = "http://www.gocomics.com/calvinandhobbes" req = urllib2.Request(url, headers={'User-Agent' : "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.30 (KHTML, like Gecko) Ubuntu/11.04 Chromium/12.0.742.112 Chrome/12.0.742.112 Safari/534.30"}) con = urllib2.urlopen(req) print con.read() Note to Self : Should ask more doubts to Rahul Relate Posts: Beehives problem Re-visiting AWMRS My Sudoku Checking Program Tags: forbidden error python server problems calvin and hobbes Posted in useful Python | 评论关闭 Recent Comments LocalCashNow:Installment loans pr... riffcliff.mymagicspa:Fascinating blog! Is... Weight Loss Offers:I was suggested this... Encante Serum:If some one needs ex... search engine ranker:Spot on with this wr... Categories useful(14) Life, Life and nothing else(19) Python(12) great ideas(8) Movies, books and other great art(8) Awesomness(14) science(5) We are all stories in the end.(10) games(2) Matt