Curl Php Error 404
Contents |
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site About Us Learn more about Stack Overflow php curl detect 404 the company Business Learn more about hiring developers or posting ads with us Stack php curl check for 404 Overflow Questions Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of curl php error handling 4.7 million programmers, just like you, helping each other. Join them; it only takes a minute: Sign up Easy way to test a URL for 404 in PHP? up vote 115 down vote favorite 37 php iis error 404 I'm teaching myself some basic scraping and I've found that sometimes the URL's that I feed into my code return 404, which gums up all the rest of my code. So I need a test at the top of the code to check if the URL returns 404 or not. This would seem like a pretty straightfoward task, but Google's not giving me any answers. I worry I'm searching for the wrong
Php Http Error 404
stuff. One blog recommended I use this: $valid = @fsockopen($url, 80, $errno, $errstr, 30); and then test to see if $valid if empty or not. But I think the URL that's giving me problems has a redirect on it, so $valid is coming up empty for all values. Or perhaps I'm doing something else wrong. I've also looked into a "head request" but I've yet to find any actual code examples I can play with or try out. Suggestions? And what's this about curl? php http validation http-headers http-status-code-404 share|improve this question edited May 1 '09 at 6:58 bignose 9,92363561 asked Jan 3 '09 at 0:44 bflora add a comment| 13 Answers 13 active oldest votes up vote 202 down vote accepted If you are using PHP's curl bindings, you can check the error code using curl_getinfo as such: $handle = curl_init($url); curl_setopt($handle, CURLOPT_RETURNTRANSFER, TRUE); /* Get the HTML or whatever is linked in $url. */ $response = curl_exec($handle); /* Check for 404 (file not found). */ $httpCode = curl_getinfo($handle, CURLINFO_HTTP_CODE); if($httpCode == 404) { /* Handle 404 here. */ } curl_close($handle); /* Handle $response here. */ share|improve this answer edited Jan 3 '09 at 1:25 answered Jan 3 '09 at 0:56 strager 58.8k1499150 I'm not familiar with cURL yet, so I'm
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site About Us Learn more
Eclipse Php Error 404
about Stack Overflow the company Business Learn more about hiring developers or posting ads apache error 404 with us Stack Overflow Questions Jobs Documentation Tags Users Badges Ask Question Page Not Found This question was removed from Stack html error 404 Overflow for reasons of moderation. Please refer to the help center for possible explanations why a question might be removed. Here are some similar questions that might be relevant: Prevent curl generating fatal error http://stackoverflow.com/questions/408405/easy-way-to-test-a-url-for-404-in-php for larger files php curl returns 400 Bad Request if does in a loop How CURL Login with Captcha and Session POST JSON with PHP cURL PHP curl: Can't get site content - cookies issue? Make a curl request to a url having no file extension? php Curl posting to PHPBB Get data from asp.net page using curl php curl return http_code 404 with content, browser okay cURL not working http://stackoverflow.com/questions/25864496/curl-returned-http-code-404 sometimes and gives empty result Try a Google Search Try searching for similar questions Browse our recent questions Browse our popular tags If you feel something is missing that should be here, contact us. about us tour help blog chat data legal privacy policy work here advertising info mobile contact us feedback Technology Life / Arts Culture / Recreation Science Other Stack Overflow Server Fault Super User Web Applications Ask Ubuntu Webmasters Game Development TeX - LaTeX Programmers Unix & Linux Ask Different (Apple) WordPress Development Geographic Information Systems Electrical Engineering Android Enthusiasts Information Security Database Administrators Drupal Answers SharePoint User Experience Mathematica Salesforce ExpressionEngine® Answers Cryptography Code Review Magento Signal Processing Raspberry Pi Programming Puzzles & Code Golf more (7) Photography Science Fiction & Fantasy Graphic Design Movies & TV Music: Practice & Theory Seasoned Advice (cooking) Home Improvement Personal Finance & Money Academia more (8) English Language & Usage Skeptics Mi Yodeya (Judaism) Travel Christianity English Language Learners Japanese Language Arqade (gaming) Bicycles Role-playing Games Anime & Manga more (18) Mathematics Cross Validated (stats) Theoretical Computer Science Physics MathOverflow Chemistry Biology Computer Science Philosophy more (3) Stack Apps Meta Stack Exchange Area 51 Stack Overflow Careers site design / logo © 2016 Stack Exch
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss http://stackoverflow.com/questions/17476828/curl-returns-404-while-the-page-is-found-in-browser the workings and policies of this site About Us Learn more about Stack Overflow the company Business Learn more about hiring developers or posting ads with us Stack Overflow http://www.heckdesigns.com/tutorials/tutorial-check-for-a-404-with-php/ Questions Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of 4.7 million programmers, just like you, helping each error 404 other. Join them; it only takes a minute: Sign up cURL returns 404 while the page is found in browser up vote 1 down vote favorite there is already similar questions on stackoverflow, but none of their solutions have been working for me. I'm trying to grab a page on LoveIt.com with cURL, but it returns me a 404 curl php error error, while the url works fine in the browser : $url = 'http://loveit.com/loves/P0D1jlFaIOzzZfZqj_bY3KV'; $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"); curl_setopt ($curl, CURLOPT_HEADER, false); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true); curl_setopt($curl, CURLOPT_REFERER,'http://loveit.com/'); Here's the header I receive : Array ( [url] => http://loveit.com/loves/P0D1jlFaIOzzZfZqj_bY3KV [content_type] => text/html; charset=utf-8 [http_code] => 404 [header_size] => 667 [request_size] => 172 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 0.320466 [namelookup_time] => 0.000326 [connect_time] => 0.119046 [pretransfer_time] => 0.119089 [size_upload] => 0 [size_download] => 499 [speed_download] => 1557 [speed_upload] => 0 [download_content_length] => 499 [upload_content_length] => 0 [starttransfer_time] => 0.320438 [redirect_time] => 0 [certinfo] => Array ( ) [primary_ip] => --- [primary_port] => 80 [local_ip] => --- [local_port] => 53837 [redirect_url] => ) I read that some website had protections against this kind of scripts; and I did test some solutions proposed, but none worked for me (CURLOPT_USERAGENT,CURLOPT_REFERER...) Any ideas of what's happening here ? I would like to backup my LoveIt account, that's why i'm
this will allow you to check if the page you are linking to or trying to get information from is a valid page. Here is the code before I break it down: In my code I am simply checking to see if my twitter page is a valid URL. If you were to add an extra character anywhere in "heckdesigns" such as an extra "s" at the end then you would trigger the invalid URL message. Now lets see how that is done: $ch = curl_init('http://twitter.com/heckdesigns'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); What this is doing is initializing http://twitter.com/heckdesigns and then it is setting cURL option CURLOPT_RETURNTRANSFER to true. This is simply taking all the information sent by the URL (such as HTTP codes) and converting it into a text string we can pull information from later on in the code. To read up on what options you can add to curl_setopt() you can visit php.net. Lets take a look at that line that will execute the options we've set: curl_exec($ch); This is simply executing the cURL that was initialized by curl_init, again instead of returning the URL this will instead return it as a string. Now that we have the information from the URL lets look at the code that will actually interpret that string and see if there is a 404 error in it. if(curl_getinfo($ch, CURLINFO_HTTP_CODE) == 404) { echo 'not a valid url'; }else { echo 'valid url'; } This is getting the information from the $ch string, specifically it is looking for the CURLINFO_HTTP_CODE and checking if it is 404. If it is then it simply returns that is it not a valid URL otherwise it returns that it is a valid URL. If you would like to learn about what other options o