How to specify user IP in Curl Request ?

Discussion in 'PHP' started by skyview, Apr 13, 2010.

  1. #1
    I am creating a extractor/crawler its almost done, My code is loaded at server with IP xx.xx.xx.xx

    when my crawler goes for crawling records its using server IP, which can lead to block of my IP

    I want when user start crawling from their browser, it should take their IP address yy.yy.yy.yy

    it can be done by CURL or other method ?

    Please advise
     
    skyview, Apr 13, 2010 IP
  2. georgiivanov

    georgiivanov Member

    Messages:
    62
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    25
    #2
    No, it is not possible with server-side crawler which is your case.

    Your best option is to create a list of working proxy servers on your host and set them in random order when crawling.

     
    georgiivanov, Apr 13, 2010 IP
  3. skyview

    skyview Peon

    Messages:
    21
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    will this work curl_setopt($ch, CURLOPT_INTERFACE,"client IP address");
     
    skyview, Apr 13, 2010 IP
  4. skyview

    skyview Peon

    Messages:
    21
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    any free proxy which we can try and test
     
    skyview, Apr 13, 2010 IP
  5. georgiivanov

    georgiivanov Member

    Messages:
    62
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    25
    #5
    Just google for "proxy list" - there are plenty of them. You will also need to write a method that check if the proxy is working or not. For e.g. put a file test.txt on your server and try to fetch it through the proxy :)
     
    georgiivanov, Apr 13, 2010 IP