1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Downloading all tweets from 5k twitter users into textfile?

Discussion in 'PHP' started by zach catskilson, Feb 28, 2017.

0
  1. #1
    I need to download ALL tweets from my list of 5k twitter users (I have their ID's + screenname) into textfile/s in order to later search through all the content to find specific data.

    One of ideas was creating a PHP curl script that will grab data off website called allmytweets(dot)net which basically displays all tweets published by user.
    The main problem is that their page is divided into pages, you need to scroll down for it to show the next page which could cause problems for the curl script.

    Is there any other simpler/more effective way of doing this?
     
    zach catskilson, Feb 28, 2017 IP
  2. sarahk

    sarahk iTamer Staff

    Messages:
    28,500
    Likes Received:
    4,460
    Best Answers:
    123
    Trophy Points:
    665
    #2
    If you know who they are you can access their feeds one by one and download that. Sounds painful but when you do it from a script it's not so bad.
     
    sarahk, Feb 28, 2017 IP
  3. eritrea1

    eritrea1 Active Member

    Messages:
    182
    Likes Received:
    9
    Best Answers:
    2
    Trophy Points:
    70
    #3
    As someone who has done similar things in the past (not the same site) this is very doable, but you better use the API, or inspect the element and see what happens when you scroll down the page. It is most likely sending a request with numbers like getTweet/?page=... from that you can fetch more
     
    eritrea1, Mar 3, 2017 IP