I am trying to download tweets from a specific account using the code below (python, package tweepy)
import tweepy
import pandas as pd
import time
# Credenziali del 08/07/2024 Pagamento personale
consumer_key = "consumer key >"
consumer_secret = "consumer secret>"
access_token = "access token "
access_secret = "access secret "
bearer_token = "bearer token "
#AUT with client (V2)
client = tweepy.Client(bearer_token=bearer_token, consumer_key=consumer_key, consumer_secret=consumer_secret, access_token=access_token, access_token_secret=access_secret,wait_on_rate_limit=True) # add wait_on_rate_limit
# search tweets produced by a specific user relying on user ID
end_time="2024-06-22T23:59:01Z"
start_time="2024-06-10T23:59:01Z"
user_id="insert user id here"
result = []
tweets = [] # Define containers to use in the loop defined below
for response in tweepy.Paginator(client.get_users_tweets,
user_id=user_id,
end_time=end_time,
tweet_fields = ['id','referenced_tweets','text','context_annotations','created_at','geo','author_id','lang','public_metrics'],
user_fields =['id','name','username','url','public_metrics'],
start_time= start_time):
for tweet in response.data:
tweets.append(tweet)
result.append({'id': tweet.id,
'author_id': tweet.author_id,
'lang': tweet.lang,
'text': tweet.text,
'created_at': tweet.created_at,
'retweets': tweet.public_metrics['retweet_count'],
'replies': tweet.public_metrics['reply_count'],
'likes': tweet.public_metrics['like_count'],
'quote_count': tweet.public_metrics['quote_count'],
'referenced_tweets': tweet.referenced_tweets,
'context_annotations': tweet.context_annotations,
'tweet_url': f"https://twitter.com/{tweet.author_id}/status/{tweet.id}"
})
time.sleep(1) # introduce a delay of 1 second between each request
df = pd.DataFrame(result)
df.to_csv("mydf.csv", index=False)
the script works as long as the number of requests becomes higher. I have added a time.sleep(1)
so to add an extra second pause. However, I keep receiving the same message Rate limit exceeded. Wait for xx seconds
. I really do not know why, I have tried searching in many repo but I still not get what I am doing wrong.
Any suggestion is more than welcome!
As mentioned I have searched in some repo and the suggestion was to add a time.sleep() to the for loop. Even increasing the time.sleep(2) the result is always the same. I keep receiving the same sleep message mentioned above.
Bene_ma_non_Benissimo is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.