How I made a Twitter Bot with Face Detection and so can you!

So I started researching on how to make a Twitter Bot and decided to make three features from a simple “Hello World!” message, to telling users information about themselves from their public profile, all the way to implementing face detection in Tweeted photos!

If you’d just like to have a look at the code, you can find the GitHub repo here. Or if you’d like to check out the bot for yourself, click here! :D

So, how do you make a Twitter Bot?

The Twitter Bot I have made uses the Twitter API 2.0, the Tweepy Python library, and the cv2 Python library by OpenCV for face detection.

Step 1: Get a Twitter Developer Account

First, you have to decide whether you want the bot to post for you or from a different account altogether. If you want the former, you can go directly to Twitter’s developer website and apply for a developer account from your account. If you’d like the bot to post from a separate account, create a new Twitter account on the main website (like how you’d make one normally), and while signed into that account, apply for a developer account on Twitter’s developer website. The application process was pretty easy, and there wasn’t any waiting time for me.

Twitter Developers banner (Source: Twitter)

Step 2: Creating a Twitter Project and App

Once you’ve verified your email address, you can click on the “Create Project” button. While doing this, you’ll get an API key, an API secret key, and a Bearer token. Store these in a safe place — you’ll need them later.

Now, you can go to your App’s settings page and go down to the “User authentication settings” section and click on “Set up” to set up Oauth 2.0 and 1.0a. Select the permissions you’d like. Example: I wanted my bot to read as well as write (post) tweets, so I chose that option. You’ll also have to give a callback URI and a website URL.

Depending on your use case, you might also need to apply for Elevated Access to allow your bot to post tweets. Once again, the application process was super smooth, and I got approved almost instantly.

Step 3: Coding the Bot

“Hello World!”

This is where the fun part begins. To start, import the tweepy library. You’ll first need to create an object of the Tweepy API class. This object basically acts like a connection to Twitter and through it you can post Tweets, retrieve Tweets, reply to others and more, but right now we are going to get it to post a “Hello World!” Tweet for us. Something like this -

Creating the API object requires you to pass in your Consumer Key, Consumer Secret Key, Access Key and the Access Secret (remember those?) as variables — I used environment variables but you can just type them directly in if you’re sure you will not make your bot script public — and using those run the OAuthHandler method. The OAuthHandler gives an auth object which we can provide our access credentials to, and create our API object. Here we have also used the verify_credentials() method to make sure our connection was successful.

import tweepy
def createApi():
    '''Function to connect and authenticate to the Twitter API'''
    # Getting the env variables
    CLIENT_ID = os.getenv("CLIENT_ID")
    CLIENT_SECRET = os.getenv("CONSUMER_KEY")
    CONSUMER_KEY = os.getenv("CONSUMER_KEY")
    CONSUMER_SECRET = os.getenv("CONSUMER_SECRET")
    ACCESS_KEY = os.getenv("ACCESS_KEY")
    ACCESS_SECRET= os.getenv("ACCESS_SECRET")     # Authorizing
    auth = tweepy.OAuthHandler(CONSUMER_KEY,CONSUMER_SECRET)
    auth.set_access_token(ACCESS_KEY,ACCESS_SECRET)
    api = tweepy.API(auth, wait_on_rate_limit=True)
    # Checking the connection
    try:
        api.verify_credentials()
        print("Authorized!")
    except:
        print("An Error occurred during authentication")
    return api

Tweets in the Tweepy library are called statuses (from old twitter lingo). So to post our little “Hello World” message, we use -

api.update_status("Hello World!")

Congratulations! Your bot just posted its’ first Tweet. 😎

Replying to users

Before we get into learning how to reply to users, let’s have a look at the following code:

# Grabbing the mentions timeline
timeline = api.mentions_timeline()
for tweet in timeline:
    print(f"Tweet ID: {tweet.id} - {tweet.user.name} said{ tweet.text}")

This little code snippet shows us all account mentions. We first call the mentions_timeline() method to get a list of the Tweets in your mentions timeline and then use a for..in.. loop to iterate and print through the Tweets (and their properties like ID, the Tweeter’s name, and the text of the Tweet itself).

Replying to users is pretty easy if you know their Tweet ID, which we can find using the above method. We can add a reply to an existing Tweet as follows-

api.update_status(reply_tweet_message, in_reply_to_status_id = tweet.id)

Where reply_tweet_message is your reply Tweet text as a string and tweet.id is a Tweet ID.

We’re almost ready to start replying (or spamming) “Hello World!” to your friends but there’s one tiny, little problem — We need a way to store the Tweet ID we last replied to, otherwise our bot would reply to everyone whom it’s already replied to all over again every time we run the bot’s script. We also need to retrieve this Tweet ID every time we start so that our bot knows which Tweets it has to actually reply to.

We can do this using a simple text file named last_seen_id.txt and the following two basic functions-

def getLastSeenID(file_name):
    fobj = open(file_name, 'r')
    last_seen_id = int(fobj.read().strip())
    fobj.close()
    return last_seen_iddef storeLastSeenID(last_seen_id, file_name):
    fobj = open(file_name, 'w')
    fobj.write(str(last_seen_id))
    fobj.close()
    return

Now we can finally put all this together. We can now add an if condition to check for certain keywords when the bot is mentioned/tagged.

Here I’ve checked for “#HelloWorld” in any capitalization. Also, let’s just wrap this up in a function called replyToTweets. I’ll explain why soon.

def replyToTweets():
    # Getting the last_seen_ID
    last_seen_id = getLastSeenID("last_seen_id.txt")     # Grabbing the mentions timeline
    timeline = api.mentions_timeline(since_id = last_seen_id, tweet_mode='extended') # Extended to allow for a tweet's full_text
    for tweet in reversed(timeline):
          print(f"Tweet ID: {tweet.id} - {tweet.user.name} said {tweet.full_text}")
          storeLastSeenID(tweet.id, "last_seen_id.txt")
          if '#helloworld' in tweet.full_text.lower():
              reply_tweet = " #HelloWorld back to you!! " + "@" + tweet.user.screen_name
              api.update_status(reply_tweet, in_reply_to_status_id = tweet.id)
              print("A Hello World Response Has Been Sent!")

The reason we made this as a function was so that we can call it in the following manner along with a time.sleep(15) function to run our bot every 15 seconds to check and automatically reply to any mentioned tweets containing “#HelloWorld”.

import time
if __name__=="__main__":
    api = createApi()
    while True:
        replyToTweets()
        time.sleep(15)

Let’s create another reply function to learn about the Tweepy user class and reply to a user with information from their profile.

Each Tweepy status has a user attribute which returns an instance of the user class. We can easily retrieve this user’s public profile information from this instance of the user class’s name, description, and location attributes.

Just to clean up the code, I’ve made the following function which takes in the user object as a parameter, then formats and returns a string with the user’s information which we can send as a reply Tweet.

def infoAboutMe(user):
    '''Function that returns a string with user's name, description and location.'''
    location_present = True # Flag Variable to check if the user has publicly set their location.
    if user.location == '':
        location_present = False
    if location_present:
        user_details = f" 𝐍𝐚𝐦𝐞: {user.name} \n 𝐃𝐞𝐬𝐜𝐫𝐢𝐩𝐭𝐢𝐨𝐧: {user.description} \n 𝐋𝐨𝐜𝐚𝐭𝐢𝐨𝐧: {user.location}"
    else:
        user_details = f" 𝐍𝐚𝐦𝐞: {user.name} \n 𝐃𝐞𝐬𝐜𝐫𝐢𝐩𝐭𝐢𝐨𝐧: {user.description} \n 𝐋𝐨𝐜𝐚𝐭𝐢𝐨𝐧: Unknown 🤫"
    return user_details

I also added a little additional logic to check whether the user’s location is not public and change the location part of the message to display “Unknown 🤫”.

Adding this following code snippet under and after the if "#helloworld" in tweet.full_text.lower() in replyToTweets(), we have successfully added another feature to your awesome bot!

elif '#aboutme' in tweet.full_text.lower():
            reply_tweet = infoAboutMe(tweet.user) + '\n' + "@" + tweet.user.screen_name
            api.update_status(reply_tweet, in_reply_to_status_id = tweet.id)
            print("An About Me Response Has Been Sent.")

Adding Face Detection

Before we get started on making this feature, let us understand how it works through a diagram:

Diagram explaining how we are going to implement the face detection function

So, we are going to use the Twitter API to retrieve Tweets mentioning our bot and “#FaceDetection”, we will save those images, then use OpenCV — A library with tools for computer vision — To scan the image for faces, create a new image with the faces marked and send it back as a reply to the original Tweet using the Twitter API.

We will be using the frontal face HAAR cascades with OpenCV to detect faces in the image. You will need to download this XML file (that basically gives OpenCV the instructions of how to detect faces) into your directory so that OpenCV can use it. The XML file is available in my repository here. How these HAAR cascades work is really interesting, but maybe a topic for another blog post :)

First, we need to grab the media (photos), if any, when “#FaceDetection” is present while our bot is mentioned. If you’ve followed through the previous two features, this should be easy. We add the following to our code-

elif '#facedetection' in tweet.full_text.lower():

Now we need to grab any media present in the Tweet. Media is present as an entity in our Tweet/Status object. Let’s also add a media_files list so that we can grab all the images if multiple images are present in the Tweet.

# Grabbing any media in the tweet
media_files = []
media = tweet.entities.get('media', [])

We then check if there are no media attached to the Tweet, and send a response accordingly-

if(len(media) == 0):
    # No picture attached with tweet
    reply_tweet = " I could not find an image with your tweet to analyse, " + "@" + tweet.user.screen_name
    api.update_status(reply_tweet, in_reply_to_status_id = tweet.id)
    print("A Face Detection Response Has Been Sent.")

Now, we have the following steps:

1. Download the image(s) for further processing.

2. Detecting the faces and marking them.

3. Replying to the user with the new image.

Step 1: Downloading the images

We can use the Python requests library to get and download the image. This returns an object from which we can extract the content and write/save it to our machine.

import requests
# Downloading the image for further processing               image_link = requests.get(media_files[-1])
f = open('that_image.jpg','wb')                f.write(image_link.content)
f.close()

Step 2: Detecting the faces and marking them.

For this step, I created a separate Python file with a function to perform this task.

We import the cv2 library for python (you will have to use pip install opencv-python in your terminal to get the library for the first time) and create a function that takes in the imagePath (where our downloaded image is stored) and the cascPath (where our cascade XML file is stored) as parameters.

import cv2
def detectFaces(imagePath, cascPath = "haarcascade_frontalface_default.xml"):

For OpenCV to analyze our image, it needs to first convert the image into gray-scale — using a built-in function in the library — so that it can apply its machine learning algorithms. Let’s read the image and convert it into gray-scale-

# Reading the imageimage = cv2.imread(imagePath)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

Then, we can create a faceCascade object and detect faces in the image-

# Detecting faces in the image 
   
faceCascade = cv2.CascadeClassifier(cascPath)
faces = faceCascade.detectMultiScale(gray,scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))

Next, we draw a rectangle on the faces detected-

# Drawing a rectangle around the faces    
for (x, y, w, h) in faces:
        cv2.rectangle(image, (x, y), (x+w, y+h), (255, 0, 0), 2)

Extra Tip: The fourth parameter in the rectangle method used above specifies the color of the rectangle

Now, we save this modified image with the faces marked.

# Saving the new image with the rectangles on faces
new_image_path = "detected-"+imagePath    cv2.imwrite(new_image_path, image)

Lastly, I wanted the function to return both the new image path and the number of faces, so-

# Returning the new image path    
return new_image_path, len(faces)

So that wraps up our separate python file and our function. Phew!

Back in the main python script, we simply call this function to get the updated image path and the number of faces.

# Detecting faces                
final_image_path, number_of_faces = detectFaces('that_image.jpg')

We now have our image ready. All that’s left now is to prepare a reply tweet.

Step 3: Replying to the user with the new image

Replying with an image works slightly differently here. We first need to upload the image to Twitter’s servers and then include it in our reply function.

We can upload our image to Twitter like this:

# Uploading the image to twitter                
uploaded_media = api.media_upload(final_image_path)

Pretty simple, right?

The method call api.media_upload() returns an object from which we can grab the media_id attribute. After that, we simply add our media_id to our tweet and voilà.

# Posting the tweet with the image
reply_tweet = f"I found {number_of_faces} faces in this image! " + "@" + tweet.user.screen_name
api.update_status(status=reply_tweet, media_ids=[uploaded_media.media_id],in_reply_to_status_id = tweet.id)
print("A Face Detection Response Has Been Sent.")

And that’s it! Congratulations on making it so far and making your own Twitter bot!

Photo by Eilis Garvey on Unsplash

Add on any new features as per your need with the concepts explained above :)

I have linked all the amazing webpages that helped me with this project below. Thanks for reading my blog post! See you around!

• Offical Twitter Developers Page on How to make a Twitter bot

• Real Python’s “How to make a Twitter Bot using Tweepy”

• HubSpot’s “How to make a Twitter Bot”

• Real Python’s “Face detection using Python”

• Adarsh Menon’s “Face Detection using OpenCV & Python”

• StackOverflow “How to filter images from tweets”

• MattCrampton’s “Posting photos to Twitter using Python”

• GeeksforGeeks’ “cv2.imwrite() method guide”

Reposted from my medium blog!