Automating Repository Monitoring and Notification with Python and Twilio

Introduction

In this blog post, we will explore a Python script that automates the monitoring of repository creation on GitHub. The script periodically checks for new repositories, clones them locally, and sends notifications using Twilio. While this project may seem whimsical, it can be a useful tool for staying updated on repository changes. Let's dive into the code and understand its implementation.

Prerequisites

Before creating this project, there are a few prerequisites that you should have in place.

• Python and Required Libraries: Ensure that you have Python installed on your system. The code is written in Python, so having a compatible Python version is essential. Additionally, you need to have the following libraries installed:

  • requests: Install it using pip install requests.

  • beautifulsoup4: Install it using pip install beautifulsoup4.

  • twilio: Install it using pip install twilio.

• GitHub Account and Credentials: To monitor repositories on GitHub, you'll need a GitHub account. Make sure you have an account and the necessary credentials (username and password). The target username, which you want to monitor, should also be known.

• Twilio Account and Credentials: To send SMS notifications, you'll need a Twilio account. If you don't have one, sign up at https://www.twilio.com/try-twilio.

Source code - Here

Code Walkthrough

#stalker.py

1. Importing Libraries:

    import requests
    from bs4 import BeautifulSoup
    import time
    from sms import send_sms
    from clone import clone_repository
   

   The script begins by importing the necessary libraries. requests is used to make HTTP requests, BeautifulSoup for HTML parsing, time for adding delays, and the custom modules sms and clone for sending notifications and cloning repositories, respectively.

2. Setting Configuration:

    USERNAME = "TARGET_USERNAME"
    prevCount = 0
   

   Define the target GitHub username (USERNAME) you want to monitor and initialize a variable prevCount to track the previous count of repositories.

3. Monitoring Loop:

    while True:
   

   The while True loop is responsible for continuously monitoring the repositories. It runs indefinitely until manually stopped.

4. Fetching the Repository Page:

        url = f"https://github.com/{USERNAME}?tab=repositories"
        response = requests.get(url)
        soup = BeautifulSoup(response.text, "html.parser")
        repositories = soup.find_all("a", {"itemprop": "name codeRepository"})
   

   Construct the URL for the target GitHub user's repository page and make an HTTP GET request using requests library. The response is then parsed using BeautifulSoup to extract repository information.

5. Checking for New Repositories:

        currCount = len(repositories)
        if currCount > prevCount and prevCount != 0:
   

   Count the number of repositories found on the page (currCount) and compare it with the previous count (prevCount). If currCount is greater than prevCount and not equal to zero, it means a new repository has been added.

6. Cloning the Repository:

                clone_repository(
                    f"https://github.com{repositories[0]['href']}.git",
                    "YOUR_USERNAME",
                    "YOUR_PASSWORD",
                    f"./{repositories[0].text.strip()}",
                )
   

   Invoke the clone_repository function from the clone.py module to clone the newly added repository locally. Pass the repository URL, your GitHub username and password, and the destination path as arguments.

7. Sending SMS Notification:

            send_sms(
                repositories[0].text.strip(),
                f"https://github.com{repositories[0]['href']}",
                USERNAME,
            )
   

   Use the send_sms function from the sms.py module to send an SMS notification using Twilio. Provide the repository name, URL, and target GitHub username as parameters.

8. Updating Count and Sleeping:

        prevCount = currCount
        time.sleep(60*60*24)
   

   Update prevCount with the current repository count (currCount). Sleep the script for 24 hours using time.sleep(60*60*24) to repeat the monitoring process.

#clone.py

The clone.py file contains the functionality to clone a repository locally using the git clone command. Let's go through the code and understand it step by step.

1. Importing Libraries:

    import subprocess
   

   The subprocess module is imported to execute shell commands from within Python.

2. Cloning Repository Function:

    def clone_repository(url, username, password, destination):
        command = ["git", "clone", url, destination]
   
        try:
            process = subprocess.Popen(
                command,
                stdin=subprocess.PIPE,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
                universal_newlines=True,
            )
   
            input_data = username + "\n" + password + "\n"
   
            output, error = process.communicate(input=input_data)
   
            if process.returncode == 0:
                print("Repository cloned successfully.")
            else:
                print(f"Error cloning repository: {error}")
        except subprocess.CalledProcessError as e:
            print(f"Error cloning repository: {e}")
   

   The clone_repository function takes four arguments: url, username, password, and destination.

   • The command list is created with the shell command and its parameters required to clone the repository using git clone.

   • Inside a try block, subprocess.Popen is used to execute the git clone command.

   • The input_data variable contains the username and password that will be passed as input to the git clone command through the stdin of the process.

   • The output and error variables store the output and error messages from the cloning process.

   • If the returncode of the process is 0, it indicates successful cloning. Otherwise, an error message is printed.

   • Exceptions are handled using subprocess.CalledProcessError to catch any errors that may occur during the cloning process.

#sms.py

1. Importing Libraries:

    from twilio.rest import Client
   

   The Client class from the twilio.rest module is imported to interact with the Twilio REST API.

2. Sending SMS Notification Function:

    def send_sms(repo_name, repo_url, username):
        account_sid = "TWILIO_ACCOUNT_SID"
        auth_token = "TWILIO_AUTH_TOKEN"
        from_phone_number = "TWILIO_PHONE_NUMBER"
        to_phone_number = "YOUR_PHONE_NUMBER"
   
        client = Client(account_sid, auth_token)
   
        message = f"Username: {username}\nRepository: {repo_name}\nURL: {repo_url}\nCloned successfully."
   
        client.messages.create(body=message, from_=from_phone_number, to=to_phone_number)
   

   The send_sms function takes three arguments: repo_name, repo_url, and username.

   • The account SID, authentication token, and phone numbers (from and to) are stored in respective variables. Replace the placeholders with the actual values from your Twilio account.

   • An instance of the Client class is created using the account SID and authentication token.

   • The message variable is created, containing the notification message with the repository details.

   • The client.messages.create method is called to send the SMS, providing the message body, from phone number, and to phone number as parameters.

Conclusion

In conclusion, this Python script harnesses the power of automation and integration with Twilio to monitor GitHub repository creations, clone them locally, and send SMS notifications. It showcases the versatility of Python in automating tasks and demonstrates the potential for creating personalized notification systems.