# Automating Repository Monitoring and Notification with Python and Twilio
## Introduction
In this blog post, we will explore a Python script that automates the monitoring of repository creation on GitHub. The script periodically checks for new repositories, clones them locally, and sends notifications using Twilio. While this project may seem whimsical, it can be a useful tool for staying updated on repository changes. Let's dive into the code and understand its implementation.
### Prerequisites
Before creating this project, there are a few prerequisites that you should have in place.
* **Python and Required Libraries:** Ensure that you have Python installed on your system. The code is written in Python, so having a compatible Python version is essential. Additionally, you need to have the following libraries installed:
* `requests`: Install it using `pip install requests`.
* `beautifulsoup4`: Install it using `pip install beautifulsoup4`.
* `twilio`: Install it using `pip install twilio`.
* **GitHub Account and Credentials:** To monitor repositories on GitHub, you'll need a GitHub account. Make sure you have an account and the necessary credentials (username and password). The target username, which you want to monitor, should also be known.
* **Twilio Account and Credentials:** To send SMS notifications, you'll need a Twilio account. If you don't have one, sign up at [**https://www.twilio.com/try-twilio**](https://www.twilio.com/try-twilio).
Source code - [**Here**](https://github.com/kartikmehta8/Gitective)
## Code Walkthrough
### `#stalker.py`
1. Importing Libraries:
```python
import requests
from bs4 import BeautifulSoup
import time
from sms import send_sms
from clone import clone_repository
```
The script begins by importing the necessary libraries. `requests` is used to make HTTP requests, `BeautifulSoup` for HTML parsing, `time` for adding delays, and the custom modules `sms` and `clone` for sending notifications and cloning repositories, respectively.
2. Setting Configuration:
```python
USERNAME = "TARGET_USERNAME"
prevCount = 0
```
Define the target GitHub username (`USERNAME`) you want to monitor and initialize a variable `prevCount` to track the previous count of repositories.
3. Monitoring Loop:
```python
while True:
```
The `while True` loop is responsible for continuously monitoring the repositories. It runs indefinitely until manually stopped.
4. Fetching the Repository Page:
```python
url = f"https://github.com/{USERNAME}?tab=repositories"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
repositories = soup.find_all("a", {"itemprop": "name codeRepository"})
```
Construct the URL for the target GitHub user's repository page and make an HTTP GET request using `requests` library. The response is then parsed using `BeautifulSoup` to extract repository information.
5. Checking for New Repositories:
```python
currCount = len(repositories)
if currCount > prevCount and prevCount != 0:
```
Count the number of repositories found on the page (`currCount`) and compare it with the previous count (`prevCount`). If `currCount` is greater than `prevCount` and not equal to zero, it means a new repository has been added.
6. Cloning the Repository:
```python
clone_repository(
f"https://github.com{repositories[0]['href']}.git",
"YOUR_USERNAME",
"YOUR_PASSWORD",
f"./{repositories[0].text.strip()}",
)
```
Invoke the `clone_repository` function from the [`clone.py`](http://clone.py) module to clone the newly added repository locally. Pass the repository URL, your GitHub username and password, and the destination path as arguments.
7. Sending SMS Notification:
```python
send_sms(
repositories[0].text.strip(),
f"https://github.com{repositories[0]['href']}",
USERNAME,
)
```
Use the `send_sms` function from the [`sms.py`](http://sms.py) module to send an SMS notification using Twilio. Provide the repository name, URL, and target GitHub username as parameters.
8. Updating Count and Sleeping:
```python
prevCount = currCount
time.sleep(60*60*24)
```
Update `prevCount` with the current repository count (`currCount`). Sleep the script for 24 hours using `time.sleep(60*60*24)` to repeat the monitoring process.
### `#clone.py`
The [`clone.py`](http://clone.py) file contains the functionality to clone a repository locally using the `git clone` command. Let's go through the code and understand it step by step.
1. Importing Libraries:
```python
import subprocess
```
The `subprocess` module is imported to execute shell commands from within Python.
2. Cloning Repository Function:
```python
def clone_repository(url, username, password, destination):
command = ["git", "clone", url, destination]
try:
process = subprocess.Popen(
command,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True,
)
input_data = username + "\n" + password + "\n"
output, error = process.communicate(input=input_data)
if process.returncode == 0:
print("Repository cloned successfully.")
else:
print(f"Error cloning repository: {error}")
except subprocess.CalledProcessError as e:
print(f"Error cloning repository: {e}")
```
The `clone_repository` function takes four arguments: `url`, `username`, `password`, and `destination`.
* The `command` list is created with the shell command and its parameters required to clone the repository using `git clone`.
* Inside a `try` block, `subprocess.Popen` is used to execute the `git clone` command.
* The `input_data` variable contains the username and password that will be passed as input to the `git clone` command through the `stdin` of the process.
* The `output` and `error` variables store the output and error messages from the cloning process.
* If the `returncode` of the process is 0, it indicates successful cloning. Otherwise, an error message is printed.
* Exceptions are handled using `subprocess.CalledProcessError` to catch any errors that may occur during the cloning process.
### `#sms.py`
1. Importing Libraries:
```python
from twilio.rest import Client
```
The `Client` class from the [`twilio.rest`](http://twilio.rest) module is imported to interact with the Twilio REST API.
2. Sending SMS Notification Function:
```python
def send_sms(repo_name, repo_url, username):
account_sid = "TWILIO_ACCOUNT_SID"
auth_token = "TWILIO_AUTH_TOKEN"
from_phone_number = "TWILIO_PHONE_NUMBER"
to_phone_number = "YOUR_PHONE_NUMBER"
client = Client(account_sid, auth_token)
message = f"Username: {username}\nRepository: {repo_name}\nURL: {repo_url}\nCloned successfully."
client.messages.create(body=message, from_=from_phone_number, to=to_phone_number)
```
The `send_sms` function takes three arguments: `repo_name`, `repo_url`, and `username`.
* The account SID, authentication token, and phone numbers (from and to) are stored in respective variables. Replace the placeholders with the actual values from your Twilio account.
* An instance of the `Client` class is created using the account SID and authentication token.
* The `message` variable is created, containing the notification message with the repository details.
* The `client.messages.create` method is called to send the SMS, providing the message body, from phone number, and to phone number as parameters.
## Conclusion
In conclusion, this Python script harnesses the power of automation and integration with Twilio to monitor GitHub repository creations, clone them locally, and send SMS notifications. It showcases the versatility of Python in automating tasks and demonstrates the potential for creating personalized notification systems.