Building a Real-Time Financial Data Alert System on AWS

A step-by-step guide to stream Quandl data into Discord channels.

AWS server room financial charts home office

Highlights

Setup All Necessary Accounts: AWS, Quandl, and Discord
Code Detailed Tasks: From fetching data and calculating technical indicators to sending Discord alerts
Deployment & Scheduling: Hosting everything on AWS with Lambda (or EC2) and scheduling updates

Overview

This guide is meant to help the owner of a Discord channel focused on financial insights create a system that takes real-time financial data from Quandl, verifies specific criteria — such as trading volume thresholds or when a stock becomes heavily oversold based on the Relative Strength Index (RSI) — and sends alert updates directly to Discord channels. The system is hosted on AWS and will involve multiple steps including account setup, environment configuration, coding, and deployment.

Step-by-Step Guide

Step 1: Set Up Your Necessary Accounts

1.1: Create an AWS Account

Visit AWS Website, click on “Create a Free Account” and follow the instructions:

Enter your email, choose a password, and provide a name for your account.
Enter payment details (a credit card is required, but you can use the AWS Free Tier).
Verify your email address and complete your account creation.

1.2: Sign Up with Quandl

Create an account on Quandl.
Navigate to your account settings and generate an API key, then save it securely.

1.3: Create a Discord Server/Channel and Set Up Webhooks

If you already own a Discord channel, you simply need to create a webhook for your updates:

Open Discord, navigate to your server, and select the channel where you want to post updates.
Click the gear icon (Edit Channel), go to the Integrations tab, and click “Create Webhook.”
Name your webhook and copy the webhook URL for later use. This URL will be used by the system to send messages to Discord.

Step 2: Set Up AWS Environment

2.1: Create an IAM User/Role for Lambda

You need to configure permissions for your AWS Lambda function.

Login to the AWS console and navigate to the IAM section.
Create a new IAM Role by selecting “Create role” and assigning it as a trusted entity for Lambda.
Attach the AWSLambdaBasicExecutionRole policy to allow logging and basic execution.
Name it something like lambda-quandl-discord-role then save.

2.2: Create a Lambda Function

AWS Lambda will serve as the hosting platform for your script:

Navigate to AWS Lambda in the AWS console and click “Create function.”
Choose “Author from scratch” and name your function (for example, quandl-discord-alert).
Select Python (preferably Python 3.9 or later) as the runtime.
Select “Use an existing role” and pick the IAM role created earlier.
After creating the function, you should see the function dashboard with a code editor ready to use.

Step 3: Develop Your Python Code

The Python code will perform the following tasks:

Fetch data from Quandl (or use a substitute like yfinance if desired)
Process the data and calculate indicators such as RSI
Check for alert conditions (volume threshold or oversold RSI)
Send an alert message to the Discord channel via the webhook

3.1: Write the Python Code – Detailed Breakdown

Create a file named lambda_function.py and enter the following code. Comments are included to clearly label each step:


# Importing necessary libraries
import json
import os
import requests
import yfinance as yf  # Used if you decide to use yfinance for demonstration purposes

def lambda_handler(event, context):
    """
    Main handler function used by AWS Lambda.
    """
    # <b></b>* Configuration Section <b></b>*
    # Define the stock symbol and criteria
    STOCK_SYMBOL = 'AAPL'  # Modify this stock symbol as needed.
    RSI_OVERSOLD_THRESHOLD = 30  # Threshold for oversold RSI condition
    VOLUME_THRESHOLD = 100000000  # Example: 100 million shares to trigger alert
    # Retrieve the Discord webhook URL from environment variables
    DISCORD_WEBHOOK_URL = os.environ.get('DISCORD_WEBHOOK_URL')
    
    # <b></b>* Fetching Stock Data <b></b>*
    try:
        # Fetch 1-day historical data for the specified stock using yfinance
        stock = yf.Ticker(STOCK_SYMBOL)
        data = stock.history(period="1d")
        
        if data.empty:
            raise ValueError("No data found for the given stock symbol: {}".format(STOCK_SYMBOL))
        
        # Retrieve latest closing price and volume data
        current_price = data['Close'].iloc[-1]
        current_volume = data['Volume'].iloc[-1]
        
        # Calculate RSI using our custom function
        rsi = calculate_rsi(data['Close'])
        
    except Exception as e:
        print("Error fetching data: {}".format(e))
        return {'statusCode': 500, 'body': json.dumps("Error: " + str(e))}
    
    # <b></b>* Alert Condition Evaluation <b></b>*
    alert_message = None
    # Check condition: oversold stock based on RSI
    if rsi < RSI_OVERSOLD_THRESHOLD:
        alert_message = f"Alert: {STOCK_SYMBOL} appears oversold (RSI = {rsi:.2f}). Price: ${current_price:.2f}"
    # Alternatively check condition: high trading volume
    elif current_volume > VOLUME_THRESHOLD:
        alert_message = f"Alert: {STOCK_SYMBOL} trading volume is high at {current_volume} shares. Price: ${current_price:.2f}"
    
    # <b></b>* Sending Discord Notification <b></b>*
    if alert_message:
        send_discord_message(alert_message, DISCORD_WEBHOOK_URL)
        return {'statusCode': 200, 'body': json.dumps("Alert sent to Discord: " + alert_message)}
    else:
        return {'statusCode': 200, 'body': json.dumps("No alert triggered at this time.")}

def calculate_rsi(prices, period=14):
    """
    Function to calculate the Relative Strength Index (RSI).
    Uses exponential weighted moving averages for smooth results.
    """
    # Compute price differences
    delta = prices.diff()
    # Remove the first NaN value
    delta = delta[1:]
    
    # Separate positive (up) and negative (down) price changes
    up = delta.clip(lower=0)
    down = -1 * delta.clip(upper=0)
    
    # Compute exponential moving averages for up and down swings
    roll_up = up.ewm(span=period, adjust=False).mean()
    roll_down = down.ewm(span=period, adjust=False).mean()
    
    # Calculate the Relative Strength (RS) and RSI
    RS = roll_up / roll_down
    RSI = 100.0 - (100.0 / (1.0 + RS))
    return RSI.iloc[-1]

def send_discord_message(message, webhook_url):
    """
    Function that sends a POST request to the Discord webhook with a message.
    """
    data = {"content": message}
    try:
        response = requests.post(webhook_url, json=data)
        response.raise_for_status()  # Check for any errors from Discord
        print("Discord alert sent successfully.")
    except requests.exceptions.RequestException as e:
        print("Failed to send Discord alert: {}".format(e))

3.2: Create a Deployment Package and Manage Dependencies

AWS Lambda does not come with all external libraries pre-installed. You need to package them:

Create a folder (e.g., deployment_package) on your computer.

Run the following commands from your terminal to install dependencies:


# In your terminal, navigate to your deployment package folder
pip install yfinance -t .
pip install requests -t .

Place your lambda_function.py inside this folder.
Zip all contents inside the folder (ensure you include the libraries installed, not the folder itself) to create your deployment package.
Upload the zip package to your AWS Lambda function under the function code section.

3.3: Environment Variable Configuration

It is important that sensitive information is not hardcoded. Instead, set environment variables:

In the AWS Lambda console, select “Configuration” then “Environment variables”.
Add a variable with the key DISCORD_WEBHOOK_URL and paste your Discord webhook URL as the value.
You may also add extra keys such as QUANDL_API_KEY if you customize your code to use Quandl directly.

Step 4: Scheduling and Triggering the Function

4.1: Using CloudWatch Events (EventBridge)

To run your Lambda function automatically at set intervals:

In the AWS Lambda console, click on “Add trigger”.
Select "EventBridge (CloudWatch Events)" as the trigger.
Create a new rule, give it a name (e.g., quandl-discord-schedule), and set a schedule expression. For instance, to run every 5 minutes, you can use:
cron(0/5 * ? * * *)
Enable the trigger and save your settings.

4.2: Alternative Deployment Using EC2 (Optional)

While AWS Lambda is serverless and generally preferred, you may also opt to use an AWS EC2 instance:

Create an EC2 instance (select a free tier eligible instance such as t2.micro).
SSH into your instance and install Python 3, pip, and any necessary packages (similar to the dependency installation described above).

Deploy your Python script on the instance and use a cron job to schedule its execution. For instance, use:


# Open the crontab editor
crontab -e

# Add an entry to run the script every 1 minute
*/1 * * * * python3 /path/to/your/lambda_function.py

Step 5: Testing and Validation

5.1: Local Testing

Before deploying on AWS, test your script locally:

Ensure that the Python script runs without errors.
Manually trigger the function to verify that it detects the required conditions (RSI below 30 or volume over threshold) and sends a test message to Discord.

5.2: Testing on AWS

After deployment:

Use AWS Lambda’s built-in “Test” functionality by creating a test event (the actual event content can be empty as your code does not depend on its input).
Check the Lambda logs (via CloudWatch Logs) for output messages to ensure the function runs as expected.
Monitor your Discord channel to verify that alerts are received appropriately.

Integration Table: System Components and Their Roles

Component	Role	Details
AWS Account	Hosting & Scheduling	Utilize AWS Lambda for serverless execution or EC2 for persistent jobs.
IAM Role	Permissions	Grants Lambda necessary permission to execute and log events.
Quandl API	Data Source	Provides the financial data including trading volumes and prices.
Python Code	Data Processing & Alerts	Fetches data, calculates RSI, verifies conditions, and sends alerts.
Discord Webhook	Notification	Sends real-time alerts to designated Discord channels.
CloudWatch/EventBridge	Scheduling	Triggers AWS Lambda at scheduled intervals (e.g., every 5 minutes).

Additional Detailed Instructions & Considerations

Extra Coding Tips

When writing your Python code, be sure to follow these extra guidelines:

Environment Variables: Always avoid hardcoding sensitive data such as the Discord webhook URL or API keys. Use environment variables in AWS Lambda.
Error Handling: Wrap your API calls and data processing code in try/except blocks to gracefully handle exceptions. Log errors using print statements or AWS CloudWatch Logs.
Comments & Documentation: Comment your code adequately, labeling sections with headers such as “Configuration”, “Data Fetching”, “Condition Evaluation”, and “Alerting”.
Modularity: Separate functionality into functions. This makes your code more readable and easier to debug.
Testing: Test each component individually. For example, verify that your RSI calculation function returns valid values before integrating it.

Combining All Elements

Once everything is tested and running locally, the next step is to integrate and deploy:

Zip your code and dependencies as described; then go to your AWS Lambda console.
Upload the zip file, verify that the Lambda handler is properly set (for instance, lambda_function.lambda_handler).
Set up the environment variables as directed.
Add your CloudWatch Event trigger to run at your desired frequency.
Monitor both CloudWatch logs for errors and Discord for alert messages.

Deployment Alternatives

If AWS Lambda is not a perfect fit for your system needs, consider:

Deploying the script on an AWS EC2 instance, installing necessary packages, and configuring a cron job.
Using specialized third-party integrations that combine cloud functions with webhook management.
Experiment with containerized deployments using AWS Fargate or similar services.