Bash Script for Weather Display without curl

A comprehensive guide to creating a weather terminal script with wget and netcat

Key Highlights

Alternative Tools: Utilizing wget or netcat in place of curl for fetching web data.
Script Flexibility: Extracting weather information from online services with effective parsing methods.
Error Handling & Parsing: Ensuring robustness using grep, sed, or jq with proper error management.

Overview

This guide provides a detailed explanation of how to create a Bash script that retrieves and displays weather information from an online source without using curl. Instead, the script leverages alternative command-line tools, specifically wget and netcat. By building on example approaches from a well-known thread on an antiX forum post, this guide integrates insights and examples from multiple sources to offer a robust solution for Unix-like environments.

Understanding the Objective

The goal is to develop a Bash script that simulates the behavior of Lead Farmer's weather script—originally designed with curl—with modifications so that it utilizes wget or other networking tools. Specifically, we target weather data from websites such as weather.com or an API like OpenWeatherMap. The script will retrieve HTML or JSON data, then parse out specific weather details including temperature, humidity, wind speed, and a brief description. In this solution, we explore various approaches, corrections to previous bugs (for instance, ensuring proper display of negative temperatures), and ideas to seamlessly extract the relevant information.

Using wget to Fetch Weather Data

The simplest and most common alternative to curl for fetching web pages in a Bash script is wget. wget is widely available in most Unix-based systems and allows you to silently download web pages or API responses. Below, we explain an example that uses wget to fetch HTML content from a weather website and employs grep with a regular expression to extract the temperature. This approach echoes improvements noted in previous forum discussions, including handling negative values correctly.

Step-by-Step Explanation

The following script demonstrates fetching data from a hypothetical weather URL using wget. It extracts the temperature by using grep with a Perl-compatible regular expression (PCRE). The regex is designed to correctly handle negative values; if temperatures drop below zero, the minus sign is retained.

#!/bin/bash

# URL Configuration: Replace YOUR_LOCATION_CODE with your actual location code.
WEATHER_URL="https://weather.com/weather/today/l/YOUR_LOCATION_CODE"

# Use wget to retrieve weather data silently. The -q flag suppresses wget progress output.
weather_data=$(wget -qO- "$WEATHER_URL")

# Use grep with a Perl regex flag to extract the temperature.
# This regex looks for an optional minus sign followed by one or more digits immediately followed by a <span tag.
temperature=$(echo "$weather_data" | grep -Po '(-?\d+(?=<span))')

# Check if temperature data was successfully extracted.
if [ -z "$temperature" ]; then
    echo "Error: Unable to fetch temperature data. Please check the URL or your network connection."
    exit 1
fi

# Display the temperature data.
echo "Current Temperature: $temperature°C"

Additional Parsing Options

If you prefer a more versatile approach, consider using additional utilities like sed or awk to extract other weather details such as humidity or weather condition phrases.

#!/bin/bash

# URL for the weather data (HTML format expected).
WEATHER_URL="https://weather.com/weather/today/l/YOUR_LOCATION_CODE"

# Fetch weather page content.
data=$(wget -qO- "$WEATHER_URL")

# Extract temperature using grep with PCRE.
temperature=$(echo "$data" | grep -Po '(-?\d+(?=<span class="CurrentConditions--tempValue))')

# Extract current weather condition phrase.
condition=$(echo "$data" | grep -Po '(?<=<div class="CurrentConditions--phraseValue--)[^<]+')

if [ -z "$temperature" ] || [ -z "$condition" ]; then
    echo "Error: Unable to retrieve full weather information."
    exit 1
fi

echo "Current Temperature: $temperature°C"
echo "Weather Condition: $condition"

The above script uses two grep commands: one for temperature that accounts for negative values, and another for current conditions. Be aware that HTML structures change over time, and you may require modifications to the regex patterns if the website's format is updated.

Using netcat as an Alternative

For environments where wget might not be available or if you seek a deeper understanding of HTTP transactions, netcat (nc) offers an alternative. This tool allows you to manually craft HTTP requests and send them to the server. While netcat is more low-level compared to wget, it provides a hands-on approach to learning how HTTP works.

Crafting an HTTP GET Request

The following script demonstrates constructing an HTTP GET request manually and using netcat to send it to the weather server. Although this method is less straightforward than using wget, it is educational and powerful in controlled environments.

#!/bin/bash

# The weather URL components.
URL="https://weather.com/weather/today/l/YOUR_LOCATION_CODE"
HOST=$(echo "$URL" | sed -E 's!https?://!!' | cut -d'/' -f1)
PATH=$(echo "$URL" | sed -E 's!https?://[^/]+!!')

# Construct an HTTP GET request string.
request="GET $PATH HTTP/1.0\r\nHost: $HOST\r\n\r\n"

# Use netcat to send the request and capture the response.
weather_response=$(echo -e "$request" | nc "$HOST" 80)

# Extract temperature information from the HTTP response.
temperature=$(echo "$weather_response" | grep -Po '(-?\d+(?=<span))')

if [ -z "$temperature" ]; then
    echo "Error: Temperature data could not be extracted using netcat."
    exit 1
fi

echo "Current Temperature: $temperature°C"

Important Considerations

When using netcat, ensure that you understand the structure of HTTP responses (including headers and body). You might need to adjust your extraction regex depending on where the weather data is embedded in the response. This approach provides more flexibility but at the cost of simplicity.

Using an API with wget and jq

For a more structured approach, particularly when weather data is available in JSON format, consider using a public weather API such as OpenWeatherMap. This method leverages wget for fetching data and jq, a lightweight JSON processor, to parse and extract relevant information cleanly. This approach not only simplifies data extraction but also improves reliability since JSON responses are less prone to format changes compared to HTML.

Script to Fetch JSON Weather Data

The following script assumes you have an API key from OpenWeatherMap and demonstrates fetching and parsing weather data. It presents key weather values such as temperature, humidity, wind speed, and a short description.

#!/bin/bash

# Configuration: Replace with your OpenWeatherMap API key and desired city.
API_KEY="YOUR_OPENWEATHERMAP_API_KEY"
CITY="London"
UNITS="metric"  # Use 'imperial' for Fahrenheit

# API URL for current weather.
API_URL="http://api.openweathermap.org/data/2.5/weather?q=${CITY}&appid=${API_KEY}&units=${UNITS}"

# Fetch weather data using wget and save output to a temporary file.
wget -q -O weather.json "$API_URL"

# Check if the JSON file exists and is non-empty.
if [ ! -s weather.json ]; then
    echo "Error: Unable to fetch weather data from the API."
    exit 1
fi

# Parse the JSON file with jq to extract weather details.
temperature=$(jq -r '.main.temp' weather.json)
humidity=$(jq -r '.main.humidity' weather.json)
wind_speed=$(jq -r '.wind.speed' weather.json)
description=$(jq -r '.weather[0].description' weather.json)

# Validate the output to ensure data has been extracted.
if [ "$temperature" = "null" ] || [ "$humidity" = "null" ]; then
    echo "Error: Incomplete weather information received."
    rm weather.json
    exit 1
fi

# Display weather information.
echo "Weather in ${CITY}:"
echo "Temperature: ${temperature}°C"
echo "Humidity: ${humidity}%"
echo "Wind Speed: ${wind_speed} m/s"
echo "Condition: ${description}"

# Clean up the temporary JSON file.
rm weather.json

Setup and Troubleshooting

Ensure you have installed jq on your system. For example, on Debian-based distributions, run sudo apt-get install jq. If the network request fails or returns an empty JSON file, verify your network connectivity and that your API key is valid. The script verifies the existence and size of the JSON file to prevent processing an empty or incomplete response.

Comparing wget and netcat Approaches

Below is a comparative summary table that highlights the differences between using wget and netcat for fetching data in a Bash script:

Aspect	wget	netcat
Ease of Use	Simple and straightforward command-line utility.	Requires manual request construction and parsing.
Data Format Handling	Automatically handles HTTP and HTTPS; integrates well with JSON/HTML.	Needs manual interpretation of HTTP responses and may require additional cleanup.
Error Management	Handles connection issues gracefully; standard exit codes.	Less forgiving with raw socket communications; error handling must be explicitly coded.
Common Use Cases	General fetching of web pages, APIs, and file downloads.	Low-level network debugging and scenarios requiring custom HTTP transactions.

Additional Considerations and Enhancements

Security and Compliance

When fetching data from external websites or APIs, ensure you adhere to the terms of service of the data provider. Avoid scripts that may violate usage policies or scrape data without proper authorization. When using APIs, always secure your API keys, and consider environment variables or configuration files that are not exposed in your script.

Maintaining Robustness

The landscape of websites and API endpoints can change over time, leading to potential script breakages due to altered HTML structures or JSON formats. Incorporate error checks such as verifying HTTP response codes and file size checks after a download. This script includes basic error detection, but more complex implementations could parse HTTP headers or integrate logging mechanisms to monitor failures.

Extending Functionality

Consider extending your Bash script with additional weather information such as forecasts, sunrise and sunset times, or alerts. You could further integrate the script with desktop notification systems or even a graphical frontend. Another enhancement could be integrating a caching mechanism to reduce API calls during rapid successive requests.

Programming Best Practices

Modularize your scripts by placing configuration variables at the top, so they can be easily modified without needing deep changes. Add comments and include usage instructions, which are useful for troubleshooting and modifying the script in the future. Version control your script with systems like Git to track changes over time, and document your regex patterns to assist others in understanding your extraction strategy.

Extending the Script for Wider Use

The examples provided above focus on displaying weather information within the terminal. However, you can integrate these scripts with other applications or workflows. For instance, you could set up a cron job to periodically update weather information on a digital sign or use the script as the backend for a simple web service that displays current conditions.

Example: Scheduling with Cron

To display updated weather information every hour, you might add a cron job similar to the following:

# Open your crontab editor:
crontab -e

# Add the following line to execute the weather script hourly:
0 * * * * /path/to/weather.sh >> /path/to/weather.log 2>&1

This entry runs the script on the hour each hour, appending its output to a log file for future reference. Adjust the frequency as needed.

Additional Tools

If you wish to build a more sophisticated text-based user interface (TUI) in Bash, consider using additional libraries or utilities such as dialog or whiptail. These tools can offer menu-driven interfaces or graphical pop-ups within terminal environments, enhancing usability significantly.