Mastering Data Preparation for Darts with Forex OHLCV Data

Discover the streamlined process to convert your DataFrame into powerful time series objects

Highlights

DataFrame Setup: Ensure your DataFrame has a datetime index and correctly labeled OHLCV columns.
Conversion Process: Use TimeSeries.from_dataframe by specifying your value columns and frequency.
Multivariate Series: Create comprehensive multivariate time series objects for robust forecasting.

Preparing Your Data

When working with hourly Forex OHLCV (Open, High, Low, Close, Volume) data, the first essential step is to ensure that your data is structured appropriately within a Pandas DataFrame. The effectiveness of the Darts library’ forecasting capabilities is highly dependent on the accuracy of your DataFrame. Below are the key steps to prepare your data:

1. DataFrame Creation and Setup

Begin by importing the necessary libraries and setting up your DataFrame. It is crucial that your DataFrame contains a column with correctly formatted datetime information, and that this datetime data is set as the index. This setup allows Darts to recognize the timescale and frequency of your data.

a. Import Libraries

# Importing the required libraries
import pandas as pd
from darts import TimeSeries

b. Create and Format DataFrame

Your DataFrame should include columns for 'Open', 'High', 'Low', 'Close', and 'Volume'. Here is an example of creating such a DataFrame:

# Sample data creation
data = {
    "time": pd.date_range("2020-01-01 00:00", periods=100, freq="H"),
    "Open": [1.2 + i*0.01 for i in range(100)],
    "High": [1.3 + i*0.01 for i in range(100)],
    "Low":  [1.1 + i*0.01 for i in range(100)],
    "Close": [1.25 + i*0.01 for i in range(100)],
    "Volume": [100 + i for i in range(100)]
}
df = pd.DataFrame(data)
# Convert 'time' column to datetime (if not already)
df['time'] = pd.to_datetime(df['time'])
# Set the datetime column as the index
df.set_index('time', inplace=True)

This ensures your DataFrame uses time as the index, which is essential for time series analysis. If your DataFrame already contains a datetime index, you can bypass some of these steps.

Converting DataFrame to a Darts TimeSeries

After successfully setting up your DataFrame, the next step is to convert it into a Darts TimeSeries object. The TimeSeries.from_dataframe() method is specifically designed to handle this conversion by directly accepting your prepared DataFrame, and, if necessary, specifying the columns for values.

2. Conversion Using TimeSeries.from_dataframe()

Depending on the organization of your data, you can create either a multivariate time series object – which encapsulates all OHLCV columns – or separate objects for each component. Typically, for financial data like Forex OHLCV, a multivariate series is very beneficial for capturing the correlations between different components.

a. Creating a Multivariate Time Series

If your DataFrame's index is comprised of datetime values and all the remaining columns are part of your forecasting model, simply convert the DataFrame without specifying the time column explicitly:

# Convert the DataFrame into a multivariate TimeSeries
series = TimeSeries.from_dataframe(df, freq='H')
# The freq parameter 'H' sets the frequency as hourly

In this example, since the DataFrame index is already a datetime index, Darts will automatically interpret this as the time component. The freq parameter explicitly tells Darts that the observations are hourly.

b. Specifying Value Columns Explicitly

You might want to be explicit about which columns to include, especially if your DataFrame contains additional auxiliary information:

# Explicitly specifying OHLCV columns
series = TimeSeries.from_dataframe(df, value_cols=['Open', 'High', 'Low', 'Close', 'Volume'], freq='H')

This approach is useful when you have extra columns present in your DataFrame that are not necessary for the time series forecasting, ensuring that only the relevant Forex data is contained in the TimeSeries object.

Working with Multiple Data Series

In some cases, you might have separate time series for each Forex currency pair or want to analyze components individually. Darts offers flexibility in handling multiple time series from a single DataFrame.

3. Creating Separate Time Series for Each OHLCV Component

If you require individual TimeSeries objects for each OHLCV data column, you can convert portions of your DataFrame separately. This method is useful for components-specific analysis or when different modeling approaches are needed for each component.

Component	Code Example
Open	`series_open = TimeSeries.from_dataframe(df[['Open']], freq='H')`
High	`series_high = TimeSeries.from_dataframe(df[['High']], freq='H')`
Low	`series_low = TimeSeries.from_dataframe(df[['Low']], freq='H')`
Close	`series_close = TimeSeries.from_dataframe(df[['Close']], freq='H')`
Volume	`series_volume = TimeSeries.from_dataframe(df[['Volume']], freq='H')`

This table demonstrates how to individually convert each part of your Forex OHLCV data into separate time series. This separation can be useful if you plan to analyze the interplay between different series later or require specialized forecasting models for each.

Additional Considerations for Data Preparation

Proper data preparation extends beyond creating a TimeSeries object. Consider the following additional aspects to refine your forecasting workflow:

4. Handling Missing Values and Data Consistency

Ensure that your datetime index does not have any missing timestamps. Missing data points may lead to errors during model training or unreliable forecasts. Missing timestamps can be appropriately handled by reindexing your DataFrame and applying interpolation or imputation techniques as required.

5. Frequency Consistency and DateTime Formatting

Double-check the frequency settings of your DataFrame. The freq parameter in TimeSeries.from_dataframe() should align with your data frequency (in this case, hourly). In case your data timestamps are irregular, consider resampling your data to a uniform frequency using Pandas' resample functionality:

# Example of resampling to hourly data if necessary
df = df.resample('H').mean()
# Then proceed with setting the datetime as index and converting to TimeSeries

6. Splitting Data for Model Validation

Once you have your TimeSeries object, it is good practice to split your data into training and validation sets. This is critical for evaluating the performance of your models reliably. For instance, you might reserve the last few hours or days of data for validation:

# Splitting a TimeSeries into train/validation sets
train, val = series[:-6], series[-6:]
print("Training Series:", train)
print("Validation Series:", val)

This simple splitting allows for robust model testing and ensures that your forecasting models do not overfit the training data.

Implementing Forecasting Models using Darts

Once your data is prepared and converted to a TimeSeries object, you can easily integrate it with various forecasting models available in Darts. Here is a brief example using an exponential smoothing model, which is well-suited for handling seasonal data common in Forex markets:

7. Forecasting Model Example

# Import a forecasting model
from darts.models import ExponentialSmoothing

# Initialize the model (daily seasonality for hourly data means seasonal periods of 24)
model = ExponentialSmoothing(seasonal_periods=24)

# Fit the model using the training data
model.fit(train)

# Predict on validation data length
prediction = model.predict(len(val))
print(prediction)

This walk-through shows how seamlessly data conversion aligns with forecasting models in Darts. Adjust model selection based on your forecasting needs and complexity of data.