Comprehensive C++ Program for Directory File Logging and Comparison

Efficiently track and compare directory contents with detailed logging

Key Takeaways

• Automated Logging: Streamlines the process of recording file details across directories and subdirectories.
• Accurate Comparison: Effectively identifies added, removed, and modified files by comparing log files.
• User-Friendly Output: Generates clear and organized difference reports for easy analysis.

Introduction

Managing and monitoring the contents of directories is a common requirement in various applications. Whether it's for backup solutions, synchronization tasks, or simply tracking changes over time, having a reliable method to log and compare directory contents is invaluable. This comprehensive guide presents a C++ program designed to record detailed information about files within a directory and its subdirectories, and subsequently compare these records to identify any changes.

Program Objectives

The C++ program outlined below accomplishes the following objectives:

1. Records the filename, size (in KB), modification date, and creation date of each file in a specified directory.
2. Recursively processes all subdirectories to ensure comprehensive coverage.
3. Generates a timestamped log file containing the recorded information.
4. Finds the most recent previous log file and compares it with the current log to identify added, deleted, or modified files.
5. Outputs the differences to a corresponding ".dif" file for easy review.

Detailed C++ Implementation

1. Setting Up the Environment

To effectively utilize the program, ensure that your development environment supports C++17 or later, as the program leverages the std::filesystem library introduced in C++17 for handling file and directory operations.

2. Program Structure

The program is structured into several key components:

• File Information Struct: Defines the structure to store file details.
• Time Formatting Functions: Converts file timestamps to the desired string format.
• Directory Traversal: Recursively scans directories and collects file information.
• Logging Mechanism: Writes the collected information to a log file.
• Log Comparison: Identifies differences between the latest and previous logs.
• Difference Reporting: Outputs the changes to a ".dif" file.

3. The Complete C++ Code

Code Overview

Below is the complete C++ program that fulfills the outlined objectives. The code is thoroughly commented to explain each section's functionality.

#include <iostream>
#include <fstream>
#include <filesystem>
#include <vector>
#include <map>
#include <chrono>
#include <iomanip>
#include <sstream>
#include <algorithm>
#include <ctime>

// Namespace alias for filesystem
namespace fs = std::filesystem;

// Struct to hold file information
struct FileInfo {
    std::string filename;
    double size_kb;
    std::string modify_time;
    std::string create_time;
};

// Function to get current time as string in "YYYY-MM-DD-HH-MM-SS" format
std::string getCurrentTimeString() {
    auto now = std::chrono::system_clock::now();
    std::time_t now_c = std::chrono::system_clock::to_time_t(now);
    std::tm *tm_ptr = std::localtime(&now_c);
    std::ostringstream oss;
    oss << std::put_time(tm_ptr, "%Y-%m-%d-%H-%M-%S");
    return oss.str();
}

// Function to convert filesystem time to string
std::string timePointToString(const fs::file_time_type& ftime) {
    auto sctp = std::chrono::system_clock::now() + 
                (ftime - fs::file_time_type::clock::now());
    std::time_t tt = std::chrono::system_clock::to_time_t(sctp);
    std::tm *tm_ptr = std::localtime(&tt);
    std::ostringstream oss;
    oss << std::put_time(tm_ptr, "%Y-%m-%d-%H-%M-%S");
    return oss.str();
}

// Function to record files in a directory recursively
std::vector<FileInfo> recordFiles(const fs::path& directory) {
    std::vector<FileInfo> files;
    try {
        for (const auto& entry : fs::recursive_directory_iterator(directory)) {
            if (fs::is_regular_file(entry.status())) {
                FileInfo fi;
                fi.filename = entry.path().filename().string();
                fi.size_kb = static_cast<double>(fs::file_size(entry.path())) / 1024.0;
                fi.modify_time = timePointToString(fs::last_write_time(entry.path()));
                
                // Creation time handling
                #ifdef _WIN32
                    fi.create_time = timePointToString(fs::creation_time(entry.path()));
                #else
                    fi.create_time = fi.modify_time; // POSIX may not support creation time
                #endif
                
                files.push_back(fi);
            }
        }
    }
    catch(const fs::filesystem_error& e) {
        std::cerr << "Filesystem error: " << e.what() << '\n';
    }
    return files;
}

// Function to write file information to a log file
bool writeLogFile(const std::vector<FileInfo>& files, const std::string& log_filename) {
    std::ofstream log_file(log_filename);
    if (!log_file.is_open()) {
        std::cerr << "Failed to open log file: " << log_filename << '\n';
        return false;
    }
    for (const auto& file : files) {
        log_file << file.filename << ","
                 << std::fixed << std::setprecision(2) << file.size_kb << "KB,"
                 << file.modify_time << ","
                 << file.create_time << "\n";
    }
    log_file.close();
    return true;
}

// Function to read log file and parse into a map
std::map<std::string, FileInfo> readLogFile(const std::string& log_filename) {
    std::map<std::string, FileInfo> file_map;
    std::ifstream log_file(log_filename);
    if (!log_file.is_open()) {
        std::cerr << "Failed to open log file for reading: " << log_filename << '\n';
        return file_map;
    }
    std::string line;
    while (std::getline(log_file, line)) {
        std::istringstream iss(line);
        std::string token;
        FileInfo fi;
        if (std::getline(iss, token, ',')) {
            fi.filename = token;
        }
        if (std::getline(iss, token, ',')) {
            size_t pos = token.find("KB");
            fi.size_kb = std::stod(token.substr(0, pos));
        }
        if (std::getline(iss, token, ',')) {
            fi.modify_time = token;
        }
        if (std::getline(iss, token, ',')) {
            fi.create_time = token;
        }
        file_map[fi.filename] = fi;
    }
    log_file.close();
    return file_map;
}

// Function to find the closest previous log file
std::string findClosestLog(const fs::path& directory, const std::string& current_log_time) {
    std::string closest_log;
    long long min_diff = LLONG_MAX;
    for (const auto& entry : fs::directory_iterator(directory)) {
        if (fs::is_regular_file(entry.status()) && entry.path().extension() == ".log") {
            std::string filename = entry.path().filename().string();
            // Extract time part
            size_t pos = filename.find(".log");
            if (pos == std::string::npos) continue;
            std::string log_time = filename.substr(0, pos);
            // Convert to time_t for comparison
            std::tm tm = {};
            std::istringstream ss(log_time);
            ss >> std::get_time(&tm, "%Y-%m-%d-%H-%M-%S");
            if (ss.fail()) continue;
            std::time_t log_tt = std::mktime(&tm);
            
            // Current log time
            std::tm current_tm = {};
            std::istringstream css(current_log_time);
            css >> std::get_time(&current_tm, "%Y-%m-%d-%H-%M-%S");
            std::time_t current_tt = std::mktime(&current_tm);
            
            if (current_tt <= log_tt) continue; // Only consider earlier logs
            
            long long diff = current_tt - log_tt;
            if (diff < min_diff) {
                min_diff = diff;
                closest_log = filename;
            }
        }
    }
    return closest_log;
}

// Function to compare two log files and output differences
bool compareLogs(const std::map<std::string, FileInfo>& old_log,
                const std::map<std::string, FileInfo>& new_log,
                const std::string& diff_filename) {
    std::ofstream diff_file(diff_filename);
    if (!diff_file.is_open()) {
        std::cerr << "Failed to open difference file: " << diff_filename << '\n';
        return false;
    }
    
    // Identify added and modified files
    std::vector<std::string> added_files;
    std::vector<std::string> modified_files;
    
    for (const auto& [filename, new_info] : new_log) {
        auto it = old_log.find(filename);
        if (it == old_log.end()) {
            added_files.push_back(filename);
        }
        else {
            if (it->second.size_kb != new_info.size_kb ||
                it->second.modify_time != new_info.modify_time ||
                it->second.create_time != new_info.create_time) {
                modified_files.push_back(filename);
            }
        }
    }
    
    // Identify deleted files
    std::vector<std::string> deleted_files;
    for (const auto& [filename, old_info] : old_log) {
        if (new_log.find(filename) == new_log.end()) {
            deleted_files.push_back(filename);
        }
    }
    
    // Write differences to the .dif file
    if (!added_files.empty()) {
        diff_file << "新增檔案:\n";
        for (const auto& file : added_files) {
            diff_file << file << "\n";
        }
        diff_file << "\n";
    }
    
    if (!deleted_files.empty()) {
        diff_file << "刪除檔案:\n";
        for (const auto& file : deleted_files) {
            diff_file << file << "\n";
        }
        diff_file << "\n";
    }
    
    if (!modified_files.empty()) {
        diff_file << "修改檔案:\n";
        for (const auto& file : modified_files) {
            diff_file << file << "\n";
        }
        diff_file << "\n";
    }
    
    diff_file.close();
    return true;
}

int main(int argc, char* argv[]) {
    // Check for directory path argument
    if (argc < 2) {
        std::cerr << "Usage: " << argv[0] << " <directory_path>\n";
        return 1;
    }
    
    fs::path directory = argv[1];
    if (!fs::exists(directory) || !fs::is_directory(directory)) {
        std::cerr << "Invalid directory path provided.\n";
        return 1;
    }
    
    // Step 1: Record current directory files
    std::vector<FileInfo> current_files = recordFiles(directory);
    std::string current_time = getCurrentTimeString();
    std::string log_filename = current_time + ".log";
    
    if (!writeLogFile(current_files, (directory / log_filename).string())) {
        return 1;
    }
    std::cout << "Log file created: " << log_filename << "\n";
    
    // Step 2: Find the closest previous log file
    std::string closest_log = findClosestLog(directory, current_time);
    if (closest_log.empty()) {
        std::cout << "No previous log file found to compare.\n";
        return 0;
    }
    std::cout << "Comparing with log file: " << closest_log << "\n";
    
    // Step 3: Read both log files
    std::map<std::string, FileInfo> old_log = readLogFile((directory / closest_log).string());
    std::map<std::string, FileInfo> new_log;
    for (const auto& file : current_files) {
        new_log[file.filename] = file;
    }
    
    // Step 4: Compare logs and write differences
    std::string dif_filename = current_time + ".dif";
    if (compareLogs(old_log, new_log, (directory / dif_filename).string())) {
        std::cout << "Difference file created: " << dif_filename << "\n";
    } else {
        return 1;
    }
    
    return 0;
}
  

Code Explanation

a. Including Necessary Headers

The program begins by including essential headers for file operations, input/output handling, and string manipulation. The std::filesystem library is pivotal for directory traversal and file attribute retrieval.

b. Defining the FileInfo Struct

The FileInfo struct is a simple data structure designed to hold pertinent details about each file:

• filename: The name of the file.
• size_kb: The size of the file in kilobytes.
• modify_time: The last modification timestamp.
• create_time: The creation timestamp.

c. Time Formatting Functions

Accurate time representation is crucial for both logging and comparison. The getCurrentTimeString function captures the current system time and formats it into a string following the "YYYY-MM-DD-HH-MM-SS" pattern. The timePointToString function converts std::filesystem::file_time_type to a formatted string, ensuring consistency across different timestamps.

d. Recording Files with recordFiles

The recordFiles function traverses the specified directory recursively, capturing information about each regular file encountered. It utilizes std::filesystem::recursive_directory_iterator to traverse all subdirectories. For each file, it records the filename, computes the size in kilobytes, and retrieves both the modification and creation times.

e. Logging Mechanism with writeLogFile

After collecting file information, the writeLogFile function writes this data to a log file. Each line in the log file follows a comma-separated format:

filename,sizeKB,modify_time,create_time

This standardized format facilitates easy parsing and comparison in subsequent steps.

f. Reading Logs with readLogFile

The readLogFile function reads an existing log file and populates a std::map with the data. This map uses the filename as the key and the corresponding FileInfo as the value, enabling efficient lookups during the comparison phase.

g. Finding the Closest Previous Log with findClosestLog

To compare the current directory state with the previous one, the program needs to identify the most recent log file prior to the current one. The findClosestLog function scans the directory for log files, parses their timestamps, and determines which log file has the closest timestamp before the current log's timestamp.

h. Comparing Logs with compareLogs

The core functionality lies in the compareLogs function. It compares the old and new log maps to identify:

• Added Files: Files present in the new log but absent in the old one.
• Deleted Files: Files present in the old log but absent in the new one.
• Modified Files: Files whose size or timestamps have changed between logs.

The differences are then written to a ".dif" file, categorized under added, deleted, and modified sections for clarity.

i. Main Function Workflow

The main function orchestrates the program's workflow:

1. Validates the provided directory path.
2. Records the current state of the directory by invoking recordFiles and logs it using writeLogFile.
3. Identifies the closest previous log file using findClosestLog.
4. Reads both the current and previous log files.
5. Compares the logs and outputs the differences to a ".dif" file using compareLogs.

Enhancements and Best Practices

Error Handling

Robust error handling ensures that the program can gracefully handle unexpected scenarios, such as inaccessible directories or malformed log files. The program includes checks for file accessibility and validates the existence of directories before proceeding.

Cross-Platform Compatibility

While the program is designed to be cross-platform, certain functionalities like retrieving the creation time are platform-dependent. On Windows systems, the creation time can be accurately fetched using std::filesystem::creation_time, whereas on POSIX systems, this might not be available, and the modification time is used as a fallback.

Performance Optimization

For directories with a vast number of files, performance can be a concern. Utilizing efficient data structures like std::unordered_map instead of std::map for storing log data can enhance lookup speeds. Additionally, multithreading can be incorporated to parallelize directory traversal, further improving performance.

Extensibility

The program can be extended to include more file attributes, such as file permissions, owner information, or checksums for enhanced verification. Integrating logging libraries can also provide more detailed logging capabilities, facilitating easier debugging and maintenance.

Sample Output

Log File (.log)

Difference File (.dif)

Conclusion

This C++ program provides a robust solution for logging and comparing directory contents. By automating the recording of file attributes and facilitating straightforward comparisons between different directory states, it serves as a valuable tool for developers and system administrators alike. The structured approach ensures scalability and ease of maintenance, making it adaptable to various use cases and environments.

References

github.com

GitHub Repository: DirectoryFilesList

geeksforgeeks.org

GeeksforGeeks: C++ Program to Get List of Files in a Directory