CHANGE LOG

Change Log

INTRODUCTION

Built on n8n, this powerful automation system enables you to transform large CSV files into manageable, processed batches, facilitating more efficient data handling and analysis. The workflow automatically monitors Google Drive folders, processes incoming CSV files, and splits them into consistent batches of 1,200 records each.

This streamlined system turns cumbersome data files into organized, actionable batches that can be easily processed by downstream systems or analyzed in smaller chunks.

How It Works

At the system's core is a powerful automation workflow that handles everything from file monitoring to batch creation, with intelligent processing ensuring high-quality output.

📁 Step 1: Capture CSV Files via Google Drive

• The system activates when a new CSV file is uploaded to a monitored Google Drive folder
• The file details are captured and processed through n8n for further handling

🔄 Step 2: Automated Processing & Batching

• The system connects multiple tools for different tasks:
  • Google Drive for file monitoring and storage
  • CSV parsing for data extraction
  • Batch processing for dividing large datasets
  • File conversion for standardized output
  • Notification systems for process completion

📊 Step 3: Multi-Stage Processing Pipeline

The workflow operates through several coordinated processes:

Input Processing Path:

• Google Drive file monitoring and detection

• File download and preparation

• CSV parsing and data extraction

• Record counting and validation

  Batch Creation Path:

• Data division into consistent 1,200-record batches

• Batch conversion to standardized CSV format