Streaming Large Financial Exports: Preventing Memory Overflows

Exporting financial ledgers, invoice lists, or inventory movement logs containing tens of thousands of records can easily trigger PHP memory limit crashes (Fatal error: Allowed memory size exhausted).

Loading entire Eloquent collection models into memory prior to writing a CSV or Excel file is an anti-pattern. Here is how we engineered a chunked HTTP stream response using database cursors to output large files with a constant memory footprint under 10MB.

1. Utilizing Eloquent Cursors for Low Memory Overhead

Instead of calling ->get(), use ->cursor(). Eloquent Cursors fetch single database rows sequentially using PDO generators without hydrating all records simultaneously into memory.

php

class=class="text-emerald-500 dark:text-emerald-400 font-medium">"text-slate-500 italic">// app/Http/Controllers/FinancialExportController.php
namespace App\Http\Controllers;

use App\Models\Transaction;
use Illuminate\Http\Request;
use Symfony\Component\HttpFoundation\StreamedResponse;

class FinancialExportController extends Controller
{
    public function exportCsv(Request $request): StreamedResponse
    {
        $companyId = $request->user()->active_company_id;
        $fileName = 'financial_export_' . date('Y_m_d_His') . '.csv';

        $headers = [
            'Content-Type' => 'text/csv',
            'Content-Disposition' => "attachment; filename="{$fileName}"",
            'Pragma' => 'no-cache',
            'Cache-Control' => 'must-revalidate, post-check=0, pre-check=0',
            'Expires' => '0',
        ];

        return response()->stream(function () use ($companyId) {
            $output = fopen('php:class=class="text-emerald-500 dark:text-emerald-400 font-medium">"text-slate-500 italic">//output', 'w');

            class=class="text-emerald-500 dark:text-emerald-400 font-medium">"text-slate-500 italic">// Write CSV BOM for UTF-8 compatibility with MS Excel
            fputs($output, $bom = (chr(0xEF) . chr(0xBB) . chr(0xBF)));

            class=class="text-emerald-500 dark:text-emerald-400 font-medium">"text-slate-500 italic">// Write CSV Header Row
            fputcsv($output, ['Transaction ID', 'Date', 'Type', 'Amount', 'Status']);

            class=class="text-emerald-500 dark:text-emerald-400 font-medium">"text-slate-500 italic">// Stream rows via Lazy Query Cursor
            Transaction::where('company_id', $companyId)
                ->orderBy('created_at', 'desc')
                ->cursor()
                ->each(function (Transaction $tx) use ($output) {
                    fputcsv($output, [
                        $tx->reference_number,
                        $tx->created_at->format('Y-m-d H:i:s'),
                        strtoupper($tx->type),
                        number_format($tx->amount, 2, '.', ''),
                        $tx->status,
                    ]);
                });

            fclose($output);
        }, 200, $headers);
    }
}

2. Handling Streamed Downloads in Vue 3 with Blob Responses

To download streamed binary or text endpoints without breaking single-page application context:

class=class="text-emerald-500 dark:text-emerald-400 font-medium">"text-slate-500 italic">// services/exportService.ts
import axios from 'axios';

export async function downloadFinancialExport(): Promise<void> {
    const response = await axios.get('/api/exports/financial-csv', {
        responseType: 'blob',
    });

    const blob = new Blob([response.data], { type: 'text/csv;charset=utf-8;' });
    const link = document.createElement('a');
    const url = URL.createObjectURL(blob);
    
    link.setAttribute('href', url);
    link.setAttribute('download', class="text-emerald-500 dark:text-emerald-400 font-medium">`financial_export_${Date.now()}.csv`);
    link.style.visibility = 'hidden';
    document.body.appendChild(link);
    link.click();
    document.body.removeChild(link);
}

Streamed HTTP cursor responses enabled our platform to process 100,000+ transaction logs instantaneously with zero memory spikes or worker timeouts.

Streaming Large Financial Exports: Preventing Memory Overflows in Laravel & Vue 3