您遇到的错误消息表明您的JavaScript堆已耗尽内存。处理大文件或同时处理大量文件时可能会发生这种情况。为了解决这个问题，您可以尝试一些优化：

Increase the memory limit for Node.js: You can try increasing the memory limit for your Node.js process by setting the --max-old-space-size flag when running your script. For example:

node --max-old-space-size=4096 your-script.js

这会将内存限制增加到4GB（4096MB）。根据您的需要调整值。

Process files sequentially: Instead of processing all the files simultaneously, you can modify your code to process them one by one. This reduces the memory usage since only one file is being processed at a time. You can use a loop and an await statement to ensure each file is processed before moving on to the next one.

Use a cursor-based approach for MongoDB: Instead of loading the entire file content into memory and parsing it as a JSON object, consider using a cursor-based approach with MongoDB. This allows you to process the file in chunks without loading the entire file into memory at once. MongoDB's createReadStream() method returns a readable stream, which can be piped directly to another writable stream (e.g., a file or another database) without loading the entire content into memory.

通过实施这些优化，您可以减少内存使用并更有效地处理大文件。下面是如何修改代码的示例：

import axios from 'axios';
import AdmZip from 'adm-zip';
import { Storage } from '@google-cloud/storage';
import mongoose from 'mongoose';
import { Readable } from 'stream';

const storage = new Storage();
const bucketName = process.env.BUCKET_NAME;

async function fetchCompanyFilingsData() {
  const response = await axios.get(
    'https://www.sec.gov/Archives/edgar/daily-index/bulkdata/submissions.zip',
    { responseType: 'arraybuffer' }
  );
  if (!response.data) {
    throw new Error('No data received from URL.');
  }
  return response.data;
}

async function unzipAndUploadData(buffer: Buffer) {
  const zip = new AdmZip(buffer);
  const files = zip.getEntries().filter((file) => /\.json$/i.test(file.name));

  if (!files.length) {
    throw new Error('No JSON files found in ZIP archive.');
  }

  for (const file of files) {
    const bucket = storage.bucket(bucketName);
    const blob = bucket.file(file.name);
    const blobStream = blob.createWriteStream();

    blobStream.on('error', (err) => {
      console.error(err);
    });

    await new Promise((resolve, reject) => {
      blobStream.on('finish', resolve);
      blobStream.on('error', reject);

      const readableStream = new Readable();
      readableStream.push(file.getData());
      readableStream.push(null);

      readableStream.pipe(blobStream);
    });

    console.log(`Uploaded ${file.name} to Google Cloud Storage`);
  }
}

async function saveDataToMongoDB() {
  const filingsCollection = mongoose.connection.db.collection('edgar-filings');
  const bucket = storage.bucket(bucketName);
  const [files] = await bucket.getFiles();

  for (const file of files) {
    if (file.name.endsWith('.json')) {
      await new Promise((resolve, reject) => {
        const contentStream = file.createReadStream();
        let data = '';

        contentStream.on('data', (chunk) => {
          data += chunk.toString();
        });

        contentStream.on('end', async () => {
          const jsonData = JSON.parse(data);

          const existingFiling = await filingsCollection.findOne({ cik: jsonData.cik });
          if (!existingFiling) {
            const result = await filingsCollection.insertOne(jsonData);
            console.log('Inserted filing:', result);
          } else {
            console.log('Filing already exists:', jsonData);
          }

          resolve();
        });

        contentStream.on('error', reject);
      });
    }
  }
}

export async function processCompanyFilingsData() {
  try {
    const buffer = await fetchCompanyFilingsData();
    await unzipAndUploadData(buffer);

    await mongoose.connect(process.env.MONGODB_URI);
    await saveDataToMongoDB();
    await mongoose.disconnect();

    console.log('Data saved to MongoDB');
  } catch (error) {
    console.error(error);
  }
}

这些修改应该有助于在处理大文件并将其存储在MongoDB中时减少内存使用。