NodeJS如何从aws s3 bucket下载文件到磁盘?

oxiaedzo  于 2022-11-03  发布在  Node.js
关注(0)|答案(7)|浏览(527)

我的目标:
显示一个对话框,提示用户保存正在从aws下载的文件。
我的问题:
我目前正在使用awssum-amazon-s3创建下载流。但是我只设法将文件保存到我的服务器或流到命令行...正如您从我的代码中所看到的,我最后的尝试是尝试手动设置内容处理头,但失败了。我不能使用res.download(),因为头已经设置好了。
我怎样才能实现我的目标?
我的节点代码:

  1. app.post('/dls/:dlKey', function(req, res, next){
  2. // download the file via aws s3 here
  3. var dlKey = req.param('dlKey');
  4. Dl.findOne({key:dlKey}, function(err, dl){
  5. if (err) return next(err);
  6. var files = dl.dlFile;
  7. var options = {
  8. BucketName : 'xxxx',
  9. ObjectName : files,
  10. };
  11. s3.GetObject(options, { stream : true }, function(err, data) {
  12. // stream this file to stdout
  13. fmt.sep();
  14. data.Headers['Content-Disposition'] = 'attachment';
  15. console.log(data.Headers);
  16. data.Stream.pipe(fs.createWriteStream('test.pdf'));
  17. data.Stream.on('end', function() {
  18. console.log('File Downloaded!');
  19. });
  20. });
  21. });
  22. res.end('Successful Download Post!');
  23. });

我的代码为棱角分明:

  1. $scope.dlComplete = function (dl) {
  2. $scope.procDownload = true;
  3. $http({
  4. method: 'POST',
  5. url: '/dls/' + dl.dlKey
  6. }).success(function(data/*, status, headers, config*/) {
  7. console.log(data);
  8. $location.path('/#!/success');
  9. }).error(function(/*data, status, headers, config*/) {
  10. console.log('File download failed!');
  11. });
  12. };

此代码的目的是让用户使用生成的密钥下载一次文件。

ghhaqwfi

ghhaqwfi1#

这是在aws-sdk的最新版本上使用流的完整代码

  1. var express = require('express');
  2. var app = express();
  3. var fs = require('fs');
  4. app.get('/', function(req, res, next){
  5. res.send('You did not say the magic word');
  6. });
  7. app.get('/s3Proxy', function(req, res, next){
  8. // download the file via aws s3 here
  9. var fileKey = req.query['fileKey'];
  10. console.log('Trying to download file', fileKey);
  11. var AWS = require('aws-sdk');
  12. AWS.config.update(
  13. {
  14. accessKeyId: "....",
  15. secretAccessKey: "...",
  16. region: 'ap-southeast-1'
  17. }
  18. );
  19. var s3 = new AWS.S3();
  20. var options = {
  21. Bucket : '/bucket-url',
  22. Key : fileKey,
  23. };
  24. res.attachment(fileKey);
  25. var fileStream = s3.getObject(options).createReadStream();
  26. fileStream.pipe(res);
  27. });
  28. var server = app.listen(3000, function () {
  29. var host = server.address().address;
  30. var port = server.address().port;
  31. console.log('S3 Proxy app listening at http://%s:%s', host, port);
  32. });
展开查看全部
t9aqgxwy

t9aqgxwy2#

下面的代码对我来说适用于最新的库:

  1. var s3 = new AWS.S3();
  2. var s3Params = {
  3. Bucket: 'your bucket',
  4. Key: 'path/to/the/file.ext'
  5. };
  6. s3.getObject(s3Params, function(err, res) {
  7. if (err === null) {
  8. res.attachment('file.ext'); // or whatever your logic needs
  9. res.send(data.Body);
  10. } else {
  11. res.status(500).send(err);
  12. }
  13. });
8mmmxcuj

8mmmxcuj3#

只需从S3创建一个ReadStream,然后将WriteStream写到你想下载的位置。找到下面的代码。对我来说非常有效:

  1. var AWS = require('aws-sdk');
  2. var path = require('path');
  3. var fs = require('fs');
  4. AWS.config.loadFromPath(path.resolve(__dirname, 'config.json'));
  5. AWS.config.update({
  6. accessKeyId: AWS.config.credentials.accessKeyId,
  7. secretAccessKey: AWS.config.credentials.secretAccessKey,
  8. region: AWS.config.region
  9. });
  10. var s3 = new AWS.S3();
  11. var params = {
  12. Bucket: '<your-bucket>',
  13. Key: '<path-to-your-file>'
  14. };
  15. let readStream = s3.getObject(params).createReadStream();
  16. let writeStream = fs.createWriteStream(path.join(__dirname, 's3data.txt'));
  17. readStream.pipe(writeStream);
展开查看全部
yh2wf1be

yh2wf1be4#

您已经知道解决问题最重要的是什么:您可以将来自S3的文件流通过管道传输到任何可写流,无论是文件流...还是将发送到客户端的响应流!

  1. s3.GetObject(options, { stream : true }, function(err, data) {
  2. res.attachment('test.pdf');
  3. data.Stream.pipe(res);
  4. });

注意res.attachment的使用,它将设置正确的头。你也可以检查关于流和S3的this answer

hgc7kmma

hgc7kmma5#

使用aws SDK v3

  1. npm install @aws-sdk/client-s3

下载代码

  1. import { GetObjectCommand } from "@aws-sdk/client-s3";
  2. /**
  3. * download a file from AWS and send to your rest client
  4. */
  5. app.get('/download', function(req, res, next){
  6. var fileKey = req.query['fileKey'];
  7. var bucketParams = {
  8. Bucket: 'my-bucket-name',
  9. Key: fileKey,
  10. };
  11. res.attachment(fileKey);
  12. var fileStream = await s3Client.send(new GetObjectCommand(bucketParams));
  13. // for TS you can add: if (fileStream.Body instanceof Readable)
  14. fileStream.Body.pipe(res)
  15. });
展开查看全部
k4emjkb1

k4emjkb16#

为此,我使用React frontendnode js backend。前端我使用Axios。我用这个点击按钮下载文件。
====节点js后端代码(AWS S3)======
//在GET方法内部调用了此函数

  1. public download = (req: Request, res: Response) => {
  2. const keyName = req.query.keyName as string;
  3. if (!keyName) {
  4. throw new Error('key is undefined');
  5. }
  6. const downloadParams: AWS.S3.GetObjectRequest = {
  7. Bucket: this.BUCKET_NAME,
  8. Key: keyName
  9. };
  10. this.s3.getObject(downloadParams, (error, data) => {
  11. if (error) {
  12. return error;
  13. }
  14. res.send(data.Body);
  15. res.end();
  16. });
  17. };

======Reactjs前端代码========
//此函数处理下载按钮onClick

  1. const downloadHandler = async (keyName: string) => {
  2. const response = await axiosInstance.get( //here use axios interceptors
  3. `papers/paper/download?keyName=${keyName}`,{
  4. responseType:'blob', //very very important dont miss (if not downloaded file unsupported to view)
  5. }
  6. );
  7. const url = window.URL.createObjectURL(new Blob([response.data]));
  8. const link = document.createElement("a");
  9. link.href = url;
  10. link.setAttribute("download", "file.pdf"); //change "file.pdf" according to saved name you want, give extension according to filetype
  11. document.body.appendChild(link);
  12. link.click();
  13. link.remove();
  14. };

------ OR(如果您使用的是标准axios而不是axios拦截器)-----

  1. axios({
  2. url: 'http://localhost:5000/static/example.pdf',
  3. method: 'GET',
  4. responseType: 'blob', // very very important
  5. }).then((response) => {
  6. const url = window.URL.createObjectURL(new Blob([response.data]));
  7. const link = document.createElement('a');
  8. link.href = url;
  9. link.setAttribute('download', 'file.pdf');
  10. document.body.appendChild(link);
  11. link.click();
  12. });

有关更多信息,请参阅下面的文章1. article 1 2. article 2

展开查看全部
2q5ifsrm

2q5ifsrm7#

使用express,基于Jushua的答案和https://docs.aws.amazon.com/AmazonS3/latest/userguide/example_s3_GetObject_section.html

  1. public downloadFeedFile = (req: IFeedUrlRequest, res: Response) => {
  2. const downloadParams: GetObjectCommandInput = parseS3Url(req.s3FileUrl.replace(/\s/g, ''));
  3. logger.info("requesting S3 file " + JSON.stringify(downloadParams));
  4. const run = async () => {
  5. try {
  6. const fileStream = await this.s3Client.send(new GetObjectCommand(downloadParams));
  7. if (fileStream.Body instanceof Readable){
  8. fileStream.Body.once('error', err => {
  9. console.error("Error downloading s3 file")
  10. console.error(err);
  11. });
  12. fileStream.Body.pipe(res);
  13. }
  14. } catch (err) {
  15. logger.error("Error", err);
  16. }
  17. };
  18. run();
  19. };
展开查看全部

相关问题