c++ 如何使用boost::beast,下载一个文件没有阻止和响应

5vf7fwbs  于 2022-12-01  发布在  其他
关注(0)|答案(2)|浏览(266)

我已经从this example开始,所以不会发布所有的代码。我的目标是下载一个大文件而不阻塞我的主线程。第二个目标是得到通知,这样我就可以更新进度条。我有几种方法来运行代码。第一种是只运行ioc.run();,让它去工作,我得到的文件下载。但我找不到无论如何启动会话没有阻止。
第二种方法我可以向下调用http::async_read_some,调用成功了,但是我不能得到一个我可以使用的响应。我不知道是否有一种方法可以传递一个捕获的lambda。
#if 0..#else..#endif切换方法。我确信有一个简单的方法,但我就是看不到它。当我让它工作时,我会清理代码,比如设置本地文件名。谢谢。

std::size_t on_read_some(boost::system::error_code ec, std::size_t bytes_transferred)
    {
        if (ec);//deal with it... 
        if (!bValidConnection) {
            std::string_view view((const char*)buffer_.data().data(), bytes_transferred);
            auto pos = view.find("Content-Length:");
            if (pos == std::string_view::npos)
                ;//error
            file_size = std::stoi(view.substr(pos+sizeof("Content-Length:")).data());
            if (!file_size)
                ;//error
            bValidConnection = true;
        }
        else {
            file_pos += bytes_transferred;
            response_call(ec, file_pos);
        }
#if 0
        std::cout << "in on_read_some caller\n";
        http::async_read_some(stream_, buffer_, file_parser_, std::bind(
            response_call,
            std::placeholders::_1,
            std::placeholders::_2));
#else
        std::cout << "in on_read_some inner\n";
        http::async_read_some(stream_, buffer_, file_parser_, std::bind(
            &session::on_read_some,
            shared_from_this(),
            std::placeholders::_1,
            std::placeholders::_2));
#endif
        return buffer_.size();
    }

最主要的,凌乱不过.....

struct lambda_type {
    bool bDone = false;
    void operator ()(const boost::system::error_code ec, std::size_t bytes_transferred) {
        ;
    }
};
int main(int argc, char** argv)
{
    auto const host = "reserveanalyst.com";
    auto const port = "443";
    auto const target = "/downloads/demo.msi";
    int version = argc == 5 && !std::strcmp("1.0", argv[4]) ? 10 : 11;

    boost::asio::io_context ioc;
    ssl::context ctx{ ssl::context::sslv23_client };

    load_root_certificates(ctx);
    //ctx.load_verify_file("ca.pem");

    auto so = std::make_shared<session>(ioc, ctx);
    so->run(host, port, target, version);

    bool bDone = false;
    auto const lambda = [](const boost::system::error_code ec, std::size_t bytes_transferred) {
        std::cout << "data lambda bytes: " << bytes_transferred << " er: " << ec.message() << std::endl;
    };

    lambda_type lambda2;
    so->set_response_call(lambda);
    ioc.run();

    std::cout << "not in ioc.run()!!!!!!!!" << std::endl;

    so->async_read_some(lambda);

    //pseudo message pump when working.........
    for (;;) {
        std::this_thread::sleep_for(250ms);
        std::cout << "time" << std::endl;
    }
    return EXIT_SUCCESS;
}

我在class session中添加的内容

class session : public std::enable_shared_from_this<session>
{
        using response_call_type = void(*)(boost::system::error_code ec, std::size_t bytes_transferred);
        http::response_parser<http::file_body> file_parser_;
        response_call_type response_call;
        //
        bool bValidConnection = false;
        std::size_t file_pos = 0;
        std::size_t file_size = 0;
    
    public:
        auto& get_result() { return res_; }
        auto& get_buffer() { return buffer_; }
        void set_response_call(response_call_type the_call) { response_call = the_call; }
kd3sttzy

kd3sttzy1#

我已经更新了这个,因为我终于把它使用,我想旧的方法,我可以下载到一个文件或一个字符串。链接到如何asio工作,伟大的谈话。
CppCon 2016 Michael Caisse Asynchronous IO with BoostAsio
至于我对如何传递lambda的误解,下面是Adam Nevraumont的答案
有两种方法可以编译这个,使用一个类型来选择方法。两种方法都显示在main的开头。你可以通过选择beast解析器的类型来构造一个文件下载器或字符串下载器。解析器没有相同的构造,所以使用了if constexpr编译时条件。我检查了一下,下载器的发布版本大约是1 K,所以它的工作量相当轻。在一个小字符串的情况下,你不必处理回调。要么传递一个空的lambda,要么添加类似如下的内容:

if(response_call)
    response_call(resp_ok, test);

这看起来是一个相当干净的方式来完成这项工作,所以我已经更新了这篇文章作为11/27/2202。
代码:

//
// Copyright (c) 2016-2019 Vinnie Falco (vinnie dot falco at gmail dot com)
//
// Distributed under the Boost Software License, Version 1.0. (See accompanying
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
//
// Official repository: https://github.com/boostorg/beast

//------------------------------------------------------------------------------
//
// Example: HTTP SSL client, synchronous, usable in a thread with a message pump
// Added code to use from a message pump
// Also useable as body to a file download, or body to string
//
//------------------------------------------------------------------------------

#include <boost/beast/core.hpp>
#include <boost/beast/http.hpp>
#include <boost/beast/ssl.hpp>
#include <boost/beast/version.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ssl/error.hpp>
#include <boost/asio/ssl/stream.hpp>
#include <cstdlib>
#include <iostream>
#include <string>
#include <fstream>

//the boost shipped certificates
#include <boost/../libs/beast/example/common/root_certificates.hpp>

//TODO add your ssl libs as you would like
#ifdef _M_IX86
#pragma comment(lib, "libcrypto.lib")
#pragma comment(lib, "libssl.lib")
#elif _M_X64
#pragma comment(lib, "libcrypto-3-x64.lib")
#pragma comment(lib, "libssl-3-x64.lib")
#endif

namespace downloader {
    namespace beast = boost::beast; // from <boost/beast.hpp>
    namespace http = beast::http;   // from <boost/beast/http.hpp>
    namespace net = boost::asio;    // from <boost/asio.hpp>
    namespace ssl = net::ssl;       // from <boost/asio/ssl.hpp>
    using tcp = net::ip::tcp;       // from <boost/asio/ip/tcp.hpp>

    //specialization if using < c++17; see both 'if constexpr' below.
    //this is not needed otherwise
    //namespace detail {
    //    template<typename Type>
    //    void open_file(http::parser < false, Type>& p, const char* name, boost::system::error_code& file_open_ec) { }
    //    template<>
    //    void open_file(http::parser<false, http::file_body>& p, const char* name, boost::system::error_code& file_open_ec) {
    //        p.get().body().open(name, boost::beast::file_mode::write, file_open_ec);
    //    }
    //    template<typename Type>
    //    std::string get_string(http::parser < false, Type>& p) { return std::string{}; }
    //    template<>
    //    std::string get_string(http::parser<false, http::string_body>& p) {
    //        return p.get().body();
    //    }
    //} //namespace detail

    enum responses {
        resp_null,
        resp_ok,
        resp_done,
        resp_error,
    };
    using response_call_type = std::function< void(responses, std::size_t)>;
 
    template<typename ParserType>
    struct download {
        //as these can be set with array initialization
        const char* target_ = "/";
        const char* filename_ = "test.txt";
        const char* host_ = "lakeweb.net";

        std::string body_;
        using response_call_type = std::function< void(responses, std::size_t)>;
        response_call_type response_call;

        boost::asio::io_context     ioc_;
        ssl::context                ctx_{ ssl::context::sslv23_client };
        ssl::stream<tcp::socket>    stream_{ ioc_, ctx_ };
        tcp::resolver               resolver_{ ioc_ };
        boost::beast::flat_buffer   buffer_;
        uint64_t                     file_size_{};
        int                         version{ 11 };

        void set_response_call(response_call_type the_call) { response_call = the_call; }
        uint64_t get_file_size() { return file_size_; }
        void stop() { ioc_.stop(); }
        bool stopped() { return ioc_.stopped(); }
        std::string get_body() { return std::move(body_); }
        void run() {
            try {
                // TODO should have a timer in case of a hang
                load_root_certificates(ctx_);

                // Set SNI Hostname (many hosts need this to handshake successfully)
                if (!SSL_set_tlsext_host_name(stream_.native_handle(), host_)) {
                    boost::system::error_code ec{ static_cast<int>(::ERR_get_error()), boost::asio::error::get_ssl_category() };
                    throw boost::system::system_error{ ec };
                }
                //TODO resolve is depreciated, use endpoint
                auto const results = resolver_.resolve(host_, "443");

                boost::asio::connect(stream_.next_layer(), results.begin(), results.end());
                stream_.handshake(ssl::stream_base::client);

                // Set up an HTTP GET request message
                http::request<http::string_body> req{ http::verb::get, target_, version };
                req.set(http::field::host, host_);
                req.set(http::field::user_agent, "mY aGENT");

                // Send the HTTP request to the remote host
                http::write(stream_, req);

                // Read the header
                boost::system::error_code file_open_ec;
                http::parser<false, ParserType> p;
                p.body_limit((std::numeric_limits<std::uint32_t>::max)());

                //detail::open_file(p, filename_, file_open_ec);
                //or => c++17
                if constexpr (std::is_same_v<ParserType, http::file_body>)
                    p.get().body().open(filename_, boost::beast::file_mode::write, file_open_ec);

                http::read_header(stream_, buffer_, p);
                file_size_ = p.content_length().has_value() ? p.content_length().value() : 0;

                //Read the body
                uint64_t test{};
                boost::system::error_code rec;
                for (;;) {
                    test += http::read_some(stream_, buffer_, p, rec);
                    if (test >= file_size_) {
                        response_call(resp_done, 0);
                        break;
                    }
                    response_call(resp_ok, test);
                }

                // Gracefully close the stream
                boost::system::error_code ec;
                stream_.shutdown(ec);
                if (ec == boost::asio::error::eof)
                {
                    // Rationale:
                    // http://stackoverflow.com/questions/25587403/boost-asio-ssl-async-shutdown-always-finishes-with-an-error
                    ec.assign(0, ec.category());
                }
                if (ec)
                    throw boost::system::system_error{ ec };

                //value = detail::get_string(p);
                //or => c++17
                if constexpr (std::is_same_v<ParserType, http::string_body>)
                    body_ = p.get().body();
            }
            catch (std::exception const& e)
            {
                std::cerr << "Error: " << e.what() << std::endl;
                response_call(resp_error, -1);
            }
            ioc_.stop();
        }
    };
}//namespace downloadns

//comment to test with string body
#define THE_FILE_BODY_TEST
int main(int argc, char** argv)
{
    using namespace downloader;

#ifdef THE_FILE_BODY_TEST
    download<http::file_body> dl{"/Nasiri%20Abarbekouh_Mahdi.pdf", "test.pdf"};
#else //string body test
    download<http::string_body> dl{ "/robots.txt" };
#endif

    responses dl_response{ resp_null };
    size_t cur_size{};

    auto static const lambda = [&dl_response, &dl, &cur_size](responses response, std::size_t bytes_transferred) {
        if ((dl_response = response) == resp_ok) {
            cur_size += bytes_transferred;
            size_t sizes = dl.get_file_size() - cur_size;//because size is what is left
            //drive your progress bar from here in a GUI app
        }
    };
    dl.set_response_call(lambda);
    std::thread thread{ [&dl]() { dl.run(); } };

    //thread has started, now the pseudo message pump
    bool quit = false; //true: as if a cancel button was pushed; won't finish download
    for (int i = 0; ; ++i) {

        switch (dl_response) { //ad hoc as if messaged
        case resp_ok:
            std::cout << "from sendmessage: " << cur_size << std::endl;
            dl_response = resp_null;
            break;
        case resp_done:
            std::cout << "from sendmessage: done" << std::endl;
            dl_response = resp_null;
            break;
        case resp_error:
            std::cout << "from sendmessage: error" << std::endl;
            dl_response = resp_null;
        }//switch

        if (!(i % 5))
            std::cout << "in message pump, stopped: " << std::boolalpha << dl.stopped() << std::endl;

        std::this_thread::sleep_for(std::chrono::milliseconds(100));
        if (quit && i == 10) //the cancel message
            dl.stop();
        if (!(i % 20) && dl.stopped()) {//dl job was quit or error or finished
            std::cout << "dl is stopped" << std::endl;
            break;
        }
    }
#ifdef THE_FILE_BODY_TEST
    std::cout << "file written named: 'test.txt'" << std::endl;
#else
    std::string res = dl.get_body();
    std::cout << "body retrieved:\n" << res << std::endl;
#endif
    if (thread.joinable())//in the case a thread was never started
        thread.join();
    std::cout << "exiting, program all done" << std::endl;

    return EXIT_SUCCESS;
}
vm0i2vca

vm0i2vca2#

我强烈建议不要使用低级的[async_]read_some函数,而不要使用http::response_parser<http::buffer_body>
我确实有一个这样的例子--这个例子有点复杂,因为它还使用Boost Process来并发地解压缩正文数据,但不管怎样,它应该告诉你如何使用它:
如何使用多线程技术,一次连接就能从Internet上读取数据?
我想我可以给出更完整的代码,根据您的特定示例来定制它,但是也许上面的代码已经足够好了?还可以参见libs/beast/example/doc/http_examples.hpp中的“中继HTTP消息”,我将其用作“灵感”。
注意:缓冲区算法并不直观。我认为这是不幸的,不应该是必要的,所以请(非常)密切关注这些样本,以了解它是如何完成的。

相关问题