通常,Node.js如何处理10,000个并发请求?

abithluo  于 2022-12-03  发布在  Node.js
关注(0)|答案(9)|浏览(177)

我知道Node.js使用单线程和事件循环来处理请求,一次只处理一个请求(这是非阻塞的)。但是,这是如何工作的呢?假设有10,000个并发请求。事件循环将处理所有请求?这会不会花费太长时间?
我还不明白它怎么会比多线程web服务器快。我知道多线程web服务器在资源(内存、CPU)上会更贵,但它不是更快吗?我可能错了;请解释此单线程如何在处理大量请求时更快,以及在处理大量请求(如10,000个)时它通常会做什么(高级)。
另外,单线程是否能很好地适应这么大的数据量?请记住,我才刚刚开始学习Node.js。

ruarlubt

ruarlubt1#

If you have to ask this question then you're probably unfamiliar with what most web applications/services do. You're probably thinking that all software do this:

user do an action
       │
       v
 application start processing action
   └──> loop ...
          └──> busy processing
 end loop
   └──> send result to user

However, this is not how web applications, or indeed any application with a database as the back-end, work. Web apps do this:

user do an action
       │
       v
 application start processing action
   └──> make database request
          └──> do nothing until request completes
 request complete
   └──> send result to user

In this scenario, the software spend most of its running time using 0% CPU time waiting for the database to return.

Multithreaded network app:

Multithreaded network apps handle the above workload like this:

request ──> spawn thread
              └──> wait for database request
                     └──> answer request
request ──> spawn thread
              └──> wait for database request
                     └──> answer request
request ──> spawn thread
              └──> wait for database request
                     └──> answer request

So the thread spend most of their time using 0% CPU waiting for the database to return data. While doing so they have had to allocate the memory required for a thread which includes a completely separate program stack for each thread etc. Also, they would have to start a thread which while is not as expensive as starting a full process is still not exactly cheap.

Singlethreaded event loop

Since we spend most of our time using 0% CPU, why not run some code when we're not using CPU? That way, each request will still get the same amount of CPU time as multithreaded applications but we don't need to start a thread. So we do this:

request ──> make database request
request ──> make database request
request ──> make database request
database request complete ──> send response
database request complete ──> send response
database request complete ──> send response

In practice both approaches return data with roughly the same latency since it's the database response time that dominates the processing.
The main advantage here is that we don't need to spawn a new thread so we don't need to do lots and lots of malloc which would slow us down.

Magic, invisible threading

The seemingly mysterious thing is how both the approaches above manage to run workload in "parallel"? The answer is that the database is threaded. So our single-threaded app is actually leveraging the multi-threaded behaviour of another process: the database.

Where singlethreaded approach fails

A singlethreaded app fails big if you need to do lots of CPU calculations before returning the data. Now, I don't mean a for loop processing the database result. That's still mostly O(n). What I mean is things like doing Fourier transform (mp3 encoding for example), ray tracing (3D rendering) etc.
Another pitfall of singlethreaded apps is that it will only utilise a single CPU core. So if you have a quad-core server (not uncommon nowdays) you're not using the other 3 cores.

Where multithreaded approach fails

A multithreaded app fails big if you need to allocate lots of RAM per thread. First, the RAM usage itself means you can't handle as many requests as a singlethreaded app. Worse, malloc is slow. Allocating lots and lots of objects (which is common for modern web frameworks) means we can potentially end up being slower than singlethreaded apps. This is where node.js usually win.
One use-case that end up making multithreaded worse is when you need to run another scripting language in your thread. First you usually need to malloc the entire runtime for that language, then you need to malloc the variables used by your script.
So if you're writing network apps in C or go or java then the overhead of threading will usually not be too bad. If you're writing a C web server to serve PHP or Ruby then it's very easy to write a faster server in javascript or Ruby or Python.

Hybrid approach

Some web servers use a hybrid approach. Nginx and Apache2 for example implement their network processing code as a thread pool of event loops. Each thread runs an event loop simultaneously processing requests single-threaded but requests are load-balanced among multiple threads.
Some single-threaded architectures also use a hybrid approach. Instead of launching multiple threads from a single process you can launch multiple applications - for example, 4 node.js servers on a quad-core machine. Then you use a load balancer to spread the workload amongst the processes. The cluster module in node.js does exactly this.
In effect the two approaches are technically identical mirror-images of each other.

kmb7vmvb

kmb7vmvb2#

你似乎在想,大多数处理都是在节点事件循环中处理的。节点实际上将I/O工作转移给了线程。I/O操作通常比CPU操作要长几个数量级,所以为什么要让CPU等待呢?此外,操作系统已经可以很好地处理I/O任务了。事实上,因为节点不需要等待,所以它实现了更高的CPU利用率。
打个比方,可以把NodeJS想象成一个服务员在厨房里为顾客点菜,而I/O厨师则在厨房里为顾客做饭。其他系统有多个厨师,他们为顾客点菜、做饭、收拾table,然后才去接待下一位顾客。

oaxa6hgo

oaxa6hgo3#

单线程事件循环模型处理步骤:

  • 客户端将请求发送到Web服务器。
  • Node JS Web Server内部维护一个有限线程池,为客户端请求提供服务。
  • NodeJS Web服务器接收这些请求并将它们放入一个队列中,称为“事件队列”。
  • Node JS Web Server内部有一个Component,称为“Event Loop”,之所以取这个名字,是因为它使用了不定循环来接收请求并处理。
  • 事件循环只使用单线程,它是 NodeJS 平台处理模型的核心。
  • 事件循环检查是否将任何客户端请求放入事件队列中如果没有,则无限期等待传入请求

1.如果是,则从事件队列中选取一个客户端请求
1.启动客户端请求
1.如果客户端请求不需要任何阻塞IO操作,则处理所有内容,准备响应并将其发送回客户端。
1.如果该客户端请求需要一些阻塞IO操作(如与数据库、文件系统、外部服务交互),则将采用不同的方法
1.检查内部线程池中线程的可用性
1.选取一个线程并将此客户端请求分配给该线程。
1.该线程负责接受请求、处理请求、执行阻塞IO操作、准备响应并将其发送回事件循环
非常好地解释了@Rambabu Posa更多的解释去扔这个Link

yqkkidmi

yqkkidmi4#

据我所知,Node.js使用单线程和事件循环来处理请求,一次只处理一个请求(这是非阻塞的)。
我可能误解了您在这里所说的,但是“一次一个”听起来您可能没有完全理解基于事件的体系结构。
在“传统的”(非事件驱动的)应用程序架构中,进程会花很多时间等待某件事情的发生,而在基于事件的架构(如Node.js)中,进程不会只是等待,它可以继续其他工作。
例如:你从客户端获得一个连接,你接受它,你读取请求头(在http的情况下),然后你开始对请求采取行动。你可能会读取请求主体,你通常会把一些数据发送回客户端(这是一个有意简化的过程,只是为了演示这一点)。
在每一个阶段,大部分时间都花在等待一些数据从另一端到达--在主JS线程中处理所花的实际时间通常相当少。
当I/O对象(例如网络连接)的状态改变使得它需要处理时(例如,在套接字上接收到数据,套接字变为可写等),主Node.js JS线程被唤醒,具有需要处理的项目的列表。
它找到相关的数据结构,并在该结构上发出一些事件,这些事件导致回调运行,回调处理传入的数据,或将更多数据写入套接字,等等。一旦所有需要处理的I/O对象都得到处理,主Node.js JS线程将再次等待,直到它被告知有更多数据可用(或一些其他操作已完成或超时)。
下一次它被唤醒时,很可能是由于需要处理不同的I/O对象--例如不同的网络连接。每次,相关的回调都会运行,然后它会回到睡眠状态,等待其他事情发生。
重要的一点是,不同请求的处理是交错的,它不会从头到尾处理一个请求,然后再处理下一个请求。
在我看来,这种方法的主要优点是,慢速请求(例如,您试图通过2G数据连接向移动的电话设备发送1 MB的响应数据,或者您正在进行非常慢的数据库查询)不会阻止快速请求。
在传统的多线程Web服务器中,通常每个请求都有一个线程处理,它只处理该请求,直到它完成。如果有很多慢速请求,会发生什么?最终,很多线程都在处理这些请求,而其他请求(可能是非常简单的请求,可以非常快速地处理)则排在它们后面。
除了Node.js之外,还有很多其他基于事件的系统,它们与传统模型相比有着相似的优点和缺点。
我并不认为基于事件的系统在任何情况下或处理任何工作负载时都更快-它们往往适合处理I/O密集型工作负载,而不适合处理CPU密集型工作负载。

m2xkgtsf

m2xkgtsf5#

斯莱德曼回答:当您说Node.JS可以处理10,000个并发请求时,它们本质上是非阻塞请求,即这些请求主要与数据库查询有关。
在内部,Node.JSevent loop正在处理thread pool,其中每个线程处理non-blocking request,并且事件循环在将工作委派给thread pool的线程之一之后继续监听更多请求。当线程之一完成工作时,它向X1 M6 N1 X发送它已经完成AKA X1 M7 N1 X的信号。X1 M8 N1 X然后处理该回叫并发回响应。
由于您是NodeJS的新手,请阅读更多关于nextTick的内容,以了解事件循环内部是如何工作的。阅读http://javascriptissexy.com上的博客,当我开始使用JavaScript/NodeJS时,它们对我真的很有帮助。

3xiyfsfu

3xiyfsfu6#

多线程阻塞系统中的阻塞部分降低了系统的效率。被阻塞的线程在等待响应时不能做任何事情。
而非阻塞单线程系统则充分利用了其单线程系统。
见下图:

在厨房门口等待或在顾客选择食物时等待,是“阻塞”服务员的全部能力。在计算系统的意义上,它可以是等待IO,或DB响应或阻塞整个线程的任何东西,即使线程在等待时能够进行其他工作。
让我们看看非阻塞的工作原理:


在一个非阻塞系统中,服务员只接受订单和服务订单,不等待在任何地方。他分享他的移动的号码,给予一个回电话当他们完成订单。同样,他分享他的号码与厨房回电话时,订单准备好服务。
这就是Event循环在NodeJS中的工作方式,并且比阻塞多线程系统的性能更好。

hs1ihplo

hs1ihplo7#

添加到slebetman的答案中,以更清楚地了解执行代码时会发生什么。
nodeJs中的内部线程池默认只有4个线程,并且它不像整个请求被附加到线程池中的新线程上,请求的整个执行过程就像任何普通请求一样(没有任何阻塞任务),只是每当请求具有任何长时间运行或繁重操作(如DB调用)时,文件操作或HTTP请求,任务被排队到由libuv提供的内部线程池。由于nodeJ默认在内部线程池中提供4个线程,因此每第5个或下一个并发请求等待一个线程,直到该线程空闲,并且一旦这些操作结束时,回调被推送到回调队列。事件循环将拾取和,并发送回响应。
现在又来了一个信息,它不是一个单一的回调队列,有许多队列。

  1. NextTick队列
    1.微任务队列
    1.计时器队列
  2. IO回调队列(请求、文件操作、数据库操作)
  3. IO轮询队列
    1.检查阶段队列或设置立即
    1.关闭处理程序队列
    每当请求到来时,代码就按照排队的回调顺序执行。
    这不像当有阻塞请求时,它被附加到一个新的线程上。默认情况下只有4个线程。所以那里有另一个队列发生。
    每当代码中出现阻塞进程(如文件读取)时,就会调用一个利用线程池中线程的函数,一旦操作完成,回调就会传递到相应的队列,然后按顺序执行。
    所有内容都将根据回调类型进行排队,并按上述顺序进行处理。
0g0grzrc

0g0grzrc8#

下面是对medium article的一个很好的解释:
给定一个NodeJS应用程序,由于Node是单线程的,假设处理涉及Promise。所有这一切需要8秒,这是否意味着在此请求之后到来的客户端请求需要等待8秒?否。NodeJS事件循环是单线程的。NodeJS的整个服务器体系结构不是单线程的。
在进入节点服务器体系结构之前,先看一下典型的多线程请求响应模型,Web服务器将具有多个线程,当并发请求到达Web服务器时,Web服务器从threadPool中选择threadOne,threadOne处理requestOne并响应clientOne,当第二个请求进入时,Web服务器从threadPool中选取第二个线程,并选取requestTwo,处理它并响应clientTwo。threadOne负责requestOne要求的所有类型的操作,包括执行任何阻塞IO操作。
线程需要等待阻塞IO操作的事实是它效率低下的原因。在这种模型中,Web服务器只能为线程池中的线程提供尽可能多的请求。
NodeJS Web服务器维护一个有限的线程池来为客户端请求提供服务。多个客户端向NodeJS服务器发出多个请求。NodeJS接收这些请求并将其放入EventQueue中。NodeJS服务器有一个称为EventLoop的内部组件,它是一个接收请求并处理请求的无限循环。此EventLoop是单线程的。换句话说,EventLoop是EventQueue的监听器。因此,我们有一个放置请求的事件队列,还有一个监听事件队列中这些请求的事件循环。接下来会发生什么?监听器(事件循环)处理该请求并且如果它能够处理该请求而不需要任何阻塞IO操作,则事件循环将自己处理请求并将响应发送回客户端。如果当前请求使用阻塞IO操作,则事件循环查看线程池中是否存在可用的线程,从线程池中挑选一个线程并将特定请求分配给挑选的线程。该线程执行阻塞IO操作并将响应发送回事件循环。一旦响应到达事件循环,事件循环将响应发送回客户端。
NodeJS与传统的多线程请求响应模型相比有何优势?在传统的多线程请求/响应模型中,每个客户端都有一个不同的线程,而在NodeJS中,较简单的请求都直接由EventLoop处理。这是对线程池资源的优化,并且没有为每个客户端请求创建线程的开销。

wvt8vs2t

wvt8vs2t9#

In node.js request should be IO bound not CPU bound. It means that each request should not force node.js to do a lot of computations. If there are a lot of computations involved in solving a request then node.js is not a good choice. IO bound has little computation required. most of the time requests are spent in either making a call to a DB or a service.
Node.js has single-threaded event loop but it is just a chef. Behind the scene most of the work is done by the operating system and Libuv ensures the communication from the OS. From the Libuv docs:
In event-driven programming, an application expresses interest in certain events and respond to them when they occur. The responsibility of gathering events from the operating system or monitoring other sources of events is handled by libuv, and the user can register callbacks to be invoked when an event occurs.
The incoming requests are handled by the Operating system. This is pretty much correct for almost all servers based on request-response model. Incoming network calls are queued in OS Non-blocking IO queue. 'Event Loop constantly polls OS IO queue that is how it gets to know about the incoming client request. "Polling" means checking the status of some resource at a regular interval. If there are any incoming requests, evnet loop will take that request, it will execute that synchronously . while executing if there is any async call (i.e setTimeout), it will be put into the callback queue. After the event loop finishes executing sync calls, it can poll the callbacks, if it finds a callback that needs to be executed, it will execute that callback. then it will poll for any incoming request. If you check the node.js docs there is this image:

From docs phase-overview
poll: retrieve new I/O events; execute I/O related callbacks (almost all with the exception of close callbacks, the ones scheduled by timers, and setImmediate()); node will block here when appropriate.
So event loop is constantly polling from different queues. If ant request needs to an external call or disk access, this is passed to OS and OS also has 2 different queues for those. As soon as event loop detects that somehting has to be done async, it puts them in a queue. Once it is put in a queue, event-loop will process to the next task.
One thing that to mention here, event loop continuously runs. Only Cpu can move this thread out of CPU, event loop itself will not do it.
From the docs:
The secret to the scalability of Node.js is that it uses a small number of threads to handle many clients. If Node.js can make do with fewer threads, then it can spend more of your system's time and memory working on clients rather than on paying space and time overheads for threads (memory, context-switching). But because Node.js has only a few threads, you must structure your application to use them wisely.
Here's a good rule of thumb for keeping your Node.js server speedy: Node.js is fast when the work associated with each client at any given time is "small".
Note that small tasks mean IO bound tasks not CPU. Single event loop will handle the client load only if the work for each request is mostly IO work.
Context switch basically means CPU is out of resources so It needs to stop the execution of one process to allow another process to execute. OS first has to evict process1 so it will take this process from CPU and it will save this process in the main memory. Next, OS will restore process2 by loading process control block from memory and it will put it on the CPU for execution. Then process2 will start its execution. Between process1 ended and the process2 started, we have lost some time. Large number of threads can cause a heavily loaded system to spend precious cycles on thread scheduling and context switching, which adds latency and imposes limits on scalability and throughput.

相关问题