While looking at #628, I noticed a few other problems with the cluster mode.
The request is not deleted from the requests map until all responses are returned, which may not happen if a child is restarted.
It is also processing messages from the workers even if the request is done
if (message.error) {
request.done(new Error(message.error));
return;
}
message.metrics.forEach(registry => request.responses.push(registry));
request.pending--;
if (request.pending === 0) {
// finalize
requests.delete(message.requestId);
clearTimeout(request.errorTimeout);
Which means all responses from workers are processed regardless of whether another request has already sent.
The clients also aren't given a deadline, which means they have no way to abort the work if it is not processed in time.
I don't recall if there's a way to avoid the JSON.parse() call. Given the structure of the response that is unlikely.
Perhaps more importantly: every time a worker fails to deliver a response at all, the responses from its siblings are left in the lookup table in perpetuity, representing a memory leak, and a rather substantial one.
This also has implications for #645
Additionally, cluster mode does not handle benchmarking well. Multiple calls to new AggregatorRegistry() leads to addListeners() calls if more than one instance of the module is loaded - which is a frequent if unfortunate occurrence in benchmarking code.
While looking at #628, I noticed a few other problems with the cluster mode.
The request is not deleted from the requests map until all responses are returned, which may not happen if a child is restarted.
It is also processing messages from the workers even if the request is done
Which means all responses from workers are processed regardless of whether another request has already sent.
The clients also aren't given a deadline, which means they have no way to abort the work if it is not processed in time.
I don't recall if there's a way to avoid the
JSON.parse()call. Given the structure of the response that is unlikely.Perhaps more importantly: every time a worker fails to deliver a response at all, the responses from its siblings are left in the lookup table in perpetuity, representing a memory leak, and a rather substantial one.
This also has implications for #645
Additionally, cluster mode does not handle benchmarking well. Multiple calls to
new AggregatorRegistry()leads toaddListeners()calls if more than one instance of the module is loaded - which is a frequent if unfortunate occurrence in benchmarking code.