IPC/COW fork fixes - #27
Merged
Merged
Conversation
…sue in the process related to dynamic routing. The core problem is that disconnected stages -- e.g. consumers with no upstream--will automatically shutdown instantly based on logic we added. See worker.rb line 60 (# If no upstream sources, this stage is disconnected) If we remove this logic, these disconnected stages will never shut-down, unless they get a dynamic producer to connect to them (or their pipeline sends and send of stage?) The solution is something like (A) allowing stages to be declared as disconnected: true, and (B) adding various keep-alive options for stages so they have a change to receive dynamic inputs before shutting down. A 3rd option (C) is to send Nonce signal (StageConnected) and/or KeepAlive signal to the stage initially, to declare the dynamic linkage. That might require some keepalive. The yield example should be refactored to use connected stages
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
workers by encoding routing information in IPC messages
signal propagation
instead of trying to modify parent process memory