” Building a concurrent, remote and distributed System for batch processing which is fault tolerant and can scale up or scale out using Akka.NET (based on actor model)”.
A generic batch processor is used for dividing parallel work to multiple actors on the same machine, remote machine as well as on distributed network.
Job Manger is managing all the tasks and their status. This can be UI or console based appliaction.
Job Pool Controller actor has following responsibilities
- Assign the task to commander actor
- Get the response from commander actor
- update the job status
- print the job statastics
Job Scheduler is responsible for scheduling the job after each interval (10 sec ). It is also responsible for checking the job status after every 3 minutes.
Executer is the backbone the batchprocessor acgtor system. It is responsible for taking the job from job manager (Job pool controller actor) and perfor that job and after completion, update job manager.
Commander actor has following responsibilities-
- Create the broadcast router for creating coordinator instances.
- Get the task from Job pool controller and assign it to any available coordinator.
- Get the response from coordinator and update to job pool controller about the task.
Coordinator actor plays mediator role between commander and worker actor.Coordinator actor has following responsibilities-
- Create the worker actor for performing task.
- Supervise worker actor. If worker actor failed to perform any task then it will take necessary action.
- Update the commander once the task is completed by worker.
Worker actor is the last actor in the hierarchy which actully perform the task. It has following responsibilities-
- Get the task from coordinator and perform the task.
- Update the task status to coordinator
Concurrent application for Batch Processing - how to execute multiple tasks concurrently as well as paralley in Akka.NET.
Please feel free to contribute.
Please create a Github issue for any questions you might have.