The Execution Service is a distributed job scheduling system designed to manage and execute tasks across multiple worker nodes. This service is responsible for coordinating job execution, managing worker nodes, and ensuring high availability and fault tolerance.
execution-service
├── cmd
│ └── main.go # Entry point of the execution service
├── internal
│ ├── coordinator
│ │ ├── coordinator.go # Manages worker nodes and job assignments
│ │ └── worker_manager.go # Handles the lifecycle of worker nodes
│ ├── worker
│ │ ├── worker.go # Represents a worker node
│ │ └── job_executor.go # Executes jobs assigned to the worker
│ ├── models
│ │ └── job.go # Defines the Job struct
│ ├── queue
│ │ ├── queue.go # Job queuing and dequeuing logic
│ │ └── kafka_client.go # Interacts with Kafka for job messages
│ └── storage
│ ├── db.go # Database connection and setup
│ └── job_repository.go # Interacts with job storage
├── config
│ └── config.yaml # Configuration settings for the service
├── go.mod # Go module definition
├── go.sum # Module dependency checksums
└── README.md # Documentation for the project
- Job Submission: Users can submit one-time or periodic jobs for execution.
- Job Monitoring: The system provides real-time monitoring of job status (queued, running, completed, failed).
- Fault Tolerance: The service is designed to handle worker node failures and reschedule jobs accordingly.
- Scalability: Capable of handling millions of jobs efficiently across multiple worker nodes.
- Go 1.16 or higher
- Kafka (for job queuing)
- A compatible database (e.g., PostgreSQL, MongoDB)
-
Clone the repository:
git clone https://github.com/yourusername/execution-service.git cd execution-service
-
Install dependencies:
go mod tidy
-
Configure the service: Edit the
config/config.yaml
file to set up your database and Kafka connection details.
To start the execution service, run the following command:
go run cmd/main.go
- Submit Job:
POST /jobs
- Get Job Status:
GET /jobs/{job_id}
- Cancel Job:
DELETE /jobs/{job_id}
- List Pending Jobs:
GET /jobs?status=pending&user_id={user_id}
Contributions are welcome! Please open an issue or submit a pull request for any enhancements or bug fixes.
This project is licensed under the MIT License - see the LICENSE file for details.