LogScale is a high-performance, concurrent log-searching engine written in Go. Designed to search massive, gigabyte-scale log files across multiple CPU cores with near-zero memory allocation, it significantly outperforms standard UNIX utilities like grep.
🚧 Project Status: Early Development
LogScale is currently evolving from a highly optimized local search tool into a fully distributed, multi-node log orchestration platform.
LogScale aims to go beyond being a fast local alternative to grep. It is being built as a distributed Scatter-Gather search system, allowing engineers to execute complex pattern matching across an entire fleet of servers simultaneously via gRPC.
The complete system architecture will consist of two core components:
- The Worker Daemon (The Swarm): Lightweight agents deployed across target machines. When triggered, a Worker utilizes optimized file-chunking and lock-free concurrency to search multiple local log files in parallel, maximizing the host machine's disk I/O and CPU without spiking RAM usage.
- The Master CLI / TUI (The Control Plane): The user-facing command center. It allows users to query the swarm, target specific subsets of machines (using metadata, tags, or aliases), and aggregate the streaming gRPC responses into a single, cohesive terminal output.
- Local multi-threaded log searching engine
- gRPC worker server that exposes the search functionality over the network
- Master CLI that can query multiple workers and aggregate results