pampa

https://github.com/leandrosardi/pampa

Pampa is a Ruby library for async & distributing computing providing the following features: - cluster-management with dynamic reconfiguration (joining and leaving nodes); - distribution of the computation jobs to the (active) nodes; - error handling, job-retry and fault tolerance; - fast (non-direct) communication to ensure realtime capabilities. The Pampa framework may be widely used for: - large scale web scraping with what we call a "bot-farm"; - payments processing for large-scale ecommerce websites; - reports generation for high demanded SaaS platforms; - heavy mathematical model computing; and any other tasks that requires a virtually infinite amount of CPU computing and memory resources. Find documentation here: https://github.com/leandrosardi/pampa