MiniPiRack was started out of frustration that I know you and many others have had before. I just needed some servers, VMs, hosts – whatever – to test out some architectural changes or to test some new ideas. I had no idea at the time what a wild ride this would turn into.
My first major WTF moment was around June of 2015 when I was working at a well-known Austin tech firm. I was officially titled a “Technical Architect”, what ever that means. We were in the process, like all growing and future-minded companies are, of scrutinizing and potentially redesigning our stack to deliver better service for our customers with more visibility into operations while (hopefully!) reducing op and support costs.
I had some pretty big and divergent plans that took the current architecture and turned it on it’s ear. But to implement it, I needed servers – lots of them. (Lots is relative these days. For my POC, I could have used about 10.) Unfortunately, and this is absolutely no knock against WPE, getting my own servers was just impossible. I won’t go into the details here but it involved sign-offs, multi-department agreement and collaboration, etc, etc. All the same type of things you’d expect. Eventually, I gave up trying to use “proper channels” and just opened my own AWS account. And that, kids, is how I ended up with a $2,500 AWS bill with no one to blame but myself.
My second major WTF moment was around February of 2016 when I was working at a large multinational networking hardware company. We were doing a massive project that hit all the cloud bingo words: data pipelines, clustering, fault tolerant, highly-available, auto-scaling. It was a huge undertaking. However, servers (VMs in this case) were at a premium. Our minimal deploy took about 15 VMs but for some reason, the company couldn’t find extra VMs to provide the developers a sandbox. We often deployed our test code right into production spaces. Policy was “Don’t deploy anything that will break the pipelines.” Genius.
The company acknowledged that we needed test sandboxes to *cough* test our code before deploying it into the main pipelines. The policy there was to use VMs aka VirtualBox via vagrant. I’ll let you do the easy math of 15 VMs and laptops with a max of 2 gig of memory. For some reason, the management a layer up couldn’t do this same math.
Having learned my lesson with AWS, I pondered how to build my own lab. I could buy a bunch of cheap PCs and routers, install Linux, etc. Painful. Then I though of the Raspberry Pi. Yes, it was ARM-based but it didn’t take long to find that nearly everything I needed was either:
A) Interpreted
B) ARM compatible or available
C) Compilable
I didn’t need speed. I only needed deployability and data deliverability. I didn’t even need huge memory because all the components had reasonably low memory requirements as long as throughput expectations were kept low. The Raspberry Pis just might work!
I had a solution! Maybe. I’ll just get a stack of Raspberry Pis up and running. Surely it can’t be that complicated….