Spinifex: the AWS-compatible cloud you run yourself

At some point over the last decade, “the cloud” stopped meaning a way of running software and turned into a place you rent. Those really aren’t the same thing, and the gap between them has a habit of turning up at awkward moments — on the monthly invoice, halfway through a compliance review, or the day a subsea cable gets cut and a factory floor realises its control plane has been living on the other side of the world the whole time.

Spinifex is what we’re building at Mulga to deal with that. It’s an open-source platform that stands the AWS service surface — EC2, EBS, S3, VPC, IAM — up on bare metal, on-premise, and out at the edge, using the same APIs, the same Terraform and the same SDKs you already use. The only thing that changes is the hardware underneath.

This is the first of a few posts I want to write about it, so I’ll keep this one to the overview and save the internals for later.

The premise

Whether you like it or not, the AWS API surface won. Nobody’s calling it elegant — anyone who has hand-rolled a SigV4 signature will happily talk you out of that idea — but it has become the common language everything else quietly assumes. Your Terraform speaks it, your CI speaks it, and that library you pulled in last Tuesday almost certainly speaks it too. Twenty-odd years of tooling has been built on the assumption that it’s there.

So the real cost of walking away from AWS was never the compute. It was rewriting everything that had grown to depend on the API.

That’s the part we didn’t want anyone to have to pay. The question we kept coming back to was a simple one: what if the API stayed exactly where it was, and the only thing that moved was the ground underneath it?

What Spinifex actually is

Underneath, Spinifex is a control plane sitting in front of a set of storage and networking engines, and together they present an AWS-compatible surface on hardware you own.

Point the AWS CLI at it and it answers just like you’d expect:

export AWS_PROFILE=spinifex

aws ec2 run-instances \
  --image-id ami-debian13 \
  --instance-type t3.micro \
  --key-name my-keypair

There’s nothing simulated about that. It’s a genuine EC2 Query Protocol request, SigV4-signed, landing on a real gateway that starts a real QEMU/KVM virtual machine on a real box in a real rack. No emulation shim, and no half-finished translation layer that gives up the moment you pass it the fourth parameter.

We’ve kept the architecture boring on purpose, in the places where boring is a feature. An HTTPS gateway authenticates the caller and publishes the request onto a NATS subject; stateless daemons subscribe to those subjects and do the actual work. If you want more capacity, you start more daemons. There’s no etcd quorum to nurse, no Kubernetes lurking underneath the thing that’s meant to be running your Kubernetes, and no external control plane that somebody other than you can quietly switch off.

One small thing I’m fond of: RunInstances publishes to a subject scoped by instance type, so NATS only ever delivers the request to a node that actually has room for that shape. Scheduling ends up being something the message bus does for free, rather than a reject-and-retry loop we had to bolt on afterwards.

Why we’re building it

There are three reasons really, and each one has been getting harder to argue with.

1. The bill stopped making sense

Cloud pricing is wonderful right up until the day it isn’t. Between egress fees, per-GB storage margins and the going rate for scarce GPUs, a cost that used to feel variable slowly hardens into a fixed one. Plenty of teams end up paying handsomely for elasticity they never actually touch, running workloads whose shape is completely predictable — steady, well understood, and about as bursty as a filing cabinet.

2. Sovereignty stopped being a footnote

Where the data physically lives, whose laws it falls under, and who genuinely holds the keys used to be small print buried in a procurement document. These days they’re questions a board asks directly, and “trust us, it’s in the local region” is a slightly thinner answer every year.

3. The edge doesn’t have a cloud

Think ships, mines, clinics, vehicles, remote sites, deployed military. They all have compute, they all have data, and they all have a link that’s intermittent on a good day. Out there, anything that needs to phone home isn’t really infrastructure — it’s a liability you just haven’t been bitten by yet. So we built Spinifex to work offline from the very first day: nothing external to check in with, no call home, and no sulky degraded mode the second the link drops out.

The part most alternatives skip

There’s no shortage of projects that wrap an AWS-shaped API around something else and call the job done. Putting up the API is the easy half. The genuinely hard part, and the part most of them quietly skip, is the storage.

So the storage is the part we actually rebuilt:

Viperblock is our take on EBS: WAL-backed block storage, with volumes replicated across nodes and proper snapshots and recovery. The write-ahead log sits on local NVMe to keep IO off the object tier, and there’s an in-memory read/write cache in front of it. Volumes attach to VMs over NBD or virtio-blk.

Predastore is our S3, and it’s fully S3-compatible right down to SigV4 auth and multipart uploads. Data is chunked and spread across nodes with Reed-Solomon erasure coding, which buys you fault tolerance without having to store three complete copies of everything.

Northstar is the DNS layer that ties the whole lot together.

All that plumbing is the reason your software runs unchanged rather than almost unchanged. A thin API wrapper will get you through a demo quite happily. What it won’t get you through is a node dying at three in the morning with your database sitting on it.

What’s compatible today

Anything marked live is usable today. The roadmap items land under that same AWS API surface, which means code you write against AWS now should just keep working the day they arrive.

The open core

Spinifex is AGPL-3.0, with a commercial option for anyone who wants one. We thought about that licence properly rather than picking it for how it reads in a pitch — it would be a bit rich to stand here telling you to stop depending on an opaque control plane you can’t see inside, and then hand you an opaque control plane you can’t see inside.

So read it, change it, run it, keep it. The platform is genuinely yours.

Try it

Here’s a single node on Debian 13:

curl -fsSL https://install.mulgadc.com | bash

sudo /usr/local/share/spinifex/setup-ovn.sh --management
sudo spx admin init --node node1 --nodes 1
sudo systemctl start spinifex.target

export AWS_PROFILE=spinifex
aws ec2 describe-instance-types

If you’d sooner just point it at a bare machine and wander off, there’s a bootable ISO for that too.

What’s next

• Self-hosted GitHub Runners — no docker-in-docker or privileged containers, single-use JIT tokens, and GPU-based CI workflows on your own hardware.

If any of that sounds like a problem you’re currently living with, the code is at github.com/mulgadc/spinifex and the docs are at docs.mulgadc.com.