|
1 | 1 | # Distributed Database using Dstack |
2 | 2 |
|
3 | | -Based on Patroni configuration of postgres, with Zookeeper for leader management. |
| 3 | +What happens if you put an ordinary replicated database in a TEE environment? |
| 4 | +You'd expect to get a free upgrade from "omission fault tolerant" to "Byzantine fault tolerant." |
| 5 | +Possibly we'd also benefit from blockchain observability of reconfiguration. |
4 | 6 |
|
5 | | -Needs 4 nodes to tolerate 1 failure. |
| 7 | +To start this exploration, here's a minimal fault-tolerant database service using Zookeeper+Postgres. |
| 8 | + |
| 9 | +It's a docker-compose simulation environment for 3 nodes. |
| 10 | +You can crash/pause any 1 of them, and the other 2 will make progress. |
| 11 | + |
| 12 | +- `./Dockerfile` starts from postgres base image |
| 13 | +- `./docker-compose.yml` defines three services |
| 14 | +- `./patroni.yml` configures the postgres |
| 15 | +- `./start.sh` runs zookeeper, fixes directory permissions, runs patroni |
| 16 | +- `./zoo.cfg` zookeeper consensus |
| 17 | + |
| 18 | +### Try it out |
| 19 | +No configuration necessary, just run with docker compose |
| 20 | +```bash |
| 21 | +docker compose build |
| 22 | +docker compose up |
| 23 | +``` |
| 24 | +### Experiment with crashing/pausing |
| 25 | + |
| 26 | +#### Crashing |
| 27 | +This configuration can tolerate one node crashing. |
| 28 | +```bash |
| 29 | +docker compose stop dstack-database-node1-1 |
| 30 | +``` |
| 31 | +You can also rejoin a node to the network: |
| 32 | +```bash |
| 33 | +docker compose start dstack-database-node1-1 |
| 34 | +``` |
| 35 | + |
| 36 | +#### Pausing |
| 37 | +You can also pause a node, causing its connections to drop. Pausing the leader takes a while to rearrange. |
| 38 | +```bash |
| 39 | +docker compose pause dstack-database-node1-1 |
| 40 | +... {wait} ... |
| 41 | +docker compose pause dstack-database-node1-1 |
| 42 | +``` |
| 43 | + |
| 44 | +#### Example logs |
| 45 | +https://gist.github.com/amiller/c5d2c516b14760bcd4320650cb73f7e1 |
| 46 | + |
| 47 | +TODO: simulate partition |
0 commit comments