001 Notes
System Design: Three Constraints
technology
-> local mechanism
-> reusable causal ladder
-> Physics / Money / Truth
Framework
Physics
-> machines, networks, storage, and humans have finite capacity
-> distance and movement take time
-> one thing cannot do infinite work
Money
-> the best physical answer often exists
-> all-RAM, every-region, more replicas, more engineers
-> but the bill is too high
Truth
-> the same fact appears in more than one place
-> copies can disagree
-> the system must define what is real
Core Ladders
Storage hierarchy
disk / SSD
-> durable + cheaper per GB
-> block/page shaped
-> slower than memory
RAM
-> fast random access
-> volatile
-> expensive per GB
NVDIMM / persistent memory
-> memory-like persistence
-> still expensive
-> platform support is limited
CXL memory
-> expands / pools memory
-> farther than local DRAM
-> topology + cost still matter
practical storage system
-> hot data in memory
-> bulk durable data on cheaper storage
-> cache + layout + journal around the gap
=> Physics + Money + Truth
Data placement
same process
-> local memory load
-> fastest
-> not shared with other machines
same host, different process
-> process boundary
-> kernel / IPC / serialization
-> still near
same AZ cache
-> app -> NIC -> switch -> Redis/Memcached
-> network + RAM lookup
-> shared by app servers
same AZ database
-> app -> network -> DB
-> parser / planner / executor
-> indexes / locks / MVCC
-> buffer pool / storage on miss
cross-region database
-> bytes cross geography
-> speed of light is the ceiling
-> writes that need agreement coordinate across distance
nearer data
-> fewer boundaries / less distance
=> Physics
shared mutable data
-> copies can disagree
=> Truth
more placement options
-> more infra
=> Money
One-machine ceiling
one machine
-> finite CPU
-> finite RAM
-> finite disk bandwidth
-> finite network bandwidth
split work/data
-> more total capacity
-> cross-machine operations get harder
=> Physics + Truth + Money
Copies disagree
same fact in two places
-> copy A says X
-> copy B says Y
-> coordinate before answering
or answer fast and repair later
=> Truth
Producer faster than consumer
producer emits work
-> consumer slower / down / burst overloaded
-> direct call blocks, fails, or drops work
-> buffer needed
=> Physics
Append-only beats random mutation
random mutation
-> touch scattered pages
-> indexes / locks / cleanup
-> expensive write path
append
-> write at the end
-> index / compact / consume later
-> cheaper write path
=> Physics
Specialized access structure
generic scan
-> touch too much data
index / layout for query shape
-> fewer bytes touched
-> more memory / write cost
=> Physics + Money
Managed service
self-host
-> control
-> own upgrades / monitoring / failures / on-call
managed service
-> less operations
-> pay vendor
-> accept service limits
=> Money
Human coordination
more machines / services / teams
-> humans cannot track everything manually
-> package / declare / automate / observe
-> platform adds its own complexity
=> Physics of humans + Money
Identity truth
identity / permission is a fact
-> who is this user?
-> what can this app do?
-> can this token still be trusted?
=> Truth
Technology Map
Storage
Filesystems - names over blocks
filesystem
-> app wants named files
-> device exposes numbered blocks/pages
-> storage hierarchy ladder
=> Physics: block/page access and placement matter
hard disk
-> mechanical seek + rotation
-> nearby blocks faster, scattered blocks slower
-> allocator tries to preserve locality
=> Physics
crash during metadata update
-> directory and block map can disagree
-> journal / copy-on-write
=> Truth
why not RAM / persistent RAM everywhere?
-> storage hierarchy ladder
-> volatile or too expensive / platform-limited
=> Money + Truth
PostgreSQL - shared facts need rules
PostgreSQL
-> two users update shared facts
-> plain file cannot define ordering / atomicity
-> WAL + locks/MVCC + constraints define what became real
=> Truth
transactions
-> visibility checks / indexes / WAL / locks
-> more machine work per operation
=> Physics
SQLite - database without a database service
SQLite
-> app wants SQL + transactions locally
-> PostgreSQL service would add process / port / ops
-> managed-service ladder inverted: remove the service
=> Money
single-file database
-> local transactions
-> single-writer shape
=> Truth + Physics trade
MongoDB - object shape plus horizontal split
MongoDB
-> app works with whole documents
-> relational schema spreads object across tables
-> document layout removes join work for that shape
=> Physics + Money
documents outgrow one machine
-> one-machine ceiling ladder
-> shard by key
=> Physics
cross-document truth
-> copies / shards can disagree
-> weaker default model than one local RDBMS
=> Truth cost
DynamoDB - predictable key access
DynamoDB
-> request knows partition key
-> route to owning partition
-> avoid general SQL query machinery
-> one-machine ceiling ladder
=> Physics
AWS runs partitioning / repair / scaling
-> managed service ladder
=> Money
limited cross-key transactions
-> copies disagree ladder is constrained by API shape
=> Truth trade
Cassandra - every node can accept writes
Cassandra
-> write volume exceeds one leader
-> one-machine ceiling ladder
-> any node accepts writes
=> Physics
append-friendly storage
-> append-only ladder
-> high write intake
=> Physics
tunable consistency
-> copies disagree ladder
-> ONE / QUORUM / ALL choose how much agreement
=> Truth
ScyllaDB - remove runtime overhead
ScyllaDB
-> Cassandra model
-> JVM pauses / locks / cross-core contention waste cycles
-> shard-per-core + no GC
=> Physics
buy fewer machines for same throughput
-> hardware cost avoided
=> Money
Spanner / CockroachDB / TiDB / YugabyteDB - SQL across machines
distributed SQL
-> one RDBMS node hits ceiling
-> split data across machines
-> keep SQL + transactions
=> Physics + Truth
replicas need agreement
-> copies disagree ladder
-> Raft/Paxos/timestamps coordinate writes
=> Truth
coordination crosses network / regions
-> data placement ladder
=> Physics + Money
Neo4j - traverse edges directly
Neo4j
-> query shape is graph traversal
-> relational JOIN^depth touches too much data
-> specialized access structure ladder
=> Physics
clustered graph truth
-> copies disagree ladder
=> Truth cost
Time-series databases - time is the access pattern
TSDB
-> data arrives by timestamp
-> queries scan time ranges
-> specialized layout for series/time
=> Physics
raw points forever
-> storage hierarchy ladder
-> retention / compression / downsampling
=> Money
OLAP column stores - read columns, not rows
OLAP
-> query needs few columns across many rows
-> row store reads bytes it will discard
-> column layout + compression
=> Physics
precompute every answer / keep all in RAM
-> storage hierarchy ladder
-> too expensive
=> Money
Druid / Pinot - fresh analytical serving
Druid / Pinot
-> user-facing dashboard wants fresh aggregations
-> warehouse path is too heavy for interactive filtering
-> pre-indexed real-time segments
=> Physics
serving cluster + indexes
-> specialized access structure ladder
=> Money
RocksDB / LevelDB - turn random writes into appends
RocksDB
-> many small updates
-> random mutation ladder is expensive
-> append memtable/log, flush SSTables, compact later
=> Physics
whole map in RAM
-> storage hierarchy ladder
-> volatile / too expensive
=> Money + Truth
S3 / GCS / Azure Blob - cheap durable bulk storage
object storage
-> petabytes do not fit behind one app server
-> one-machine ceiling ladder
-> distribute objects across storage fleet
=> Physics
own disks / repairs / replication
-> managed service ladder
=> Money
whole-object API
-> not local random I/O
-> limited transaction shape
=> Truth + Physics trade
FoundationDB - one transactional substrate
FoundationDB
-> higher databases need correct cross-key transactions
-> copies disagree ladder
-> strict serializable ordered key-value core
=> Truth
single-node store
-> one-machine ceiling ladder
=> Physics
Caching
Redis - shared RAM-ish lookup over the network
Redis
-> app reads shared hot key
-> same-AZ cache in data placement ladder
-> network + Redis RAM lookup
-> avoids DB parser/planner/index/MVCC/buffer-pool path
=> Physics
all durable data in RAM
-> storage hierarchy ladder
-> volatile / expensive
=> Money + Truth
cached copy
-> copies disagree ladder
-> stale or lost unless managed carefully
=> Truth
Memcached - disposable shared bytes
Memcached
-> same-AZ cache in data placement ladder
-> key -> bytes -> expiry
-> less machinery than Redis
=> Physics + Money
cache disappears
-> rebuild from source of truth
=> Truth trade
Varnish - reuse HTTP responses
Varnish
-> many users request same response
-> repeated origin render wastes CPU / DB work
-> cache response near origin path
=> Physics
cache forever
-> stale or personalized data leaks
-> copies disagree ladder
=> Truth
CDN - nearby copies beat distance
CDN
-> user far from origin
-> data placement ladder
-> copy cacheable content nearer
=> Physics
full app in every city
-> many deployments + mutable data everywhere
-> managed service + copies disagree ladders
=> Money + Truth
Messaging
Kafka - durable event buffer
Kafka
-> producer faster than consumer
-> direct call blocks / fails / drops work
-> append-only durable log
=> Physics
consumer offsets / replay
-> delivery effect is still application state
-> duplicates possible
=> Truth
retained logs + brokers
-> storage hierarchy + managed/self-host cost
=> Money
RabbitMQ - routed reliable work
RabbitMQ
-> sender and worker availability differ
-> producer/consumer ladder
-> broker stores, routes, acks, redelivers
=> Physics + Truth
rich routing / per-message behavior
-> more broker machinery than append-only log
=> Money + Physics cost
SQS - queue without queue operations
SQS
-> need producer/consumer buffer
-> do not want broker operations
-> managed service ladder
=> Money
standard vs FIFO
-> choose throughput or ordering/deduplication
-> copies disagree / delivery truth ladder
=> Truth
NATS - tiny fast message path
NATS
-> messages need low overhead
-> durable log machinery would dominate
-> direct pub-sub subjects
=> Physics
need persistence
-> add JetStream
-> now storage / truth machinery appears
=> Truth + Money
Pulsar - separate broker from storage
Pulsar
-> Kafka broker owns compute + storage together
-> storage and traffic scale differently
-> split brokers from BookKeeper storage
=> Physics
old retention on object storage
-> storage hierarchy ladder
=> Money
ZeroMQ - messaging without infrastructure
ZeroMQ
-> need message patterns
-> broker would add process / hop / operations
-> library-level sockets
=> Physics + Money
no central durable broker
-> messages can be lost
=> Truth trade
Stream And Batch Processing
Flink - stateful stream truth
Flink
-> unbounded events + growing keyed state
-> failures lose local memory
-> checkpoints + event-time watermarks
=> Truth
state larger than heap / continuous input
-> one-machine ceiling + storage hierarchy ladders
=> Physics
Spark - avoid disk-heavy distributed pipelines
Spark
-> dataset exceeds one machine
-> one-machine ceiling ladder
-> distribute computation
=> Physics
MapReduce writes every step to disk
-> storage hierarchy ladder
-> keep intermediate data in memory when useful
=> Physics + Money
Kafka Streams - stream processing without another cluster
Kafka Streams
-> need stream transforms
-> separate Flink/Spark cluster is extra operations
-> library inside app + Kafka changelog
=> Money
local RocksDB state
-> storage hierarchy ladder
=> Physics + Truth
Search
Elasticsearch / OpenSearch / Solr - word to documents
search engine
-> query wants documents containing terms
-> generic scan touches every document
-> inverted index: term -> document IDs
=> Physics
primary DB write vs search index refresh
-> copies disagree ladder
=> Truth
Algolia - search operations outsourced
Algolia
-> search relevance / typo tolerance / replicas need expertise
-> managed service ladder
=> Money
index in vendor cloud
-> data placement ladder
=> Physics + control trade
Vector databases - nearby vectors
vector DB
-> exact nearest neighbor scans every vector
-> specialized access structure ladder
-> HNSW / IVF reduce comparisons
=> Physics
approximate search
-> may miss exact nearest result
=> Truth
Network And Platform
Nginx / HAProxy - one service over many backends
load balancer
-> one backend hits CPU/socket/network ceiling
-> one-machine ceiling ladder
-> spread requests
=> Physics
retries / health checks / routing
-> request effect may duplicate or move
=> Truth
Envoy - consistent service traffic behavior
Envoy
-> every service reimplements timeouts/retries/TLS/metrics
-> human coordination ladder
-> shared proxy behavior
=> Money + Truth
sidecar / proxy hop
-> data placement ladder adds boundary
=> Physics
Istio / Linkerd - fleet-wide service rules
service mesh
-> many services / languages / teams
-> human coordination ladder
-> central mTLS / policy / telemetry
=> Money + Truth
proxy on calls
-> extra boundary in data placement ladder
=> Physics
Kubernetes - desired state for machines
Kubernetes
-> many containers / machines / failures
-> humans cannot place/restart/roll out manually
-> human coordination ladder
=> Physics of humans + Money
desired state vs actual state
-> controllers reconcile what is real
=> Truth
Docker - package the environment
Docker
-> app depends on libraries/files/runtime shape
-> host environments differ
-> image packages dependency truth
=> Truth + Money
VM per app
-> stronger isolation
-> more duplicated kernel/resources
=> Physics + Money
Lambda / Cloud Functions - pay when code runs
serverless
-> workload is bursty / idle often
-> provisioned server burns money while idle
-> managed service ladder
=> Money
start on demand / platform limits
-> data placement + runtime boundary
=> Physics
DNS - name to current location
DNS
-> users need stable names
-> IPs / locations change
-> cached name indirection
=> Physics + Money
resolver cache
-> old answer can survive until TTL
-> copies disagree ladder
=> Truth
Coordination
ZooKeeper / etcd / Consul - agreement on small facts
coordination service
-> who is leader / which config / who owns shard
-> copies disagree ladder
-> Raft/Paxos over small state
=> Truth
quorum nodes + messages
-> data placement ladder
=> Physics + Money
Observability
Prometheus - recent metrics cheaply
Prometheus
-> many processes expose counters
-> humans need current system state
-> scrape + time-series storage
=> Physics of collection + humans
keep every metric forever
-> storage hierarchy ladder
=> Money
Grafana - shared visual operational memory
Grafana
-> raw metrics/logs are hard for humans under pressure
-> human coordination ladder
-> dashboards over many sources
=> Physics of humans + Money
Jaeger / Zipkin / OpenTelemetry - request history across services
tracing
-> one request crosses many services
-> local logs do not show full path
-> trace ID + spans reconstruct path
=> Truth
record every span forever
-> storage hierarchy ladder
=> Money + Physics
ELK / Loki - logs survive machines
log aggregation
-> logs born on many machines
-> machines/containers disappear
-> centralize logs
=> Truth
index every word forever
-> specialized access + storage hierarchy ladders
=> Physics + Money
Protocols
HTTP/2 - one connection, many streams
HTTP/2
-> many page resources
-> many HTTP/1.1 connections repeat setup/headers
-> multiplex + header compression
=> Physics
HTTP/3 / QUIC - streams over lossy networks
HTTP/3
-> TCP gives one ordered byte stream
-> missing packet blocks later bytes
-> QUIC gives independent streams over UDP
=> Physics
new user-space transport
-> more implementation and ecosystem cost
=> Money
gRPC - typed compact service calls
gRPC
-> internal services need contracts
-> JSON text + hand clients waste bytes/CPU and drift
-> Protobuf + codegen + HTTP/2
=> Physics + Money + Truth
GraphQL - client-shaped response
GraphQL
-> UI needs fields across resources
-> REST fixed shapes over-fetch / under-fetch
-> client asks for exact shape
=> Physics + Money
schema / resolver truth
-> server must enforce allowed shape and cost
=> Truth
WebSocket - avoid polling
WebSocket
-> server has updates before client asks
-> polling repeats empty requests
-> persistent bidirectional channel
=> Physics
connection per client
-> server holds state
-> reconnect/order handling
=> Truth + Money
Authentication And Identity
OAuth 2.0 - delegated permission
OAuth
-> app wants limited access to user's resource
-> password sharing gives full authority
-> scoped revocable tokens
-> identity truth ladder
=> Truth
standard flow across integrations
-> avoid bespoke auth protocols
=> Money
OIDC - portable login truth
OIDC
-> OAuth says authorization, not login identity
-> app needs who the user is
-> signed ID token + standard claims
=> Truth
common provider integration
-> identity truth ladder reused
=> Money
JWT - local token verification
JWT
-> server wants auth without session lookup
-> claims signed into token
-> verify locally
-> data placement ladder avoids central store per request
=> Physics + Money
revocation
-> stateless token remains valid until expiry
-> identity truth ladder
=> Truth
Major Patterns
Sharding
sharding
-> one machine cannot hold/serve all data
-> one-machine ceiling ladder
-> split by key/range/tenant/region
=> Physics
cross-shard operation
-> copies / partitions need coordination
=> Truth + Money
Replication
replication
-> one copy can fail
-> extra copies survive failure / serve near reads
-> data placement + copies disagree ladders
=> Physics + Truth
more copies
-> more storage / bandwidth
=> Money
Caching everywhere
cache
-> repeated answer needed
-> keep copy closer
-> data placement ladder
=> Physics
copy can be stale
-> copies disagree ladder
=> Truth
CAP theorem
network partition
-> replicas cannot communicate
-> serve anyway or wait
-> cannot guarantee one shared truth while isolated
=> Truth under Physics failure
Saga
saga
-> business action crosses services
-> one global transaction blocks on every participant
-> local commits + compensations
=> Truth trade under Physics + Money constraints
Event sourcing
event sourcing
-> current state alone loses history
-> append facts that produced state
-> append-only ladder
=> Truth
history storage / replay
=> Money + Physics
CQRS
CQRS
-> write shape != read shape
-> specialized access structure ladder
-> separate command model and query model
=> Physics
read model trails write model
-> copies disagree ladder
=> Truth
Backpressure
backpressure
-> producer faster than consumer
-> queue grows until memory fails
-> consumer capacity signals upstream
=> Physics
Circuit breaker / bulkhead / hedged requests
slow dependency
-> threads/sockets/queues fill
-> healthy work gets trapped
-> isolate, stop, or race requests
=> Physics
duplicates / rejected requests
=> Truth trade
Distributed locks
distributed lock
-> many machines need one owner
-> ownership is a shared fact
-> copies disagree ladder
=> Truth
lock service round trips / leases
-> data placement ladder
=> Physics
Idempotency keys
retry after unknown result
-> request may have already succeeded
-> store key -> result
-> duplicate retry returns same effect
=> Truth
key storage / lookup
=> Money + Physics
Microservices vs monolith
monolith
-> one deployable / fewer network calls
-> cheaper for small teams
=> Money
microservices
-> many teams need independent ownership
-> human coordination ladder
=> Physics of humans
Fourth-Constraint Test
Security
authentication / authorization
-> identity truth ladder
=> Truth
encryption / DDoS defense
-> CPU + network capacity
=> Physics + Money
Maintainability
hard-to-change code
-> humans cannot hold hidden behavior
-> human coordination ladder
=> Physics of humans + Money
Compliance
delete / audit / encrypt
-> which copies exist, what happened, who can read
=> Truth
fines / audits
=> Money
User experience
screen waits / jumps / shows wrong data
-> machine work + human perception + state correctness
=> Physics + Truth
research / design time
=> Money
Vendor lock-in
switching provider
-> migration work / retraining / contract risk
=> Money
Reliability
survive failure
-> extra copies + failover + repair
-> copies disagree + data placement ladders
=> Truth + Physics + Money
Scalability
more users / data / teams
-> one-machine ceiling + human coordination ladders
=> Physics
capacity must be affordable
=> Money
Decision Template
1. What local mechanism is failing?
disk movement, distance, duplicate state, hot key, slow consumer,
random mutation, human coordination, query scan, idle server cost
2. Which ladder does that mechanism use?
storage hierarchy, data placement, one-machine ceiling, copies disagree,
producer/consumer, append-only, specialized index, managed service,
human coordination, identity truth
3. How does the ladder reduce to Physics / Money / Truth?
show the final arrow explicitly
4. What does the technology trade?
name the cost, not only the benefit
technology
-> local mechanism
-> ladder
-> Physics / Money / Truth
Closing Claim
app server
-> same-AZ network
-> Redis RAM lookup
-> skips DB query/correctness machinery
=> Physics
producer faster than consumer
-> append-only durable log
-> consumer offsets
=> Physics + Truth + Money
origin far from user
-> bytes cross geography
-> copy content nearer
=> Physics + Truth + Money
Comments