language: en kae3g ← back to index
(unlisted) — This essay is not shown on the main index but remains accessible via direct link

kae3g 9594: Build Systems - From Source to Binary

Phase 1: Foundations & Philosophy | Week 4 | Reading Time: 16 minutes

What You'll Learn

Prerequisites

From Seeds to Harvest

Source code (text files): Dormant potential, like seeds

Build process: Transformation into executable, like growing plants

Binary executable: Running program, like harvested crop

Plant lens: "Build systems are recipes for growing crops from seeds—taking source (seeds) and producing binaries (harvest)."

Compilation vs Interpretation

Compiled Languages

C, Rust, Go:

Source code (.c, .rs, .go)
    ↓ compile
Machine code (binary)
    ↓ run
Executable program

Pros:

Cons:

Interpreted Languages

Python, JavaScript, Ruby:

Source code (.py, .js, .rb)
    ↓ run
Interpreter (reads source, executes)

Pros:

Cons:

Hybrid (JVM, Clojure)

Source code (.clj, .java)
    ↓ compile
Bytecode (.class)
    ↓ run
JVM (interprets/JITs bytecode)

Best of both: Fast-ish (JIT compilation), portable (bytecode works anywhere).

The Compilation Pipeline

For C program:

Step 1: Preprocessing

// hello.c
#include <stdio.h>
#define MESSAGE "Hello, Valley!"

int main() {
    printf("%s\n", MESSAGE);
}

Preprocessor expands macros, includes headers:

gcc -E hello.c -o hello.i

# hello.i now has entire stdio.h contents + "Hello, Valley!" inline

Step 2: Compilation

C → Assembly:

gcc -S hello.i -o hello.s

# hello.s contains assembly code:
#   mov edi, OFFSET FLAT:.LC0
#   call puts
#   ...

Step 3: Assembly

Assembly → Machine code:

as hello.s -o hello.o

# hello.o is binary (object file)
# Contains machine instructions, but not yet executable

Step 4: Linking

Combine object files + libraries:

ld hello.o /usr/lib/libc.so -o hello

# hello is now executable!

Or all at once:

gcc hello.c -o hello
# (gcc runs all 4 steps internally)

Build Tools: Automating the Process

Make (1976, Stuart Feldman)

Makefile:

# Target: dependencies
#     command

hello: hello.o
	gcc hello.o -o hello

hello.o: hello.c
	gcc -c hello.c -o hello.o

clean:
	rm -f hello hello.o

Run:

make hello
# Output:
# gcc -c hello.c -o hello.o
# gcc hello.o -o hello

make clean
# rm -f hello hello.o

Benefit: Incremental (only rebuilds what changed).

Problem: Imperative (you specify HOW to build, not just WHAT).

Ninja (2012, Evan Martin)

Faster than Make:

Not hand-written (too low-level). Tools generate build.ninja files.

Bazel (Google)

Scalable (handles huge codebases):

Used by: Google (entire codebase), large projects.

Problem: Complex (learning curve, overhead for small projects).

Nix (The Ultimate!)

Declarative, reproducible, isolated:

# default.nix
{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation {
  name = "hello";
  src = ./.;
  buildInputs = [ pkgs.gcc ];
  
  buildPhase = ''
    gcc hello.c -o hello
  '';
  
  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin/
  '';
}

Build:

nix-build
# Result: ./result/bin/hello

Benefits:

This is sovereignty (Essay 9503, 9960 - grainhouse strategy!).

Why Reproducible Builds Matter

Problem: "Works on my machine!"

Developer:   Builds fine ✅
CI server:   Build fails ❌
Production:  Different binary ⚠️

Causes:

Solution: Reproducible builds (same source + same environment → bit-identical binary).

Nix guarantees this (hermetic builds, locked dependencies).

Why it matters:

Plant lens: "Reproducible builds are like saving seeds—plant the same seed in same soil → get identical plant."

Incremental Builds

Problem: Rebuilding everything wastes time.

Solution: Track dependencies, rebuild only what changed.

Example (make):

main: main.o utils.o
	gcc main.o utils.o -o main

main.o: main.c utils.h
	gcc -c main.c

utils.o: utils.c utils.h
	gcc -c utils.c

Change main.c:

make
# Only recompiles main.c → main.o
# Then relinks main
# Skips utils.c (unchanged!)

Nix approach: Hash-based (content-addressed):

The Nix Build Model

Nix is special (Essay 9504 mentioned it):

Hermetic Builds

Isolated from system:

Traditional build:
  gcc hello.c -o hello
  # Uses: system gcc, system libc, system headers
  # (Depends on what's installed!)

Nix build:
  Uses: /nix/store/abc123-gcc-11.2/bin/gcc
        /nix/store/def456-glibc-2.35/
  # Everything explicit, isolated
  # (Doesn't depend on system state!)

Result: Same Nix expression → same binary (always).

Content-Addressed

Derivations are hashed:

/nix/store/abc123-hello-1.0
           └─────┘
           Hash of: source + dependencies + build script

If ANY input changes (source, dependencies, script), hash changes → rebuild.

If nothing changed, use cached result → instant!

This is perfect for grainhouse strategy (Essay 9960).

Try This

Exercise 1: Manual Compilation

# Write simple C program
cat > hello.c <<EOF
#include <stdio.h>
int main() {
    printf("Hello, Valley!\n");
    return 0;
}
EOF

# Compile step-by-step
gcc -E hello.c -o hello.i     # Preprocess
gcc -S hello.i -o hello.s     # Compile to assembly
as hello.s -o hello.o          # Assemble to object file
gcc hello.o -o hello           # Link to executable

# Run
./hello

Observe: Four distinct steps (usually hidden by gcc hello.c -o hello).

Exercise 2: Make Incremental Build

# Create Makefile
cat > Makefile <<EOF
hello: hello.o
	gcc hello.o -o hello

hello.o: hello.c
	gcc -c hello.c -o hello.o

clean:
	rm -f hello hello.o
EOF

# First build
make hello

# No changes, rebuild:
make hello
# Output: make: 'hello' is up to date.

# Change source:
echo "// comment" >> hello.c

# Rebuild (incremental!)
make hello
# Only recompiles hello.c

Observe: Make tracks timestamps, rebuilds only what's needed.

Exercise 3: Nix Build (if you have Nix)

# Create simple Nix expression
cat > default.nix <<EOF
{ pkgs ? import <nixpkgs> {} }:

pkgs.writeScriptBin "hello" ''
  echo "Hello from Nix!"
''
EOF

# Build
nix-build

# Run
./result/bin/hello

Observe: Nix handles dependencies, isolation, caching automatically.

Going Deeper

Related Essays

External Resources

Reflection Questions

  1. Why do build systems exist? (Can't we just gcc *.c? What breaks at scale?)
  2. Is reproducibility always achievable? (Timestamps, randomness, network - how to handle?)
  3. Should all builds be hermetic? (Nix says yes - but what's the cost? Disk space, complexity)
  4. What if source code directly executed? (Interpreted languages do this - trade-offs?)
  5. How would Nock build systems work? (Pure functions (noun → noun) - deterministic by definition!)

Summary

Build Systems Transform:

Compilation Pipeline:

  1. Preprocess: Expand macros, include headers
  2. Compile: C → Assembly
  3. Assemble: Assembly → Object code
  4. Link: Object files → Executable

Build Tools:

Key Concepts:

Why Reproducibility:

Nix Advantages:

In the Valley:

Plant lens: "Build systems are recipes—transform seeds (source) into crops (binaries) through systematic cultivation (compilation pipeline)."

Next: We'll explore package managers—how to manage dependencies at scale, the problem Nix solves beautifully, and why dependency hell exists!

Navigation:
← Previous: 9594 (concurrency threads parallelism) | Phase 1 Index | Next: 9596 (package managers dependency resolution)

Metadata:

Copyright © 2025 kae3g | Dual-licensed under Apache-2.0 / MIT
Competitive technology in service of clarity and beauty

← back to index