All posts
A murder scene from a medieval illuminated manuscript.
You're next in line for splitting!

I refuse to split .c and .h

There's no hiding it. C belongs to a very small family of languages where every module has to live in at least two files. For decades now C developers created .c and .h files together as pairs to solve a simple problem: there can only be one definition of a function. This thing even has it's own name: One Definition Rule.

Of course in other languages you also cannot define stuff twice! But in almost any other popular programming language you also get a proper module system. Instead in C we have #include. And what #include does is it just pastes code.

To illustrate that let's look at a simple example:

// add.h
int add(int a, int b); // This is a declaration

// add.c
#include "add.h"
int add(int a, int b) { // This is a definition
  return a + b;
}

// mul.h
int mul(int a, int b);

// mul.c
#include "mul.h"
int mul(int a, int b) {
  return a * b;
}

// main.c
#include <stdio.h>
#include "add.h"
#include "mul.h"
int main(void) {
  printf("%d, %d", add(1, 2), mul(3, 4));
}

When the compiler gets to processing main.c the preprocessor will simply paste the included file contents into the file being compiled. Which allows the main function to know how to call add and mul, but it also allows it to call printf, since the contents of stdio.h are also pasted into main.c.

// main.c translation unit in compiler's memory
int printf(const char* format, ...);
// [...] other contents of stdio.h omitted for brevity
int add(int a, int b);
int mul(int a, int b);
int main(void) {
  printf("%d, %d", add(1, 2), mul(3, 4));
}

To finish with our example let's compile and run the program. VoilĂ 

$ cc main.c add.c mul.c -o program && ./program
3, 12

I find it funny that wanting multiple files not only necessitates creating those skeleton .h files, but also requires enumerating all the .c files that you care about in the compiler invocation. Some people split this even further, invoking the compiler separately for each file and then once again to link. And then they create build systems in order to manage all that complexity.

At this point we ought to ask ourselves a basic question. Why do we even want to have more than one file in the first place? And I suppose there really are two main answers. For one, there might be some benefit to the compilation; Maybe the compile times are faster when we split; Or maybe if your file is too big the compilation can't continue for some reason; Maybe splitting the code into multiple files unlocks some performance gains? On the other hand, splitting into multiple files might help us with the structure of our code. Let's examine those points, and then come back to think of something better.

Does the compiler care?

To answer this question I decided to do a little experiment and measure the results myself. I wrote a python script that generates a desired volume of C code spread across a number of files. It then measures how long it takes to compile said code. The actual code being compiled is very simple, it's just a lot of functions that call each other. See func_70 for example:

int func_70(int arg) {
  int count = func_69(arg) % 1000 + 3;
  if (count < 3) count = 3;
  for (int i = 0; i < count; i++) {
    arg = (arg ^ 1842945148) * 33 + arg;
  }
  return arg;
}

You can check out the script output here for yourself. Below I have also compiled the results for you. Here they are for GCC:

Lines 1 file 10 files 100 files 1,000 files
100 0.039s 0.168s n/a n/a
1,000 0.076s 0.202s 1.433s n/a
10,000 0.428s 0.573s 1.902s 14.275s
100,000 4.142s 4.175s 5.591s 18.954s
1,000,000 43.487s 41.653s 41.618s 56.299s
10,000,000 crashed 436.216s 415.737s 417.158s

And here are the results for Clang:

Lines 1 file 10 files 100 files 1,000 files
100 0.053s 0.216s n/a n/a
1,000 0.062s 0.231s 1.850s n/a
10,000 0.174s 0.338s 1.965s 18.331s
100,000 1.336s 1.479s 3.221s 19.673s
1,000,000 13.088s 13.118s 14.681s 32.142s
10,000,000 129.596s 130.726s 131.908s 146.841s

Of course those results are from my machine so only the relative times and orders of magnitude matter in the general case.

Personally, I consider compile times under 5 seconds to be acceptable. Which means that somewhere between 100k and 1M lines of code a full rebuild on every change would slow me down and investing in an incremental build would be worthwhile.

At the same time the results clearly show that the compile times grow with the number of files. The 100 and 1000 file setups show big increases in compile times for all code sizes except 10M. The 10M line codebase seems to also be the most problematic, causing an outright crash on GCC and producing compile times that I don't wish on my enemies.

It's not obvious to me how to measure the incremental build case. Compiling part of the code and then linking would definitely speed things up. Even with 10M lines with the build lasting minutes the link time was only around 3 seconds. At the same time the compile time after each change depends greatly on how many files are actually affected. I suspect that how I write my files (we'll get to that, I promise) might make things a bit worse compared to having .c and .h separate. But again, the problems would only manifest with a codebase sized in hundreds of thousands of lines of code.

Do we care?

Does it actually benefit the programmer to split the codebase into so many files? Before writing C full time I wrote TypeScript. And there was a period where I thought that every function should live in it's own file. Organizing all those files was it's own ordeal. One had to create deep hierarchies of folders, often with index.ts reexporting items as to not force the consumer of a given module to import all those functions from different places.

When I rediscovered C, I also found out that files that are thousands of lines long are quite common in this world. At first I thought it was insane. How can one even find their way around? With time however I started to see the large benefits of this approach.

The first benefit is quite obvious. Since you have less files to deal with they can also live in a flat or flatter hierarchy. Similar to how smaller companies don't need tons of middle management.

Then you start to notice how pleasant it is to use a module that lives in a single file. A single import or include gives you everything. When you want to see how something is implemented all the context lives next to what you're looking at. And for the module to become a part of your project you just copy over the file!

And finally the experience of working on the module itself. You no longer have to focus on how to organize your code into small files and those files into folders and worry about everything forming a nice acyclic graph. Now everything inside the module can depend on anything else inside the module trivially. The code can really go anywhere in the file and the order is usually obvious. If function a calls function b then b should probably be close to a.

After discovering that you can program like this, even before I switched to C I started making my files larger and a weight has been lifted from my shoulders. I wholeheartedly recommend this approach. Now whenever I see that some functionality is related I just make it part of the same file. A graphics library? One file. A testing framework? One file.

But wasn't the blog post about .c and .h?

Trust me we're getting there. As part of my journey I stumbled upon the STB libraries. Not only do they embody the idea of a single file per module they also double down on it by bundling .c and .h together. But how?

We need to talk quickly about header guards. When you #include a file it's actually a preprocessor directive. And there are more directives, for example #define. So to prevent a file from being pasted twice when included we can wrap it in a header guard. When the file is included the second time the guard is already defined so the contents are skipped.

// myheader.h
#ifndef MYHEADER_H
#define MYHEADER_H

typedef int MyInt;
MyInt my_add(MyInt a, MyInt b);

#endif

We can actually reuse the guard idea for the implementation. Instead of it being automatic, we are going to ask the user of the module to define if they want the implementation or not.

// mylibrary.h
#ifndef MYLIBRARY_H
#define MYLIBRARY_H

typedef int MyInt;
MyInt my_add(MyInt a, MyInt b);

#endif
#ifdef MYLIBRARY_IMPLEMENTATION

MyInt my_add(MyInt a, MyInt b) {
  return a + b;
}

#endif

This is the exact approach of libraries like stb_image. I wasn't fully satisfied with it however. If this approach were to scale to the entire codebase we need to fix two issues. For ease of use it would be great to request implementations of all modules at once. Additionally, including the header multiple times after requesting the implementation shouldn't add the implementation multiple times.

Without further ado, here's what I currently use in my projects:

// mymodule.h
#ifndef MYMODULE_H
#define MYMODULE_H

// =-=-=-=-=-=-=-=-=-=-=-=-=
// HEADER
// =-=-=-=-=-=-=-=-=-=-=-=-=

#endif
#if defined(IMPLEMENTATION) || defined(MYMODULE_IMPLEMENTATION)
#ifndef MYMODULE_I
#define MYMODULE_I

// =-=-=-=-=-=-=-=-=-=-=-=-=
// IMPLEMENTATION
// =-=-=-=-=-=-=-=-=-=-=-=-=

#endif
#endif

Every module looks for the generic IMPLEMENTATION which when defined enables all implementations. The implementation itself is also guarded by it's own guard. But the best of it all is what you can't see until you use this. When modules #include other modules their implementations are automatically inserted too. Which means that you can get away with code like this:

// main.c
#include "game.h"
// game.h   includes engine.h
// engine.h includes input.h
// engine.h includes gfx.h
// engine.h includes logger.h
// engine.h includes gltf.h
// gltf.h   includes json.h

int main(void) {
  game_init();
  game_run();
}

Even though there are multiple files, all the dependencies are resolved automatically and the final compile command just has to reference the main file:

$ cc main.c -o game

Which I think is just lovely. Since the compiler sees a single translation unit in the end I get the best of both worlds: fast compile times and multiple files. I also don't shy away from files that are thousands of lines long. Bring them on!

In a future blog post I'd like to talk about how we can squeeze even more into this one file by adding tests. But that's a story for another time.