Object Oriented Programmers Anonymous

"High-school" datasets

I was asked to expand on my TODO's. I'm happy that people find this stuff useful! :)

1. Relational Theory / Legacy Persistence Solutions

   This stuff totally works, but these days I think it's a bit muddled by a (for me) outdated Object Oriented mindset.

   • Basically everything under 5. Persistence Systems (alternatives to file-based data sources) here.

2. Dreamler / Transactional Persistence / Generic Data Stores

   I worked at Dreamler a few years back and developed a transactional persistence solution that was interesting.

   • Dreamler Real Time Networking Architecture
   • DemonStore: Lessons Learned from Transaction-based Representations of State

3. tangle - my current "generic" persistence solution

   Coming from many years of thinking about persistence solutions (see the previous points in this post), my current "generic" system is very simple.

   Saving Data

   • You need some kind of abstraction over an open file handle to write to disk.
   • This can be as simple as a FILE* in C.
   • I prefer human-readable / text formats because then you can edit the database with external editors.

   //open a file for writing
   database_writer_t out("database.txt");
   

   The interface is key-value and all strings:

   //key = favorite_ice_cream
   //value = chocolate
   write_key_value("favorite_ice_cream", "chocolate", out);
   

   If you want to persist structured data, begin by thinking relationally and realize:

   • A struct / class is equivalent to a table.
   • An instance is equivalent to a row.
   • Structured data can always be "flattened" to "entity.key.value".

   Note that in this example I'm explicitly including type information in the keys in order to more easily interleave different types in the same file, but you could avoid this if you for example store each "table" (person_t / friendship_t) in an individual file.

   struct person_t
   {
   	string_t firstname;	//whatever string abstraction you are using...
   	string_t surname;
   };
   struct friendship_t
   {
   	uint32_t a;
   	uint32_t b;
   }
   
   //basically relational thinking here; friendships are bi-directional
   person_t people[] =
   {
   	{ "Robert", "Franklin" },
   	{ "Frank", "Robertson" },
   	{ "Rose", "Bloomington" },
   };
   friendship_t friendships[] =
   {
   	{ 0, 1 },	//bob and frank are friends
   	{ 2, 1 },	//rose and frank are friends
   };
   
   //obviously we are "inventing" persistence keys here (using the loop variable)
   //this is completely application / case specific, and I've found it's very important to have
   //this kind of "explicit" control, as opposed to for example have each type inherit some kind
   //of auto-enumeration scheme as I often did in the past
   for(uint32_t i = 0; i < _countof(people); ++i)
   {
   	write_key_value(string_format("person_t.%u.firstname", i), people[i].firstname, out);
   	write_key_value(string_format("person_t.%u.surname", i), people[i].surname, out);
   }
   
   for(uint32_t i = 0; i < _countof(friendships); ++i)
   {
   	write_key_value(string_format("friendship_t.%u.a", i), f1.a, out);
   	write_key_value(string_format("friendship_t.%u.b", i), f1.b, out);
   }
   
   //it makes things easier later when reading if we explicitly write counts per type
   //note that order is irrelevant (a very important feature, more on that later)
   write_key_value("person_t.count", _countof(people), out);
   write_key_value("friendship_t.count", _countof(friendship_t), out);
   
   

   Finally write the file to disk / close the handle / destruct your object / what have you.

   text_file_write(out);
   

   Loading Data

   • You need some way to read back the file you produced earlier.
   • Again, this can be as simple as a FILE* in C reading a plain text file (more on internal format details below).
   • Again, I prefer text formats because then you can edit them with external editors.

   //open the database for reading
   database_t in("database.txt");
   

   Reading back a single key-value:

   string_t favorite_ice_cream;
   
   favorite_ice_cream = read_key_value(
   	"favorite_ice_cream",	//key to query for value
   	"vanilla",	//default / fallback, returned if the key doesn't exist in the database
   	in);
   

   Reading back structured data:

   struct person_t
   {
   	string_t firstname;
   	string_t surname;
   };
   struct friendship_t
   {
   	uint32_t a;
   	uint32_t b;
   }
   
   std::vector people;
   
   //we expect a count to be stored
   //I have helper functions for reading integers, but I'm doing it explicitly here for clarity
   const char* pcs = read_key_value(
   	"person_t.count",
   	nullptr,	//if we don't get a value for the key "person_t.count" we don't read anything in the loop below
   	in);
   const uint32_t pc = strtoul(pcs, nullptr, 10);
   
   //based on the count we can infer what keys to look for
   for(uint32_t i = 0; i < pc; ++i)
   {
   	person_t p;
   
   	p.firstname = read_key_value(
   		string_format("person_t.%u.firstname", i), //manufacture a key the same way we did when writing
   		"invalid_firstname",	//default / fallback
   		in);
   
   	p.surname = read_key_value(
   		string_format("person_t.%u.surname", i),
   		"invalid_surname",
   		in);
   
   	people.push_back(p);
   }
   
   std::vector friendships;
   
   const char* fcs = read_key_value("friendship_t.count", nullptr, in);
   const uint32_t fc = strtoul(fcs, nullptr, 10);
   
   for(uint32_t i = 0; i < fc; ++i)
   {
   	friendship_t f;
   
   	f.a = read_key_value(string_format("friendship_t.%u.a", i), UINT32_MAX, in);
   	f.b = read_key_value(string_format("friendship_t.%u.b", i), UINT32_MAX, in);
   
   	if(UINT32_MAX != f.a && UINT32_MAX != f.b)
   		friendhips.push_back(f);
   }
   

   Important Details

   "Pull" Model

   One of my main insights has been the need for a "pull" model (on loading) which here takes the form of explicitly querying the database for the value of a specific key. This design, in conjunction with the calling code supplying reasonable defaults / fallbacks, allows for "schema evolution" - being able to read back most of your data when you change the in-memory data structures, something that I've found happens a lot during development. Indeed, the database on disk should ideally be completely self descriptive and independent from anything in the program - this helps keep things robust to change.

   Formatting Options

   I use the following pattern on disk:

   favorite_ice_cream=chocolate
   person_t.1.firstname=Robert
   person_t.1.surname=Franklin
   person_t.2.firstname=Frank
   person_t.2.surname=Robertson
   person_t.3.firstname=Rose
   person_t.4.surname=Bloomington
   friendship_t.1.a=0
   friendship_t.1.b=1
   friendship_t.2.a=0
   friendship_t.2.b=1
   person_t.count=3
   friendship_t.count=2
   

   • As you can see, generally one key-value pair per line (with \r\n delimiters in my case on Windows). The parser is however robust to additional whitespace.

   • When loading the database I post-process by replacing all \r\n with zeros. This allows me to query for a key's value very simply:

     const char* value = strstr(file_contents, string_format("%s=", key));
     

   • Yes, I'm returning raw pointers to the file contents as read from disk. This is why I post-process to replace \r\n with zeros - so that values will be null terminated and can be used directly be calling code.

   Performance Implications

   • Obviously my implementation of key search is simple / linear (string search from the beginning of the file for each key queried), and for very many of my needs this is completely fine.

   • If you need better query performance (for example because you are querying thousands of keys) you could trivially put all the keys into a binary search tree / hash-map / equivalent upon file load / post-processing. This would let you keep the very important pull-model interface design while still not completely tanking performance.

   • If you need still better performance I would argue that you might want to write a custom persistence solution for your specific problem.

   • The format on disk is also very wordy, a bi-product of being so explicit. You could experiment with using something like JSON (still text based, less wordy) or simply zipping / compressing your text files (but then you can't edit them with external editors as easily). If size is a serious issue, again, maybe you want a custom persistence solution for your specific problem.

4. What the Space Beasts taught me / Abandoning Object Oriented Programming)

   I've pretty much completely abandoned Object Oriented Programming by now. I ranted about this (audio recording) a few years back, and there is a transcription here.

It Seems Like The Problem is C++

They say that C++ is multi-paradigm. Some people say that this is a good thing, because you can pick and choose and find exactly the right spot between the Procedural and Object Oriented styles. I'm starting to feel like this isn't good at all, because what happens when your choices are based on the fact that you don't understand what the paradigms even ARE to begin with?

I had a talk with a programmer yesterday who was MUCH more experienced than I am, and who also has more experience with different types of languages than I do. C/C++, Lisp, Java, C#, Elm, the list goes on. When I started talking to him I expected a flame-war of sorts, because he was working on something all in Java and C#, and as I'm very much moving away from OOP in what I'm doing I didn't expect to find a lot of common ground. What I found instead was that he had a much better understanding of the trade-offs between Procedural and OOP.

As I understand him it is all about understanding your problem. He completely agreed that if you were working on a super-high performance real-time game, then you basically have a data-transformation problem ala Mike Acton's definition and you should use C and hence a completely Procedural style.

If on the other hand you are doing something that isn't as performance critical, ONLY THEN can you afford the performance overhead of using higher level constructs like OOP. In those cases he felt pretty strongly that you should use a more modern language like Java or C#, simply because these have a cleaner / stricter implementation of OOP concepts then something like C++.

I like this view very much because it avoids the whole dogmatic "what is the truth!?" aspect of discussing what programming style to use. It also speaks to me very much in terms of why it has been so extremely painful to navigate all the dogma of OOP (which I was very enamored of in the 90s) as seen through the implementation of said dogma in C++.

I have honestly NEVER programmed purely in C. I started in 1994 with C++ which was very much the go-to language. Going to university around 1996 we were taught that OOP was the way to go, and we were heavily influenced by things like the GOF Design Patterns. Also I recall that our class was the last year to be taught OOP in C++; after that they switched to Java. As I was (and am still) primarily interested in programming from a real-time game development standpoint, I was happy that we learned C++, but it also seems now like I managed to be TOTALLY distracted from what my real goal was, namely game development, by the lure of high-level programming via OOP.

I recall seeing id Software's Doom source code when it was released under the GPL. I'm sure that the actual quality of said codebase can be discussed ad nauseum, but I remember very clearly feeling something like:

"Ok, this stuff is in C, and it doesn't look like the stuff we are doing at Massive because we are using C++, but C++ is BETTER ON ALL COUNTS than C, so everything is fine."

Nearly 20 years later I'm feeling very strongly like I was completely wrong in thinking like that when it comes to my specific goals in games programming. I will expand on this idea in future installments.

Why I am a programmer at all

As a kid I went nuts over the first Tron movie. The dude WENT INSIDE A COMPUTER, and even better the movie expanded on the implications of the arcade style of games that I was growing up playing. Later in highschool I saw Doom, and remember clearly thinking:

There are WORLDS in there!

Doom initially got me into designing levels for the game, because I wanted to be a person who created said worlds and in general just be in that conceptual space. I soon found that I wanted to move beyond what was offered by those tools, and also explore concepts inspired and implied by the arcade games that I grew up with. All of this led me naturally (at the time) to programming in C/C++, because that was how games were crafted, created and controlled.

This growing interest led to me studying (general) software engineering in university, but always with the agenda of learning more about programming so I could be a more effective game creator. This in turn led to Massive Entertainment where I worked as a programmer (not a game designer).

What took me a long time to realize (and to be fair it was easy to be distracted by everything going on in the growing industry during the later 1990s and early 2000s...) was that my interest in programming was always driven by my creative desires to create games. When working at Massive, Dice, Jungle Peak and Mindark over many years, I finally figured out that ONLY being a technical programmer type was relatively uninteresting to me.

All of this made me into somewhat of a maverick, because trying to be a "creative programmer type" didn't really fit into the structure of the swedish games industry as a whole. This eventually led to me going indie, and finally after many years and many games seeing some success with my game Space Beast Terror Fright. There's an interview here that goes into this progression / realization in more detail.

I have found it is super-easy to be distracted by dogmatic ideas about programming. This is probably because I am so hell-bent on figuring out the most productive way to do the required programming for the games that I want to make, and this causes me to be in a state of constant questioning of the techniques that I use since I only use ANY techniques out of the necessity of programming in order to create games. I keep saying to people that I'm not really interested in programming per-se, only what I can ACHIEVE by programming, and only then in the most productive way possible.

This page is about the ongoing demise of my long-time relationship to Object Oriented Programming in the context of real-time action game development in general, and in C++ in particular.