Introduction
This blog will be some ideas that I reassemble of how I think about programming. These are subjected to change the more I grow. These are pure personal and opinionated, it’s just a public memo that I would like to revisit from time to time and track how the way I think about programming have changed each time I look at this. A worth noting, that I am not a subject expert. So I touch the surface of different topics. I won’t claim that I know the smallest details about all of the things. Just enough knowledge that get the job done.
Philosophy and Rules
When I develop software, I like to keep the design simple. Although, there is a quote that I like a lot.
"For every complex problem there is an answer that is clear, simple, and wrong." — H. L. MenckenSome problems will force complexity into the design. I like simple tooling too, one reason that I hate JavaScript ecosystem is the amount of things you have to include in the project to produce a working app, that most of the time you don't even know what is going on. That it, every time I see something that using many things to do a simple stuff, I take a step back and ask if I really need to work with this. Any ecosystem that dump a lot of stuff into the project is kind of a deal breaker for me.
For the library choice, the first principle that I follow is Go proverb A little copying is better than a little dependency. which is do I really need a library for this? If so what I look at is the number of dependencies that this library have, the fewer that better. Then I look at performance of it, I will always Go with libraries that have 0 allocations for example. Also, most important part is maintainability of this library, when was last time this was maintained? If I really like it, I fork it and maintain my own copy of it. Although, for security related stuff, I always go for higher security regardless of all points mentioned before.
Also, I tend to like the libraries that are very specific to solving a set problems. Not something that is trying to be a generalist.
Also, it’s very very rare, that I rely on a framework, a framework is just someone else opinions being forced on me. I like to stick to standard library and libraries that check my boxes. So if I need something for example for my web projects, I write my own framework for that.
Meta-programming, I am a big fan of it. I like to have generated code when I can, rewriting same code is nothing but a waste of time. Also there is performance aspect to it, relying on runtime reflection is not only risky, but also expensive as it hurts the performance in magnitudes. So you will see me tending to use tools that will generate boilerplate code like sqlc, grpc gen protos.
From minimal code I can generate more code. For example I can define my struct and let a general purpose app generate necessary methods for my struct.
Also meta programming, allow generating code from variable stuff at compile time. So if I have dynamic stuff to maintain dynamically in config files, instead of doing a map at runtime map[string]struct{} just to make sure that I am passing correct dynamic string for example I have X and Y dynamic configurations, at compile time, I generate operationOnX(x TypeOfX), operationOnY(y TypeOfY)
Waste in software, many interpreted languages, generate a ton of waste instructions, for which will be one instruction in compiled peer. That’s it, I won’t use any interpreted language for critical parts of the system. Performance always matters for me. Although, at this moment I don’t do systems programming, but I like to do so. Because at the end of the day the code is executed on hardware, and this hardware have semantics, designing software to have data structures that are memory and cache lines friendly is my end goal to master. So my software design will dramatically relies on this fact. At time of writing this, there is data oriented programming which I would like to learn more about, and start designing my data structures based on that. Another benefit of compiled languages, is that you don’t get left with surprised during runtime, altho there are transpilers these days on top of Python and Javascript for example, but still after transpilation layer you left with original code, which is just full of waste.
Concurrency and parallelism, I always try to leverage them, although I may leave a note to handle concurrency later, if I am in rush. And revisit later. CPU, OS and languages themselves, each provide a way to handle these. CPU have physical cores, OS have threads, and programming languages have fiber threads/coroutines. Each share same spirit and managed at a different levels. Something I take care about is the context switches and locking. These can hurt performance sometimes instead of gaining it. So when I design a concurrent data structure, I try to avoid locking as much as I can. And if I have to lock, that shouldn’t be frequent. If so the routines will end up, locking each other and doing context switches so it end up worst.
Distributed systems, altho I like distributed in theory, in practice they are hard, a lot of engineering time will be spent solving hard problems. One quote I like is
"You can have a second computer once you’ve shown you know how to use the first one" — Paul BarhamSo first question I ask, do we really need to distribute? Although, I won't claim that at the moment of writing this, I know how to measure exactly how much the server is able to handle load before I go distributed. That's something definitely I like to learn about. From my studying of distributed systems, when the system is stateful, the best data structure that fit the system is state machine that is single threaded, which means only one routing can alter the state of the machine at a time. A talk I watched that I will try to apply in my own state machines is [Release Party | Ways To Do Things with Peter Bourgon](https://www.youtube.com/watch?v=LHe1Cb_Ud_M) state machines fits so well with `WAL` and append-only log systems. Mainly you read from an append-only log and apply the actions to your state machine. Also, if this state machine get it state from external events, then the events should be validated and translated to domain events, for example I get UUID as string from external world, their should be a layer of protection around the state machine, this received string should be validated and parsed into domain object which is UUID type, after that the state machine get updated to reflect the new state.
Observability and Monitoring, without these we can’t understand the system that we are working on, without monitoring SQL queries for example, I can’t know which queries are under performing so we can optimize. Without numbers I can’t understand my system. Usually when you develop an app the debugger is your best friend, not only to catch bugs, but to also understand the system you are working on. In web development with distributed async flows, the debugger can’t save us most of the times. So web ecosystem have some tools like otel which provide a framework to capture events in the system with their time taken, which if done correctly ( which is the hard part of it ), the traces and logs, can to a degree give some powerfulness that can be gained in debugger, although it’s not even close to the full benefits of the debugger.
AI, in my workflow I do use AI, but I have disabled it as auto complete, mainly I use AI as replacement of google, and the neat part, is that I don’t need to leave my editor, I have a shortcut that open an agent chat, without leaving my editor, that’s so powerful, before googling wanted to leave the editor and go over many resources to get a useful information, in AI it’s just text, so can be integrated so nicely with editor. I do chat with AI about some designs, also I have added some shortcuts that I found useful, like providing a code review for my staged code, or generate commit messages ( I use this one a lot, and I love it! ). In general, I like AI, it has proved to increase productivity, and search engines these days are not so good to use. So AI is better and I use as advanced smart search.