Starting a business is hard. Starting a business is very hard. There are tons of problems to solve and a minimal amount of time for it. A technical stack for the startup could be the problem or enabler, depending on how you would approach it. The most important trait when choosing a technological stack should be pure pragmatism. Today I'd like to talk about the approach I took for Yarkly tech stack.
It is a very opinionated post that definitely will not fit every project, especially in a hard tech and in fields where the cost of error is high. However, the given approach and reasoning were helpful in my case, and I believe many software startups will find something useful among the lines.
Different product stages have different requirements
Different project stages often require entirely different approaches to product building, and tools used in the beginning will be replaced later on.
In the early stages, when you don't have any clue about customers and their needs, the most crucial feature is the ability to iterate as fast as possible. There is no need to build a colossal architecture that could scale to the level of the billion-dollar company if you will ever have close to zero customers.
Most of the modern programming languages allow you to build almost anything. It is almost always better to use the technology you are the most familiar with at this stage. There is no difference from the product perspective what kind of language or framework will power your business. Facebook started with PHP, and GitHub started with Rails, both are very successful companies.
I started a few projects over the last few years, and for some of them, I pick fancy technologies just for fun. I start facing some problems and limitations of the chosen tools very early on, and I had no experience in solving given issues with these tools. In the end, my poor choice added needless matters to the project — no need to explain how bad it ends up. Once I even had to rewrite the whole project to a more familiar platform just because there was no way I could keep building it in a fast and efficient manner.
There are no strict rules here except the one. The product should iterate as fast as it can to get to the product-market fit as soon as possible. I know some very successful teams that hack everything together and even skips writing a single test in favor of quick iterations and getting to the customers as fast as possible.
Most of the startups start with a monolithic architecture, where everything is written inside one big application. This approach allows saving a lot of time on infrastructure support because all the parts are in one place. Having all the code in one place could significantly improve development speed because the framework development team addresses most of the common problems, and there is no need to care about between services communications.
Now when you have some customers willing to pay you, and there is quite some technology that is build—time to start slowly become more rigor in terms of codebase quality and processes.
Most of the companies slowly start increasing the engineering headcount at the given stage. It requires some additional rigorousness in the development process. Some tests have to be added, and some ad hoc solutions should be replaced with a more reliable solution. However, the main focus is still on fast iterations and learning from the customers.
Quite often, it is still possible to keep growing with monolithic application architecture. There are two main reasons to give up with monolithic applications: too many engineers working at the same code parts and cause issues to one another, and it is impossible to scale hardware vertically.
Most of the startups never get to this stage, and one of the reasons for that is the fact that they tried to solve problems related to this phase during the prototyping period. The main objective at this point is how to serve all the customers willing to use the service and keep the servers online at the same time.
Unfortunately, it is tough to comment on what precisely should be done at the given stage because every organization has a different background and issues in different places, so they should approach the given stage differently.
Usually, companies kill monolithic applications they have, solve the reliability issues, and make the development process as smooth and independent as possible. Every single organizational dependency slows down the company and costs thousands of euros/dollars/pounds/etc. so they should be addressed as soon as possib
Yarkly tech stack
Given the fact I am still in the prototyping stage, I start with the tools I am more familiar with. I have many years of experience, so I automatically adjust some parts of my development process to scale in the future. Still, it is crucial to mention improvements do not require any significant efforts from my side.
I use git as my primary code versioning system because I use it for quite many years and quite efficient with it. Nowadays, repositories can be hosted online for free; in my case, I use Gitlab as a git repo hosting provider. In addition, they provide 2000 minutes of CI/CD pipelines for free, so altogether it makes my development process very lean and efficient and placed in one place.
My primary hosting platform is Heroku. Despite the fact it is somehow more expensive than other platforms, there is straightforward reasoning why I use it. It allows me to iterate very fast and save a significant amount of time on infrastructure.
I made simple napkin calculations and decided that, in my case, it is much more affordable to use Heroku than hire someone to take care of my servers and deployments.
The additional benefit is a straightforward deployment process. I spend probably less than an hour and set up automatic deployments off all the code I push into my Gitlab repository.
Languages and frameworks
Java & Spring Boot
I use Java for many years. It is battle-tested, proven to be a fast, reliable, and extremely scalable production-ready solution. It was a no brainer for me to use it as a core component of my application.
To organize my code and solve the most common problems, I use the Spring Boot framework. I write and arrange my codebase in a slightly unconventional form. I call it a "scalable monolith." The code I write is placed in a single application; however, it is divided into separate modules based on the core functionality. This approach allows me to iterate fast and use all the benefits of monolithic application architecture. At the same time, I could split the application into functional microservices in a matter of hours.
Typescript and Node.js
Despite the fact I have significant expertise in using Typescript and NodeJS, I would avoid it as much as I can. Unfortunately, there is a business need to check data on the web pages. There are two mainstream solutions out there: Selenium and Puppeteer. Both of them have advantages and disadvantages, but I decided to stick with Puppeteer because it offers me a faster start and better transferability.
I have two completely independent parts of the application written using completely different languages and platforms, and it causes some hurdles. One of them is integration and communication between these components. There are a few scenarios of how systems could communicate with each other.
Over the persistence layer
One of the easiest and more reliable ways to integrate the components is to use a shared database. There is a massive set of issues with this approach. However, it is a fast and convenient way to integrate applications. Unfortunately, many teams abuse it, making them less efficient later on, when scalability issues matter.
Another widely adopted approach is to use direct HTTP calls from one service to another. However, some corner cases have to be kept in mind. For example, applications should be able to handle the situation when the target service is unreachable. Another vital consideration is HTTP calls authentication and authorization because we don't want our API to be available and exposed to everyone in the world.
One of the most popular and scalable way to communicate between distributed components is to use some messaging systems. One of the most popular solutions to that problem is Apache Kafka, luckily Heroku offers a hosted version of Kafka for a very reasonable price.
I don't want to invent a bicycle here, I have some other stuff to do, so I decided to use well known, and battle-tested technologies. My DB of choice is Postres, and luckily Heroku offers a hosted version of PostgreSQL 12, so I have a great fit here.
Error handling and monitoring
I try to keep everything as simple as possible, so I rely mostly on the Heroku provided metrics for hardware monitoring. At my scale, it is a sufficient source of data, and it covers all of my needs.
I use Sentry.io as my primary error monitoring solution. There are two main reasons why this solution is so appealing to me. First of all, Sentry is an open-source platform, so potentially I could host it by myself. It is an attractive and very nice feature. Anyway, I don't plan to do so because I have much more important stuff to do. Secondly, Sentry provides a free tier for small apps so I could use it for some time without any expenses. In the later stages, my small monetary contribution in exchange for service will contribute to the neat open source project.
Documentation and project management
There is probably an endless amount of project management software. In my case, I use Basecamp because it lets me keep all the necessary documentation, customer interviews, research, tasks, and everything else in the same place. There are some disadvantages, some things are very opinionated, and I am not used to some of them, but it is a great tool that solves it's a problem very well.
I try to stay as lean and agile with my technical stack as possible. At the same time, I have to optimize for fast iterations and customer development. Leveraging my experience and PaaS platform, such as Heroku, allows me to build my project swiftly and efficiently.
In the next series
That is all for today. Subscribe to my newsletter, and you will get the next post from my startup diary. See you next time when we will talk about being laser-focused and work on one project at a time.