At OpenWeb, our proprietary technology is built for performance. Our infrastructure was designed for speed and scale to support billions of conversations in real-time.
In this Q&A, Erez Levi, Tech Lead of the Infrastructure Team at OpenWeb, provides an in-depth look at the way our engineering team simplified dependency management on our Go services and got better visibility into code dependencies.
Here you can find the full example on Github.
Here you can find a link to Uber’s FX repository.
Who am I, and why should you care?
I’m Erez, the Tech Lead of the Infrastructure Team at OpenWeb, and I have been writing Go for the past 3 years.
I came to Go from a C# background, where you get a lot of “magic” behind the scenes with libraries like Unity Dependency Injection. When I made the move to this new language, I missed the ability to inject what I need without worrying about all the constructors I’ll have to change.
As the services I wrote got more complex, I got to a point where it was crucial to simplify the dependency injection to ease testing, clear the dependency graph, and get better visibility on our code dependencies.
Why is dependency injection in Go so difficult?
At face value, managing dependencies on your own isn’t such a difficult task.
Your database needs a logger so that you can trace what’s happening, so you pass a logger to it. Your business service needs the database so that it can access data, so you pass the database to it. Then you realize that you also want to use your logger in the business logic service; tracing is very important after all. And to be more efficient, you want to add a cache to your service so that you aren’t always going to the database. That cache should probably have a logger as well.
All of these things add up, and you end up passing instances of everything here and there, getting lost in the mayhem of it all. After some time working on your project, you’re left with a complicated mess, and that’s just to start up the service.
So here are the problems we face:
- Long initialization functions.
- Difficulties injecting the same dependencies that only need to be initialized only once (loggers, etc.)
- Complicated dependency graph.
- Adding new dependencies forces you to change multiple function signatures.
- Replacing real dependencies with mocks while testing forces you to duplicate all of the initialization functions for the mocks
So, how can we solve these problems? Let’s dive in
First, let’s create a simple server using fasthttp and fasthttprouter:
This doesn’t seem so tricky, so let’s spice things up by caching the result and logging any errors we might encounter. In order to do this, we’ll start by converting the functional handler into a struct that contains our dependencies, a logger and a cache:
And once we have this, we implement a Handle method that matches the signature of a fasthttp Handler:
Now, we have our handler, but we need to make sure to make some simple initializers for the logger and the cache client (we’ll be using Redis for our cache in this case):
Let’s put all of this together in a main function and see how it looks now:
We can see here that we have a handler and a cache. Both have dependencies, some shared (logger) and some exclusive (Redis client). Of course, in real-life scenarios, we can have a lot more. So what can we do to simplify this?
Fx to the rescue!
Fx has two main functionalities, Providers and Invokers.
Providers are our “Constructors”, they provide a dependency to others. On startup, Fx creates a container, and we register our constructors to it as Providers. That would look something like this:
Once we have our Providers, we create our Invokers. These are methods we want to run using the provided values that we just gave Fx. The Invokers can receive what Fx calls a Lifecycle, a struct that allows us to subscribe to specific behaviors such as OnStart and OnStop.
Let’s take the end of our main function, where we created the HTTP server, and wrap it in a small function that can be used as an Fx Invoker. We’ll pass in the Fx Lifecycle, and the main dependency that the server needs.
Now we add this function as an Invoker in our Fx container:
And finally, just to make things interesting, let’s add a simple listener on signals so we can quickly stop our server, and we’ll be ready to go!
So, let’s take a look at all of our problems from the beginning, and see how we tackled them.
1. Long initialization functions.
Fx simplified our initializations since we could easily inject edge dependencies of the dependency graph without knowing the full lifecycle.
2. Difficulties injecting the same dependencies that only need to be initialized only once (loggers, etc.).
Fx was a game-changer when it came to initializing once and injecting a lot. We only had to create a Provider one time, and Fx took care of passing it to all of the necessary places.
3. Complicated dependency graph.
Fx didn’t actually solve the complexity of the dependency graph, but it gives us the ability to care less about it. We only had to tell Fx what each constructor needed, and it took care of the rest!
4. Adding new dependencies forces you to change multiple function signatures.
Fx solved the ease of adding or using dependencies that are already registered to the container; all we needed to do was add the dependencies in the constructor of the type, and Fx took care of injecting all necessary dependencies to each constructor for us.
5. Replacing real dependencies with mocks while testing forces you to duplicate all of the initialization functions for the mocks.
Fx solved the ease of replacing real dependencies with mock ones; we could create a separate container for tests that uses the mock dependencies to test various scenarios.
If You Want To Dive In A Little Deeper:
- Fx is a heavy user of reflection. We were ready to pay the cost of reflection during the application startup time, but this is important to remember. Reflection in Go has a heavy cost.
- Using reflection, Fx gets the type of the Provider that needs to be injected. When using interfaces, you can automatically inject only one implementation of that interface. This can easily be solved in a few different ways, but it is something to keep in mind as you consider the move to Fx.
- The cost of reflection is always important to consider when it comes to low latency systems if used on runtime.
* If you want to dive into a more complete example, take a look here.