Logic World Wednesdays: The Edition With A Short URL

by @MouseHatGamesDeveloper1 year ago

The metrics endeavour - Felipe

The logicworld.net site is composed of many independent services, and each of them has a number of metrics. A metric in this context is a value (usually a number) that changes over time, such as how many comments users have submitted, how many views the site got, etc. I’ve set up a metric monitoring system called Prometheus: its job is to collect metrics from all the services that expose them then aggregate and store them for later querying. Prometheus works on a “pull” fashion, where instead of the services sending the metrics to the Prometheus instance, Prometheus asks each service for metrics at a given interval.

Our cloud infrastructure consists of a Kubernetes cluster with (currently) two nodes. These nodes are regular droplets with their own IP, and each runs some services as orchestrated by the Kubernetes master. Each service in a node is contained in a pod with its own IP and can expose ports and connect to other services through their IP. In order to provide metrics Prometheus will send the service an HTTP request asking for data, so each service needs a tiny HTTP server to respond to the request.

The biggest issue was the fact that Prometheus needs to know the address of each service in order to send them a request, but since the Prometheus instance is running on a separate droplet outside the cluster, it’s on a different network than the services and thus can’t find or talk to them.

I solved this using a PushProx fork, which acts like a bridge between the Prometheus instance and the cluster network. It has two parts: a client and a server. The server runs on the same droplet as the Prometheus instance and it registers the client (or clients) when they connect to it, which is possible since the droplet has a stable and known IP. The client runs inside the cluster and it forwards any requests it receives to the appropriate service, then sends back the response to the server who then forwards it to the Prometheus instance.

All of this allows us to run a Grafana instance that queries the Prometheus instance and renders some cool graphs!

metrics graphs.png

It’s not very impressive yet, as I am still in the process of configuring and adding more metrics to the panel, but in the future this will serve as a centralized source of data to aid us in decision making.

This brings me to my last point: data persistence. Prometheus can store data for a certain amount of time, but it isn’t made for long-term data storage. This is where VictoriaMetrics comes in, which is a time series database just like Prometheus but designed for long(er)-term storage as evidenced by the lower disk usage. It’s fully compatible with the Prometheus protocol, which means that Prometheus can replicate the data it receives to the VictoriaMetrics instance, which will then efficiently store it.

We can then configure Grafana to use the VictoriaMetrics instance as a data source when we need to look up older data, and (in theory) all of the queries will not need to be changed!

If you’re still reading, thank you for suffering through this journey with me. I decided to skip some longer and boring parts to make it easier to read, so hopefully I succeeded and you enjoyed it!

Short links - Felipe

I also made a URL shortening service, which now lets us use short URLs like this one: https://logic.world/pst-5c6211b5. Short URLs are useful for sharing links on social media, and soon I’ll add a button to get a short link to any piece of content on the site. In the future, short links will work for lots of things beyond just posts and comments – uploaded creations, user mods, etc.

Note from Jimmy: getting ownership of the https://logic.world/ domain was super difficult, I’m delighted that we’re now using it for something! :D

A (nearly!) finished refactor - Jimmy

Last week I gave an in-depth look into a big refactor I was working on. At the time I felt I was almost done, but unfortunately facts don’t care about my feelings.

I ended up reworking a few parts of my new architecture, particularly the way parent-child relationships between components are managed. I also went through and added XML documentation to all the new stuff, so folks working with it (myself & Felipe, but also future modders) will have an easier time. The refactor has taken up most of my work this week, but it is almost, almost done. I believe there is just one remaining bug: a nasty issue where component peg information isn’t being properly deserialized, which I have been trying to figure out for like 4 hours now and which is slowly driving me insane. But hopefully I’ll figure that one out tonight and we’ll be on to features with more flash and fun for next week’s blog :)

On a personal note – I haven’t talked about this much publicly, but the start of this year was really difficult for me, and I suffered pretty severe productivity losses. I’ve been struggling for a while to climb out of that hole, but as of this week I think I’ve done it. I’m as excited and passionate as ever about Logic World, I’m thrilled every day to sit down and start coding, and I’m getting lots of great work done. If I can keep up this pace, we might very well be looking at a release date sometime before 2030.

We’ll keep releasing these weekly updates right up until the game comes out. To make sure you don’t miss them, you can sign up for our newsletter. Be sure also to wishlist Logic World on Steam and join the official Discord.

See you next Wednesday!

More Logic World Wednesdays

@Ecconia1 year ago

Logic World: The pessimistic edition…
Pipe01: If you’re still reading,
Jimmy: release date sometime before 2030

getting ownership of the https://logic.world/ domain was super difficult

What was the story here, sounds like someone owned it and you had to negotiate with that person.

short URLs like this one: https://logic.world/pst-5c6211b5

What does the pst prefix (which we are using for a time) stand for?

@JimmyDeveloper1 year ago

What was the story here, sounds like someone owned it and you had to negotiate with that person.

Yeah, that’s pretty much the story lol. The difficulty came because the owner was Chinese and spoke very little English. It took a while to agree to the sale and get the transfer completed.

What does the pst prefix (which we are using for a time) stand for?

“Post”! There are two other prefixes in use – “frm” for forums (i.e. https://logic.world/frm-devupdts) and “cmt” for comments (i.e. https://logic.world/cmt-40e807bd)

@ForLoveOfCats1 year ago

@Jimmy I’m glad to hear that you’re doing better mentally :)

@JimmyDeveloper1 year ago

Thank you, I am glad too <3