Logic World Wednesdays: The Aligned Edition

by @MouseHatGamesDeveloper2 months ago (edited2 months ago)

I’m back! - Felipe

Hello everyone, I’m back! I’ve been busy with the final exams but now I’m free and back to working on the game. I know the site has been very unreliable lately and I’ve already diagnosed a possible solution to this; hopefully I’ll implement it in the coming days. I’ve also been working on rendering the site on the server, but I haven’t quite finished it yet so I’ll talk about it in more detail in next week’s LWW.

Customizable Alignments - Jimmy

On Tuesday I was watching this delightful beta stream. I was surprised and excited by how the two players were using Labels to collaborate and share ideas, writing out equations and truth tables and whatnot. However, it was a bit awkward for them to do this; the Label text was always centered, and so to line things up properly they had to insert a lot of whitespace.

So, I added alignment options to Labels! The text can now be aligned on both the horizontal and vertical axes.

alignment options.png

Smooth text boxes - Jimmy

Last Halloween I added smooth scrolling to the game’s scroll views. However, internally a very different system is used for regular scroll views and for editable text boxes that scroll (nothing I can do about this, it’s a Unity thing). It has irked me ever since that scrolly text boxes were unsmooth.

Well this week I finally ported the scroll view smoothing to text boxes! Finally, the tyranny of unsmooth scrolling is gone from Logic World for good.

New save format - Jimmy

Soon I’ll be implementing a new binary save file format to the game. Due to the new circuit simulation logic and the nearly-done subassembly system, the old save file format is no longer suitable.

I’ve mostly finished speccing this out, and I plan to start coding it in the next few days. A lot of y’all were very interested in the old save file format – even going so far as to write code for interacting with the saves! – and so I’m sharing my draft of the new file format: you can check it out here. If you’re the kind of nerd who has opinions on binary save file formats, then please share that feedback with me :)

We’ve decided to call this the Blotter File Format, as a tribute to a lost component.


We’ll keep releasing these weekly updates right up until the game comes out. To make sure you don’t miss them, you can sign up for our newsletter. Be sure also to wishlist Logic World on Steam and join the official Discord.

See you next Wednesday!

More Logic World Wednesdays


8 comments
@DaMastaCoda1 month ago

Just an opinion: for the text alignment, I believe a 3x3 (total 9) grid of options makes sense more than two separate options since the button’s location would match the text, and vertical alignment would match the vertical change in buttons.

@JimmyDeveloper1 month ago

Hm, perhaps. I was trying to model that UI off of existing text alignment UI so that users can benefit by already being familiar with it. The two separate controls for vertical and horizontal alignment is typically how I see alignment options implemented in other software.

@Ecconia2 months ago (edited2 months ago)

I just finished reading the save-file documentation V5.
And it was fully understandable to me. However I also brought a lot of required context along.

I did have a bunch of questions which got answered later on in the documentation.
For a documentation this small that might be okay. But if it grows, the layout should be reconsidered.
Most documentations have the atomic types at the top (int float etc). And work their way up.
Combined with a good index one then can jump around.
I do like the linking of certain terms. But I think that terms like “circuit (state)” should be explained much more early. Also terms like subassemblies could be linked to its definition.


PLEASE FIX THIS ISSUE:

Just like Version4 of the save file, Version5 comes with the same problem:
The mod TextID’s are missing as well as its version. This is not acceptable. Change it before release!

  • One can argue that the prefix of a component is the mods TextID. And technically that is okay. But then LW should disallow mods to add components with a different TextID. Reason being, that if LW encounters a component prefix/mods TextID which it does not know yet, it should not say “Idk how to open that”, but instead “Hey, that uses mod XYZ, want to look on the LW website for a download?”

  • The biggest problem here is the missing mod version though. Mods also have updates. It is not just LogicWorld. And that also means, that the file format of a mods component can change. And it would just be amazing, if mods can simply query the version of itself when its own components got exported into a save. That allows the usage of automated save-loading converter, supplied by the mod. Mods can workaround this issue, but saving its own version in each and every of the custom data fields of each instances of its component. I am pretty sure you can not give me a single reasonable argument in favor of that.


Everything that follows now, is things that I would probably do different without good arguments against it.


Notes/Questions:

  • (Repost) As @Red3D already said, mark the save-file type values over 3 as reserved.
    Is this meant to be a bitfield, or why not start at 0?
    How will LW handle an extension to the file-format (values over 2)? Just “save-file is created by a newer version of LogicWorld” I assume.

  • There currently is a maximum of 65534 component types. And probably no mod in near future wants to challenge that. So it is probably an okay-choice to use 2 bytes for the component id. It slightly irritates me though. I wonder if mods can reach that limit. What is your feeling regarding this? (See Integer compression later on.)

  • Which characters are allowed in the TextID of a component? Is it literally any Unicode?
    LogicWorld is written in C# a high-level language with built-in unicode support. Many tools will have such language support too.
    Some tools however might not have that support. And further unicode leaves a huge room for exploits and edge-cases. (Zero-Padding-Space etc…).
    Primary I fear that this might be ugly on the website. First component to do this: “Is tͭhͪaͣtͭ aͣ goͦoͦdͩ cͨhͪoͦiͥcͨeͤ tͭoͦ aͣlloͦw aͣll tͭhͪeͤ uͧniͥcͨoͦdͩeͤ?”
    I fear it is easier for every participant to default to ASCII+Symbols you won’t have edge cases with that.
    Yes it does limit the creativity of some few users, but it increase the usability of LW for all other users.


Shrink the filesize (Why do you even save that byte?):

  • I still do not quite understand why there is the component-address instead of an index referencing the order of component, which could easily be generated while loading the file. Especially since this directly becomes technically obsolete, if there is also the rule “parent ID must be one of the previous components”. The only justifiable reason I see here is to prevent corruption or to allow reconstruction.

  • Why is the exclusiveness of pegs still not a bitfield, but a boolean/byte list?

  • Related to that, wires also use the component address, instead of a connector index, which could also be generated while saving/loading.
    The only downside is to set a maximum size of signed integers for connector/peg amount. I lack a statistic of how high these numbers go.
    It is probably fine to do it as you do, while you could spare memory there. But that is just an idea. Just keep it as is. What are your thoughts?

Ideas:

  • Subassemblies store the circuit state as list of ON-states. In case that the OFF-states list is smaller, one could store that. And provide one boolean as toggle.

  • Presonally I still see no reason, why a core part such as input/output count is stored per component, instead of included as an extension to the component-ID map.
    As in provide an additional map which references into the component-ID map, which component then reference to. That map would contain the I/O counts.
    This literally saves 2 bytes for all components. At a most of the time small price. (Kind of like creating an entry for each prefab - but ignoring the size of a component, since that grows too exponentially echem boards.)

Suggestions:

There is a form of variable-width integers: You basically just read a byte and if the highest bit in the byte is set, you read another byte, until the highest bit is no longer set. This allows you to encode numbers from 0 to 127 within 1 byte.
The component ID would gain a lot from this, as well as connector/peg count (Probs also circuit-state ID - not sure). Would shrink the file-size a bit further.
The save file is already working a lot with variable-length sections, so this will not add much complexity. The only downside is a slightly reduced encoding/decoding time.


Simulation state storage:

As you already wrote in the documentation, you are using the internal format of LogicWorld.
That is pretty much okay, and it looks nice. However in terms of file-size is it not desirable.
The primary reason to stick with it, is to reduce the loading time. While I doubt that the conversion takes much time even for big worlds.
However the inconsistency of visual connections and exposed internal circuits smells, in a negative sense.
Yes not your responsibility if an external tool messes that up. But how is LW handling it, if circuit-state ID’s are completely different from the actual representation of the circuit network?

I also thought a bit about the note how the states can be captured.
- Currently you store the state per “cluster”. That is okay, but a little bit redundant, but contains all the information without extra work.
- Less redundant is to only store the output-peg clusters state. And it is easy to get the connected cluster states as well as the inputs of component states from that.
- Storing only the input-pegs works too, here the outputs need to be calculated by the component’s simulation though, which is highly undesired.

You may do as you please here. But that would for corruption and file-size reasons not be my choice.


I think that was everything I had in my mind. If you did not understand my point of a certain point, feel free to ask.

@JimmyDeveloper2 months ago

Thanks for the long and thoughtful comment! Took me a while to get to it but here we go:


The mod TextID’s are missing as well as its version.

Thanks for pointing this out. I’ve added a list of mods, with versions, to the file format. Mods will be able to opt-in to be included here, and are opted-in by default if they add any component types. There will also be an API for save converters that mods can use.

How will LW handle an extension to the file-format (values over 2)? Just “save-file is created by a newer version of LogicWorld” I assume.

Correct

There currently is a maximum of 65534 component types. And probably no mod in near future wants to challenge that. So it is probably an okay-choice to use 2 bytes for the component id. It slightly irritates me though. I wonder if mods can reach that limit. What is your feeling regarding this? (See Integer compression later on.)

I do not anticipate any realistic scenario where a game install gets even remotely close to this limit, unless someone intentionally tries to reach it just for the sake of breaking things. We probably could use more bytes for this, but I really don’t think it’s necessary.

Which characters are allowed in the TextID of a component? Is it literally any Unicode?

Yes. We encourage alphanumeric + ., but theoretically there’s no inherent restriction. I do not anticipate this being an issue.

I still do not quite understand why there is the component-address instead of an index referencing the order of component

Why is the exclusiveness of pegs still not a bitfield, but a boolean/byte list?

Related to that, wires also use the component address, instead of a connector index, which could also be generated while saving/loading.

Please see the section at the bottom of the doc, “A note on seemingly-bizarre design choices”. The save file format uses the same data structures that are used at runtime. This is to make saving and loading both fast and simple, but it does mean that the save file format appears a little weird.

Component addresses are used because if a component is deleted, we don’t want to have to change the address/index of other components.

Subassemblies store the circuit state as list of ON-states. In case that the OFF-states list is smaller, one could store that. And provide one boolean as toggle.

This is additional complexity, and I want the code for saving/loading to be as simple as possible. I am not worried about the size of files, they are acceptably small in their current state.

Presonally I still see no reason, why a core part such as input/output count is stored per component, instead of included as an extension to the component-ID map.

Some components have variable peg counts, like AND gates and Displays. Storing the data this way also simplifies conversion to/from the data structures.

variable-width integers

I don’t want to introduce the extra complexity.

But how is LW handling it, if circuit-state ID’s are completely different from the actual representation of the circuit network?

The circuit model is recalculated when the save is loaded, it is not based on the saved circuit states. The saved circuit states are applied to the circuit model after the model has been calculated.

Also keep in mind that the state of all pegs/wires needs to be stored at least in subassemblies so that the client can display an accurate visual of them.

@tokumei2 months ago

I would also try compressing the files instead of using varints. Something like LZ4 would easily beat the space saved from varints, and it would be much less complex to add another layer of I/O versus a custom function for reading and writing integers that has to be applied everywhere. Time overhead is definitely larger, but based on the results from this benchmark it could easily be around 5ms per megabyte.

@Ecconia2 months ago (edited2 months ago)

Indeed “varints” make most sense if you do not compress the files.
While I think even then you could get out some optimization. Information that is not there, does not need to be compressed.

Personally I would compress not this file, but the whole save-folder. But that is a different story.
(Here there is the problem that users will not be able to compress it again after somehow uncompressing the folder).
While when applied per file, there is no reason for them to compress without a LW-Specific-Tool.

But as long as the LW-LoadMenu can load the information fast enough on the fly - smash compression on top!

@Ry2 months ago

Footer The final 16 bytes of every Blotter file are 72-65-64-73-74-6F-6E-65-20-73-75-78-20-6C-6F-6C , which is UTF-8 for “redstone sux lol”. thats so funny

@JimmyDeveloper2 months ago

We like to have fun here