Hello CubeCrafters!
Welcome back to Behind the Cube! We've been on hiatus for a while and been busy cracking out other updates over the last few years, but today I want to step back a couple years to our 1.19 server update. This was our biggest, most ambitious update we've ever performed on the network. This has allowed us to deliver far more feature-complete updates across both networks simultaneously, and completely transformed our developer experience.
This edition will dive deep into the internals our server update to 1.19, some the challenges that we encountered during our migration, and some
spicy changes we’ve made on the tech side that have greatly improved our updating process. This is a juicy one, so grab a bowl of popcorn 
and let’s dive right in!
As many of you may be aware, we initially communicated our goals of updating the network to 1.19 waaaaay back in September of 2022, however in actuality, work began far earlier, prior to 1.19 even being released. Updating nearly 10 Minecraft versions was no easy feat, and required a lot of background planning and backend work.
Previously, our server platform was running on a 1.9.2 Spigot jar we updated to 1.12 and modded Bedrock support into back in 2021. Some of you may recall our Translator system we discussed in our Feelin’ Lucky Behind the Cube which sat on top of our BungeeCords. This system allowed us to support Bedrock Edition users on our existing Java infrastructure.
As we grew throughout 2020 and 2021, it became a major technical challenge for us to maintain as we started implementing native Bedrock features that are now integral to our server, such as resource packs, custom entities and custom items / blocks. We decided to go the route of removing the Translator, and turning our Java server into a full-on Bedrock server.
Moving away from our Translator proved to have a large number of benefits, including reducing player latency by 50ms - 100ms across the board. However, this came with the downside of maintaining a significant amount of code in our server software.
Rather than going the route of updating our 1.9-but-really-it-was-actually-1.12-server jar to 1.18, we decided to start fresh by forking Paper 1.18. The first commits to our new CubeTap project (internally known as CubeTap Junior) were in March of 2022.
The first few months of work primarily revolved around getting our new CubeTap version up to speed with our 1.12 version. We also took the opportunity to redesign core parts of our Bedrock support, such as moving to Bedrock’s server-authoritative inventory system, implementing chunk caching, and adding proper custom blocks. We also added new APIs to integrate nicely with these systems.
First look at chunks rendering for Bedrock Edition on our new software.
Much of the work to update the CubeCraft codebase began in July of 2022. For those of you who have worked with Spigot or Paper before may know that in 1.13, there were massive, backwards-compatible breaking changes in the Bukkit API. The internal NMS code also saw major changes, going from being heavily obfuscated to fully deobfuscated using Mojang’s code mappings. Additionally, from 1.13 - 1.18, huge amounts of the server code was rewritten.
While most of these changes were positive, essentially most anything using NMS was going to break (and boooyyy did we use a lot!) and most all 1.13+ Bukkit plugins were going to no longer be functional.
After some time, the first project we completed updating was GameFramework, which is one of our core frameworks that all our games are built on top of.
First look at a simple TNT Run game running on a local server using GameFramework.
Getting GameFramework to work was a massive milestone and meant that within a short period of time, we could actually start testing CubeCraft code on 1.19!
Within just a few weeks from this point, we had many of our other core libraries updated, such as Loot, Armoury, Play Again, etc. and by early August, had a version of Survival Games running for Java Edition
As bug free as you get…
At the speed things were going, it looked as if we were going to have 1.19 out by December, just in time for our 10th anniversary! Right? Right..? Well, we’ll see… let’s not get too ahead of ourselves
As work continued to get all our code updated to 1.19, a question arose: what did we want to get out of this update?
I think the obvious answer everyone arrived to was: offer new features and to build potential for future content. While these are and have always been at the root of any content update at CubeCraft, we were starting to see major technical debt problems arise on our existing network.
At the time, our codebases for both Java and Bedrock were very disjointed. We maintained four separate Git branches for each project:
This ultimately meant that if a bug fix was found in team handling for instance that was present on both Java and Bedrock, that would mean a commit fixing the issue would need to be pushed to eight branches (╯°□°)╯︵ ┻━┻
Also at the time, our update process typically involved us writing the code for Java Edition first, testing it, and then merging it into our Bedrock branches, often encountering multiple merge conflicts, then updating or removing any features not relevant to Bedrock, and testing again. This is also the reason that in times past, we often released all major content updates for Java first (or even exclusively in some cases), then later Bedrock. This made any major update incredibly cumbersome and often led to very "Java-first" updates that disregarded features on Bedrock.
Since we were already making major technical changes, we decided now would be the best time to reconsider how we conducted our updates on a technical side.
Shortly after getting Survival Games functional on our Java Edition development network, @rubik_cube_man, @Alemiz, @Austin and I sat down and discussed plans on how we wanted to approach this problem. We decided that modularising the code was the best approach, creating a common module for all common code, then separate java and bedrock modules for features that vary for each platform. Rather than maintaining 4 branches, we would only need 2: one for production code and one for development code.
We then pitched the idea to the rest of the dev team and decided on a course of action.
Modularising all our Bedrock and Java code together essentially meant that all updates we did would only ever need to be written once, rather than one time for Java, then tested, then updated in bits and places for Bedrock (or in some cases like MinerWare, substantially), then tested again, then finally released. We could instead streamline the process and do simultaneous testing on both platforms, allowing greater flexibility between updates and the ability to offer much more content. These changes also meant that general bug fixes in the game logic only needed to be written once, rather than individually for each platform.
SkyWars project layout today.
While the results of this modularisation have been incredibly beneficial for us today, it took us a long time to get there. There was a large amount of scope creep, unexpected challenges and many, many delays.
As we began to modularise our codebase and integrate everything together, we began to realize that many of our core libraries needed massive rewrites to cooperate well with both Java and Bedrock, or needed to be rewritten from the ground up. Systems such as our menu library, game voting system, loot, and other similar systems needed massive overhauls.
Here is a quick look at how we now create menus on our network now using the UI library we wrote as part of this migration:
The code used to render the Loot home menu in the Bedrock module.
The code used to render the Loot home menu in the Java module.
And in our common module, we can have a common open method, even if it is rendered differently on both platforms
In other instances, we found that simply having a feature switch for the player platform in the common code would be sufficient, as demonstrated with the following code:
This is a method in our floating text library, which instructs the system to use Text Display entities for Java, but Armor Stands for Bedrock, as Text Displays do not exist on Bedrock.
As amazing as these new libraries were from a development perspective, they required significant time investments to design, iterate upon, and eventually adopt across the board. For menus, CubeCraft has over 100 hundred different menus across both Java and Bedrock, whether it be the loot menu, VIP Levels menu, SkyWars features or Parkour map selection, and as you can imagine, migrating every one of them took quite a long time.
Menus were just one part of this - we also ran into various other challenges implementing platform-specific features, such as chest skins into the game. Despite the fact that we have code split up between two modules, all of the actual chest and loot table code is in the same place. We had to modify the core system responsible for this so chest skins could work fine on Bedrock without breaking Java Edition.
Our loot system needed to have significant refactors as well to support loading both Java and Bedrock loot in the same system. Certain features such as VIP Levels were only ever present on Bedrock, and required large changes to the core loot library to ensure it fit in well.
Additionally, Skyblock, one of our most technically challenging games to maintain at the time, was still using our legacy game library and needed a migration to GameFramework on top of being updated to 1.19 and integrating with our new systems. Skyblock also relied very heavily on obfuscated NMS code to function.
BlockLightStone? Don’t they know it’s called Glowstone! Silly Bukkit using 2010 names!
Skyblock was by far one of the most technically challenging games to migrate and required a lot of QA to ensure features like generators migrated over properly. The custom Skyblock world format we use needed updating to support newer game features. Migrating systems such as generators and entities away from NMS and into proper APIs added additional overhead too.
And to top it off, these migrations were only the first part of the update. Most games were still awaiting a content update too!
Unfortunately, these delays and the challenges we encountered meant that we would not have a release out before our birthday celebration
whew, okay, let’s fast forward a bit
As we crunched through the migration and started to build new content on top of the all the new systems and APIs available to us, we found that our ability to deliver updates has drastically improved. No longer were we limited by legacy codebases, limited APIs and old versions. As we got towards the end of March, over three months after our initial target, we found ourselves ready to start preparing for a soft launch on Java Edition.
As we did our soft launch on Java Edition in the beginning of April, we were immediately hit with stability issues and memory leaks. While we certainly expected a number of bugs and bumps at the beginning, many of the issues we encountered were only evident at scale. Thankfully, we were able to resolve most of these within just a few days and in the span of a few weeks things seemed to be pretty in a fairly solid state
And that concludes this edition of Behind the Cube! As a team, we learned a whole lot during this migration and truly learned what it was like to push CubeCraft to it’s limits! We appreciate all of you who supported us through the migration and shared feedback, constructive criticism and simply just played on the server!
Looking back on this years later, we see the value this migration has provided for us, whether it be all the modern features in Skyblock, or games like Pillars of Fortune which give you random items from even the latest versions. Until next time
Welcome back to Behind the Cube! We've been on hiatus for a while and been busy cracking out other updates over the last few years, but today I want to step back a couple years to our 1.19 server update. This was our biggest, most ambitious update we've ever performed on the network. This has allowed us to deliver far more feature-complete updates across both networks simultaneously, and completely transformed our developer experience.
This edition will dive deep into the internals our server update to 1.19, some the challenges that we encountered during our migration, and some
spicy changes we’ve made on the tech side that have greatly improved our updating process. This is a juicy one, so grab a bowl of popcorn and let’s dive right in!
The 1.19 1.18 update…?
The As many of you may be aware, we initially communicated our goals of updating the network to 1.19 waaaaay back in September of 2022, however in actuality, work began far earlier, prior to 1.19 even being released. Updating nearly 10 Minecraft versions was no easy feat, and required a lot of background planning and backend work.
For a bit of background…
Previously, our server platform was running on a 1.9.2 Spigot jar we updated to 1.12 and modded Bedrock support into back in 2021. Some of you may recall our Translator system we discussed in our Feelin’ Lucky Behind the Cube which sat on top of our BungeeCords. This system allowed us to support Bedrock Edition users on our existing Java infrastructure.
As we grew throughout 2020 and 2021, it became a major technical challenge for us to maintain as we started implementing native Bedrock features that are now integral to our server, such as resource packs, custom entities and custom items / blocks. We decided to go the route of removing the Translator, and turning our Java server into a full-on Bedrock server.
Moving away from our Translator proved to have a large number of benefits, including reducing player latency by 50ms - 100ms across the board. However, this came with the downside of maintaining a significant amount of code in our server software.
Updating to 1.18…
Rather than going the route of updating our 1.9-but-really-it-was-actually-1.12-server jar to 1.18, we decided to start fresh by forking Paper 1.18. The first commits to our new CubeTap project (internally known as CubeTap Junior) were in March of 2022.
The first few months of work primarily revolved around getting our new CubeTap version up to speed with our 1.12 version. We also took the opportunity to redesign core parts of our Bedrock support, such as moving to Bedrock’s server-authoritative inventory system, implementing chunk caching, and adding proper custom blocks. We also added new APIs to integrate nicely with these systems.
First look at chunks rendering for Bedrock Edition on our new software.
Updating CubeCraft!
Updating CubeCraft!Much of the work to update the CubeCraft codebase began in July of 2022. For those of you who have worked with Spigot or Paper before may know that in 1.13, there were massive, backwards-compatible breaking changes in the Bukkit API. The internal NMS code also saw major changes, going from being heavily obfuscated to fully deobfuscated using Mojang’s code mappings. Additionally, from 1.13 - 1.18, huge amounts of the server code was rewritten.
While most of these changes were positive, essentially most anything using NMS was going to break (and boooyyy did we use a lot!) and most all 1.13+ Bukkit plugins were going to no longer be functional.
After some time, the first project we completed updating was GameFramework, which is one of our core frameworks that all our games are built on top of.
First look at a simple TNT Run game running on a local server using GameFramework.
Getting GameFramework to work was a massive milestone and meant that within a short period of time, we could actually start testing CubeCraft code on 1.19!
Within just a few weeks from this point, we had many of our other core libraries updated, such as Loot, Armoury, Play Again, etc. and by early August, had a version of Survival Games running for Java Edition
As bug free as you get…
At the speed things were going, it looked as if we were going to have 1.19 out by December, just in time for our 10th anniversary! Right? Right..? Well, we’ll see… let’s not get too ahead of ourselves
What did we want out of this update?
What did we want out of this update?As work continued to get all our code updated to 1.19, a question arose: what did we want to get out of this update?
I think the obvious answer everyone arrived to was: offer new features and to build potential for future content. While these are and have always been at the root of any content update at CubeCraft, we were starting to see major technical debt problems arise on our existing network.
At the time, our codebases for both Java and Bedrock were very disjointed. We maintained four separate Git branches for each project:
- production - Contains all code live on our Java Edition server
- production-mco - Contains all code live on our Bedrock Edition server
- master - Contains all code live on our Java Edition development server
- master-mco - Contains all code live on our Bedrock Edition development server
This ultimately meant that if a bug fix was found in team handling for instance that was present on both Java and Bedrock, that would mean a commit fixing the issue would need to be pushed to eight branches (╯°□°)╯︵ ┻━┻
Also at the time, our update process typically involved us writing the code for Java Edition first, testing it, and then merging it into our Bedrock branches, often encountering multiple merge conflicts, then updating or removing any features not relevant to Bedrock, and testing again. This is also the reason that in times past, we often released all major content updates for Java first (or even exclusively in some cases), then later Bedrock. This made any major update incredibly cumbersome and often led to very "Java-first" updates that disregarded features on Bedrock.
Since we were already making major technical changes, we decided now would be the best time to reconsider how we conducted our updates on a technical side.
Modularising the code
Modularising the codeShortly after getting Survival Games functional on our Java Edition development network, @rubik_cube_man, @Alemiz, @Austin and I sat down and discussed plans on how we wanted to approach this problem. We decided that modularising the code was the best approach, creating a common module for all common code, then separate java and bedrock modules for features that vary for each platform. Rather than maintaining 4 branches, we would only need 2: one for production code and one for development code.
We then pitched the idea to the rest of the dev team and decided on a course of action.
Initial slide from our pitch deck in August of 2022
Modularising all our Bedrock and Java code together essentially meant that all updates we did would only ever need to be written once, rather than one time for Java, then tested, then updated in bits and places for Bedrock (or in some cases like MinerWare, substantially), then tested again, then finally released. We could instead streamline the process and do simultaneous testing on both platforms, allowing greater flexibility between updates and the ability to offer much more content. These changes also meant that general bug fixes in the game logic only needed to be written once, rather than individually for each platform.
SkyWars project layout today.
While the results of this modularisation have been incredibly beneficial for us today, it took us a long time to get there. There was a large amount of scope creep, unexpected challenges and many, many delays.
Delays and challenges…
Delays and challenges…As we began to modularise our codebase and integrate everything together, we began to realize that many of our core libraries needed massive rewrites to cooperate well with both Java and Bedrock, or needed to be rewritten from the ground up. Systems such as our menu library, game voting system, loot, and other similar systems needed massive overhauls.
Here is a quick look at how we now create menus on our network now using the UI library we wrote as part of this migration:
The code used to render the Loot home menu in the Bedrock module.
The code used to render the Loot home menu in the Java module.
And in our common module, we can have a common open method, even if it is rendered differently on both platforms
In other instances, we found that simply having a feature switch for the player platform in the common code would be sufficient, as demonstrated with the following code:
This is a method in our floating text library, which instructs the system to use Text Display entities for Java, but Armor Stands for Bedrock, as Text Displays do not exist on Bedrock.
As amazing as these new libraries were from a development perspective, they required significant time investments to design, iterate upon, and eventually adopt across the board. For menus, CubeCraft has over 100 hundred different menus across both Java and Bedrock, whether it be the loot menu, VIP Levels menu, SkyWars features or Parkour map selection, and as you can imagine, migrating every one of them took quite a long time.
Menus were just one part of this - we also ran into various other challenges implementing platform-specific features, such as chest skins into the game. Despite the fact that we have code split up between two modules, all of the actual chest and loot table code is in the same place. We had to modify the core system responsible for this so chest skins could work fine on Bedrock without breaking Java Edition.
Our loot system needed to have significant refactors as well to support loading both Java and Bedrock loot in the same system. Certain features such as VIP Levels were only ever present on Bedrock, and required large changes to the core loot library to ensure it fit in well.
Additionally, Skyblock, one of our most technically challenging games to maintain at the time, was still using our legacy game library and needed a migration to GameFramework on top of being updated to 1.19 and integrating with our new systems. Skyblock also relied very heavily on obfuscated NMS code to function.
BlockLightStone? Don’t they know it’s called Glowstone! Silly Bukkit using 2010 names!
Skyblock was by far one of the most technically challenging games to migrate and required a lot of QA to ensure features like generators migrated over properly. The custom Skyblock world format we use needed updating to support newer game features. Migrating systems such as generators and entities away from NMS and into proper APIs added additional overhead too.
And to top it off, these migrations were only the first part of the update. Most games were still awaiting a content update too!
Unfortunately, these delays and the challenges we encountered meant that we would not have a release out before our birthday celebration
A light at the end of the tunnel
A light at the end of the tunnelwhew, okay, let’s fast forward a bit
As we crunched through the migration and started to build new content on top of the all the new systems and APIs available to us, we found that our ability to deliver updates has drastically improved. No longer were we limited by legacy codebases, limited APIs and old versions. As we got towards the end of March, over three months after our initial target, we found ourselves ready to start preparing for a soft launch on Java Edition.
As we did our soft launch on Java Edition in the beginning of April, we were immediately hit with stability issues and memory leaks. While we certainly expected a number of bugs and bumps at the beginning, many of the issues we encountered were only evident at scale. Thankfully, we were able to resolve most of these within just a few days and in the span of a few weeks things seemed to be pretty in a fairly solid state
And that concludes this edition of Behind the Cube! As a team, we learned a whole lot during this migration and truly learned what it was like to push CubeCraft to it’s limits! We appreciate all of you who supported us through the migration and shared feedback, constructive criticism and simply just played on the server!
Looking back on this years later, we see the value this migration has provided for us, whether it be all the modern features in Skyblock, or games like Pillars of Fortune which give you random items from even the latest versions. Until next time
