Porting Surreal Engine C++ to WebAssembly

In this post I will describe how I ported the Surreal Engine game engine to WebAssembly.

Surreal Engine is a game engine that implements most of the functionality of the Unreal Engine 1 engine, famous games on this engine are Unreal Tournament 99, Unreal, Deus Ex, Undying. It belongs to the classic engines that worked mainly in a single-threaded execution environment.

My initial idea was to take on a project that I couldn’t complete in any reasonable amount of time, thus showing my Twitch followers that there are projects that even I can’t do. On my very first stream, I suddenly realized that the task of porting Surreal Engine C++ to WebAssembly using Emscripten was doable.

Surreal Engine Emscripten Demo

A month later I can show my fork and assembly of the engine on WebAssembly:
https://demensdeum.com/demos/SurrealEngine/

The controls are the same as in the original, using the keyboard arrows. Next, I plan to adapt it to mobile controls (touch), add correct lighting and other graphic features of the Unreal Tournament 99 render.

Where to start?

The first thing I want to say is that any project can be ported from C++ to WebAssembly using Emscripten, the only question is how complete the functionality will be. Choose a project whose library ports are already available for Emscripten, in the case of Surreal Engine, you are very lucky, because the engine uses the SDL 2, OpenAL libraries – they are both ported to Emscripten. However, Vulkan is used as a graphics API, which is currently not available for HTML5, work is underway to implement WebGPU, but it is also in the draft stage, and it is also unknown how simple the further port from Vulkan to WebGPU will be, after its full standardization. Therefore, I had to write my own basic OpenGL-ES / WebGL renderer for Surreal Engine.

Building the project

The build system in Surreal Engine is CMake, which also simplifies porting, since Emscripten provides its own native builders – emcmake, emmake.
The Surreal Engine port was based on the code of my last game on WebGL/OpenGL ES and C++ called Death-Mask, because of this the development went much easier, all the necessary build flags were with me, code examples.

One of the most important points in CMakeLists.txt is the build flags for Emscripten, below is an example from the project file:


-s MAX_WEBGL_VERSION=2 \

-s EXCEPTION_DEBUG \

-fexceptions \

--preload-file UnrealTournament/ \

--preload-file SurrealEngine.pk3 \

--bind \

--use-preload-plugins \

-Wall \

-Wextra \

-Werror=return-type \

-s USE_SDL=2 \

-s ASSERTIONS=1 \

-w \

-g4 \

-s DISABLE_EXCEPTION_CATCHING=0 \

-O3 \

--no-heap-copy \

-s ALLOW_MEMORY_GROWTH=1 \

-s EXIT_RUNTIME=1")

The build script itself:


emmake make -j 16

cp SurrealEngine.data /srv/http/SurrealEngine/SurrealEngine.data

cp SurrealEngine.js /srv/http/SurrealEngine/SurrealEngine.js

cp SurrealEngine.wasm /srv/http/SurrealEngine/SurrealEngine.wasm

cp ../buildScripts/Emscripten/index.html /srv/http/SurrealEngine/index.html

cp ../buildScripts/Emscripten/background.png /srv/http/SurrealEngine/background.png

Next, we’ll prepare index.html, which includes the project file system preloader. For posting on the web, I used Unreal Tournament Demo version 338. As you can see from the CMake file, the unpacked game folder was added to the build directory and linked as a preload-file for Emscripten.

Major code changes

Then it was necessary to change the game loop, you can’t run an infinite loop, it leads to the browser freezing, instead you need to use emscripten_set_main_loop, I wrote about this feature in my 2017 note “Porting SDL C++ game to HTML5 (Emscripten)
We change the code for the condition for exiting the while loop to if, then we output the main class of the game engine, which contains the game loop, to the global scope, and write a global function that will call the step of the game loop from the global object:


#include <emscripten.h>

Engine *EMSCRIPTEN_GLOBAL_GAME_ENGINE = nullptr;

void emscripten_game_loop_step() {

	EMSCRIPTEN_GLOBAL_GAME_ENGINE->Run();

}

#endif

After this, you need to make sure that there are no background threads in the application. If there are, then get ready to rewrite them for single-thread execution, or use the phtread library in Emscripten.
The background thread in Surreal Engine is used to play music, the main thread of the engine receives data about the current track, about the need to play music, or its absence, then the background thread via mutex receives a new state and starts playing new music, or pauses. The background thread is also used to buffer music during playback.
My attempts to build Surreal Engine under Emscripten with pthread were unsuccessful, because the SDL2 and OpenAL ports were built without pthread support, and I did not want to rebuild them for the sake of music. Therefore, I moved the background music thread functionality to single-thread execution using a loop. Having removed pthread calls from the C++ code, I moved buffering, music playback to the main thread, so that there were no delays, I increased the buffer by several seconds.

Next I will describe specific implementations of graphics and sound.

Vulkan is not supported!

Yes, Vulkan is not supported in HTML5, although all the advertising brochures point out cross-platform and wide support on platforms as the main advantage of Vulkan. For this reason, I had to write my own basic graphics renderer for a simplified type of OpenGL – ES, it is used on mobile devices, sometimes does not contain fashionable features of modern OpenGL, but it is very well transferred to WebGL, this is what Emscripten implements. Writing a basic tile renderer, bsp rendering, for the simplest display of GUI, and drawing models + maps, was possible in two weeks. This was probably the most difficult part of the project. There is still a lot of work ahead to implement the full functionality of the Surreal Engine rendering, so any help from readers in the form of code and pull requests is welcome.

OpenAL is supported!

It was a great stroke of luck that Surreal Engine uses OpenAL for audio output. After writing a simple hello world in OpenAL and building it in WebAssembly using Emscripten, it became clear to me how simple it all is, and I set out to port the audio.
After several hours of debugging, it became obvious that there are several bugs in the OpenAL implementation of Emscripten, for example, when initializing the reading of the number of mono channels, the method returned an infinite number, and after trying to initialize a vector of infinite size, C++ crashes with the exception vector::length_error.
This was circumvented by hardcoding the number of mono channels to 2048:


		alcGetIntegerv(alDevice, ALC_STEREO_SOURCES, 1, &stereoSources);



#if __EMSCRIPTEN__

		monoSources = 2048; // for some reason Emscripten's OpenAL gives infinite monoSources count, bug?

#endif



Is there a network?

Surreal Engine currently does not support network play, play with bots is supported, but someone is needed to write AI for these bots. Theoretically, it is possible to implement network play on WebAssembly/Emscripten using Websockets.

Conclusion

In conclusion, I would like to say that porting Surreal Engine turned out to be quite smooth due to the use of libraries for which there are Emscripten ports, as well as my previous experience implementing a game in C++ for WebAssembly on Emscripten. Below are links to sources of knowledge, repositories on the topic.
M-M-M-MONSTER KILL!

Also, if you want to help the project, preferably with WebGL/OpenGL ES render code, then write to me in Telegram:
https://t.me/demenscave

Links

https://demensdeum.com/demos/SurrealEngine/
https://github.com/demensdeum/SurrealEngine-Emscripten

https://github.com/dpjudas/SurrealEngine

Flash Forever – Interceptor 2021

Recently, it turned out that Adobe Flash works quite stably under Wine. During a 4-hour stream, I made the game Interceptor 2021, which is a sequel to the game Interceptor 2020, written for the ZX Spectrum.

For those who are not in the know – the Flash technology provided interactivity on the web from 2000 to around 2015. Its shutdown was prompted by an open letter from Steve Jobs, in which he wrote that Flash should be consigned to history because it lagged on the iPhone. Since then, JS has become even more sluggish than Flash, and Flash itself has been wrapped in JS, making it possible to run it on anything thanks to the Ruffle player.

You can play it here:
https://demensdeum.com/demos/Interceptor2021

Video:
https://www.youtube.com/watch?v=-3b5PkBvHQk

Source code:
https://github.com/demensdeum/Interceptor-2021

Masons-DR Demo Games

Masonry-AR is an augmented reality game where you need to move around the city in the real world and collect Masonic knowledge from books, earning currency and capturing territory for your Masonic order. The game has no relation to any real organizations, all coincidences are random.

Demo games:
https://demensdeum.com/demos/masonry-ar/client

Wiki:
https://demensdeum.com/masonry-ar-wiki-ru/

Source code:
https://github.com/demensdeum/Masonry-AR

Flame Steel Engine Runner

I present to your attention Flame Steel Engine Runner – a platform for launching multimedia applications based on the Flame Steel Engine toolkit. The supported platforms are Windows, MacOS, Linux, Android, iOS, HTML 5. The emphasis of application code development has shifted towards scripting – at the moment, JavaScript support has been added using TinyJS, the toolkit and engine itself will continue to be developed in languages ​​close to hardware (C, C++, Rust, etc.)
Flame Steel Engine Runner Demo
On the page below you can spin the cube, write JavaScript code, upload models, sounds, music, code using the Upload files button, and start from the main.js file using the Run button.
https://demensdeum.com/demos/FlameSteelEngineRunner/

Space Jaguar Action RPG 0.0.4

First prototype of Space Jaguar Action RPG for Webassembly:

It will take a long time to load (53 MB) without loading indication.

Space Jaguar Action RPG – a space pirate life simulator. At the moment, the simplest game mechanics are available:

  • fly in space
  • to die
  • eat
  • sleep
  • hire a team
  • look at the restless, fast-flying flow of time

Due to poor optimization of 3D scene rendering in the web version, it is not possible to walk in a 3D environment. Optimization will be added in future versions.

The source code of the engine, game, scripts is available under the MIT license. All suggestions for improvement are accepted extremely positively:

https://gitlab.com/demensdeum/space-jaguar -action-rpg

Screenshots from the native Linux version:

ZX Spectrum Game Development in C

This is a silly post about developing a game for an old ZX Spectrum computer in C. Let’s take a look at the beauty:

It began production in 1982 and was produced until 1992. Technical characteristics of the machine: 8-bit Z80 processor, 16-128 KB of memory and other extensions, such as the AY-3-8910 sound chip.

As part of the Yandex Retro Games Battle 2019 competition, I wrote a game called Interceptor 2020 for this machine. Since I didn’t have time to learn assembler for the Z80, I decided to develop it in C. As a toolchain, I chose the ready-made set – z88dk, which contains C compilers and many auxiliary libraries to speed up the implementation of applications for the Spectrum. It also supports many other Z80 machines, such as MSX, Texas Instruments calculators.

Next I will describe my superficial flight over the computer architecture, the z88dk toolchain, and show how I managed to implement the OOP approach and use design patterns.

Installation Features

Installation of z88dk should be carried out according to the manual from the repository, however for Ubuntu users I would like to note a feature – if you already have compilers for Z80 installed from deb packages, then you should remove them, since z88dk will access them from the bin folder by default, due to incompatibility of the toolchain compiler versions you most likely will not be able to build anything.

Hello World

Writing Hello World is very simple:

#include 

void main()
{
    printf("Hello World");
}

It’s even easier to assemble a tap file:

zcc +zx -lndos -create-app -o helloworld helloworld.c

To run, use any ZX Spectrum emulator with tap file support, for example online:
http://jsspeccy.zxdemo.org/

Draw on the picture on the whole screen

tl;dr Pictures are drawn in tiles, tiles of size 8×8 pixels, the tiles themselves are embedded in the spectrum font, then the picture is printed in a line of indices.

The sprite and tile output library sp1 outputs tiles using UDG. The image is translated into a set of individual UDGs (tiles), then assembled on the screen using indices. It should be remembered that UDG is used to output text, and if your image contains a very large set of tiles (for example, more than 128 tiles), then you will have to go beyond the set boundaries and erase the default Spectrum font. To get around this limitation, I used a base of 128 – 255 by simplifying the images, leaving the original font in place. About simplifying the images below.

To draw full-screen images, you need to arm yourself with three utilities:
Gimp
img2spec
png2c-z88dk

There is a way for real ZX men, real retro warriors: open a graphic editor using the spectrum palette, knowing the features of image output, prepare it manually and upload it using png2c-z88dk or png2scr.

The simpler way is to take a 32-bit image, switch the number of colors to 3-4 in Gimp, edit it a little, then import it into img2spec so as not to work with color restrictions manually, export png and convert it to a C array using png2c-z88dk.

Please remember that for successful export each tile cannot contain more than two colors.

As a result, you will receive an h file that contains the number of unique tiles. If there are more than ~128, then simplify the image in Gimp (increase the repeatability) and perform the export procedure again.

After exporting, you literally load the “font” from the tiles and print the “text” from the tile indices on the screen. Here is an example from the render “class”:

// грузим шрифт в память
    unsigned char *pt = fullscreenImage->tiles;

    for (i = 0; i < fullscreenImage->tilesLength; i++, pt += 8) {
            sp1_TileEntry(fullscreenImage->tilesBase + i, pt);
    }

    // ставим курсор в 0,0
    sp1_SetPrintPos(&ps0, 0, 0);

    // печатаем строку
    sp1_PrintString(&ps0, fullscreenImage->ptiles);

Drawing sprites on the screen

Next I will describe the method of drawing 16-16 pixel sprites on the screen. I did not get to animation and color changes, because at this stage, as I suppose, I simply ran out of memory. Therefore, the game contains only transparent monochrome sprites.

Draw a monochrome png image 16×16 in Gimp, then use png2sp1sprite to translate it into an asm assembler file, declare arrays from the assembler file in C code, and add the file at the build stage.

After the stage of declaring the sprite resource, it must be added to the screen in the desired position, here is an example of the code for the “class” of the game object:

    struct sp1_ss *bubble_sprite = sp1_CreateSpr(SP1_DRAW_MASK2LB, SP1_TYPE_2BYTE, 3, 0, 0);
    sp1_AddColSpr(bubble_sprite, SP1_DRAW_MASK2,    SP1_TYPE_2BYTE, col2-col1, 0);
    sp1_AddColSpr(bubble_sprite, SP1_DRAW_MASK2RB,  SP1_TYPE_2BYTE, 0, 0);
    sp1_IterateSprChar(bubble_sprite, initialiseColour);

From the names of the functions you can roughly understand the meaning – we allocate memory for the sprite, add two columns 8-8, add color for the sprite.

In each frame the position of the sprite is set:

sp1_MoveSprPix(gameObject->gameObjectSprite, Renderer_fullScreenRect, gameObject->sprite_col, gameObject->x, gameObject->y);

Emulating OOP

There is no syntax for OOP in C, what to do if you still really want it? You need to connect your mind and be enlightened by the thought that such a thing as OOP equipment does not exist, everything ultimately comes to one of the machine architectures, in which there is simply no concept of an object and other related abstractions.

This fact for a long time prevented me from understanding why OOP is needed at all, why it should be used if in the end everything comes to machine code.

However, having worked in product development, I discovered the charms of this programming paradigm, mainly, of course, the flexibility of development, protective mechanisms of the code, with the right approach, reducing entropy, simplifying work in a team. All the listed advantages follow from the three pillars – polymorphism, encapsulation, inheritance.

It is also worth noting the simplification of solving issues related to application architecture, because 80% of architectural problems were solved by computer scientists in the last century and described in the literature on design patterns. Next, I will describe ways to add OOP-like syntax to C.

It is more convenient to use C structures as a basis for storing class instance data. Of course, you can use a byte buffer, create your own structure for classes, methods, but why reinvent the wheel? After all, we are already reinventing the syntax.

Class data

Example of GameObject “class” data fields:

struct GameObjectStruct {
    struct sp1_ss *gameObjectSprite;
    unsigned char *sprite_col;
    unsigned char x;
    unsigned char y;
    unsigned char referenceCount;
    unsigned char beforeHideX;
    unsigned char beforeHideY;
};
typedef struct GameObjectStruct GameObject;

We save our class as “GameObject.h”, #include “GameObject.h” in the right place and use it.

Class Methods

Let’s take the experience of Objective-C language developers into account, the class method signature will be functions in the global scope, the first argument will always be the data structure, then the method arguments. Below is an example of a “method” of the “class” GameObject:

void GameObject_hide(GameObject *gameObject) {
    gameObject->beforeHideX = gameObject->x;
    gameObject->beforeHideY = gameObject->y;
    gameObject->y = 200;
}

The method call looks like this:

GameObject_hide(gameObject);

Constructors and destructors are implemented in the same way. It is possible to implement a constructor as an allocator and field initializer, but I prefer to control object allocation and initialization separately.

Working with memory

Manual memory management of the form using malloc and free wrapped in new and delete macros to match C++:

#define new(X) (X*)malloc(sizeof(X))
#define delete(X) free(X)

For objects that are used by several classes at once, semi-manual memory management is implemented based on reference counting, in the image and likeness of the old Objective-C Runtime ARC mechanism:

void GameObject_retain(GameObject *gameObject) {
    gameObject->referenceCount++;
}

void GameObject_release(GameObject *gameObject) {
    gameObject->referenceCount--;

    if (gameObject->referenceCount < 1) { sp1_MoveSprAbs(gameObject->gameObjectSprite, &Renderer_fullScreenRect, NULL, 0, 34, 0, 0);
        sp1_DeleteSpr(gameObject->gameObjectSprite);
        delete(gameObject);
    }
}

Thus, each class must declare the use of a shared object with retain, and release ownership with release. Modern ARC uses automatic placement of retain/release calls.

We’re making noise!

The Spectrum has a tweeter capable of playing 1-bit music; composers of that time were able to play up to 4 sound channels on it simultaneously.

Spectrum 128k contains a separate sound chip AY-3-8910, which can play tracker music.

A library is provided for using the buzzer in z88dk

What is yet to be discovered

I was interested in getting acquainted with the Spectrum, implementing the game using z88dk, learning many interesting things. I still have a lot to learn, for example, the Z80 assembler, since it allows you to use the full power of the Spectrum, working with memory banks, working with the AY-3-8910 sound chip. I hope to participate in the contest next year!

Links

https://rgb.yandex
https://vk.com/sinc_lair
https://www.z88dk.org/forum/

Source code

https://gitlab.com/demensdeum/ zx-projects/tree/master/interceptor2020

Death Mask Wild Beta

Death-Mask game enters public beta (wild beta)
The game’s main menu screen has been redesigned, a view of the blue zone of the techno-labyrinth has been added, with pleasant music in the background.

Next, I plan to rework the gameplay controller, add smooth movement like in old shooters, high-quality 3D models of boxes, weapons, enemies, the ability to move to other levels of the techno-labyrinth not only through portals (elevators, doors, falling through holes in the floor, holes in the walls), I will add a little variety to the environment of the generated labyrinth. I will also work on the game balance.
Skeletal animation will be added during the polishing phase before release.