Slowride Storage Benchmark Utility

Slowride – utility benchmark utility for block devices for POSIX operation systems with root access to /dev/sd*. You can check blocks read speed in terminal, set threshold for operations to diagnose problems with block device.
Command to perform 100mb blocks read of entire device with 2 seconds threshold:


sudo ./slowride /dev/sda 100 2000

Source Code

https://gitlab.com/demensdeum/slowride

Space Jaguar 3D Action RPG

For a long time I have not announced new projects) The next project I’m starting to work on is a 3D action RPG called Space Jaguar. A story in a sci-fi setting about a tough guy named Jag and his difficult adventure in search of a missing father. There will be 3D graphics on the Flame Steel Engine (or possibly on any other popular), using the achievements of past projects (Death Mask, Cube Art Project), a comedy story with many references, arcade battles and bosses. I’m not ready to talk about release date of the full version, I plan to release the game in parts.

Project Repository:
https://gitlab.com/demensdeum/space-jaguar-action-rpg

Backend server C++ FCGI

Quick note about how I wrote the backend for 3D Cube Art Project editor, the server needs to store and display the user work from web version, giving them a short URL on the save button. At first I wanted to use the Swift/PHP/Ruby/JS or some similar modern language for the backend, but looking at the characteristics of my VPS was decided to write a server on the C/C ++.
First you need to install on the server and libfcgi fcgi support module for your web server, for example Ubuntu and Apache:


sudo apt install libfcgi libapache2-mod-fcgid

Next, configure the module in the config file:

FcgidMaxProcessesPerClass – the maximum number of processes in the class, I put 1 process because it does not count on a big load.
AddHandler fcgid-script .fcgi – file extension which should start fcgi module.
Add in the config directory which will run cgi applications:

Next, write an application to the C/C ++ with fcgi support, collect it, and it is copied to the folder /var/www/html/cgi-bin.
Examples code and build script:
https://gitlab.com/demensdeum/cube-art-project-server/-/blob/master/src/cubeArtProjectServer.cpp
https://gitlab.com/demensdeum/cube-art-project-server/-/blob/master/src/build.sh
Then you need to restart your web server:


systemctl restart apache2

Next, put down the necessary rights for the execution of cgi-bin folder via chmod.
After that, your cgi program should work through the browser to the link, for example Cube Art Project Server:
http://192.243.103.70/cgi-bin/cubeArtProject/cubeArtProjectServer.fcgi
If something does not work, then see the web server logs, or connect the debugger to a running process, the debugging process should not differ from the usual process of debugging the client application.

References

https://habr.com/ru/post/154187/
http://chriswu.me/blog/writing-hello-world-in-fcgi-with-c-plus-plus/

Source Code

https://gitlab.com/demensdeum/cube-art-project-server

Cube Art Project

Cube Art Project – cubes based 3D editor.
You can build, remove cubes, move around scene by WSAD + E controls, use mouse wheel to change cube color. Only 16 colors are supported, but there will be enhancements in future.

Web version
https://demensdeum.com/games/CubeArtProjectWEB/

Windows
https://demensdeum.com/games/CubeArtProjectReleases/CubeArtProjectWin32.zip

macOS
https://demensdeum.com/games/CubeArtProjectReleases/CubeArtProjectMacOS.zip

Linux (x86-64)
https://demensdeum.com/games/CubeArtProjectReleases/CubeArtProjectLinux86_64.zip

Android
(Proof of concept, usb-mouse required)
https://demensdeum.com/games/CubeArtProjectReleases/CubeArtProject.apk

Source Code
https://gitlab.com/demensdeum/cube-art-project-bootstrap
https://gitlab.com/demensdeum/cube-art-project-server

Technologies: SDL, Emscripten, MinGW, Glew, GLM, Cpp-JSON

Porting C++ SDL Application To Android

In this article I will describe his experience porting 3D editor prototype Cube Art Project on Android.
First, look at the result, the emulator running the editor with 3D cursor red cube:

To build a successful have to do the following:

  1. Install the latest Android SDK and NDK (National Palace of Culture version of the fresher the better).
  2. Download the code SDL2, take out a template to build an application for android.
  3. Add SDL Image, SDL Mixer for assembly.
  4. Add the library of my game engine and toolkit, their dependencies (GLM, JSON for Modern C++)
  5. Adapt the assembly files for Gradle.
  6. Adapt the C ++ code to be compatible with Android, changes were platform specific components (OpenGL ES, the graphics context initialization)
  7. Assemble and test the project on the emulator.

Project Template

Load the source code SDL, SDL Image, SDL Mixer:
https://www.libsdl.org/download-2.0.php
The docs folder contains detailed instructions on working with project android template; copy the directory android-project in a separate folder, make a symlink or copy the SDL folder in android-project / app / jni.
Substitute the correct identifier for avd flag, run the emulator from the Android Sdk directory:


cd ~/Android/Sdk/emulator
./emulator -avd Pixel_2_API_24

Specify the path to the script, pick a project:


rm -rf app/build || true
export ANDROID_HOME=/home/demensdeum/Android/Sdk/
export ANDROID_NDK_HOME=/home/demensdeum/Android/android-ndk-r21-beta2/
./gradlew clean build
./gradlew installDebug

Should meet SDL project template with the code in C from a file


android-sdl-test-app/cube-art-project-android/app/jni/src/YourSourceHere.c

Dependencies

Download the source code archives for SDL_image, SDL_mixer:
https://www.libsdl.org/projects/SDL_image/
https://www.libsdl.org/projects/SDL_mixer/

Load depending on your project, for example, my shared library:
https://gitlab.com/demensdeum/FlameSteelCore/
https://gitlab.com/demensdeum/FlameSteelCommonTraits
https://gitlab.com/demensdeum/FlameSteelBattleHorn
https://gitlab.com/demensdeum/FlameSteelEngineGameToolkit/
https://gitlab.com/demensdeum/FlameSteelEngineGameToolkitFSGL
https://gitlab.com/demensdeum/FSGL
https://gitlab.com/demensdeum/cube-art-project

All of this was discharged into app / jni, each “module” in a separate folder, such as app / jni / FSGL. Next, you have the option to find workers generators Application.mk and Android.mk files, I have not found, but maybe there’s a simple solution based on CMake. Click on a link and start to get acquainted with the format of the build files for Android NDK:

https://developer.android.com/ndk/guides/application_mk
https://developer.android.com/ndk/guides/android_mk

You should also read about different APP_STL implementation NDK:
https://developer.android.com/ndk/guides/cpp-support.html

After reviewing create for each “module” file Android.mk, further example of an assembly file shared library Cube-Art-Project:


LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)

APP_STL := c++_static
APP_CPPFLAGS := -fexceptions
LOCAL_MODULE := CubeArtProject
LOCAL_C_INCLUDES := $(LOCAL_PATH)/src $(LOCAL_PATH)/../include $(LOCAL_PATH)/../include/FlameSteelCommonTraits/src/FlameSteelCommonTraits
LOCAL_EXPORT_C_INCLUDES = $(LOCAL_PATH)/src/

define walk
  $(wildcard $(1)) $(foreach e, $(wildcard $(1)/*), $(call walk, $(e)))
endef

ALLFILES = $(call walk, $(LOCAL_PATH)/src)
FILE_LIST := $(filter %.cpp, $(ALLFILES))
$(info CubeArtProject source code files list)
$(info $(FILE_LIST))
LOCAL_SRC_FILES := $(FILE_LIST:$(LOCAL_PATH)/%=%)

LOCAL_SHARED_LIBRARIES += FlameSteelCore
LOCAL_SHARED_LIBRARIES += FlameSteelBattleHorn
LOCAL_SHARED_LIBRARIES += FlameSteelCommonTraits
LOCAL_SHARED_LIBRARIES += FlameSteelEngineGameToolkit
LOCAL_SHARED_LIBRARIES += FlameSteelEngineGameToolkitFSGL
LOCAL_SHARED_LIBRARIES += FSGL
LOCAL_SHARED_LIBRARIES += SDL2
LOCAL_SHARED_LIBRARIES += SDL2_image

LOCAL_LDFLAGS := -static-libstdc++
include $(BUILD_SHARED_LIBRARY)

CMake any experienced user realizes this configuration with the first string formats are very similar in Android.mk no GLOB_RECURSIVE, so we have to recursively search for source files with walk functions.

Change Application.mk, Android.mk with-but to build C++ instead of C code:


APP_ABI := armeabi-v7a arm64-v8a x86 x86_64
APP_PLATFORM=android-16
APP_STL := c++_static
APP_CPPFLAGS := -fexceptions

Rename YourSourceHere.c -> YourSourceHere.cpp, grep for entry, change the path in the assembly, for example:


app/jni/src/Android.mk:LOCAL_SRC_FILES := YourSourceHere.cpp

Next, try to build the project, if you see an error from the compiler about the absence of Heather, check the correctness of the ways in Android.mk; if mistakes are kind of linker “undefined reference”, check the correctness of indications of source code files in assemblies ottreysit lists possible by specifying $ (info $ (FILE_LIST)) in Android.mk file. Do not forget the double-linking mechanism, with modules in LOCAL_SHARED_LIBRARIES key and correct linking through the LD, for example FSGL:


LOCAL_LDLIBS := -lEGL -lGLESv2

Adaptation And Launching

I had to change some things, for example to remove GLEW of assemblies for iOS and Android, to rename part of OpenGL calls, adding a suffix EOS (glGenVertexArrays -> glGenVertexArraysOES), include macro missing modernistic features debug, the cherry on the cake is the implicit include vulnerability GLES2 Heather indicating macro GL_GLEXT_PROTOTYPES 1 :


#define GL_GLEXT_PROTOTYPES 1
#include "SDL_opengles2.h"

Also, a black screen in the first starts with an error type “E / libEGL: validate_display: 255 error 3008 (EGL_BAD_DISPLAY)”, has changed initialize SDL window, GL profile initialization and it worked:


SDL_DisplayMode mode;
SDL_GetDisplayMode(0,0,&mode);
int width = mode.w;
int height = mode.h;

window = SDL_CreateWindow(
            title,
            0,
            0,
            width,
            height,
            SDL_WINDOW_OPENGL | SDL_WINDOW_FULLSCREEN | SDL_WINDOW_RESIZABLE
        );

SDL_GL_SetAttribute( SDL_GL_CONTEXT_PROFILE_MASK, SDL_GL_CONTEXT_PROFILE_ES );

On the default emulator application is installed with the “Game” icon SDL and name.

I needed to explore the possibility of automatically generating assembly files based on CMake, or else migrate assembly for all platforms on Gradle; CMake but remains the de facto choice for the current development in C++.

Source Code

https://gitlab.com/demensdeum/android-sdl-test-app
https://gitlab.com/demensdeum/android-sdl-test-app/tree/master/cube-art-project-android

Sources

https://developer.android.com/ndk/guides/cpp-support.html
https://developer.android.com/ndk/guides/application_mk
https://developer.android.com/ndk/guides/android_mk
https://lazyfoo.net/tutorials/SDL/52_hello_mobile/android_windows/index.php
https://medium.com/androiddevelopers/getting-started-with-c-and-android-native-activities-2213b402ffff

LazyFoo Productions Website

Perhaps it is worth mentioning the website from which almost always start my projects and new development, it LazyFoo Productions website where you can find answers to the enough hardcore topics: Examples of the use of sophisticated the API, learn how to combine the seemingly incompatible systems (Android / C ++) with a detailed an explanation of the principles of work, working code examples.

https://lazyfoo.net/

Flipped World

To develop a new project Cube Art Project has adopted a methodology for the development of Test Driven Development. In this approach, implemented first test for a particular functional application, and then implemented this functionality. A great advantage in this approach, I believe the final implementation of the interfaces, the most uninitiated in the details of implementation, prior to the development of the functional. With this approach, the test dictates the further implementation, added to all the advantages of contract programming when interfaces are contracts for the implementation.

Cube Art Project – 3D editor in which the user builds the shape of cubes, not so long ago, this genre was very popular. Since this graphic application, I decided to add tests to the validation of screenshots.

For screenshot validation you need to get them from OpenGL context first, it is done with the help of glReadPixels function. Description of function arguments are simple – the starting position, width, height, format (. RGB / RGBA / etc.), a pointer to the output buffer, anyone working with SDL or having experience with data buffers in C will simply substitute the correct arguments. However consider it necessary to describe an interesting feature of the output buffer glReadPixels, the pixels stored therein upwards and in SDL_Surface all basic operations taking place downwards.

That is, the reference by uploading a screenshot from the png file, I could not compare the two buffers in the forehead, as one of them upside down.

To flip the output buffer of OpenGL you need to fill it up taking away the height of the screenshot to coordinate Y. However, cost to take into account that there is a chance to go beyond the buffer limits, if not take the unit to fill the time, which will lead to memory corruption.

Since I’m all over the place trying to use OOP paradigm “programming interfaces”, instead of a direct C-like memory access at the sign, when you try to write data outside of the buffer object I reported it to the method of borders through validation.

Final code method of obtaining screenshots in the style of top-down:


    auto width = params->width;
    auto height = params->height;

    auto colorComponentsCount = 3;
    GLubyte *bytes = (GLubyte *)malloc(colorComponentsCount * width * height);
    glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, bytes);

    auto screenshot = make_shared(width, height);

    for (auto y = 0; y(redColorByte, greenColorByte, blueColorByte, 255);
            screenshot->setColorAtXY(color, x, height - y - 1);
        }
    }

    free(bytes);

Sources

https://community.khronos.org/t/glreadpixels-fliped-image/26561
https://stackoverflow.com/questions/8346115/why-are-bmps-stored-upside-down

Source Code

https://gitlab.com/demensdeum/cube-art-project-bootstrap

Longest Common Substring

In this article I will describe an algorithm for solving the longest common substring. Suppose we try to decipher the encrypted binary data, first try to find the common patterns by searching the largest substring.
The input string for example: adasDATAHEADER??jpjjwerthhkjbcvkDATAHEADER??kkasdf
We are looking for a line repeated twice: DATAHEADER??

Prefixes

To begin write method for comparing prefixes of two rows, let returns the resulting string in which a left prefix symbols are the symbols of the right prefix.
For example, for strings:


        val lhs = "asdfWUKI"
        val rhs = "asdfIKUW"

The resulting string – asdf
Example of Kotlin:


fun longestPrefix(lhs: String, rhs: String): String {
        val maximalLength = min(lhs.length-1, rhs.length -1)
        for (i in 0..maximalLength) {
            val xChar = lhs.take(i)
            val yChar = rhs.take(i)
                if (xChar != yChar) {
                    return lhs.substring(0, i-1)
                }
        }
        return lhs.substring(0,maximalLength)
}

Brute Force

When it is impossible for a good, should resort to brute force. Using the method longestPrefix pass on row two cycles, the first line takes from i to the end of the second i + 1 to the end, transmits them to the search for the largest prefix. The time complexity of the algorithm is approximately equal to O (n ^ 2) ~ O (n * ^ 3).
Example of Kotlin:


fun searchLongestRepeatedSubstring(searchString: String): String {
        var longestRepeatedSubstring = ""
        for (x in 0..searchString.length-1) {
            val lhs = searchString.substring(x)
            for (y in x+1..searchString.length-1) {
                val rhs = searchString.substring(y)
                val longestPrefix = longestPrefix(lhs, rhs)
                if (longestRepeatedSubstring.length < longestPrefix.length) {
                    longestRepeatedSubstring = longestPrefix
                }
            }
        }
        return longestRepeatedSubstring
}

Suffix array

For a more elegant solution, we need a tool - a data structure called "suffix array." This data structure is an array of substrings filled in a loop, where every substring starts the next character string to the end.
For example for a row:


adasDATAHEADER??

Suffix array looks like this:


adasDATAHEADER??
dasDATAHEADER??
asDATAHEADER??
sDATAHEADER??
DATAHEADER??
ATAHEADER??
TAHEADER??
AHEADER??
HEADER??
EADER??
ADER??
DER??
ER??
R??
??
?

Solve sorting

Sort the suffix array, and then go through all the elements of the cycle where the left hand (lhs) the current item on the right (rhs) and calculate the next longest prefix using longestPrefix method.
Example of Kotlin:


fun searchLongestRepeatedSubstring(searchString: String): String {
    val suffixTree = suffixArray(searchString)
    val sortedSuffixTree = suffixTree.sorted()

    var longestRepeatedSubstring = ""
    for (i in 0..sortedSuffixTree.count() - 2) {
        val lhs = sortedSuffixTree[i]
        val rhs = sortedSuffixTree[i+1]
        val longestPrefix = longestPrefix(lhs, rhs)
        if (longestRepeatedSubstring.length < longestPrefix.length) {
            longestRepeatedSubstring = longestPrefix
        }
    }
    return longestRepeatedSubstring
}

Time complexity O (N log N), which is much better than brute force algorithm.

References

https://en.wikipedia.org/wiki/Longest_common_substring_problem

Source Code

https://gitlab.com/demensdeum/algorithms