Techie – Demens Deum

Writing stuff in Assembly for Sega Genesis #1

The first article dedicated to writing games for the classic Sega Genesis console in Motorola 68000 Assembler.

Let’s write the simplest endless loop for Sega. For this we need: an assembler, an emulator with a disassembler, a favorite text editor, a basic understanding of the structure of the Sega rom.

For development, I use my own Gen68KryBaby assembler/disassembler:

https://gitlab.com/demensdeum/gen68krybaby/

The tool is developed in Python 3, for assembly, a file with the extension .asm or .gen68KryBabyDisasm is supplied to the input, the output is a file with the extension .gen68KryBabyAsm.bin, which can be run in the emulator or on a real console (carefully, step away, the console may explode!)

Roms disassembling is also supported, for this you need to put a rom file to the input, without the .asm or .gen68KryBabyDisasm extensions. Opcode support will increase or decrease depending on my interest in the topic, the participation of contributors.

Structure

The Sega rom header occupies the first 512 bytes. It contains information about the game, name, supported peripherals, check sum, and other system flags. I suppose that without a title, the prefix will not even look at the rom, thinking that it is incorrect, like “what are you giving me here?”

After the header comes the Reset subroutine, from which the m68K processor starts its work. Well, it’s just a small matter – to find the opcodes, namely, the execution of nothing (!) And the transition to the subroutine at the address in memory. Googling, you can find the NOP opcode that does nothing and the JSR opcode that performs an unconditional jump to the argument address, that is, it just moves the carriage to where we ask for it, without any whims.

Putting It All Together

One of the games in the Beta version was the donor of the title for the rom, at the moment it is recorded in the form of hex data.

 ROM HEADER:

 00 ff 2b 52 00 00 02 00 00 00 49 90 00 00 49 90 00 00 49 90 00 ... etc.

The program code so-but is a declaration of the Reset / EntryPoint subroutine in 512 (0x200) bytes, NOP, carriage return to 0x00000200, so we get an infinite loop.

Assembly code of Subroutine Reset / EntryPoint:

 SUBROUTINE_EntryPoint:
    NOP
    NOP
    NOP
    NOP
    NOP
    JSR 0x00000200

Complete example along with rom title:

https://gitlab.com/demensdeum/segagenesissamples/-/blob/main/1InfiniteLoop/1infiniteloop.asm

Next, assembly:

 python3 gen68krybaby.py 1infiniteloop.asm

Run rom 1infiniteloop.asm.gen68KryBabyAsm.bin in debugger mode of Exodus / Gens emulator, see that m68K correctly reads NOP, and endlessly jumps to EntryPoint at 0x200 on JSR

Sonic should be showing V here, but he left for Wacken .

Links

https://gitlab.com/demensdeum/gen68krybaby/

https://gitlab.com/demensdeum/segagenesissamples

https://www.exodusemulator.com/downloads/release-archive

Sources

ROM Hacking Demo – Genesis and SNES games in 480i < / p>

http://68k.hax.com/

https://www.chibiakumas.com/68000/genesis.php

https://plutiedev.com/rom-header

https: //blog.bigevilcorporation.co.uk/2012/02/28/sega-megadrive-1-getting-started/

https : //opensource.apple.com/source/cctools/cctools-836/as/m68k-opcode.h.auto.html

How I didn’t hit the guy on the pole or the story of amazing ingenuity

In this post I will write about the importance of architectural decisions in development, application support, in a team development environment.

Self-Operating Napkin – Rube Goldberg

During my youth, I worked on an app for ordering a taxi. In the program, you could choose a pickup point, a drop point, calculate the cost of the trip, the type of tariff, and, in fact, order a taxi. I got the application at the last stage of the pre-launch, after adding several fixes, the application was released in the AppStore. Already at that stage, the whole team understood that it was implemented very poorly, design patterns were not used, all components of the system were tightly connected, in general, it could be written into one large continuous class (God object), nothing would have changed, so how classes mixed their boundaries of responsibility and, in their mass, overlapped each other in a dead cohesion. Later, the management decided to write the application from scratch, using the correct architecture, which was done and the final product was implemented by several dozen B2B clients.

However, I will describe a curious incident from past architecture, from which I sometimes wake up in a cold sweat in the middle of the night, or suddenly remember in the middle of the day and start laughing hysterically. The thing is that I could not hit the guy on the pole the first time, and this brought down most of the application, but first things first.

It was an ordinary working day, one of the customers received a task to slightly modify the application design – it is banal to move a few pixels up the icon in the center of the screen on the pickup address selection screen. Well, having professionally evaluated the task in 10 minutes, I raised the icon 20 pixels up, suspecting nothing at all, I decided to check the taxi order.

What? Does the app no longer show the order button? How did it happen?

I could not believe my eyes, after raising the icon by 20 pixels, the application stopped showing the continue ordering button. Rolling back the change, I saw the button again. Something was wrong here. After spending 20 minutes in the debugger, I got a little tired of unwinding spaghetti from calls to overlapping classes, but I found that * moving the picture really changes the logic of the application *

It was all about the icon in the center – the man on the pole, when he moved the map, he jumped to animate the camera movement, this animation was followed by the disappearance of the button below. Apparently the program thought that the man shifted by 20 pixels was in a jump, so, according to internal logic, it hid the confirmation button.

How can this happen? Does * the state * of the screen depend not on the pattern of the state machine, but on the * representation * of the guy’s position on the pole?

Everything turned out to be so, every time the map was drawn, the application * visually pick * in the middle of the screen and checked what was there, if there was a man on a pole, then this means that the animation of the map shift was over and the button had to be shown. In the case when the man is not there, it means that the map is shifting, and the button must be hidden.

In the example above, everything is fine, firstly, this is an example of the Goldberg Machine (abstruse machines), secondly, an example of the developer’s unwillingness to somehow interact with other developers in the team (try to figure it out without me), thirdly, you can list all the problems by SOLID, patterns (code smell), MVC violation, and more.

Try not to do this, develop in all possible directions, help your colleagues in their work. Happy new year everyone)

Links

https://en.wikipedia.org/wiki/Rube_Goldberg_machine

https://en.wikipedia.org/wiki/SOLID

https://refactoring.guru/refactoring/smells

https://en.wikipedia.org/wiki/Model-view-controller

https://refactoring.guru/design-patterns/state

Guess Band

In this post I will describe how to work with the fasttext text classifier.

Fasttext is a machine learning library for text classification. Let’s try to teach her to identify a metal band by the name of the song. For this, we use supervised learning using a dataset.

Let’s create a dataset of songs with group names:

__label__metallica the house jack built
__label__metallica fuel
__label__metallica escape
__label__black_sabbath gypsy
__label__black_sabbath snowblind
__label__black_sabbath am i going insane
__label__anthrax anthrax
__label__anthrax i'm alive
__label__anthrax antisocial
[etc.]

Training sample format:

{__label__class} {example from class}

Let’s train fasttext and save the model:

model = fasttext.train_supervised ("train.txt")
model.save_model ("model.bin")

Let’s load the trained model and ask to identify the group by the song name:

model = fasttext.load_model ("model.bin")
predictResult = model.predict ("Bleed")
print (predictResult)

As a result, we will get a list of classes that this example looks like, indicating the level of similarity by a number, in our case, the similarity of the Bleed song name to one of the dataset groups.
In order for the fasttext model to be able to work with a dataset that goes beyond the boundaries of the training sample, the autotune mode is used using a validation file (test file). During autotune, fasttext selects the optimal model hyperparameters, validating the result on a sample from the test file. The autotune time is limited by the user independently by passing the autotuneDuration argument.
An example of creating a model using a test file:

model = fasttext.train_supervised ("train.txt", autotuneValidationFile = "test.txt", autotuneDuration = 10000)

Sources

https://fasttext.cc
https://gosha20777.github.io / tutorial / 2018/04/12 / fasttext-for-windows

Source code

https://gitlab.com/demensdeum/ MachineLearning / – / tree / master / 6bandClassifier

x86_64 Assembler + C = One Love

In this article I will describe the process of calling C functions from assembler.
Let’s try to call printf ( “Hello World \ n!”); and exit (0);

section .rodata
    message: db "Hello, world!", 10, 0

section .text
    extern printf
    extern exit
    global main

main:
    xor	rax, rax
    mov	rdi, message    
    call printf
    xor rdi, rdi
    call exit

Everything is much simpler than it seems, in the section .rodata we describe the static data, in this case the string “Hello, world!”, 10 it is a newline character, and will not forget it annihilate the.

The section of code declare outside of the printf function, exit libraries, stdio, stdlib, also declare main entry function:

section .text
    extern printf
    extern exit
    global main

In the case of the return function rax pass 0, can be used mov rax, 0; but to accelerate the use xor rax, rax; Further, in the first argument is a pointer to a string:

rdi, message

Next call external C functions printf:

main:
    xor	rax, rax
    mov	rdi, message    
    call printf
    xor rdi, rdi
    call exit

By analogy, transfer case 0 in the first argument and calling exit:

    xor rdi, rdi
    call exit

As the Elves say:
Who does not listen
He eats plov @Alexander Pelevin

References

https://www.devdungeon.com/content/how-mix-c-and-assembly
https://nekosecurity.com/x86-64-assembly/part-3-nasm-anatomy-syscall-passing-argument
https://www.cs.uaf.edu/2017/fall/cs301/reference/x86_64.html

Source Code

https://gitlab.com/demensdeum/assembly-playground

Hello World x86_64 Assembly

In this article I will describe the IDE configuration process, writing the first Hello World assembler x86_64 for Ubuntu Linux operating system.
Let’s start with IDE SASM plant assembler nasm:

sudo apt install sasm nasm

Next, invoke SASM and write Hello World:

global main

section .text

main:
    mov rbp, rsp      ; for correct debugging
    mov rax, 1        ; write(
    mov rdi, 1        ;   STDOUT_FILENO,
    mov rsi, msg      ;   "Hello, world!\n",
    mov rdx, msglen   ;   sizeof("Hello, world!\n")
    syscall           ; );

    mov rax, 60       ; exit(
    mov rdi, 0        ;   EXIT_SUCCESS
    syscall           ; );

section .rodata
    msg: db "Hello, world!"
    msglen: equ $-msg

Hello World code is taken from the blog James Fisher, Adapted for assembling and debugging SASM. In SASM documentation states that the entry point must be a function named main, otherwise debug and compile code is incorrect.
What we did in this code? Made the call syscall – an appeal to the Linux operating system kernel with the correct arguments in registers, a pointer to a string in the data section.

Zoom Enhance

Consider the code details:

global main

global – assembler directive allows you to set global symbols with string names. A good analogy – interface header files C / C ++ languages. In this case, we ask the main character for the input function.

section .text

section – assembler directive allows define sections (segments) of code. Section directive or a segment equal. The .text section is placed code.

main:

Announces the beginning of the main function. The assembler function called subroutines (subroutine)

mov rbp, rsp

The first machine instruction mov – puts the value of the argument 1 to argument 2. In this case, we transfer the register value in rbp rsp. Of comments you can understand that this line added SASM to simplify debugging. Apparently that is a personal affair between SASM and debugger gdb.

Next, look at the code to .rodata data segment, two call syscall, first outputs Hello World string exits from the second application with the correct code 0.

Let us imagine that the registers are variables with names rax, rdi, rsi, rdx, r10, r8, r9. By analogy with the high-level language, turn from vertical to horizontal view of the assembly, then the call syscall will look like this:

syscall(rax, rdi, rsi, rdx, r10, r8, r9)

Then the call to print text:

syscall(1, 1, msg, msglen)

Calling the exit with the correct code 0:

syscall(60, 0)

Consider the arguments in more detail in the header asm/unistd_64.h file find function __NR_write – 1, then look in the documentation for the arguments write:
ssize_t write (int fd, const void * buf, size_t count);

The first argument – the file descriptor, the second – the buffer with the data, the third – the counter bytes to write to a file handle. We are looking for the number of file descriptor for standard output, in the manual on stdout find the code 1. Then the case for small, to pass a pointer to the Hello World string buffer from the data section .rodata – msg, byte count – msglen, transfer registers rax, rdi, rsi, rdx correct has argument and call syscall.

Designation constant length lines and is described in manual nasm:

message db 'hello, world'
msglen equ $-message

Simple enough right?

References

https://github.com/Dman95/SASM
https://www.nasm.us/xdoc/2.15.05/html/nasmdoc0.html
http://acm.mipt.ru/twiki/bin/view/Asm/HelloNasm
https://jameshfisher.com/2018/03/10/linux-assembly-hello-world/
http://www.ece.uah.edu/~milenka/cpe323-10S/labs/lab3.pdf
https://c9x.me/x86/html/file_module_x86_id_176.html
https://www.recurse.com/blog/7-understanding-c-by-learning-assembly
https://ru.wikipedia.org/wiki/%D0%9F%D1%80%D0%BE%D0%BB%D0%BE%D0%B3_%D0%BF%D1%80%D0%BE%D1%86%D0%B5%D0%B4%D1%83%D1%80%D1%8B
https://www.tutorialspoint.com/assembly_programming/assembly_basic_syntax.html
https://nekosecurity.com/x86-64-assembly/part-3-nasm-anatomy-syscall-passing-argument
https://man7.org/linux/man-pages/man2/syscall.2.html
https://en.wikipedia.org/wiki/Write_(system_call)

Source Code

https://gitlab.com/demensdeum/assembly-playground

Hash Table

Hash table data structure allows to realize an associative array (dictionary), with an average capacity of O (1) to insert, delete, search.

Below is an example of a simple implementation of a hash mapy on nodeJS:

How it works? Watching the hands:

Inside is an array of hash mapy
Inside the element of the array is a pointer to the first node of a linked list
Partitioning the memory to an array of pointers (e.g. 65,535 cells)
Implement the hash function, the input dictionary is the key, and at the outlet it can do just about anything, but in the end returns the array index

How does the record:

At the entrance there is a pair of key – value
The hash function returns the index on
Get node linked list from an array by index
Check whether it matches the key
If it matches, then replace the value
If it does not, then move on to the next node, until we find or do not find the node with the correct key.
If the node has not found, we create it at the end of a linked list

How does the search key:

At the entrance there is a pair of key – value
The hash function returns the index on
Get node linked list from an array by index
Check whether it matches the key
If it matches, the return value
If it does not, then move on to the next node, until we find or do not find the node with the correct key.

Why do we need a linked list in the array? Because of possible conflicts in the calculation of the hash function. In such a case several different key-value pairs will be located on the same index in the array, in such a case is carried out by extending the linked list with the search key necessary.

References

https://en.wikipedia.org/wiki/Hash_table
https://www.youtube.com/watch?v=wg8hZxMRwcw

Source Code

https://gitlab.com/demensdeum/datastructures

Resources access through NDK C++ Android

To work with resources in Android through ndk – C ++ there are several options:

Use access to the resources of the apk file using AssetManager
Download resources from the Internet and extract them in the application directory, used by standard methods C ++
Combined method – to get access to the archive with resources apk through AssetManager, unpack them in the application directory, then use with standard C ++ techniques

Next, I will describe the combination of access methods using the game engine Flame Steel Engine.
When using SDL can facilitate access to the resources of the apk, library wraps the calls to AssetManager, offering a similar interface to the stdio (fopen, fread, fclose, etc.)


SDL_RWops *io = SDL_RWFromFile("files.fschest", "r");

After the file download from the apk to the buffer, you need to change the current working directory to the application directory, it is available to the application without additional permits. For this we use a wrapper on SDL:


chdir(SDL_AndroidGetInternalStoragePath());

Next, write down the file from the clipboard to the current working directory using fopen, fwrite, fclose. After the archive will be available in the directory for C ++, unpack it. Archives zip can extract a combination of two libraries – minizip and zlib, the first structure can work with files, the second decompresses the data.
For more control, simplicity, portability, I realized own archive format with zero compression called FSChest (Flame Steel Chest). This format supports the directory archiving files, and unpacking; Support folder hierarchy is missing, can work only with files.
Connecting the library header FSChest, unpack the archive:


#include "fschest.h" 
FSCHEST_extractChestToDirectory(archivePath, SDL_AndroidGetInternalStoragePath());

After unpacking, C / C ++ interfaces will be available files from the archive. So I did not have to rewrite all the work with the files in the engine, and add only unpacking files at startup.

References

https://developer.android.com/ndk/reference/group/asset

Source Code

https://gitlab.com/demensdeum/space-jaguar-action-rpg
https://gitlab.com/demensdeum/fschest

Stack Machine and RPN

Let’s say we need to implement a simple bytecode interpreter, which approach to the implementation of this task to choose?

Stack data structure provides an opportunity to implement a simple bytecode machine. Features and realization of machines stack described in numerous articles of Western and domestic Internet, just mention that the Java virtual machine is an example of a stack machine.

The principle of operation of the machine is simple, the input is a program containing data and opcodes (opcodes), using the stack manipulations performed realization of necessary operations. Consider the example of the program bytecode my stack machine:


пMVkcatS olleHП

At the output we get the string “Hello StackVM”. Stack machine reads the program from left to right, character by character by uploading data onto the stack, with the appearance of the opcode to symbol – performs implementation team using the stack.

An example of the implementation of the stack machine to nodejs:

Reverse polish notation (RPN)

Also, stacking machine is easy to use for the implementation of calculators, this is done using RPN (postfix notation).
An example of a conventional infix:
2*2+3*4

Converted into RPN:
22*34*+

To calculate postfix notation use a stack machine:
2 – at the top of the stack (stack 2)
2 – on top of the stack (Stack: 2.2)
* – get the top of the stack twice, multiply the result is sent to the top of the stack (stack of 4)
3 – on top of the stack (the stack 4, 3)
4 – on top of the stack (a stack of 4, 3, 4)
* – get the top of the stack twice, multiply the result is sent to the top of the stack (stack of 4, 12)
+ – get the top of the stack twice, add up the results, go to the top of the stack (stack 16)

As you can see – the result of operations 16 remains on the stack, it can be derived by implementing opcodes stack printing, for example:
п22*34*+П

П – print start opcode stack, n – opcode closure print stack and sending the final line in rendering.
For conversion from arithmetic operations in postfix infix used Edsger Dijkstra algorithm called “Shunting-yard algorithm”. An example implementation can be found above or in the project repository on the machine stack nodejs below.

References

https://tech.badoo.com/ru/article/579/interpretatory-bajt-kodov-svoimi-rukami/
https://ru.wikipedia.org/wiki/Обратная_польская_запись

Source Code

https://gitlab.com/demensdeum/stackvm/