Very often during the discussions of the correctness of the work of some kind of software feature, I encounter such a situation that the functionality from the user looked strange, illogical.Discussion with the naval product looked something like this:
– There is clearly a problem in behavior
– Well, we’ll start and when users begin to complain then and fix it
– ???Well ok …
It seems to be a working scheme?A fairly optimal algorithm for commands with a small budget, compressed deadlines, insufficient research/absence of the UI/UX specialist.Users will complain if that, nothing to worry.
Search in Google shows that the source of these methods comes from the article – “Complaint -Driven Development” from Coding Horror
Once I pierced products, including a doctoral sausage for 300 rubles.Through the terminal in the supermarket, he left the store with this sausage in full confidence that it was paid – the terminal offered not to print the check and I agreed not to spend precious paper on this check.During the process of “breaking through” the goods for each product, the terminal published a squeak, which signals that everything worked correctly.Plus with a sound notification, the terminal winked out a backlight from the barcode scanner.
The next day I went for groceries in the supermarket again, struck the products through the terminal.At the exit, I was met by a man of the southern appearance with a thick beard, stretching out the smartphone, he said-“Is this you on the camera?”, I looked into his phone and saw myself in the Melodic-Death-Metal T-shirt with skulls with skulls and all that, reasons to doubtThere was no.
“Yes, this is me, but what’s the matter?”, The man, very squinting, said “you didn’t break through the sausage yesterday” … Ukhta
After short -term clarifications who he is, and how he made these conclusions, he showed me a video that hangs on the ceiling of the store, in the video I break through sausage, the terminal flashes the scanner backlight, I put the sausage in the bag.
– The video shows how the scanner worked
– I did not work out anything, pay sausage!
Having a little taken aback from such an attitude, I demanded a complaint book to write that the terminal requires a finalization of the software, since it makes all the signs of the correct work, but in fact it just buggy, without signaling it on the screen.
After 10 minute skirmishes with him, and his boss, who immediately came running to the defense of his employee and a bad working terminal, they decided to call the administrator girl so that she had already brought a complaint book and pierces a doctoral sausage.
On this day, I understood how actually it is difficult for users to complain about hardware, software products, and that most likely the mantra “people will complain – we’ll fix” very poorly.The main reason is people who protect broken robots, broken software solutions, for simplicity I propose to introduce new terms – a defender of a broken robot and a defender of broken systems.
Ordinary users cannot complain about the failure of the terminals since they are interfered with the ribbons that are clipping and starting to love the cars with which they work, perhaps considering them some kind of animated entities, forgetting that there is nothing alive there.
A similar situation happens with the dungeons, these people can protect some stupid flaws in frameworks, programming languages or any other software product with foam at the mouth, despite the complaints of users and other developers.
A typical conversation with the dazelik is as follows:
– something here does not work, according to the documentation, it seems that everything is correct
– Ah, so you did not read that manual from 2005, where it is written below in small letters that you need to add program_start: 6969
– ???uh
Such people may not understand how they themselves contribute to the dissemination of problems, mistakes, losses of time and means of their and other people.Because of them, everyone suffers, because digital transformation is impossible when silenting neo-obviousness, problems of software and hardware solutions.
I know a recent story with a mistake in the Horizon British mail, which has been driving people into debts for decades, destroyed marriages and lives of people.All this continued due to the connivance of people who were silent about the problems in the software, thus “protecting” him.
Friends, do not be litters and alights, treat the tools with which you work with a share of skepticism, otherwise you are threatened with the total enslavement of horseradish, broken systems, like hostages in the new digital world of the future.For those who cannot – at least do not prevent other people from trying to pay attention to non -working, interfering with software/hardware, because the developers of these products agreed – “when users begin to complain then and fix it”
Sources
https://blog.codinghorror.com/complaint-driven-development/
https://habr.com/ru/articles/554404/
https://en.wikipedia.org/wiki/British_Post_Office_scandal
Author Archives: demensdeum
Building a BGFX Emscripten application.
In this note, I will describe the method of assembling BGFX applications for the web (Webassembly) via Emscripten.
The installation platform is Linux X86-64, such as Arch Linux.
To begin with, we will install the EMSCRIPEN version 3.1.51, otherwise you will not succeed, everything is due to changes in the type of dynamic libraries in the latest version of EMScripten.You can read more here:
https://github.com/bkaradzic/bgfx/discussions/3266
This is done like this:
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install 3.1.51
./emsdk activate 3.1.51
source ./emsdk_env.sh
We will collect the BGFX for Webassembly – Emscripten:
mkdir bgfx-build-test
cd bgfx-build-test
git clone https://github.com/bkaradzic/bx.git
git clone https://github.com/bkaradzic/bimg.git
git clone https://github.com/bkaradzic/bgfx.git
cd bgfx
emmake make wasm-debug
As a result, in the .Build folder you will have Bitcode files with the .bc extension that will need to be licked with your BGFX application.
Should be bgfx.bc, bx.bc, bimg.bc;In different assemblies, a different name for these files, depending on the type of assembly (Release/Debug)
Add to cmakelists.txt the file with .bc files, for example, absolute ways to files from the BGFX-Experiment project:
target_link_libraries(${PROJECT_NAME} SDL2 GL /home/demensdeum_stream/Sources/bgfx-build/bgfx/.build/wasm/bin/bgfxDebug.bc /home/demensdeum_stream/Sources/bgfx-build/bgfx/.build/wasm/bin/bxDebug.bc /home/demensdeum_stream/Sources/bgfx-build/bgfx/.build/wasm/bin/bimgDebug.bc)
Now change National Window Handle in Platform Data on the initialization of BGFX:
bgfx::PlatformData platformData{};
platformData.context = NULL;
platformData.backBuffer = NULL;
platformData.backBufferDS = NULL;
platformData.nwh = (void*)"#canvas";
You also need to replace the type of render with Opengl:
bgfx::Init init;
init.type = bgfx::RendererType::OpenGL;
init.resolution.width = screenWidth;
init.resolution.height = screenHeight;
init.resolution.reset = BGFX_RESET_VSYNC;
init.platformData = platformData;
if (!bgfx::init(init))
{
throw std::runtime_error("Failed to initialize bgfx");
}
Cross -sleeping GLSL shaders under 120:
shaderc -f "VertexShader.vs" -o "VertexShader.glsl" --type "v" -p "120"
shaderc -f "FragmentShader.fs" -o "FragmentShader.glsl" --type "f" -p "120"
EU-but .GLSL Files must be added to cmakelists.txt as -preload-file:
set(CMAKE_CXX_FLAGS ... <Остальная часть>
--preload-file VertexShader.glsl \
--preload-file FragmentShader.glsl \
It remains to replace the main cycle of the render in your application with While to call the function through EMSCRIPTEN_SETN_MAIN_LOOP.
You can read about this here:
https://demensdeum.com/blog/ru/2017/03/29/porting-sdl-c-game-to-html5-emscripten/
Next, collect your EMSCRIPTEN project for the usual one, everything should work.
Of the interesting – in the assembly EMSCRIPTEN 3.1.51 it seems that there is no Openal (or only mine).
The source code of the project that is correctly assembled with BGFX and EMSCRIPTEN:
https://github.com/demensdeum/bgfx-experiments/tree/main/2-emscripten-build
Sources
https://github.com/bkaradzic/bgfx/discussions/3266
https://bkaradzic.github.io/bgfx/build.html
https://emscripten.org/docs/getting_started/downloads.html
https://demensdeum.com/blog/ru/2017/03/29/porting-sdl-c-game-to-html5-emscripten/
https://llvm.org/docs/BitCodeFormat.html
Surreal Engine C++ WebAssembly Port
In this note, I will describe how I ported the Surreal Engine game engine to WebAssembly.
Surreal Engine is a game engine that implements most of the functionality of the Unreal Engine 1. Well-known games on this engine include Unreal Tournament 99, Unreal, Deus Ex, and Undying. It is a classic engine that primarily worked in a single-threaded execution environment.
Initially, I had the idea to take on a project that I would not be able to complete in any reasonable timeframe, thus showing my Twitch followers that there are projects that even I cannot accomplish. On the very first stream, I suddenly realized that porting the Surreal Engine C++ to WebAssembly using Emscripten was achievable.
A month later, I can demonstrate my fork and build of the engine on WebAssembly:
https://demensdeum.com/demos/SurrealEngine/
The controls, as in the original, are carried out on the keyboard arrows. Next, I plan to adapt for mobile control (touch), add correct lighting, and other graphical features of the Unreal Tournament 99 renderer.
Where to Start?
The first thing I want to say is that any project can be ported from C++ to WebAssembly using Emscripten; the question is only how complete the functionality will be. Choose a project whose library ports are already available for Emscripten. In the case of Surreal Engine, it was very fortunate because the engine uses the SDL 2 and OpenAL libraries, both of which are ported to Emscripten. However, the graphical API used is Vulkan, which is currently not available for HTML5. Work is underway to implement WebGPU, but it is also in draft stage, and it is unknown how simple the further port from Vulkan to WebGPU will be after its full standardization. Therefore, I had to write my own basic OpenGL ES / WebGL renderer for Surreal Engine.
Project Build
The build system in Surreal Engine is CMake, which also simplifies porting since Emscripten provides its native builders – emcmake, emmake.
The Surreal Engine port was based on the code of my latest game on WebGL/OpenGL ES and C++ called Death-Mask, which made the development much easier; all necessary build flags and code examples were with me.
One of the most important moments in CMakeLists.txt is the build flags for Emscripten. Below is an example from the project file:
set(CMAKE_CXX_FLAGS "-s MIN_WEBGL_VERSION=2 \
-s MAX_WEBGL_VERSION=2 \
-s EXCEPTION_DEBUG \
-fexceptions \
--preload-file UnrealTournament/ \
--preload-file SurrealEngine.pk3 \
--bind \
--use-preload-plugins \
-Wall \
-Wextra \
-Werror=return-type \
-s USE_SDL=2 \
-s ASSERTIONS=1 \
-w \
-g4 \
-s DISABLE_EXCEPTION_CATCHING=0 \
-O3 \
--no-heap-copy \
-s ALLOW_MEMORY_GROWTH=1 \
-s EXIT_RUNTIME=1")
The build script itself:
clear
emmake make -j 16
cp SurrealEngine.data /srv/http/SurrealEngine/SurrealEngine.data
cp SurrealEngine.js /srv/http/SurrealEngine/SurrealEngine.js
cp SurrealEngine.wasm /srv/http/SurrealEngine/SurrealEngine.wasm
cp ../buildScripts/Emscripten/index.html /srv/http/SurrealEngine/index.html
cp ../buildScripts/Emscripten/background.png /srv/http/SurrealEngine/background.png
Next, prepare index.html, which includes the project file system preloader. For web deployment, I used the Unreal Tournament Demo version 338. As can be seen from the CMake file, the unpacked game folder was added to the build directory and linked as a preload-file for Emscripten.
Main Code Changes
Then it was necessary to change the game’s main loop; launching an infinite loop is not allowed as it causes the browser to hang. Instead, you need to use emscripten_set_main_loop. I wrote about this feature in my 2017 note “Porting an SDL C++ game to HTML5 (Emscripten)”
Change the while loop exit condition to if, then output the main game engine class that contains the game loop to the global scope, and write a global function that will call the game loop step from the global object:
#if __EMSCRIPTEN__ #includeEngine *EMSCRIPTEN_GLOBAL_GAME_ENGINE = nullptr; void emscripten_game_loop_step() { EMSCRIPTEN_GLOBAL_GAME_ENGINE->Run(); } #endif
After this, make sure that there are no background threads in the application. If there are, be prepared to rewrite them for single-threaded execution or use the pthread library in Emscripten.
A background thread in Surreal Engine is used for music playback. From the main engine thread, data about the current track and the need to play or stop music is received. The background thread then gets the new state via mutex and starts playing new music or pauses. The background thread is also used for music buffering during playback.
My attempts to build Surreal Engine under Emscripten with pthread were unsuccessful because the SDL2 and OpenAL ports were built without pthread support, and I didn’t want to rebuild them for the sake of music. Therefore, I transferred the background music thread functionality to single-threaded execution using a loop. By removing pthread calls from the C++ code, I moved buffering and music playback to the main thread, increasing the buffer by a few seconds to avoid delays.
Next, I will describe the specific implementations of graphics and sound.
Vulkan is Not Supported!
Yes, Vulkan is not supported in HTML5, although all advertising brochures present cross-platform and wide support on platforms as the main advantage of Vulkan. For this reason, I had to write my own basic graphics renderer for the simplified type of OpenGL – ES. It is used on mobile devices, sometimes lacking modern OpenGL features, but it is very well portable to WebGL, which is implemented by Emscripten. Writing the basic renderer for tiles, bsp rendering, simple GUI display, and models + maps took two weeks. This was perhaps the most challenging part of the project. There is still a lot of work ahead to implement the full rendering functionality of Surreal Engine, so any help from readers in the form of code and pull requests is welcome.
OpenAL is Supported!
It was very fortunate that Surreal Engine uses OpenAL for sound output. Writing a simple hello world in OpenAL and building it on WebAssembly with Emscripten made it clear how simple everything is, and I set off to port the sound.
After a few hours of debugging, it became apparent that there are several bugs in the Emscripten OpenAL implementation. For example, when initializing to read the number of mono channels, the method returned an infinite number, and after attempting to initialize a vector of infinite size, C++ crashed with a vector::length_error exception.
This was bypassed by hardcoding the number of mono channels to 2048:
alcGetIntegerv(alDevice, ALC_MONO_SOURCES, 1, &monoSources);
alcGetIntegerv(alDevice, ALC_STEREO_SOURCES, 1, &stereoSources);
#if EMSCRIPTEN
monoSources = 2048; // for some reason Emscripten's OpenAL gives infinite monoSources count, bug?
#endif
Is there a Network?
Surreal Engine currently does not support network play. Bot play is supported, but someone needs to write AI for these bots. Theoretically, network play on WebAssembly/Emscripten can be implemented using Websockets.
Conclusion
In conclusion, I want to say that porting Surreal Engine was quite smooth due to the use of libraries that have Emscripten ports, as well as my previous experience in implementing a C++ game for WebAssembly on Emscripten. Below are links to sources of knowledge and repositories on the topic.
M-M-M-MONSTER KILL!
If you want to help the project, preferably with WebGL/OpenGL ES renderer code, write to me on Telegram:
https://t.me/demenscave
Links
https://demensdeum.com/demos/SurrealEngine/
https://github.com/demensdeum/SurrealEngine-Emscripten
https://github.com/dpjudas/SurrealEngine
Flash Forever – Interceptor 2021
Recently, it turned out that Adobe Flash works quite stably under Wine. During a 4-hour stream, I made the game Interceptor 2021, which is a sequel to the game Interceptor 2020, written for the ZX Spectrum.
For those who are not in the know – the Flash technology provided interactivity on the web from 2000 to around 2015. Its shutdown was prompted by an open letter from Steve Jobs, in which he wrote that Flash should be consigned to history because it lagged on the iPhone. Since then, JS has become even more sluggish than Flash, and Flash itself has been wrapped in JS, making it possible to run it on anything thanks to the Ruffle player.
You can play it here:
https://demensdeum.com/demos/Interceptor2021
Video:
https://www.youtube.com/watch?v=-3b5PkBvHQk
Source code:
https://github.com/demensdeum/Interceptor-2021
Masonry-AR Game Demo
Masonry-AR is an augmented reality game where you need to move around the city in the real world and collect Masonic knowledge from books, obtaining currency and capturing territory for your Masonic order. The game has no relation to any real organizations, all matches are random.
Game demo:
https://demensdeum.com/demos/masonry-ar/client
Wiki:
https://demensdeum.com/masonry-ar-wiki-en/
Source code:
https://github.com/demensdeum/Masonry-AR
CRUD repository
In this note, I will describe the basic principles of the famous classic Crud pattern, the implementation in Swift. Swift is an open, cross -platform language available for Windows, Linux, MacOS, iOS, Android.
There are many decisions of abstracting the storage of data and the logic of the application.One of these solutions is the CRUD approach, this is acronym from C – Create, R -read, U – Update, D – Delete.
Typically, the implementation of this principle is ensured by implementing the interface to the database in which work with elements occurs using a unique identifier, such as ID.An interface is created for each letter Crud – Create (Object, ID), Read (ID), Update (Object, ID), Delete (Object, ID).
If the object contains ID inside itself, then the ID argument can be missed in terms of the methods (Create, Update, Delete), since the object is transmitted there entirely with its field – ID.But for – Read requires ID, as we want to get an object from the database by the identifier.
All names are fictitious
Imagine that the Assistantai hypothetical application was created using an Etherrelm free SDK database, the integration was simple, the API was very convenient, as a result, the application was released into markets.
Suddenly, the SDK Etherrelm developer decides to make it paid, setting the price of $ 100 per year per one user of the application.
What?Yes!What now to do to developers from Assistantai, because they already have 1 million active users!Pay $ 100 million?
Instead, it is decided to evaluate the transfer of the application to the Rootdata database, according to programmers, such a transfer will take about six months, this without taking into account the implementation of new features in the application.After short thoughts, it is decided to remove the application from the markets, rewrite it on another free cross -platform framework with a Buems built -in database, this will solve the problem with the database + will simplify the development for other platforms.
A year later, the application is rewritten to Buems, but then suddenly the developer of the framework decides to make it paid.It turns out that the team got into the same trap twice, whether they will manage to get out for the second time, this is a completely different story.
Abstraction to help
These problems would be avoided if the developers used the abstraction of interfaces inside the application.For three whales of OOP – polymorphism, incapsulation, inheritance, not so long ago added one more abstraction.
The abstraction of the data allows you to describe ideas, models in general terms, with a minimum of details, while quite accurate for the implementation of specific implementations that are used to solve business abomes.
How can we abstract the work with the database so that the logic of the application does not depend on it?We use CRUD!
Simplified UML CRUD scheme looks like this:
An example with a fictional data database:
An example with a real SQLite database:
As you have already noticed, when you switch the database, only it changes, the CRUD interface with which the application interacts remains unchanged.CRUD is an option for the implementation of the GOF pattern – adapter, becauseWith it, we adapt the application interfaces to any database, combine incompatible interfaces.
The words are empty, show me the code
To implement abstractions in programming languages, interfaces/protocols/abstract classes are used.All these are phenomena of the same order, however, in interviews you may be asked to name the difference between them, I personally think that there is much sense in this becauseThe only purpose of use is the implementation of data abstraction, otherwise it is an interviewer memory test.
CRUD is often implemented as part of the repository pattern, the repository, however, can implement the CRUD interface, or may not be implemented, it all depends on the ingenuity of the developer.
Consider a rather typical SWIFT code of the Book structures repository, which works directly with the userDefaults database:
import Foundation
struct Book: Codable {
let title: String
let author: String
}
class BookRepository {
func save(book: Book) {
let record = try! JSONEncoder().encode(book)
UserDefaults.standard.set(record, forKey: book.title)
}
func get(bookWithTitle title: String) -> Book? {
guard let data = UserDefaults.standard.data(forKey: title) else { return nil }
let book = try! JSONDecoder().decode(Book.self, from: data)
return book
}
func delete(book: Book) {
UserDefaults.standard.removeObject(forKey: book.title)
}
}
let book = Book(title: "Fear and Loathing in COBOL", author: "Sir Edsger ZX Spectrum")
let repository = BookRepository()
repository.save(book: book)
print(repository.get(bookWithTitle: book.title)!)
repository.delete(book: book)
guard repository.get(bookWithTitle: book.title) == nil else {
print("Error: can't delete Book from repository!")
exit(1)
}
The code above seems simple, however, we calculate the number of violations of the principle of Dry (do not repeat youurself) and the connected code:
Continence to the userdefaults database
The connectedness of the json -jsonencoder, jsondecoder encoders and decoders
Correspondence with the Book structure, and we need an abstract repository in order not to create a repository class for each structure that we will store in the database (Dry Violation)
I meet such a CRUD code of the repository quite often, it is possible to use it, but high connectedness, duplication of code, lead to the fact that over time its support will become very complicated.This will be especially felt when trying to switch to another database, or when the internal logic of working with the database in all repository created in the application.
Instead of duplicating the code, keeping high connectedness – we will write a protocol for the CRUD repository, thus abstracting the interface of the database and business logic of the application, observing DRY, carrying out low connectedness:
protocol CRUDRepository {
typealias Item = Codable
typealias ItemIdentifier = String
func create(id: CRUDRepository.ItemIdentifier, item: T) async throws
func read(id: CRUDRepository.ItemIdentifier) async throws -> T
func update(id: CRUDRepository.ItemIdentifier, item: T) async throws
func delete(id: CRUDRepository.ItemIdentifier) async throws
}
The Crudrepository protocol describes the interfaces and associated data types for the further implementation of a particular CRUD repository.
Next, write a specific implementation for the UserDefaults database:
class UserDefaultsRepository: CRUDRepository {
private typealias RecordIdentifier = String
let tableName: String
let dataTransformer: DataTransformer
init(
tableName: String = "",
dataTransformer: DataTransformer = JSONDataTransformer()
) {
self.tableName = tableName
self.dataTransformer = dataTransformer
}
private func key(id: CRUDRepository.ItemIdentifier) -> RecordIdentifier {
"database_\(tableName)_item_\(id)"
}
private func isExists(id: CRUDRepository.ItemIdentifier) async throws -> Bool {
UserDefaults.standard.data(forKey: key(id: id)) != nil
}
func create(id: CRUDRepository.ItemIdentifier, item: T) async throws {
let data = try await dataTransformer.encode(item)
UserDefaults.standard.set(data, forKey: key(id: id))
UserDefaults.standard.synchronize()
}
func read(id: CRUDRepository.ItemIdentifier) async throws -> T {
guard let data = UserDefaults.standard.data(forKey: key(id: id)) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
let item: T = try await dataTransformer.decode(data: data)
return item
}
func update(id: CRUDRepository.ItemIdentifier, item: T) async throws {
guard try await isExists(id: id) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
let data = try await dataTransformer.encode(item)
UserDefaults.standard.set(data, forKey: key(id: id))
UserDefaults.standard.synchronize()
}
func delete(id: CRUDRepository.ItemIdentifier) async throws {
guard try await isExists(id: id) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
UserDefaults.standard.removeObject(forKey: key(id: id))
UserDefaults.standard.synchronize()
}
}
The code looks long, but contains a complete specific implementation of CRUD repository containing weak contact, details further.
Typealias’s are added to make code smug.
Weakness and strong connectivity
The damping from a specific structure (Struct) is implemented using geneticist T, which in turn should implement CODable protocols.Codable allows you to transform structures using classes that implement the Toplevelencoder and ToplevelDecoder protocols, such as JSONENCODER and JSONDECODER, using the basic types (Int, String, Float, etc.), there is no need to write additional code for transforming structures.
The damping from a particular encoder and decoder occurs using abstracting in the Datatransformer protocol:
protocol DataTransformer {
func encode(_ object: T) async throws -> Data
func decode(data: Data) async throws -> T
}
Using the implementation of the Date-transformer, we implemented an abstraction of the enkoder and decoder interfaces, having implemented a weak connectedness to ensure work with various types of data formats.
The following is the code of a particular Datatransformer, namely for JSON:
class JSONDataTransformer: DataTransformer {
func encode(_ object: T) async throws -> Data where T : Encodable {
let data = try JSONEncoder().encode(object)
return data
}
func decode(data: Data) async throws -> T where T : Decodable {
let item: T = try JSONDecoder().decode(T.self, from: data)
return item
}
}
And so it was possible?
What has changed?Now it is enough to initiate a specific repository for working with any structure that the CODBLE protocol implits, in this way the need for the duplication of code disappears, and a faint connected application is realized.
An example of a client CRUD with a specific repository, the database is userdefaults, the JSON data format, the Client structure, also an example of the recording and reading of the array:
import Foundation
print("One item access example")
do {
let clientRecordIdentifier = "client"
let clientOne = Client(name: "Chill Client")
let repository = UserDefaultsRepository(
tableName: "Clients Database",
dataTransformer: JSONDataTransformer()
)
try await repository.create(id: clientRecordIdentifier, item: clientOne)
var clientRecord: Client = try await repository.read(id: clientRecordIdentifier)
print("Client Name: \(clientRecord.name)")
clientRecord.name = "Busy Client"
try await repository.update(id: clientRecordIdentifier, item: clientRecord)
let updatedClient: Client = try await repository.read(id: clientRecordIdentifier)
print("Updated Client Name: \(updatedClient.name)")
try await repository.delete(id: clientRecordIdentifier)
let removedClientRecord: Client = try await repository.read(id: clientRecordIdentifier)
print(removedClientRecord)
}
catch {
print(error.localizedDescription)
}
print("Array access example")
let clientArrayRecordIdentifier = "clientArray"
let clientOne = Client(name: "Chill Client")
let repository = UserDefaultsRepository(
tableName: "Clients Database",
dataTransformer: JSONDataTransformer()
)
let array = [clientOne]
try await repository.create(id: clientArrayRecordIdentifier, item: array)
let savedArray: [Client] = try await repository.read(id: clientArrayRecordIdentifier)
print(savedArray.first!)
At the first check, CRUD has been processed an exception in which the reading of the remote Aitem will already be unavailable.
Switch the database
Now I will show how to transfer the current code to another database.For example, I will take the code of the SQLite repository that ChatGPT generated:
import SQLite3
class SQLiteRepository: CRUDRepository {
private typealias RecordIdentifier = String
let tableName: String
let dataTransformer: DataTransformer
private var db: OpaquePointer?
init(
tableName: String,
dataTransformer: DataTransformer = JSONDataTransformer()
) {
self.tableName = tableName
self.dataTransformer = dataTransformer
self.db = openDatabase()
createTableIfNeeded()
}
private func openDatabase() -> OpaquePointer? {
var db: OpaquePointer? = nil
let fileURL = try! FileManager.default
.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: false)
.appendingPathComponent("\(tableName).sqlite")
if sqlite3_open(fileURL.path, &db) != SQLITE_OK {
print("error opening database")
return nil
}
return db
}
private func createTableIfNeeded() {
let createTableString = """
CREATE TABLE IF NOT EXISTS \(tableName) (
id TEXT PRIMARY KEY NOT NULL,
data BLOB NOT NULL
);
"""
var createTableStatement: OpaquePointer? = nil
if sqlite3_prepare_v2(db, createTableString, -1, &createTableStatement, nil) == SQLITE_OK {
if sqlite3_step(createTableStatement) == SQLITE_DONE {
print("\(tableName) table created.")
} else {
print("\(tableName) table could not be created.")
}
} else {
print("CREATE TABLE statement could not be prepared.")
}
sqlite3_finalize(createTableStatement)
}
private func isExists(id: CRUDRepository.ItemIdentifier) async throws -> Bool {
let queryStatementString = "SELECT data FROM \(tableName) WHERE id = ?;"
var queryStatement: OpaquePointer? = nil
if sqlite3_prepare_v2(db, queryStatementString, -1, &queryStatement, nil) == SQLITE_OK {
sqlite3_bind_text(queryStatement, 1, id, -1, nil)
if sqlite3_step(queryStatement) == SQLITE_ROW {
sqlite3_finalize(queryStatement)
return true
} else {
sqlite3_finalize(queryStatement)
return false
}
} else {
print("SELECT statement could not be prepared.")
throw CRUDRepositoryError.databaseError
}
}
func create(id: CRUDRepository.ItemIdentifier, item: T) async throws {
let insertStatementString = "INSERT INTO \(tableName) (id, data) VALUES (?, ?);"
var insertStatement: OpaquePointer? = nil
if sqlite3_prepare_v2(db, insertStatementString, -1, &insertStatement, nil) == SQLITE_OK {
let data = try await dataTransformer.encode(item)
sqlite3_bind_text(insertStatement, 1, id, -1, nil)
sqlite3_bind_blob(insertStatement, 2, (data as NSData).bytes, Int32(data.count), nil)
if sqlite3_step(insertStatement) == SQLITE_DONE {
print("Successfully inserted row.")
} else {
print("Could not insert row.")
throw CRUDRepositoryError.databaseError
}
} else {
print("INSERT statement could not be prepared.")
throw CRUDRepositoryError.databaseError
}
sqlite3_finalize(insertStatement)
}
func read(id: CRUDRepository.ItemIdentifier) async throws -> T {
let queryStatementString = "SELECT data FROM \(tableName) WHERE id = ?;"
var queryStatement: OpaquePointer? = nil
var item: T?
if sqlite3_prepare_v2(db, queryStatementString, -1, &queryStatement, nil) == SQLITE_OK {
sqlite3_bind_text(queryStatement, 1, id, -1, nil)
if sqlite3_step(queryStatement) == SQLITE_ROW {
let queryResultCol1 = sqlite3_column_blob(queryStatement, 0)
let queryResultCol1Length = sqlite3_column_bytes(queryStatement, 0)
let data = Data(bytes: queryResultCol1, count: Int(queryResultCol1Length))
item = try await dataTransformer.decode(data: data)
} else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
} else {
print("SELECT statement could not be prepared")
throw CRUDRepositoryError.databaseError
}
sqlite3_finalize(queryStatement)
return item!
}
func update(id: CRUDRepository.ItemIdentifier, item: T) async throws {
guard try await isExists(id: id) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
let updateStatementString = "UPDATE \(tableName) SET data = ? WHERE id = ?;"
var updateStatement: OpaquePointer? = nil
if sqlite3_prepare_v2(db, updateStatementString, -1, &updateStatement, nil) == SQLITE_OK {
let data = try await dataTransformer.encode(item)
sqlite3_bind_blob(updateStatement, 1, (data as NSData).bytes, Int32(data.count), nil)
sqlite3_bind_text(updateStatement, 2, id, -1, nil)
if sqlite3_step(updateStatement) == SQLITE_DONE {
print("Successfully updated row.")
} else {
print("Could not update row.")
throw CRUDRepositoryError.databaseError
}
} else {
print("UPDATE statement could not be prepared.")
throw CRUDRepositoryError.databaseError
}
sqlite3_finalize(updateStatement)
}
func delete(id: CRUDRepository.ItemIdentifier) async throws {
guard try await isExists(id: id) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
let deleteStatementString = "DELETE FROM \(tableName) WHERE id = ?;"
var deleteStatement: OpaquePointer? = nil
if sqlite3_prepare_v2(db, deleteStatementString, -1, &deleteStatement, nil) == SQLITE_OK {
sqlite3_bind_text(deleteStatement, 1, id, -1, nil)
if sqlite3_step(deleteStatement) == SQLITE_DONE {
print("Successfully deleted row.")
} else {
print("Could not delete row.")
throw CRUDRepositoryError.databaseError
}
} else {
print("DELETE statement could not be prepared.")
throw CRUDRepositoryError.databaseError
}
sqlite3_finalize(deleteStatement)
}
}
Or CRUD Code for a repository for a file system that also generated ChatGPT:
import Foundation
class FileSystemRepository: CRUDRepository {
private typealias RecordIdentifier = String
let directoryName: String
let dataTransformer: DataTransformer
private let fileManager = FileManager.default
private var directoryURL: URL
init(
directoryName: String = "Database",
dataTransformer: DataTransformer = JSONDataTransformer()
) {
self.directoryName = directoryName
self.dataTransformer = dataTransformer
let paths = fileManager.urls(for: .documentDirectory, in: .userDomainMask)
directoryURL = paths.first!.appendingPathComponent(directoryName)
if !fileManager.fileExists(atPath: directoryURL.path) {
try? fileManager.createDirectory(at: directoryURL, withIntermediateDirectories: true, attributes: nil)
}
}
private func fileURL(id: CRUDRepository.ItemIdentifier) -> URL {
return directoryURL.appendingPathComponent("item_\(id).json")
}
private func isExists(id: CRUDRepository.ItemIdentifier) async throws -> Bool {
return fileManager.fileExists(atPath: fileURL(id: id).path)
}
func create(id: CRUDRepository.ItemIdentifier, item: T) async throws {
let data = try await dataTransformer.encode(item)
let url = fileURL(id: id)
try data.write(to: url)
}
func read(id: CRUDRepository.ItemIdentifier) async throws -> T {
let url = fileURL(id: id)
guard let data = fileManager.contents(atPath: url.path) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
let item: T = try await dataTransformer.decode(data: data)
return item
}
func update(id: CRUDRepository.ItemIdentifier, item: T) async throws {
guard try await isExists(id: id) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
let data = try await dataTransformer.encode(item)
let url = fileURL(id: id)
try data.write(to: url)
}
func delete(id: CRUDRepository.ItemIdentifier) async throws {
guard try await isExists(id: id) else {
throw CRUDRepositoryError.recordNotFound(id: id)
}
let url = fileURL(id: id)
try fileManager.removeItem(at: url)
}
}
We replace the repository in the client code:
import Foundation
print("One item access example")
do {
let clientRecordIdentifier = "client"
let clientOne = Client(name: "Chill Client")
let repository = FileSystemRepository(
directoryName: "Clients Database",
dataTransformer: JSONDataTransformer()
)
try await repository.create(id: clientRecordIdentifier, item: clientOne)
var clientRecord: Client = try await repository.read(id: clientRecordIdentifier)
print("Client Name: \(clientRecord.name)")
clientRecord.name = "Busy Client"
try await repository.update(id: clientRecordIdentifier, item: clientRecord)
let updatedClient: Client = try await repository.read(id: clientRecordIdentifier)
print("Updated Client Name: \(updatedClient.name)")
try await repository.delete(id: clientRecordIdentifier)
let removedClientRecord: Client = try await repository.read(id: clientRecordIdentifier)
print(removedClientRecord)
}
catch {
print(error.localizedDescription)
}
print("Array access example")
let clientArrayRecordIdentifier = "clientArray"
let clientOne = Client(name: "Chill Client")
let repository = FileSystemRepository(
directoryName: "Clients Database",
dataTransformer: JSONDataTransformer()
)
let array = [clientOne]
try await repository.create(id: clientArrayRecordIdentifier, item: array)
let savedArray: [Client] = try await repository.read(id: clientArrayRecordIdentifier)
print(savedArray.first!)
The initialization of userdefaultsrepository is replaced by FilesyStemrepository, with compatible arguments.
After launching the second version of the client code, you will find in the document folder the Clients Database Directory, which will contain a massif serialized in JSON with one Client structure.
Switching data storage format
Now we ask Chatgpt to generate an encoder and a decoder for XML:
class XMLDataTransformer: DataTransformer {
let formatExtension = "xml"
func encode(_ item: T) async throws -> Data {
let encoder = PropertyListEncoder()
encoder.outputFormat = .xml
return try encoder.encode(item)
}
func decode(data: Data) async throws -> T {
let decoder = PropertyListDecoder()
return try decoder.decode(T.self, from: data)
}
}
Thanks to built -in types in SWIFT, the task for the neural network becomes elementary.
Replace JSON with XML in the client code:
import Foundation
print("One item access example")
do {
let clientRecordIdentifier = "client"
let clientOne = Client(name: "Chill Client")
let repository = FileSystemRepository(
directoryName: "Clients Database",
dataTransformer: XMLDataTransformer()
)
try await repository.create(id: clientRecordIdentifier, item: clientOne)
var clientRecord: Client = try await repository.read(id: clientRecordIdentifier)
print("Client Name: \(clientRecord.name)")
clientRecord.name = "Busy Client"
try await repository.update(id: clientRecordIdentifier, item: clientRecord)
let updatedClient: Client = try await repository.read(id: clientRecordIdentifier)
print("Updated Client Name: \(updatedClient.name)")
try await repository.delete(id: clientRecordIdentifier)
let removedClientRecord: Client = try await repository.read(id: clientRecordIdentifier)
print(removedClientRecord)
}
catch {
print(error.localizedDescription)
}
print("Array access example")
let clientArrayRecordIdentifier = "clientArray"
let clientOne = Client(name: "Chill Client")
let repository = FileSystemRepository(
directoryName: "Clients Database",
dataTransformer: XMLDataTransformer()
)
let array = [clientOne]
try await repository.create(id: clientArrayRecordIdentifier, item: array)
let savedArray: [Client] = try await repository.read(id: clientArrayRecordIdentifier)
print(savedArray.first!)
The client code has changed only one expression jsondatatransformer -> XMldatratransformer
Result
CRUD repository is one of the design patterns that can be used to implement the weakness of the components of the application architecture.Another solution is the use of ORM (object mapping), if briefly, the ORM uses an approach in which the structures are completely tolerate on the database, and then changes with models should be displayed (flashing (!)) On the database.
But this is a completely different story.
The full implementation of CRUD repositories for SWIFT is available on the link:
https://gitlab.com/demensdeum/crud-example
By the way, SWIFT has long been supported outside MacOS, the code from the article was written and tested on Arch Linux.
Sources
https://developer.apple.com/documentation/combine/topleveldecoder
https://developer.apple.com/documentation/combine/toplevelencoder
https://en.wikipedia.org/wiki/Create,_read,_update_and_delete
dd input/output error
If you receive an input/output error when copying a normal disk using dd in Linux, here are the steps you should take:
The situation is quite sad, but solvable. Most likely, you are dealing with a faulty disk that contains bad blocks that can no longer be used, written to, or read from.
Be sure to check such a disk using S.M.A.R.T., it will most likely show you disk errors. This was the case for me, the number of bad blocks was so large that I had to say goodbye to the old hard drive and replace it with a new SSD.
The problem was that this disk had a fully functional system with licensed software necessary for work. I attempted to use partimage to quickly copy the data, but found that the utility only copied a third of the disk, then terminated either with a segfault or some other amusing C/C++ quirk.
Then I tried copying the data using dd, and found that dd reached about the same point as partimage, then encountered an input/output error. Various fun flags like conv=noerr, skip, or other such things did not help at all.
However, I was able to copy the data to another disk without any issues using a GNU utility called ddrescue.
ddrescue /dev/sda1 /dev/sdb1
After that, my hair became silky, my wife returned, the kids, and the dog stopped biting the sofa.
A big advantage of ddrescue is the built-in progress bar, so there is no need to hack together some workarounds like pv and other not very pretty dd flags. Also, ddrescue shows the number of attempts to read the data; the wiki also says that the utility has some super algorithm for reading damaged data, we’ll leave that for people who like to dig into the source code, we are not among them, right?
https://en.wikipedia.org/wiki/Ddrescue
https://www.gnu.org/software/ddrescue/ddrescue.html
ChatGPT Overview
Hi all! In this article, I want to talk about ChatGPT, a powerful language modeling tool from OpenAI that can help solve various text-processing tasks. I will show how this tool works and how it can be used in practical situations. Let’s get started!
At the moment, ChatGPT is one of the best neural network-based language models in the world. It was created to help developers create intelligent systems that can generate natural language and communicate with people in it.
One of the key advantages of ChatGPT is its ability for contextual text modeling. This means that the model takes into account the previous dialogue and uses it to better understand the situation and generate a more natural response.
You can use ChatGPT for a variety of tasks such as customer support automation, chatbot creation, text generation, and more.
The neural networks behind ChatGPT have been trained on huge arrays of text to provide highly accurate predictions. This allows the model to generate natural text that can support dialogue and answer questions.
With ChatGPT, you can create your own chatbots and other intelligent systems that can interact with people in natural language. This can be especially useful in industries such as travel, retail, and customer support.
In conclusion, ChatGPT is a powerful tool for solving various language modeling problems. Its ability for contextual modeling makes it especially useful for building chatbots and intelligent systems.
In fact, everything that ChatGPT wrote above was completely written by itself. What? Yes? I’m shocked myself!
The network itself can be tested here:
https://chat.openai.com/chat