Category: Notes

  • Local image generation: ComfyUI and FLUX model

    Nowadays, you don’t have to rely on cloud services: you can generate high-quality images entirely on your own hardware. In this post, I will describe how to run the modern FLUX model locally on your computer using ComfyUI.

    ComfyUI uses node-based architecture. This allows you to:
    – Totally control every stage of generation.
    – Easily share ready-made “workflows”

    FLUX is a large model, so the hardware requirements are higher than SD 1.5 or SDXL:
    Video card (GPU): Nvidia RTX with 12 GB VRAM or higher (for comfortable work). If you have 8 GB or less, you will have to use the quantized versions (GGUF or NF4).
    Random access memory (RAM): minimum 16 GB (preferably 32 GB and above).
    Disk Space: Approximately 20–50 GB for models and components.

    The easiest way to start FLUX is to use a ready-made template. Just search for flux text to image in the workflows window and install.

    Write a prompt in English in the `Text to Image (Flux.1 Dev)` node, select the resolution (FLUX works well with 1024×1024 and even higher) and press RUN.

    The first generation may take time as the models will be loaded into the video card memory.

    https://github.com/comfyanonymous/ComfyUI

  • Local Vibe coding: LM Studio, VS Code and Continue

    If you had a desire to use neural networks to help write code (so-called Vibe coding), and you have a fairly powerful computer, for example with an Nvidia RTX video card, then you can deploy the entire environment absolutely free on your machine. This solves problems with paid subscriptions and allows you to safely work with projects under NDA, since your code is not sent anywhere. In this post I will describe how to assemble a local bundle of LM Studio, VS Code and the Continue extension.

    Tools for local Vibe coding

    For comfortable work we need three main components:
    LM Studio: a convenient application for downloading and running local LLMs. It takes on all the complexity of working with GGUF models and puts up a local server compatible with the OpenAI API.
    VS Code: a popular and familiar code editor.
    Continue: extension for VS Code that integrates neural networks directly into the work environment. Allows you to chat, highlight code for refactoring, and supports autocomplete.

    Hardware requirements

    Local language models are memory intensive:
    Video card (GPU): Nvidia with 8 GB VRAM or higher (for comfortable work with models with 7-8 billion parameters). Heavier models will require 16 GB of VRAM.
    Disk space: about 500 GB for storing various downloaded models.

    Configuring the link

    The setup process is quite simple and does not require complex manipulations in the terminal:
    1. Download and install LM Studio. Use the built-in search to find a lightweight model like Qwen Coder or gemma3:12b.
    2. In LM Studio, go to the Local Server tab and click Start Server. By default it will start on `http://localhost:1234/v1`.
    3. Open VS Code and install the Continue extension from the plugin store.
    4. Open the Continue configuration file and add a new model, specifying the `openai` provider and the address of your local server from LM Studio.

    You can then communicate with your local LLM directly in the Continue sidebar, ask questions about your code, and generate new components.

    Why does this work?

    As I wrote earlier, LLMs do better with flat structure and WET (Write Everything Twice) code. Local parameter models may be inferior to giants like GPT-4 when it comes to designing complex architectures, but they are more than capable of generating boilerplate code, refactoring simple functions, and rapid prototyping.

    Additionally, with local Vibe coding, your code never leaves the machine. This makes this combination ideal for corporate development and working with sensitive data.

    Output

    Local neural networks are not capable of fully replacing a programmer or designing a complex system. However, the combination of LM Studio + VS Code + Continue provides independence from cloud services and maintains privacy. This is a completely working auxiliary tool for routine tasks, if you are willing to put up with the limitations of small models and independently control the project architecture.

    Links

    https://code.visualstudio.com/
    https://lmstudio.ai/
    https://continue.dev/

    Sources

    https://youtu.be/IqqCwhG46jY
    https://www.youtube.com/watch?v=7AImkA96mE8

  • Local video generation: ComfyUI and LTX-2.3

    Previously, creating videos using neural networks was the prerogative of cloud services like Runway or Luma. Today, if you have a modern Nvidia graphics card, you can generate high-quality videos right on your computer. In this post, I will tell you how to set up local video generation using ComfyUI and the effective LTX-2.3 model.

    Tools for video generation

    For work we will need:
    ComfyUI: a powerful interface with a node-based architecture that allows you to flexibly customize the generation process.
    LTX-2.3: A modern model from Lightricks, optimized for creating smooth and detailed videos with relatively moderate video memory requirements.

    Hardware requirements

    Generating video is a much more resource-intensive process than working with images:
    Video card (GPU): Nvidia RTX with 8 GB VRAM is the minimum required for a resolution of 768×512. For comfortable operation and higher resolutions, it is highly desirable to have 16–24 GB of VRAM.
    Random access memory (RAM): minimum 32 GB. Video models and VAEs take up a lot of space when downloading.
    Disk space: about 500 GB for the model itself and related components.

    Setup and launch

    The process of launching LTX-2.3 in ComfyUI is as follows:
    1. Update ComfyUI: The model is relatively new, so make sure you have the latest version of the interface installed.
    2. Install Workflow: The easiest way is to find a ready-made JSON template for LTX Video. The model requires specific nodes to work with video latent space.
    3. Prompt and parameters: Enter a description of the scene in English. Note that the LTX-2.3 understands motion well (eg “camera orbits around”, “fast movement”).

    Why choose LTX-2.3?

    LTX-2.3 is notable because it delivers results comparable to proprietary cloud services, but runs locally. This gives you:
    Complete privacy: your prompts and generated videos do not go to other people’s servers.
    Control: you can experiment with frame rate (FPS), resolution and prompt strength without having to pay for each attempt.

    Local video generation is still in active development, and LTX-2.3 is a great entry into the world of “home Hollywood.”

    Links

    https://github.com/comfyanonymous/ComfyUI
    https://huggingface.co/Lightricks/LTX-Video

  • Local music generation: ComfyUI and ACE-Step-1.5 model

    Nowadays, you don’t have to rely on cloud services to create content: you can generate high-quality music entirely on your own hardware. In this post, I will describe how to run the modern ACE-Step-1.5 model locally on your computer using ComfyUI.

    ComfyUI uses node-based architecture. This allows you to:
    – Totally control every stage of audio generation.
    – Easily share ready-made “workflows”.

    ACE-Step-1.5 is an advanced model for music generation that requires significant computational resources. The hardware requirements are higher than those of many simple synthesizers:
    Video card (GPU): Nvidia RTX with 8 GB VRAM or higher (12 GB+ recommended) for comfortable work at high quality.
    Random access memory (RAM): minimum 16 GB (preferably 32 GB and above).
    Processor (CPU): Modern multi-core processor with good support for AVX/CUDA computing.
    Disk Space: Approximately 20–50 GB for models and components.

    The easiest way to run ACE-Step-1.5 is to use a ready-made audio generation template. Just search for music text to audio in the workflows window and install.

    Write a prompt describing the genre and mood (for example, “uplifting synthwave track with heavy bass”) in the `Prompt Input` node. Specify the desired duration and press RUN.
    The first generation may take time, as the models will be loaded into the video card memory and process complex acoustic patterns.

    https://github.com/comfyanonymous/ComfyUI
    https://www.youtube.com/watch?v=UAlLD5fS7-c

  • Local neural networks using ollama

    If you had a desire to launch something like ChatGPT and you have a fairly powerful computer, for example with an Nvidia RTX video card, then you can run the ollama project, which will allow you to use one of the ready-made LLM models on your local machine, absolutely free. ollama provides the ability to communicate with LLM models, in the manner of ChatGPT; also in the latest version, the ability to read images and format the output data in json format has been announced.

    I also ran the project itself on a MacBook with an Apple M2 processor, and I know that the latest models of video cards from AMD are supported.

    To install on macOS, go to the ollama website:
    https://ollama.com/download/mac

    Click “Download for macOS”, you will download an archive of the form ollama-darwin.zip, inside the archive there will be Ollama.app which needs to be copied to “Applications”. After this, launch Ollama.app, most likely the installation process will occur on the first launch. After that, in the tray you saw the ollama icon, the tray is on the top right next to the clock.

    After that, launch a regular macOS terminal, and type the command to download, install and run any ollama model. A list of available models, descriptions, and their characteristics can be seen on the ollama website:
    https://ollama.com/search

    Choose the model with the fewest parameters if it does not fit into your video card at launch.

    For example, the command to launch the llama3.1:latest model:

    ollama run llama3.1:latest
    

    Installation for Windows and Linux is generally similar, in one case there will be an ollama installer and further work with it via Powershell.
    For Linux, installation is done using a script, but I recommend using the version of your specific package manager. On Linux, ollama can also be launched via a regular bash terminal.

    Sources
    https://www.youtube.com/watch?v=Wjrdr0NU4Sk
    https://ollama.com

  • Video stabilization using ffmpeg

    If you want to stabilize videos and remove camera shake, the `ffmpeg` tool offers a powerful solution. Thanks to the built-in filters `vidstabdetect` and `vidstabtransform`, you can achieve professional results without using complex video editors.

    Preparing for work

    Before you start, make sure your `ffmpeg` supports the `vidstab` library. On Linux you can check this with the command:

    bash  
    ffmpeg -filters | grep vidstab  
    

    If the library is not installed, you can add it:

    sudo apt install ffmpeg libvidstab-dev  
    

    Installation for macOS via brew:

    brew install libvidstab
    brew install ffmpeg
    

    Now let’s move on to the process.

    Step 1: Motion Analysis

    First you need to analyze the motion of the video and create a file with stabilization parameters.

    ffmpeg -i input.mp4 -vf vidstabdetect=shakiness=10:accuracy=15 transfile=transforms.trf -f null -  
    

    Parameters:

    shakiness: Video shake level (default 5, can be increased to 10 for more complex cases).
    accuracy: Analysis accuracy (default 15).
    transfile: File name to save the motion parameters.

    Step 2: Apply Stabilization

    Now you can apply stabilization using the transformation file:

    ffmpeg -i input.mp4 -vf vidstabtransform=input=transforms.trf:zoom=5 output.mp4
    

    Parameters:

    input: Points to the file with transformation parameters (created in the first step).
    zoom: Zoom factor to remove black edges (e.g. 5 – auto zoom until artifacts are removed).

  • Turing computing machines

    I present to your attention a translation of the first pages of Alan Turing’s article “ON COMPUTABLE NUMBERS WITH AN APPLICATION TO THE PROBLEM OF RESOLUTION” from 1936. The first chapters contain a description of computers, which later became the basis for modern computing.

    The full translation of the article and explanation can be read in the book by American popularizer Charles Petzold, entitled “Reading Turing: A Journey through Turing’s Historical Article on Computability and Turing Machines” (ISBN 978-5-97060-231-7, 978-0-470-22905-7)

    Original article:
    https://www.astro.puc.cl/~rparra/tools/PAPERS/turing_1936.pdf

    ON COMPUTABLE NUMBERS WITH APPLICATION TO THE RESOLUTION PROBLEM

    A. M. TURING

    [Received May 28, 1936 – Read November 12, 1936]

    “Computable” numbers can be briefly described as real numbers whose expressions as decimal fractions are calculable in a finite number of ways. Although at first glance this article treats numbers as computable, it is almost as easy to define and explore computable functions of an integer variable, a real variable, a computable variable, computable predicates, and the like. However, the fundamental problems associated with these computable objects are the same in each case. For a detailed consideration, I chose computable numbers as a computable object because the method of considering them is the least cumbersome. I hope to soon describe the relationship of computable numbers with computable functions and so on. At the same time, research will be carried out in the field of the theory of functions of a real variable expressed in terms of computable numbers. By my definition, a real number is computable if its decimal representation can be written by a machine.

    In paragraphs 9 and 10 I give some arguments to show that computable numbers include all numbers that are naturally thought to be computable. In particular, I show that some large classes of numbers are computable. They include, for example, the real parts of all algebraic numbers, the real parts of the zeros of Bessel functions, the numbers π, e and others. However, computable numbers do not include all definable numbers, as evidenced by the following example of a definable number that is not computable.

    Although the class of computable numbers is very large and in many respects similar to the class of real numbers, it is still enumerable. In §8 I consider certain arguments that would seem to argue to the contrary. When one of these arguments is correctly applied, conclusions are drawn that, at first glance, are similar to those of Gödel*. These results have extremely important applications. In particular, as shown below (§11), the resolution problem cannot have a solution.

    In a recent article, Alonzo Church introduced the idea of ​​“effective calculability,” which is equivalent to my idea of ​​“computability” but has a completely different definition. Church also comes to similar conclusions regarding the problem of resolution. The proof of the equivalence of “computability” and “effectively calculable” is presented in the appendix to this article.

    1. Computers

    We have already said that computable numbers are those numbers whose decimal places are countable by finite means. A clearer definition is needed here. This article will make no real attempt to justify the definitions given here until we get to §9. For now, I will just note that the (logical) rationale (for this) is that human memory is, by necessity, limited.

    Let us compare a person in the process of calculating a real number with a machine that is capable of fulfilling only a finite number of conditions q1, q2, …, qR; Let’s call these conditions “m-configurations”. This (that is, so defined) machine is equipped with a “tape” (analogous to paper). This belt passing through the machine is divided into sections. Let’s call them “squares”. Each such square can contain some kind of “symbol”. At any moment, there is only one such square, say the rth one, containing the symbol that is “in this machine.” Let’s call such a square a “scanned symbol”. A “scanned character” is the only character that the machine is, so to speak, “directly aware” of. However, by changing its m-configuration, the machine can effectively remember some of the characters it has “seen” (scanned) previously. The possible behavior of the machine at any moment is determined by the m-configuration qn and the scanned symbol***. Let’s call this pair of symbols qn, “configuration”. The configuration thus designated determines the possible behavior of a given machine. In some of these configurations in which the scanned square is blank (ie, does not contain a character), the machine writes a new character on the scanned square, and in other of these configurations it erases the scanned character. This machine is also capable of moving to scan another square, but in this way it can only move to the adjacent square to the right or left. In addition to any of these operations, the m-configuration of the machine can be changed. In this case, some of the written characters will form a sequence of digits, which is the decimal part of the real number being calculated. The rest of them will be nothing more than inaccurate marks in order to “help memory”. In this case, only the above-mentioned inaccurate marks can be erased.

    I claim that the operations considered here include all those operations that are used in calculation. The rationale for this statement is easier to understand for the reader who has an understanding of machine theory. Therefore, in the next section I will continue to develop the theory in question, based on an understanding of the meaning of the terms “machine”, “tape”, “scanned”, etc.

    *Gödel “On the Formally Undecidable Sentences of the Principia Mathematics (published by Whitehead and Russell in 1910, 1912 and 1913) and Related Systems, Part I,” Journal of Mathematics. Physics, monthly bulletin in German No. 38 (for 1931, pp. 173-198.
    ** Alonzo Church, “An Undecidable Problem in Elementary Number Theory,” American J. of Math., No. 58 (1936), pp. 345-363.
    *** Alonzo Church, “A Note on the Resolution Problem,” J. of Symbolic Logic, No. 1 (1936), pp. 40-41