If you wanted to run something like ChatGPT and you have a powerful enough computer, for example with an Nvidia RTX video card, then you can run the ollama project, which will allow you to use one of the ready-made LLM models on a local machine, absolutely free. ollama provides the ability to communicate with LLM models, like ChatGPT, and the latest version also announced the ability to read images, format output data in json format.
I also launched the project on a MacBook with an Apple M2 processor, and I know that the latest models of AMD video cards are supported.
To install on macOS, visit the ollama website:
https://ollama.com/download/mac
Click “Download for macOS”, you will download an archive of the form ollama-darwin.zip, inside the archive there will be Ollama.app which you need to copy to “Applications”. After that, launch Ollama.app, most likely the installation process will occur at the first launch. After that, you saw the ollama icon in the tray, the tray is on the right top next to the clock.
After that, launch a regular macOS terminal and type the command to download, install and launch any ollama model. The list of available models, descriptions, and their characteristics can be found on the ollama website:
https://ollama.com/search
Choose the model with the least number of parameters if it does not fit into your video card at startup.
For example, the commands to launch the llama3.1:latest model:
ollama run llama3.1:latest
Installation for Windows and Linux is generally similar, in one case there will be an ollama installer and further work with it through Powershell.
For Linux, the installation is done by a script, but I recommend using the version of your specific package manager. In Linux, ollama can also be launched through a regular bash terminal.
Sources
https://www.youtube.com/watch?v=Wjrdr0NU4Sk
https://ollama.com
