Running LLama on linux CPU | Ubuntu 20.04

LLAMA.cpp, when paired with the CodeLlama 13B model, becomes a potent tool for a wide range of tasks, from code translation to natural language processing. In this comprehensive guide, we will take you through the entire process, from installing LLAMA.cpp to using it with the CodeLlama 13B model on Ubuntu 20.04.

Step 1: Prepare Your Ubuntu System

Ensure you have Ubuntu 20.04 or a compatible version installed on your device. LLAMA.cpp and the CodeLlama 13B model work seamlessly with this operating system.

Step 2: Install Required Dependencies

Before getting started, you’ll need to make sure your system has the necessary dependencies. Open a terminal and run the following command:

sudo apt-get install g++ make cmake git wget

This command installs essential tools such as g++, make, cmake, git, and wget, which are needed for the subsequent steps.

Step 3: Clone the LLAMA.cpp Repository

To begin, you need to clone the LLAMA.cpp repository from GitHub. Use the following commands to accomplish this:

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp

This action downloads the LLAMA.cpp source code to your local machine.

Step 4: Navigate to the LLAMA.cpp Source Directory

Now, navigate to the directory containing the LLAMA.cpp source code:

cd ~/llama.cpp

Step 5: Configure the Compilation Process

Set up the compilation process with CMake:

cmake .

This step ensures that the code is ready for compilation.

Step 6: Compile the Source Code

Compile the LLAMA.cpp source code with the following command:

make

This will generate an executable file named “main” in the LLAMA.cpp directory.

Step 7: Download the CodeLlama 13B Model

Head to the directory where you cloned the LLAMA.cpp repository and download the CodeLlama 13B model using wget:

cd ~/llama.cpp/models
wget https://huggingface.co/TheBloke/CodeLlama-13B-Instruct-GGUF/resolve/main/codellama-13b-instruct.Q5_K_M.gguf

This action retrieves the necessary model for code-related tasks.

Step 8: Execute CodeLlama 13B

You are now ready to harness the capabilities of CodeLlama 13B with the model you’ve downloaded. Ensure you’re in the right directory and use the following command:

cd ~/llama.cpp
./main -m ./models/codellama-13b-instruct.Q5_K_M.gguf -n 16 -p "Translate the following Python code into JavaScript:"

Feel free to adjust the -n flag for different batch sizes and -p for various prompts, depending on your specific requirements.

With these steps completed, you have LLAMA.cpp and the CodeLlama 13B model fully operational on your Ubuntu 20.04 system. You are now equipped to handle an array of tasks, from code translation to advanced natural language processing. Enjoy the power and flexibility this combination offers for your projects and tasks!