6.5.2. Python bindings for llama.cpp¶

GitHub: https://github.com/abetlen/llama-cpp-python

安装¶

前提:

# 先安装libgomp
conda install -n llamacpp -c conda-forge libgomp
# 显式设置 LIBRARY_PATH 和 LD_LIBRARY_PATH
export LIBRARY_PATH=$CONDA_PREFIX/lib:$LIBRARY_PATH
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib

# or
sudo apt-get update
sudo apt-get install libgomp1

:$LD_LIBRARY_PATH

安装包:
```
# CPU版
pip install llama-cpp-python
```

验证:

#!/bin/bash

echo "llamacpp environment is ready"
python -c "import llama_cpp; print('llama_cpp version:', llama_cpp.__version__)"
exec "$@"

High-level API¶