PrivateGPT Installation and Use
Steps on how to install and use privateGPT
Private GPT is a tool to ask questions to your documents without having an internet conection. It uses the power of LLMs, it is 100% private and no data leavs your execution environment at any point. You do not need internet connection to ask question to your document corpus.
Follow the steps to install python and setup privateGPT (For Debian 12)
1. Install python virtual environment
sudo apt install python3-venv
2. Clone privateGPT repository
git clone github.com/imartinez/privateGPT.git
3. Create a virtual environment and install required libraries
Enter the privateGPT folder and create python's virtual environment and install the requirement python dependencies
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt
4. Copy environment template
Copy the example.env template into .env. Make any changes if necessary
cp example.env .env
Here is the description of the environment file
MODEL_TYPE: supports LlamaCpp or GPT4All
PERSIST_DIRECTORY: is the folder you want your vectorstore in
MODEL_PATH: Path to your GPT4All or LlamaCpp supported LLM
MODEL_N_CTX: Maximum token limit for the LLM model
MODEL_N_BATCH: Number of tokens in the prompt that are fed into the model at a time. Optimal value differs a lot depending on the model (8 works well for GPT4All, and 1024 is better for LlamaCpp)
EMBEDDINGS_MODEL_NAME: SentenceTransformers embeddings model name (see www.sbert.net/docs/pretrained_models.html)
TARGET_SOURCE_CHUNKS: The amount of chunks (sources) that will be used to answer a question
5. Download models
Create a folder named models inside the privateGPT folder and download the following files
mkdir models
cd models
wget gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin
wget huggingface.co/Pi3141/alpaca-native-7B-ggml/resolve/397e872bf4c83f4c642317a5bf65ce84a105786e/ggml-model-q4_0.bin
6. Copy all your documents into source_documents folder
The supported extensions are:
- .csv: CSV,
- .docx: Word Document,
- .doc: Word Document,
- .enex: EverNote,
- .eml: Email,
- .epub: EPub,
- .html: HTML File,
- .md: Markdown,
- .msg: Outlook Message,
- .odt: Open Document Text,
- .pdf: Portable Document Format (PDF),
- .pptx : PowerPoint Document,
- .ppt : PowerPoint Document,
- .txt: Text file (UTF-8),
6. Now ingest all the data.
Run the following to ingest all the data that you have stored in source_documents folder
python ingest.py
The output will be something like this:
Creating new vectorstore
Loading documents from source_documents
Loading new documents: 100%|██████████████████████| 1/1 [00:01<00:00, 1.73s/it]
Loaded 1 new documents from source_documents
Split into 90 chunks of text (max. 500 tokens each)
Creating embeddings. May take some minutes...
Using embedded DuckDB with persistence: data will be stored in: db
Ingestion complete! You can now run privateGPT.py to query your documents
It will create a db folder containing the local vectorstore. Will take 20-30 seconds per document, depending on the size of the document. You can ingest as many documents as you want, and all will be accumulated in the local embeddings database. If you want to start from an empty database, delete the db folder.
Note: during the ingest process no data leaves your local environment. You could ingest without an internet connection, except for the first time you run the ingest script, when the embeddings model is downloaded.
7. Now ask a question.
Use the following command to ask a question:
python privateGPT.py
Source: