\u5176\u4e2d\u7684\u6838\u5fc3\u5728\u65bc Langchain\uff0c\u5b83\u662f\u7528\u65bc\u958b\u767c\u7531\u8a9e\u8a00\u6a21\u578b\u652f\u63f4\u7684\u61c9\u7528\u7a0b\u5f0f\u7684\u6846\u67b6\uff0cLangChain \u5c31\u50cf\u81a0\u6c34\u4e00\u6a23\uff0c\u6709\u5404\u7a2e\u63a5\u53e3\u53ef\u4ee5\u5c07 LLM \u6a21\u578b\u8207\u5176\u4ed6\u5de5\u5177\u548c\u6578\u64da\u6e90\u9023\u63a5\uff0c\u4e0d\u904e\u73fe\u5728 LangChain \u6b63\u5728\u84ec\u52c3\u767c\u5c55\u4e2d\uff0c\u8a31\u591a\u6587\u4ef6\u6216 API \u6539\u7248\u5f88\u591a\uff0c\u4ee5\u4e0b\u6211\u4f7f\u7528\u6700\u7c21\u55ae\u7684\u65b9\u5f0f\u793a\u7bc4\u3002<\/p>\n\n\n\n
\u6b65\u9a5f1. \u74b0\u5883\u8a2d\u7f6e<\/h1>\n\n\n\n
\u9996\u5148\u8a2d\u7f6e python \u74b0\u5883\uff0c\u6211\u4f7f\u7528 conda \u5275\u5efa\u74b0\u5883\uff0c\u4e26\u5b89\u88dd\u4ee5\u4e0b library\uff0c\u6211\u5728 Jupyter \u74b0\u5883\u5b8c\u6210\u7bc4\u4f8b\uff0c\u5b8c\u6574 code \u53ef\u4ee5\u5728 github \u627e\u5230\u3002<\/p>\n\n\n\n
# python=3.9
ipykernel
ipywidgets
langchain
PyMuPDF
chromadb
sentence-transformers
llama-cpp-python<\/pre>\n\n\n\n\u6b65\u9a5f2. \u8b80\u5165\u6a94\u6848\u8655\u7406\u4e26\u532f\u5165 DB<\/h1>\n\n\n
\n<\/figure>\n<\/div>\n\n\n
\u9996\u5148\u6211\u5011\u8981\u5c07\u5916\u90e8\u8cc7\u8a0a\u8655\u7406\u5f8c\uff0c\u653e\u5230 DB \u4e2d\uff0c\u4ee5\u4f9b\u4e4b\u5f8c\u67e5\u8a62\u76f8\u95dc\u77e5\u8b58\uff0c\u9019\u908a\u7684\u6b65\u9a5f\u5c0d\u61c9\u5230\u4e0a\u5716\u6846\u8d77\u4f86\u7684\u90e8\u5206\uff0c\u4e5f\u5c31\u662f\u6a58\u8272\u7684 1. Text splitter \u8207 2. Embedding<\/p>\n\n\n\n
a). \u4f7f\u7528\u6587\u4ef6\u8f09\u5165\u5668<\/h1>\n\n\n\n
Langchain \u63d0\u4f9b\u4e86\u5f88\u591a\u6587\u4ef6\u8f09\u5165\u5668\uff0c\u7e3d\u8a08\u5927\u6982\u6709 55 \u7a2e\uff0c\u5305\u542b word, csv, PDF, GoogleDrive, Youtube \u7b49\uff0c\u4f7f\u7528\u65b9\u5f0f\u4e5f\u5f88\u7c21\u55ae\uff0c\u9019\u908a\u6211\u5275\u5efa\u4e86\u4e00\u500b\u865b\u64ec\u4eba\u7269 Alison Hawk \u7684 PDF \u8cc7\u8a0a\uff0c\u4e26\u4f7f\u7528
PyMuPDFLoader<\/code>\u8b80\u5165\uff0cAlison Hawk \u7684PDF \u8cc7\u8a0a\u53ef\u4ee5\u5728 github \u67e5\u770b\u3002\u6ce8\u610f PyMuPDFLoader \u9700\u8981\u5b89\u88ddPyMuPDF<\/code> \u624d\u80fd\u4f7f\u7528\u3002<\/p>\n\n\n\nfrom langchain.document_loaders import PyMuPDFLoader
loader = PyMuPDFLoader(\"LangChain\/Virtual_characters.pdf\")
PDF_data = loader.load()<\/pre>\n\n\n\nb). \u4f7f\u7528 Text splitter \u5206\u5272\u6587\u4ef6<\/h1>\n\n\n\n
Text splitter \u6703\u5c07\u6587\u4ef6\u6216\u6587\u5b57\u5206\u5272\u6210\u4e00\u500b\u500b chunk\uff0c\u7528\u4ee5\u9810\u9632\u6587\u4ef6\u7684\u8cc7\u8a0a\u8d85\u904e LLM \u7684 tokens\uff0c\u6709\u4e00\u4e9b\u7814\u7a76\u5728\u63a2\u8a0e\u5982\u4f55\u5c07 chunk \u512a\u5316\uff0c\u53ef\u4ee5\u53c3\u8003\u6211\u7684\u53e6\u4e00\u7bc7\u6587\u7ae0\u3002<\/p>\n\n\n\n
\u9019\u6b65\u9a5f\u5e38\u7528\u7684\u5de5\u5177
RecursiveCharacterTextSplitter<\/code> \u8207CharacterTextSplitter<\/code>\uff0c\u5dee\u5225\u5728\u65bcRecursiveCharacterTextSplitter<\/code> \u5982\u679c\u5340\u584a\u5927\u5c0f\u8d85\u904e\u6307\u5b9a\u95be\u503c\uff0c\u5b83\u4e5f\u6703\u905e\u6b78\u5730\u5c07\u6587\u5b57\u5206\u5272\u6210\u66f4\u5c0f\u7684\u5340\u584a\u3002LangChain 2\u7a2e\u65b9\u5f0f\u7686\u6709\u63d0\u4f9b\uff0c\u53e6\u5916\u4e3b\u8981\u53c3\u6578\u70ba\u4ee5\u4e0b\uff1a<\/p>\n\n\n\n\n
- chunk_size\uff1a\u6c7a\u5b9a\u5206\u5272\u6587\u5b57\u6642\u6bcf\u500b\u5340\u584a\u4e2d\u7684\u6700\u5927\u5b57\u5143\u6578\u3002\u5b83\u6307\u5b9a\u6bcf\u500b\u5340\u584a\u7684\u5927\u5c0f\u6216\u9577\u5ea6\u3002<\/li>\n\n\n\n
- chunk_overlap\uff1a\u6c7a\u5b9a\u5206\u5272\u6587\u5b57\u6642\u9023\u7e8c\u5340\u584a\u4e4b\u9593\u91cd\u758a\u7684\u5b57\u5143\u6578\u3002\u5b83\u6307\u5b9a\u524d\u4e00\u500b\u5340\u584a\u7684\u591a\u5c11\u61c9\u5305\u542b\u5728\u4e0b\u4e00\u500b\u5340\u584a\u4e2d\u3002<\/li>\n<\/ul>\n\n\n\n
from langchain.text_splitter import RecursiveCharacterTextSplitter
text_splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=5)
all_splits = text_splitter.split_documents(PDF_data)<\/pre>\n\n\n\nc). \u8f09\u5165 Embedding model<\/h1>\n\n\n\n
\u63a5\u8457\u4f7f\u7528 Embedding \u5c07\u6b65\u9a5f b)\u5206\u5272\u7684 chunk \u6587\u5b57\u8f49\u63db\u70ba\u5411\u91cf\uff0cLangChain \u63d0\u4f9b\u4e86\u8a31\u591a Embedding model \u7684\u63a5\u53e3\uff0c\u5982OpenAI\u3001Cohere\u3001Hugging Face\u3001Weaviate \u7b49\uff0c\u53ef\u4ee5\u53c3\u8003 LangChain\u5b98\u7db2\u3002<\/p>\n\n\n\n
\u9019\u908a\u6211\u4f7f\u7528 Hugging Face \u7684 Sentence Transformers\uff0c\u5b83\u63d0\u4f9b\u4e86\u8a31\u591a\u7a2e pretrain \u6a21\u578b\uff0c\u53ef\u4ee5\u6839\u64da\u4f60\u7684\u9700\u6c42\u6216\u61c9\u7528\u60c5\u5883\u9078\u64c7\uff0c\u6211\u9078\u64c7
all-MiniLM-L6-v2<\/code> \uff0c\u5176\u4ed6 model \u7d30\u7bc0\u53ef\u4ee5\u770b\u5230 SBERT.net \u6216 HuggingFace\u3002\u6ce8\u610f\u8981\u5148\u5b89\u88ddsentence-Tranformers<\/code> \u624d\u80fd\u4f7f\u7528\u3002<\/p>\n\n\n\nfrom langchain.embeddings import HuggingFaceEmbeddings
model_name = \"sentence-transformers\/all-MiniLM-L6-v2\"
model_kwargs = {'device': 'cpu'}
embedding = HuggingFaceEmbeddings(model_name=model_name,
model_kwargs=model_kwargs)<\/pre>\n\n\n\nd). \u5c07 Embedding \u7d50\u679c\u532f\u5165 VectorDB<\/h1>\n\n\n\n
Embedding \u5f8c\u7684\u7d50\u679c\u6211\u5011\u6703\u5132\u5b58\u5728 VectorDB \u4e2d\uff0c\u5e38\u898b\u7684 VectorDB \u6709 Chroma\u3001Pinecone\u3001FAISS\u7b49\uff0c\u9019\u908a\u6211\u4f7f\u7528 Chroma \u4f86\u5be6\u4f5c\u3002Chroma \u8ddf LangChain \u7684\u6574\u5408\u5f97\u5f88\u597d\uff0c\u53ef\u4ee5\u76f4\u63a5\u4f7f\u7528 LangChain \u7684\u63a5\u53e3\u4f86\u505a\u3002<\/p>\n\n\n\n
# Embed and store the texts
# Supplying a persist_directory will store the embeddings on disk
from langchain.vectorstores import Chroma
persist_directory = 'db'
vectordb = Chroma.from_documents(documents=all_splits, embedding=embedding, persist_directory=persist_directory)<\/pre>\n\n\n\n\u6b65\u9a5f3. \u555f\u7528 LLM \u670d\u52d9<\/h1>\n\n\n
\n<\/figure>\n<\/div>\n\n\n
\u6709\u5169\u7a2e\u65b9\u6cd5\u555f\u52d5\u4f60\u7684 LLM \u6a21\u578b\u4e26\u9023\u63a5\u5230 LangChain\u3002\u4e00\u662f\u4f7f\u7528 LangChain \u7684 LlamaCpp \u63a5\u53e3\u4f86\u5be6\u4f5c\uff0c\u9019\u6642\u5019\u662f\u7531 LangChain \u5e6b\u4f60\u628a llama2 \u670d\u52d9\u555f\u52d5\uff1b\u53e6\u4e00\u500b\u65b9\u6cd5\u662f\u7528\u5176\u4ed6\u65b9\u5f0f\u67b6\u8a2d Llama2 \u7684 API \u670d\u52d9\uff0c\u4f8b\u5982\u4f7f\u7528 llama.cpp \u7684 server \u555f\u52d5 API \u670d\u52d9\u7b49\uff0c\u9019\u90e8\u5206\u7d30\u7bc0\u53ef\u4ee5\u770b\u5230\u6211\u7684\u53e6\u4e00\u7bc7 llama.cpp\u6559\u5b78\u3002\u6211 2 \u500b\u65b9\u5f0f\u90fd\u6703\u793a\u7bc4\uff0c\u4f60\u53ef\u4ee5\u9078\u64c7\u9069\u5408\u4f60\u7684\u65b9\u6848\u3002<\/p>\n\n\n\n
a). \u4f7f\u7528 LangChain \u7684 LlamaCpp<\/h1>\n\n\n\n
\u4f7f\u7528 LlamaCpp \u63a5\u53e3\u8f09\u5165 model\uff0c\u5b83\u6703\u5e6b\u4f60\u555f\u52d5 Llama \u7684\u670d\u52d9\uff0c\u9019\u65b9\u6cd5\u8f03\u7c21\u55ae\uff0c\u76f4\u63a5\u4f7f\u7528\u4e0b\u9762 code \u5c31\u53ef\u4ee5\u57f7\u884c\uff0cmodel_path \u6307\u5b9a\u5230\u4f60\u7684\u6a21\u578b\u4e2d\uff0c\u4f8b\u5b50\u4e2d\u6211\u4f7f\u7528\u91cf\u5316\u904e\u5f8c\u7684 Llama2 Chat\u3002\u6ce8\u610f\u9019\u908a\u8981\u5b89\u88dd
llama-cpp-python<\/code><\/p>\n\n\n\nfrom langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain_community.llms import LlamaCpp
model_path = \"llama.cpp\/models\/llama-2-7b-chat\/llama-2_q4.gguf\"
llm = LlamaCpp(
model_path=model_path,
n_gpu_layers=100,
n_batch=512,
n_ctx=2048,
f16_kv=True,
callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
verbose=True,
)<\/pre>\n\n\n\n<\/figure>\n<\/div>\n\n\n
\u53ef\u4ee5\u4f7f\u7528\u6e2c\u8a66\u770b\u770b\u6709\u6c92\u6709 llm \u670d\u52d9\u555f\u52d5\u6c92\uff1a<\/p>\n\n\n\n
llm(\"What is Taiwan known for?\")<\/pre>\n\n\n\n<\/figure>\n<\/div>\n\n\n
b). \u4f7f\u7528\u5df2\u67b6\u8a2d\u7684 API \u670d\u52d9<\/h1>\n\n\n\n
\u5982\u679c\u4f60\u5df2\u7d93\u4f7f\u7528\u5176\u4ed6\u65b9\u5f0f\u67b6\u8a2d LLM \u7684 API \u670d\u52d9\uff0c\u6216\u8005\u662f\u4f7f\u7528 openai \u7684 API \u7684\u8a71\uff0c\u4f60\u9700\u8981\u4f7f\u7528 LangChain \u7684 ChatOpenAI \u63a5\u53e3\u3002\u6211\u9019\u908a\u793a\u7bc4\u662f llama.cpp \u7684 server \u670d\u52d9 (llama.cpp\u6559\u5b78)\uff0c\u5b83\u63d0\u4f9b\u4e86\u985e\u5225OpenAI\u7684API\uff0c\u56e0\u6b64\u6211\u5011\u80fd\u76f4\u63a5\u7528\u540c\u500b\u63a5\u53e3\u4f86\u64cd\u4f5c\uff0c\u4ee5\u4e0b\u662f\u8a72\u63a5\u53e3\u7684\u4e00\u4e9b\u76f8\u95dc\u53c3\u6578\uff1a<\/p>\n\n\n\n
\n
- open_ai_key<\/strong>\uff1a\u7531\u65bc\u4e26\u6c92\u6709\u4f7f\u7528\u771f\u6b63\u7684 OpenAI API\uff0c\u56e0\u6b64\u53ef\u4ee5\u96a8\u610f\u586b\u5beb\u3002<\/li>\n\n\n\n
- openai_api_base<\/strong>\uff1a\u70ba\u6a21\u578bAPI\u7684Base URL<\/li>\n\n\n\n
- max_tokens\uff1a<\/strong>\u898f\u7bc4\u6a21\u578b\u56de\u7b54\u7684\u9577\u5ea6<\/li>\n<\/ul>\n\n\n\n
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(openai_api_key='None', openai_api_base='http:\/\/127.0.0.1:8080\/v1')<\/pre>\n\n\n\n\u6b65\u9a5f4. \u8a2d\u5b9a\u4f60\u7684 Prompt<\/h1>\n\n\n\n
\u4e00\u4e9b LLM \u53ef\u4ee5\u4f7f\u7528\u7279\u5b9a\u7684 Prompt\u3002\u4f8b\u5982\uff0cLlama \u53ef\u4f7f\u7528\u7279\u6b8a token\uff0c\u7d30\u7bc0\u53ef\u4ee5\u53c3\u8003 twitter\u3002\u6211\u5011\u53ef\u4ee5\u4f7f\u7528 ConditionalPromptSelector \u6839\u64da\u6a21\u578b\u985e\u578b\u8a2d\u5b9a Prompt\uff0c\u5982\u4ee5\u4e0b\uff1a<\/p>\n\n\n\n
from langchain.chains import LLMChain
from langchain.chains.prompt_selector import ConditionalPromptSelector
from langchain.prompts import PromptTemplate
DEFAULT_LLAMA_SEARCH_PROMPT = PromptTemplate(
input_variables=[\"question\"],
template=\"\"\"<<SYS>> \\n You are an assistant tasked with improving Google search \\
results. \\n <<\/SYS>> \\n\\n [INST] Generate THREE Google search queries that \\
are similar to this question. The output should be a numbered list of questions \\
and each should have a question mark at the end: \\n\\n {question} [\/INST]\"\"\",
)
DEFAULT_SEARCH_PROMPT = PromptTemplate(
input_variables=[\"question\"],
template=\"\"\"You are an assistant tasked with improving Google search \\
results. Generate THREE Google search queries that are similar to \\
this question. The output should be a numbered list of questions and each \\
should have a question mark at the end: {question}\"\"\",
)
QUESTION_PROMPT_SELECTOR = ConditionalPromptSelector(
default_prompt=DEFAULT_SEARCH_PROMPT,
conditionals=[(lambda llm: isinstance(llm, LlamaCpp), DEFAULT_LLAMA_SEARCH_PROMPT)],
)
prompt = QUESTION_PROMPT_SELECTOR.get_prompt(llm)
prompt<\/pre>\n\n\n\n\u4f7f\u7528 LLMChain \u5c07 prompt \u8207 llm \u63a5\u5728\u4e00\u8d77\uff0c\u53e6\u5916 LangChain \u6700\u8fd1\u7684\u6539\u7248\u4f7f\u7528
invoke<\/code> \u66ff\u4ee3run<\/code> \uff0c\u7576\u4f60\u770b\u5230\u5176\u4ed6\u6587\u7ae0\u4f7f\u7528run<\/code> \u6642\u53ef\u4ee5\u6ce8\u610f\u3002<\/p>\n\n\n\nllm_chain = LLMChain(prompt=prompt, llm=llm)
question = \"What is Taiwan known for?\"
llm_chain.invoke({\"question\": question})<\/pre>\n\n\n\n\u5be6\u6a5f\u756b\u9762\uff1a<\/p>\n\n\n
\n<\/figure>\n<\/div>\n\n\n
\u6b65\u9a5f5. Text Retrieval + Query LLM<\/h1>\n\n\n
\n<\/figure>\n<\/div>\n\n\n
\u4e0a\u9762\u6211\u5011\u5c07 PDF \u8cc7\u8a0a\u532f\u5165 DB \u4e2d\uff0c\u4e26\u4e14\u555f\u52d5\u4e86 LLM \u670d\u52d9\uff0c\u63a5\u4e0b\u4f86\u6211\u5011\u8981\u5c07\u6574\u500b RAG \u7684\u6b65\u9a5f\u4e32\u8d77\u4f86\uff1a<\/p>\n\n\n\n
\n
- User \u767c\u4f86 QA<\/li>\n\n\n\n
- \u5f9e DB \u4e2d Text Retrieval<\/li>\n\n\n\n
- \u7d50\u5408 QA \u8207 Text Retrieval \u767c\u9001\u7d66 LLM<\/li>\n\n\n\n
- LLM \u6839\u64da\u8cc7\u8a0a\u56de\u7b54<\/li>\n<\/ol>\n\n\n\n
\u9996\u5148\u6211\u5011\u8981\u5148\u5275\u5efa Retriever\uff0c\u5b83\u53ef\u4ee5\u6839\u64da\u975e\u7d50\u69cb\u5316QA \u8fd4\u56de\u76f8\u61c9\u6587\u4ef6\uff0cLangChain \u63d0\u4f9b\u4e86\u5f88\u591a\u7a2e\u65b9\u5f0f\uff0c\u4e5f\u6574\u5408\u9032\u7b2c\u4e09\u65b9\u7684\u5de5\u5177\uff0c\u76ee\u524d\u6709\u5f88\u591a\u7814\u7a76\u5728\u63a2\u8a0e\u5982\u4f55\u6839\u64da QA \u627e\u5c0b\u5c0d\u61c9\u7684\u6587\u4ef6\u3002\u6211\u9019\u908a\u4f7f\u7528\u7684\u662f\u00a0Vectorstore\u00a0\u7684\u65b9\u5f0f\uff0c\u5176\u4ed6\u7a2e\u985e\u53ef\u4ee5\u53c3\u8003\u00a0Retrievers\u3002<\/p>\n\n\n\n
\u63a5\u8457\u4f7f\u7528 RetrievalQA \u7d50\u5408 Retriever \u8207 QA \u8207 llm\uff0c\u6ce8\u610f
VectorDBQA<\/code> \u7684\u529f\u80fd\u5df2\u7d93\u88ab\u68c4\u7528\uff0c\u73fe\u5728\u90fd\u4f7f\u7528RetrievalQA<\/code>\uff0c\u5982\u679c\u770b\u5230\u5225\u7684\u6587\u7ae0\u4f7f\u7528\u53ef\u4ee5\u6ce8\u610f\u3002<\/p>\n\n\n\nretriever = vectordb.as_retriever()
qa = RetrievalQA.from_chain_type(
llm=llm,
chain_type=\"stuff\",
retriever=retriever,
verbose=True
)<\/pre>\n\n\n\n\u6b65\u9a5f6. \u4f7f\u7528\u4f60\u7684 RAG<\/h1>\n\n\n\n
\u5230\u9019\u88e1\u6211\u5011\u5c31\u4e32\u597d\u6574\u500b RAG \u7684\u6d41\u7a0b\uff0c\u63a5\u4e0b\u4f86\u6211\u5011\u4f86\u554f\u554f Alison Hawk \u7684\u8a0a\u606f(PDF \u7d00\u9304\u7684\u865b\u64ec\u4eba\u7269\u540d\u7a31)<\/p>\n\n\n\n
query = \"Tell me about Alison Hawk's career and age\"
qa.invoke(query)<\/pre>\n\n\n
![\"\"\/](\"https:\/\/i0.wp.com\/miro.medium.com\/v2\/resize%3Afit%3A700\/1%2A4iAZeoLqBT6TA9wnZBB0EA.png?w=640&ssl=1\")
<\/figure>\n<\/div>\n\n\n