{"id":7094,"date":"2024-08-22T16:54:34","date_gmt":"2024-08-22T08:54:34","guid":{"rendered":"https:\/\/aict.nkust.edu.tw\/digitrans\/?p=7094"},"modified":"2024-12-13T21:09:59","modified_gmt":"2024-12-13T13:09:59","slug":"%e4%bd%bf%e7%94%a8-gemma-2b-llm-%e5%bb%ba%e7%bd%ae-rag-%e7%ae%a1%e9%81%93%e7%9a%84%e6%ad%a5%e9%a9%9f","status":"publish","type":"post","link":"https:\/\/aict.nkust.edu.tw\/digitrans\/?p=7094","title":{"rendered":"\u4f7f\u7528 Gemma 2b LLM \u5efa\u7f6e RAG \u7ba1\u9053\u7684\u6b65\u9a5f"},"content":{"rendered":"\n<p>2024-08-22 | Superteams<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u4ecb\u7d39<\/h2>\n\n\n\n<p><strong>Gemma<\/strong> \u662f Google \u5728\u4eba\u5de5\u667a\u6167\u9818\u57df\u6700\u65b0\u63a8\u51fa\u7684 <strong>LLM<\/strong> \u7cfb\u5217\uff0c\u5b83\u4e0d\u50c5\u50c5\u662f\u53e6\u4e00\u500b\u5927\u578b\u8a9e\u8a00\u6a21\u578b\u3002\u9019\u662f\u4e00\u500b\u300c\u958b\u653e\u6a21\u578b\u300d\u7cfb\u5217\uff0c\u610f\u5473\u8457\u5b83\u5011\u7684\u6838\u5fc3\u529f\u80fd\u53ef\u4ee5\u514d\u8cbb\u5b58\u53d6\u3002\u503c\u5f97\u6ce8\u610f\u7684\u662f\uff0cGoogle \u5728\u6700\u8fd1\u7684 <strong>Gemini<\/strong> \u5347\u7d1a\u5f8c\uff0c\u7acb\u5373\u767c\u5e03\u4e86 <strong>Gemma<\/strong> \u4f5c\u70ba\u958b\u653e\u6a21\u578b\u3002<\/p>\n\n\n\n<p><strong>Gemma<\/strong> \u6709\u5169\u7a2e\u5927\u5c0f &#8211; 20 \u5104\u548c 70 \u5104\u53c3\u6578 &#8211; \u5206\u5225\u7a31\u70ba <strong>Gemma 2b<\/strong> \u548c <strong>Gemma 7b<\/strong>\u3002<strong>Gemma<\/strong> \u529f\u80fd\u5f37\u5927\uff0c\u8207 <strong>Llama2<\/strong> \u7b49\u5176\u4ed6\u958b\u6e90\u6a21\u578b\u76f8\u6bd4\uff0c\u5b83\u5728\u76f8\u5c0d\u8f03\u5c0f\u7684\u6a21\u578b\u5927\u5c0f\u4e0b\u5be6\u73fe\u4e86\u6700\u5148\u9032\u7684\u6548\u80fd\u3002\u9019\u4e9b\u8f03\u5c0f\u7684\u6a21\u578b\u751a\u81f3\u9069\u5408\u4f5c\u70ba\u7b46\u8a18\u578b\u96fb\u8166\u6216\u684c\u4e0a\u578b\u96fb\u8166\u4e0a\u7684 <strong>LLM<\/strong>\uff0c\u4f7f\u5b83\u5011\u6210\u70ba\u958b\u767c\u8005\u9032\u884c\u5be6\u9a57\u7684\u7406\u60f3\u9078\u64c7\u3002<\/p>\n\n\n\n<p>Google \u5728 <strong>Gemma<\/strong> \u4e0a\u505a\u4e86\u4e00\u4ef6\u91cd\u8981\u7684\u4e8b\u60c5\u662f\uff0c\u5b83\u5be6\u65bd\u4e86\u56b4\u683c\u7684\u6a19\u6e96\uff0c\u4ee5\u78ba\u4fdd\u5b89\u5168\u548c\u516c\u6b63\u7684\u8f38\u51fa\uff0c\u89e3\u6c7a\u4e86\u5c0d\u5927\u578b\u8a9e\u8a00\u6a21\u578b\u53ef\u80fd\u88ab\u6feb\u7528\u7684\u64d4\u6182\u3002\u6b64\u5916\uff0c\u4ed6\u5011\u9084\u70ba <strong>Kaggle<\/strong> \u548c <strong>Google Cloud<\/strong> \u7b49\u5e73\u53f0\u4e0a\u7684\u7814\u7a76\u548c\u958b\u767c\u63d0\u4f9b\u514d\u8cbb\u914d\u984d\u3002<\/p>\n\n\n\n<p>\u5728\u672c\u6587\u4e2d\uff0c\u6211\u5011\u5c07\u63a2\u7d22 <strong>Gemma 2b<\/strong>\uff0c\u4e26\u4f7f\u7528 <strong>LangChain<\/strong> \u548c\u6211\u5011\u975e\u5e38\u559c\u611b\u7684\u5411\u91cf\u8cc7\u6599\u5eab <strong>Qdrant<\/strong> \u5efa\u7acb <strong>RAG<\/strong> \u61c9\u7528\u7a0b\u5f0f\u3002<\/p>\n\n\n\n<p>\u4f46\u5728\u6b64\u4e4b\u524d\uff0c\u6211\u5011\u70ba\u4ec0\u9ebc\u9700\u8981 <strong>RAG<\/strong> \u7ba1\u9053\uff1f\u53e6\u5916\uff0c\u9084\u6709\u95dc\u65bc <strong>Gemma<\/strong> \u6a21\u578b\u7684\u6280\u8853\u7c21\u4ecb\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Gemma \u6a21\u578b\u6280\u8853\u7c21\u4ecb<\/h2>\n\n\n\n<p>Google\u5df2\u7d93\u63a1\u53d6\u4e86\u4e00\u4e9b\u63aa\u65bd\u4f86\u7c21\u5316\u958b\u767c\u4eba\u54e1\u7684\u63a1\u7528\uff0c\u4ee5\u4fbf\u958b\u767c\u4eba\u54e1\u53ef\u4ee5\u5617\u8a66\u3002<\/p>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>\u67b6\u69cb\u548c\u6846\u67b6\u652f\u6301<\/strong><\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u591a\u6846\u67b6\u5de5\u5177\uff1aGemma \u63d0\u4f9b\u8de8\u5404\u7a2e\u6846\u67b6\u7684\u63a8\u7406\u548c\u5fae\u8abf\u7684\u53c3\u8003\u5be6\u73fe\uff0c\u5305\u62ec Keras 3.0\u3001PyTorch\u3001JAX \u548c Hugging Face Transformers\u3002<\/li>\n\n\n\n<li>\u8de8\u88dd\u7f6e\u76f8\u5bb9\u6027\uff1aGemma \u53ef\u8de8\u4e0d\u540c\u88dd\u7f6e\u904b\u884c\uff0c\u4f8b\u5982\u7b46\u8a18\u578b\u96fb\u8166\u3001\u684c\u4e0a\u578b\u96fb\u8166\u3001\u7269\u806f\u7db2 (IoT) \u88dd\u7f6e\u3001\u624b\u6a5f\u548c\u96f2\u7aef\u74b0\u5883\u3002<\/li>\n\n\n\n<li>\u786c\u9ad4\u6574\u5408\uff1aGemma \u5229\u7528\u5c16\u7aef\u786c\u9ad4\u5e73\u53f0\uff0c\u7279\u5225\u662f NVIDIA GPU\uff0c\u78ba\u4fdd\u6700\u4f73\u6548\u80fd\u4e26\u8207\u5148\u9032\u6280\u8853\u6574\u5408\u3002<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>\u6027\u80fd\u548c\u5b89\u5168\u7279\u6027<\/strong><\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6a21\u578b\u8b8a\u9ad4\uff1aGemma \u63d0\u4f9b\u5169\u7a2e\u5c3a\u5bf8 \u2014 Gemma 2B \u548c Gemma 7B\uff0c\u6bcf\u7a2e\u5c3a\u5bf8\u90fd\u6709\u7d93\u904e\u9810\u5148\u8a13\u7df4\u548c\u6307\u4ee4\u8abf\u6574\u7684\u8b8a\u9ad4\u3002<\/li>\n\n\n\n<li>\u5b89\u5168\u548c\u8ca0\u8cac\u4efb\u7684\u751f\u6210\uff1aGemma \u7d93\u904e\u9810\u5148\u8a13\u7df4\uff0c\u53ef\u4ee5\u904e\u6ffe\u654f\u611f\u6216\u500b\u4eba\u8a0a\u606f\uff0c\u4e26\u4f7f\u7528\u4eba\u985e\u56de\u994b\u7684\u5f37\u5316\u5b78\u7fd2 (RLHF) \u4f86\u6700\u5927\u7a0b\u5ea6\u5730\u6e1b\u5c11\u7522\u751f\u6709\u5bb3\u5167\u5bb9\u7684\u53ef\u80fd\u6027\u3002<\/li>\n\n\n\n<li>\u624b\u52d5\u6e2c\u8a66\uff1a\u9032\u884c\u624b\u52d5\u7d05\u968a\u548c\u5c0d\u6297\u6027\u6e2c\u8a66\u4f86\u8b58\u5225\u8207 Gemma \u76f8\u95dc\u7684\u6f5b\u5728\u98a8\u96aa\u3002<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>\u53ef\u8a2a\u554f\u6027\u548c\u5354\u4f5c<\/strong><\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u958b\u6e90\uff1aGemma \u514d\u8cbb\u63d0\u4f9b\u7d66\u5168\u7403\u958b\u767c\u4eba\u54e1\u548c\u7814\u7a76\u4eba\u54e1\u793e\u7fa4\u3002<\/li>\n\n\n\n<li>\u5de5\u5177\u93c8\u652f\u63f4\uff1aGemma \u63d0\u4f9b\u8de8\u6240\u6709\u4e3b\u8981\u6846\u67b6\u7684\u63a8\u7406\u548c\u76e3\u7763\u5fae\u8abf (SFT) \u5de5\u5177\u93c8\u3002<\/li>\n\n\n\n<li>\u5373\u7528\u578b\u8cc7\u6e90\uff1aGemma \u96a8\u9644\u5373\u7528\u578b Colab \u548c Kaggle \u7b46\u8a18\u672c\uff0c\u4e26\u8207 Hugging Face\u3001MaxText\u3001NVIDIA NeMo \u548c TensorRT-LLM \u7b49\u71b1\u9580\u5de5\u5177\u6574\u5408\u3002<\/li>\n\n\n\n<li>\u514d\u8cbb\u7a4d\u5206\uff1a\u9996\u6b21\u4f7f\u7528 Google Cloud \u7684\u7528\u6236\u53ef\u4ee5\u7372\u5f97 300 \u7f8e\u5143\u7684\u7a4d\u5206\uff0c\u7814\u7a76\u4eba\u54e1\u53ef\u4ee5\u7533\u8acb\u984d\u5916\u7684 Google Cloud \u7a4d\u5206\uff0c\u6700\u9ad8\u53ef\u9054 50 \u842c\u7f8e\u5143\u3002<\/li>\n<\/ul>\n\n\n\n<p>\u622a\u81f3\u76ee\u524d\uff0cGemma \u50c5\u5c08\u6ce8\u65bc\u6587\u672c\u5230\u6587\u672c\u7684\u4efb\u52d9\uff0c\u8207\u5176\u524d\u8eab Gemini \u4e0d\u540c\uff0cGemini \u662f\u591a\u6a21\u5f0f\u7684\u3002<\/p>\n\n\n\n<p>\u5118\u7ba1\u5982\u6b64\uff0cGemma \u4f3c\u4e4e\u8868\u73fe\u51fa\u4e86\u8207\u5176\u5c3a\u5bf8\u76f8\u95dc\u7684\u51fa\u8272\u6027\u80fd\uff0c\u5728\u95dc\u9375\u57fa\u6e96\u4e0a\u8d85\u8d8a\u4e86\u66f4\u5927\u7684\u578b\u865f\uff0c\u540c\u6642\u4fdd\u6301\u4e86\u56b4\u683c\u7684\u5b89\u5168\u548c\u8cac\u4efb\u6a19\u6e96<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u4e86\u89e3 RAG \u7ba1\u9053<\/h2>\n\n\n\n<p><em>\uff08\u5982\u679c\u60a8\u5df2\u7d93\u719f\u6089 RAG \u53ca\u5176\u91cd\u8981\u6027\uff0c\u5247\u53ef\u4ee5\u8df3\u904e\u672c\u7bc0\u3002\uff09<\/em><\/p>\n\n\n\n<p>\u96d6\u7136LLM\u64c5\u9577\u8655\u7406\u8a9e\u8a00\u548c\u751f\u6210\u5275\u610f\u6587\u672c\uff0c\u4f46\u4ed6\u5011\u5728\u5c07\u81ea\u5df1\u7684\u56de\u7b54\u7acb\u8db3\u65bc\u73fe\u5be6\u4e16\u754c\u7684\u4e8b\u5be6\u548c\u77e5\u8b58\u65b9\u9762\u5b58\u5728\u5c40\u9650\u6027\u3002<\/p>\n\n\n\n<p>\u9019\u5c31\u662f RAG \u7ba1\u9053\u7684\u7528\u6b66\u4e4b\u5730\uff0c\u5b83\u900f\u904e\u89e3\u6c7a\u9019\u4e9b\u9650\u5236\uff0c\u5728 LLM \u7684\u300c\u57fa\u790e\u300d\u65b9\u9762\u767c\u63ee\u8457\u81f3\u95dc\u91cd\u8981\u7684\u4f5c\u7528\u3002\u5b83\u5011\u7d50\u5408\u4e86\u5169\u500b\u5f37\u5927\u7684 AI \u7d44\u4ef6\u7684\u512a\u52e2\uff1a\u5927\u578b\u8a9e\u8a00\u6a21\u578b (LLM) \u548c\u5411\u91cf\u8cc7\u6599\u5eab\u3002\u4ee5\u4e0b\u662f\u5b83\u5011\u7684\u5de5\u4f5c\u539f\u7406\u7684\u8a73\u7d30\u8aaa\u660e\uff1a<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>\u7b2c 1 \u6b65 &#8211; \u77e5\u8b58\u6e96\u5099<\/strong><\/h3>\n\n\n\n<p><strong>\u6587\u4ef6\u6536\u96c6<\/strong>\uff1a\u76f8\u95dc\u6587\u4ef6\uff08\u4f8b\u5982\u6587\u7ae0\u3001\u5831\u544a\u6216\u7a0b\u5f0f\u78bc\uff09\u88ab\u6536\u96c6\u4e26\u5132\u5b58\u5728\u8cc7\u6599\u5eab\u4e2d\u3002<\/p>\n\n\n\n<p><strong>\u5411\u91cf\u5316<\/strong>\uff1a\u4f7f\u7528\u5d4c\u5165\u6a21\u578b\u5c07\u6bcf\u500b\u6587\u4ef6\u8f49\u63db\u70ba\u7a31\u70ba\u300c\u5411\u91cf\u300d\u7684\u6578\u5b57\u8868\u793a\u3002\u5c07\u5176\u60f3\u50cf\u70ba\u5728\u591a\u7dad\u7a7a\u9593\u4e2d\u6355\u7372\u6587\u4ef6\u7684\u672c\u8cea\u3002<\/p>\n\n\n\n<p><strong>\u5411\u91cf\u8cc7\u6599\u5eab<\/strong>\uff1a\u6240\u6709\u6587\u4ef6\u5411\u91cf\u90fd\u6709\u6548\u5730\u5132\u5b58\u5728\u5c08\u70ba\u5feb\u901f\u76f8\u4f3c\u6027\u641c\u5c0b\u800c\u8a2d\u8a08\u7684\u5c08\u7528\u8cc7\u6599\u5eab\u4e2d\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>\u6b65\u9a5f 2. \u4f7f\u7528\u8005\u4ea4\u4e92<\/strong><\/h3>\n\n\n\n<p><strong>\u4f7f\u7528\u8005\u67e5\u8a62<\/strong>\uff1a\u60a8\u5411LLM\u63d0\u51fa\u554f\u984c\u6216\u63d0\u4f9b\u8aaa\u660e\u3002<\/p>\n\n\n\n<p><strong>\u67e5\u8a62\u5411\u91cf\u5316<\/strong>\uff1a\u60a8\u7684\u67e5\u8a62\u4e5f\u6703\u4f7f\u7528\u76f8\u540c\u7684\u5d4c\u5165\u6a21\u578b\u8f49\u63db\u70ba\u5411\u91cf\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>\u6b65\u9a5f3.\u8cc7\u8a0a\u6aa2\u7d22<\/strong><\/h3>\n\n\n\n<p><strong>\u76f8\u4f3c\u6027\u641c\u5c0b<\/strong>\uff1aLLM \u7684\u67e5\u8a62\u5411\u91cf\u8207\u8cc7\u6599\u5eab\u4e2d\u7684\u6240\u6709\u6587\u4ef6\u5411\u91cf\u9032\u884c\u6bd4\u8f03\u3002\u6aa2\u7d22\u5177\u6709\u6700\u63a5\u8fd1\u5411\u91cf\u8868\u793a\u7684\u6587\u6a94\uff0c\u8868\u793a\u5b83\u5011\u5305\u542b\u76f8\u95dc\u8cc7\u8a0a\u3002<\/p>\n\n\n\n<p><strong>\u904e\u6ffe\u548c\u91cd\u65b0\u6392\u540d\uff08\u53ef\u9078\uff09<\/strong>\uff1a\u5176\u4ed6\u6a21\u578b\u53ef\u4ee5\u6839\u64da\u76f8\u95dc\u6027\u3001\u4e0a\u4e0b\u6587\u6216\u5176\u4ed6\u6a19\u6e96\u7d30\u5316\u6aa2\u7d22\u5230\u7684\u6587\u4ef6\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>\u6b65\u9a5f 4. \u77e5\u8b58\u6574\u5408<\/strong><\/h3>\n\n\n\n<p><strong>\u4e0a\u4e0b\u6587\u8c50\u5bcc<\/strong>\uff1a\u6aa2\u7d22\u5230\u7684\u6587\u4ef6\u5c07\u4f5c\u70ba\u9644\u52a0\u8cc7\u8a0a\u63d0\u4f9b\u7d66LLM\u3002\u9019\u6709\u52a9\u65bcLLM\u4e86\u89e3\u60a8\u67e5\u8a62\u7684\u4e0a\u4e0b\u6587\u4e26\u5f9e\u76f8\u95dc\u4f86\u6e90\u5b58\u53d6\u7279\u5b9a\u8a73\u7d30\u8cc7\u8a0a\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>\u6b65\u9a5f 5. \u97ff\u61c9\u751f\u6210<\/strong><\/h3>\n\n\n\n<p><strong>\u5de5\u4f5c\u4e2d\u7684LLM<\/strong>\uff1a\u900f\u904e\u8c50\u5bcc\u7684\u4e0a\u4e0b\u6587\uff0cLLM\u53ef\u4ee5\u7522\u751f\u56de\u61c9\u3002\u9019\u53ef\u4ee5\u662f\u60a8\u554f\u984c\u7684\u7b54\u6848\u3001\u6839\u64da\u60a8\u7684\u6307\u793a\u64b0\u5beb\u7684\u5275\u610f\u6587\u672c\uff0c\u6216\u662fLLM\u63a5\u53d7\u57f9\u8a13\u7684\u4efb\u4f55\u5176\u4ed6\u8f38\u51fa\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u554f\u984c\u9673\u8ff0\uff1a\u5efa\u7acb RAG \u7ba1\u9053\u4f86\u67e5\u8a62\u5c65\u6b77<\/h2>\n\n\n\n<p>\u5728\u63a5\u4e0b\u4f86\u7684\u6b65\u9a5f\u4e2d\uff0c\u6211\u5011\u5c07\u4f7f\u7528\u5927\u91cf\u5019\u9078\u4eba\u7684\u5c65\u6b77\u4f5c\u70ba\u6211\u5011\u7684\u8cc7\u6599\u96c6\u3002\u9019\u662f\u4e00\u500b\u5f88\u597d\u7684\u61c9\u7528\u5834\u666f\uff0c\u56e0\u70ba\u4eba\u529b\u8cc7\u6e90\u4e3b\u7ba1\u7d93\u5e38\u9700\u8981\u700f\u89bd\u6578\u5343\u4efd\u5019\u9078\u4eba\u7684\u5c65\u6b77\uff0c\u624d\u80fd\u627e\u5230\u9069\u5408\u67d0\u500b\u8077\u4f4d\u7684\u61c9\u5fb5\u8005\u3002<\/p>\n\n\n\n<p>\u501f\u52a9 <strong>LLM<\/strong> \u548c <strong>RAG<\/strong> \u61c9\u7528\u7a0b\u5f0f\uff0c\u53ef\u4ee5\u986f\u8457\u63d0\u5347\u4eba\u529b\u8cc7\u6e90\u4e3b\u7ba1\u7684\u5de5\u4f5c\u6548\u7387\uff0c\u4e26\u70ba\u4ed6\u5011\u63d0\u4f9b\u4e00\u500b\u7c21\u55ae\u7684\u5c65\u6b77\u67e5\u8a62\u5de5\u5177\u3002\u5be6\u969b\u4e0a\uff0c\u6211\u5011\u4e4b\u524d\u5df2\u7d93\u900f\u904e <strong>Mistral 7b LLM<\/strong> \u90e8\u7f72\u4e86\u9019\u6a23\u7684\u89e3\u6c7a\u65b9\u6848\u3002<\/p>\n\n\n\n<p>\u5728\u672c\u6587\u4e2d\uff0c\u6211\u5011\u5c07\u5617\u8a66\u5c0d <strong>Gemma 2b<\/strong> \u9032\u884c\u76f8\u540c\u7684\u64cd\u4f5c\u3002\u76f8\u8f03\u4e4b\u4e0b\uff0c\u9019\u500b <strong>LLM<\/strong> \u7684\u898f\u6a21\u8f03\u5c0f\uff0c\u4f46\u5e0c\u671b\u7d50\u679c\u540c\u6a23\u51fa\u8272\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u7b2c1\u6b65\uff1a\u5b89\u88dd<\/h3>\n\n\n\n<p>\u8b93\u6211\u5011\u4f7f\u7528\u60a8\u559c\u6b61\u7684\u4efb\u4f55\u7b46\u8a18\u578b\u96fb\u8166\u74b0\u5883\u3002\u5c31\u6211\u5011\u800c\u8a00\uff0c\u6211\u5011\u50be\u5411\u65bc\u5728 <strong>Google Cloud<\/strong> \u4e0a\u555f\u52d5\u4e00\u500b\u7bc0\u9ede\uff0c<strong>Google Cloud<\/strong> \u6177\u6168\u5730\u70ba\u6211\u5011\u63d0\u4f9b\u4e86\u514d\u8cbb\u7684\u555f\u52d5\u914d\u984d\u3002\u5728\u672c\u4f8b\u4e2d\uff0c\u6211\u5011\u5c07\u4f7f\u7528\u5df2\u7d93\u904b\u884c\u7684 <strong>4xT4<\/strong> \u7bc0\u9ede\u3002\u70ba\u4e86\u9023\u63a5\u5230\u9060\u7aef\u7bc0\u9ede\uff0c\u6211\u5011\u5c07\u4f7f\u7528 <strong>VS Code Remote Explorer<\/strong> \u64f4\u5145\u529f\u80fd\uff0c\u4e26\u5efa\u7acb <strong>SSH<\/strong> \u9023\u7dda\u3002<\/p>\n\n\n\n<p>\u5118\u7ba1\u9019\u500b\u6a21\u578b\u6709\u53ef\u80fd\u5728\u6211\u7684\u7b46\u8a18\u578b\u96fb\u8166\u4e0a\u904b\u884c\uff0c\u4f46\u6211\u60f3\u5c55\u793a\u7684\u662f\u5728\u5177\u6709\u5927\u91cf\u8a18\u61b6\u9ad4\u548c\u5927\u578b\u8cc7\u6599\u96c6\u7684 <strong>GPU<\/strong> \u4e0a\u5efa\u7acb\u7684\u5be6\u969b\u7528\u4f8b\u3002<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/cdn.prod.website-files.com\/640a0e9bc826600a975af76c\/66c7042a461cfa692c0d44bc_65d72189abcb4cc8b26b5cd5_Sbom1ijJowAQxvIZrsEyCHjEii79BrylAheBEYNYqSd8fs402LFgaHDDLMX4i_IeNIpit9wHM3plsZ2LahD4kW_BlRM901a7tJb32Nx0YBeSnPEpi0WiAYTj4Yyn2tNoZtmkY0c3PLEdPsXRD5QYopE.png?w=640&#038;ssl=1\" alt=\"\"\/><\/figure>\n<\/div>\n\n\n<p>\u5b89\u88dd\u64f4\u5145\u5f8c\uff0c\u5efa\u7acb\u4e00\u500b\u65b0\u7684 ssh \u9023\u7dda\u3002\u6b65\u9a5f\u5f88\u660e\u986f\uff0c\u5b83\u6703\u8b93\u4f60\u9032\u5165\u6a5f\u5668\u3002\u7136\u5f8c\uff0c\u6211\u559c\u6b61\u7e3d\u662f\u5275\u5efa\u4e00\u500b\u540d\u70ba\u201cworkspace\u201d\u7684\u8cc7\u6599\u593e\uff0c\u7136\u5f8c\u5275\u5efa\u4e00\u500b\u985e\u4f3c\u201cgemma_rag\u201d\u7684\u8cc7\u6599\u593e\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir -p ~\/workspace\/gemma_rag<\/code><\/pre>\n\n\n\n<p>\u5728\u8a72\u8cc7\u6599\u593e\u4e2d\u5efa\u7acb\u4e00\u500b\u7b46\u8a18\u672c\uff0c\u4f8b\u5982\u300cgemma_test.ipynb\u300d\u3002\u9019\u5c07\u70ba\u60a8\u63d0\u4f9b\u5f37\u5927\u7684 GPU \u652f\u63f4\u7684\u7b46\u8a18\u578b\u96fb\u8166\u74b0\u5883\u3002\u63a5\u4e0b\u4f86\uff0c\u9078\u64c7\u4e00\u500b\u6838\u5fc3\u3002\u7576\u60a8\u9078\u64c7\u53f3\u5074\u7684\u6838\u5fc3\u6642\uff0cVS Code \u5c07\u5141\u8a31\u60a8\u9060\u7aef\u5efa\u7acb\u865b\u64ec\u74b0\u5883\u3002\u9019\u5c07\u5728\u540c\u4e00\u76ee\u9304\u4e2d\u5efa\u7acb\u4e00\u500b .venv \u8cc7\u6599\u593e\u3002<\/p>\n\n\n\n<p>\u73fe\u5728\u8b93\u6211\u5011\u5b89\u88dd\u4e00\u4e9b\u51fd\u5f0f\u5eab\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>!pip install tiktoken\n!pip install qdrant-client langchain pypdf<\/code><\/pre>\n\n\n\n<p>\u63a5\u4e0b\u4f86\u662f import \u8a9e\u53e5\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><strong>import<\/strong> os\n<strong>import<\/strong> getpass\n<strong>from<\/strong> operator <strong>import<\/strong> itemgetter\n<strong>from<\/strong> langchain_community.document_loaders <strong>import<\/strong> PyPDFDirectoryLoader\n<strong>from<\/strong> langchain.vectorstores <strong>import<\/strong> Qdrant\n<strong>from<\/strong> langchain_core.prompts <strong>import<\/strong> ChatPromptTemplate, PromptTemplate\n<strong>from<\/strong> langchain_core.output_parsers <strong>import<\/strong> StrOutputParser\n<strong>from<\/strong> langchain_core.runnables <strong>import<\/strong> RunnablePassthrough, RunnableParallel\n<strong>from<\/strong> langchain.schema <strong>import<\/strong> format_document\n<strong>from<\/strong> langchain.llms <strong>import<\/strong> HuggingFacePipeline\n<strong>from<\/strong> langchain.document_loaders <strong>import<\/strong> TextLoader\n<strong>from<\/strong> langchain.text_splitter <strong>import<\/strong> RecursiveCharacterTextSplitter\n<strong>from<\/strong> langchain.embeddings <strong>import<\/strong> HuggingFaceEmbeddings\n<strong>from<\/strong> langchain.chains <strong>import<\/strong> RetrievalQA<\/code><\/pre>\n\n\n\n<p>\u6211\u4f7f\u7528 VS Code \u672c\u8eab\u5efa\u7acb\u4e86\u4e00\u500b\u540d\u70ba\u300cdata\u300d\u7684\u8cc7\u6599\u593e\u3002\u4e4b\u5f8c\uff0c\u6211\u53ea\u662f\u5c07\u5927\u7d04\u4e00\u5343\u4efd\u5c65\u6b77\u62d6\u5230\u8a72\u8cc7\u6599\u593e\u4e2d\u3002\u8db3\u5920\u9069\u5408\u6e2c\u8a66\u4e86\u3002<\/p>\n\n\n\n<p>\u73fe\u5728\uff0c\u8b93\u6211\u5011\u52a0\u8f09\u9019\u4e9b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>loader = PyPDFDirectoryLoader(\".\/data\")\ndocs = loader.load()\nprint(len(docs))<\/code><\/pre>\n\n\n\n<p>\u9019\u61c9\u8a72\u6703\u5217\u5370\u5df2\u5f9e\u76ee\u9304\u8f09\u5165\u7684\u6587\u4ef6\u7e3d\u6578\u3002\u63a5\u4e0b\u4f86\uff0c\u6211\u5011\u5c07\u4f7f\u7528 HuggingFaceEmbeddings() \u51fd\u6578\u5c07\u9019\u4e9b\u6587\u4ef6\u8f49\u63db\u70ba\u5411\u91cf\u5d4c\u5165\u3002\u6211\u5011\u5c07\u628a\u5b83\u4fdd\u5b58\u5230\u4e00\u500b\u540d\u70ba\u300c\u5c65\u6b77\u300d\u7684\u96c6\u5408\u4e2d\u3002\u5728\u9019\u88e1\uff0c\u6211\u5011\u53ea\u662f\u5728\u201c\u5167\u5b58\u201d\u6a21\u5f0f\u4e0b\u4f7f\u7528 Qdrant\u3002\u7406\u60f3\u60c5\u6cc1\u4e0b\uff0c\u60a8\u61c9\u8a72\u900f\u904e docker \u555f\u52d5 Qdrant\uff0c\u6216\u4f7f\u7528\u4ed6\u5011\u7684\u96f2\u7aef\u3002<\/p>\n\n\n\n<p>\u9996\u5148\u78ba\u4fdd\u60a8\u5b89\u88dd\u4e86\u53e5\u5b50\u8f49\u63db\u5668\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>!pip install sentence-transformers<\/code><\/pre>\n\n\n\n<p>\u73fe\u5728\u7e7c\u7e8c\u9032\u884c\u5411\u91cf\u5d4c\u5165\u8f49\u63db\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><strong>from<\/strong> langchain.text_splitter <strong>import<\/strong> RecursiveCharacterTextSplitter\n\n# initialise embeddings used to convert text to vectors\nmodel_name = \"sentence-transformers\/all-mpnet-base-v2\"\nmodel_kwargs = {\"device\": \"cuda\"}\n\nembeddings = HuggingFaceEmbeddings(model_name=model_name, model_kwargs=model_kwargs)\n\n# Split documents into chunks so it fits in context\n\ntext_splitter = RecursiveCharacterTextSplitter(chunk_size = 500, chunk_overlap = 0)\nall_splits = text_splitter.split_documents(docs)\n\n# create a qdrant collection - a vector based index of all resumes\nqdrant_collection = Qdrant.from_documents(\n    all_splits,\n    embeddings,\n    location=\":memory:\", # Local mode with in-memory storage only\n    collection_name=\"resumes\",\n)\n\n# construct a retriever on top of the vector store\nqdrant_retriever = qdrant_collection.as_retriever()<\/code><\/pre>\n\n\n\n<p>\u63a5\u4e0b\u4f86\uff0c\u5728\u5c07 RAG \u7ba1\u9053\u6574\u5408\u5728\u4e00\u8d77\u4e4b\u524d\uff0c\u6211\u5011\u5c07\u5617\u8a66 Gemma 2b \u6a21\u578b\u3002\u70ba\u6b64\uff0c\u6211\u5011\u5fc5\u9808\u5148\u78ba\u4fdd\u4f7f\u7528\u5347\u7d1a\u5f8c\u7684\u8b8a\u58d3\u5668\u5eab\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>!pip install -U \"transformers==4.38.0\" --upgrade<\/code><\/pre>\n\n\n\n<p>\u63a5\u4e0b\u4f86\uff0c\u6211\u5011\u5c07\u6e2c\u8a66 Gemma-2b\u3002\u5728\u7e7c\u7e8c\u57f7\u884c\u4e0b\u4e00\u7d44\u7a0b\u5f0f\u78bc\u4e4b\u524d\uff0c\u60a8\u9700\u8981\u63a5\u53d7\u8a31\u53ef\u8b49\u4e26\u6709\u6b0a\u5b58\u53d6\u8a72\u6a21\u578b\u3002\u70ba\u6b64\uff0c\u60a8\u61c9\u8a72\u8a2a\u554f\u4ee5\u4e0b URL\u3002<br><br>https:\/\/huggingface.co\/google\/gemma-2b-it&nbsp;<\/p>\n\n\n\n<p><br>\u7372\u5f97\u5b58\u53d6\u6b0a\u9650\u5f8c\uff0c\u60a8\u53ef\u4ee5\u4f7f\u7528 HuggingFace \u4ee4\u724c\uff0c\u4e5f\u53ef\u4ee5\u5728\u672c\u6a5f\u4e0b\u8f09\u6a21\u578b\u3002\u6211\u5011\u8981\u505a\u524d\u8005\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><strong>from<\/strong> transformers <strong>import<\/strong> AutoTokenizer, pipeline\n<strong>import<\/strong> torch\n\nhf_access_token = 'hf......'\nmodel = \"google\/gemma-2b-it\"\n\n# Code below is to first test out the model\n\ntokenizer = AutoTokenizer.from_pretrained(model, token=hf_access_token)\npipeline = pipeline(\n    \"text-generation\",\n    model=model,\n    model_kwargs={\"torch_dtype\": torch.bfloat16},\n    device=\"cuda\",\n    max_new_tokens=512\n)\n\nmessages = &#91;\n    {\"role\": \"user\", \"content\": \"Where is Milan?\"},\n]\nprompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\noutputs = pipeline(\n\tprompt,\n\tmax_new_tokens=256,\n\tadd_special_tokens=True,\n\tdo_sample=True,\n\ttemperature=0.7,\n\ttop_k=50,\n\ttop_p=0.95\n)\nprint(outputs&#91;0]&#91;\"generated_text\"]&#91;len(prompt):])<\/code><\/pre>\n\n\n\n<p>\u5982\u679c\u5230\u76ee\u524d\u70ba\u6b62\u60a8\u5df2\u7d93\u5b8c\u6210\u4e86\u6240\u6709\u64cd\u4f5c\uff0c\u60a8\u5c07\u770b\u5230\u4ee5\u4e0b\u8f38\u51fa\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Milan <strong>is<\/strong> located <strong>in<\/strong> the Lombardy region of Italy. It <strong>is<\/strong> the largest city <strong>in<\/strong> the region <strong>and<\/strong> the second-largest city <strong>in<\/strong> Italy after Rome.<\/code><\/pre>\n\n\n\n<p>\u63a5\u4e0b\u4f86\uff0c\u8b93\u6211\u5011\u4f7f\u7528 Qdrant \u548c\u6211\u5011\u7684\u5c65\u6b77\u8cc7\u6599\u96c6\u5c07 RAG \u7ba1\u9053\u63a5\u5728\u4e00\u8d77\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gemma_llm = HuggingFacePipeline(\n    pipeline=pipeline,\n    model_kwargs={\"temperature\": 0.7},\n)\n\nqa = RetrievalQA.from_chain_type(\n    llm=gemma_llm,\n    chain_type=\"stuff\",\n    retriever=qdrant_retriever\n)\n\nquery = \"Which resumes have Python experience?\"\nqa.invoke(query)<\/code><\/pre>\n\n\n\n<p>\u9019\u61c9\u8a72\u5df2\u7d93\u70ba\u4f60\u5e36\u4f86\u4e86\u5f88\u597d\u7684\u7d50\u679c\uff0c\u4f60\u7684LLM\u4ee5\u6578\u64da\u70ba\u57fa\u790e\u3002<\/p>\n\n\n\n<p>\u73fe\u5728\u8b93\u6211\u5011\u5728\u9802\u90e8\u653e\u7f6e\u4e00\u500b Gradio UI\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>!pip install --upgrade gradio<\/code><\/pre>\n\n\n\n<p>\u6211\u5011\u73fe\u5728\u5c07\u628a\u5b83\u5011\u653e\u5728\u4e00\u8d77\uff0c\u7528\u5c65\u6b77\u8cc7\u6599\u96c6\u5efa\u7acb\u6211\u5011\u7684 RAG \u7ba1\u9053\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><strong>import<\/strong> gradio <strong>as<\/strong> gr\n<strong>import<\/strong> os\n<strong>from<\/strong> shutil <strong>import<\/strong> copyfile\n\n<strong>with<\/strong> gr.Blocks() <strong>as<\/strong> demo:\n\tchatbot = gr.Chatbot()\n\tmsg = gr.Textbox()\n\tclear = gr.ClearButton(&#91;msg, chatbot])\n\n\t<strong>def<\/strong> <strong>respond<\/strong>(message, chat_history):\n  \t\tbot_message = qa.invoke(message)\n  \t\tchat_history.append((message, bot_message))\n  \t\t<strong>return<\/strong> \"\", chat_history\n\n\tmsg.submit(respond, &#91;msg, chatbot], &#91;msg, chatbot])\ndemo.launch(share=True)<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">\u7d50\u679c<\/h2>\n\n\n\n<p>\u95dc\u65bc Gemma \u6a21\u578b\u6700\u6709\u5e0c\u671b\u7684\u4e8b\u5be6\u4e4b\u4e00\u662f\uff0c\u5118\u7ba1\u8cc7\u6e90\u8f03\u5c11\uff0c\u4f46\u5b83\u7684\u8868\u73fe\u537b\u975e\u5e38\u597d\u3002\u9019\u5f88\u5f37\u5927\uff0c\u56e0\u70ba\u6211\u5011\u5c07LLM\u8996\u70ba\u5404\u7a2e\u5834\u666f\u4e2d\u7684\u81ea\u7136\u8a9e\u8a00\u5f15\u64ce\uff0c\u5f9e\u4f01\u696d\u7d1a\u804a\u5929\u6a5f\u5668\u4eba\u5230\u5728\u624b\u6a5f\u548c\u7b46\u8a18\u578b\u96fb\u8166\u4e0a\u904b\u884c\u7684\u672c\u5730LLM\u3002\u524d\u8005\u53ef\u4ee5\u5e6b\u52a9\u7c21\u5316\u4f01\u696d\u5de5\u4f5c\u6d41\u7a0b\uff0c\u800c\u5f8c\u8005\u5c07\u70ba\u60a8\u7684\u65e5\u5e38\u751f\u6d3b\u63d0\u4f9b\u5e6b\u52a9\u3002<\/p>\n\n\n\n<p>\u8c37\u6b4c\u900f\u904eGemma\u5c55\u793a\u4e86\u5728\u8f03\u5c0f\u7684\u6a21\u578b\u898f\u6a21\u4e0b\u5be6\u73fe\u8d85\u8d8a\u57fa\u6e96\u6e2c\u8a66\u7684\u54c1\u8cea\u662f\u53ef\u80fd\u7684\uff0c\u6211\u5011\u9810\u8a08\u5b83\u5c07\u53d6\u4ee3\u6211\u5011\u6700\u559c\u611b\u7684\u53e6\u5916\u5169\u500b\u6a21\u578b\u2014\u2014Mistral 7B \u548c Llama2 7B\uff0c\u6210\u70ba\u65b0\u7684\u4f7c\u4f7c\u8005<\/p>\n\n\n\n<p>\u8cc7\u6599\u4f86\u6e90: <a href=\"https:\/\/www.superteams.ai\/blog\/steps-to-build-a-rag-pipeline-using-gemma-2b-llm\">https:\/\/www.superteams.ai\/blog\/steps-to-build-a-rag-pipeline-using-gemma-2b-llm<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>2024-08-22 | Superteams \u4ecb\u7d39 Gemma \u662f Google \u5728\u4eba\u5de5\u667a\u6167\u9818\u57df\u6700\u65b0\u63a8\u51fa\u7684 LLM \u7cfb\u5217\uff0c\u5b83\u4e0d\u50c5\u50c5\u662f\u53e6\u4e00\u500b\u5927\u578b\u8a9e\u8a00\u6a21\u578b\u3002\u9019\u662f\u4e00\u500b\u300c\u958b\u653e\u6a21\u578b\u300d\u7cfb\u5217\uff0c&hellip;<\/p>\n","protected":false},"author":4,"featured_media":7095,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[579,4],"tags":[204],"class_list":["post-7094","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-579","category-industry-news","tag-204"],"gutentor_comment":0,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/aict.nkust.edu.tw\/digitrans\/wp-content\/uploads\/2024\/10\/%E8%9E%A2%E5%B9%95%E6%93%B7%E5%8F%96%E7%95%AB%E9%9D%A2-2024-10-04-165701.png?fit=740%2C410&ssl=1","jetpack-related-posts":[],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=\/wp\/v2\/posts\/7094","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7094"}],"version-history":[{"count":2,"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=\/wp\/v2\/posts\/7094\/revisions"}],"predecessor-version":[{"id":7121,"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=\/wp\/v2\/posts\/7094\/revisions\/7121"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=\/wp\/v2\/media\/7095"}],"wp:attachment":[{"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7094"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7094"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aict.nkust.edu.tw\/digitrans\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7094"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}