coconut0115 님의 블로그

sudo apt update sudo apt install cmake git clone https://github.com/ggerganov/llama.cpp.gitcd llama.cpppip install -r requirements.txt mkdir build cd buildcmake ..make cd ~/llama.cpp python convert_hf_to_gguf.py "변환할 모델 경로" --outfile "gguf 저장 경로/model.gguf"양자화 q8_0, q6_k, q4_0~/llama.cpp/build/bin/llama-quantize "gguf 저장 경로/model.gguf" "경로/model-q8_0.gguf" q8_0

https://arxiv.org/html/2502.02797v1 Upweighting Easy Samples in Fine-Tuning Mitigates ForgettingThe majority of the approaches for mitigating forgetting assume task-specific knowledge access to different extents; either (a subset of) the pre-training dataset itself or some information/statistic computed from pre-training data. Below, we describe thearxiv.org 코드https://github.com/sanyalsunny111/F..

system_prompt = """당신은 답변에 대한 질문을 추가 생성하는 챗봇 어시스턴트 입니다.당신에게는 \\n으로 구분된 여러개의 질문과 하나의 답변이 제공됩니다. 당신의 임무는 해당 질문과 답변을 참고하여 서술어를 바꾼 간결한 30개의 추가 질문을 생성하는 것입니다.""" from openai import OpenAIclient = OpenAI()# 추가 질문을 생성하는 함수def generate_additional_questions(question, answer):    prompt = f"질문: {question}\n답변: {answer}\n\n{system_prompt}"        try:        # GPT API 호출        response = client.chat.compl..

from langchain_community.vectorstores import FAISSfrom langchain.retrievers import BM25Retriever, EnsembleRetrieverfrom langchain.schema import Documentfrom konlpy.tag import Oktfrom kiwipiepy import Kiwikiwi = Kiwi()def kiwi_tokenize(text):    return [token.form for token in kiwi.tokenize(text)]kiwi_bm25 = BM25Retriever.from_texts(texts, preprocess_func=kiwi_tokenize)faiss = FAISS.from_document..

from llama_cpp import Llamadef load_model(repo_id, filename, chat_format = 'llama-3'):    model = Llama.from_pretrained(        # model_path="./Phi-3-mini-4k-instruct-q4.gguf", # 다운 받은 후 사용할 때        repo_id=repo_id,        filename=filename,        n_gpu_layers=-1,        chat_format = chat_format,        n_ctx = 4000,    )    return modeldef generate_chat_completion(user_content, system_conten..