LLM LangChain

Using LangChain

LangChain is a framework that helps you use LM (Large Language) conveniently.

Basic

LangChain Basic explains sample code for each configuration of LangChain.

Using LangChain on Falcon FM

Describes how to apply LangChain to a SageMaker Endpoint created with Falcon FM. Use the SageMaker Endpoint (e.g. jumpstart-dft-hf-llm-falcon-7b-instruct-bf16) obtained from Installing Falcon FM with SageMaker JumpStart.

LangChain declaration for SageMaker Endpoint

Referring to Falcon's input and output, register transform_input and transform_output of ContentHandler as shown below.

 from langchain import PromptTemplate , SagemakerEndpoint
from langchain . llms . sagemaker_endpoint import LLMContentHandler

class ContentHandler ( LLMContentHandler ):
    content_type = "application/json"
    accepts = "application/json"

    def transform_input ( self , prompt : str , model_kwargs : dict ) -> bytes :
        input_str = json . dumps ({ 'inputs' : prompt , 'parameters' : model_kwargs })
        return input_str . encode ( 'utf-8' )
      
    def transform_output ( self , output : bytes ) -> str :
        response_json = json . loads ( output . read (). decode ( "utf-8" ))        
        return response_json [ 0 ][ "generated_text" ]

Register the llm for Sagemaker Endpoint using endpoint_name, aws_region, parameters, and content_handler as shown below.

 endpoint_name = 'jumpstart-dft-hf-llm-falcon-7b-instruct-bf16'
aws_region = boto3 . Session (). region_name
parameters = {
    "max_new_tokens" : 300
}
content_handler = ContentHandler ()

llm = SagemakerEndpoint (
    endpoint_name = endpoint_name , 
    region_name = aws_region , 
    model_kwargs = parameters ,
    content_handler = content_handler
)

You can check the operation of llm as follows.

 llm ( "Tell me a joke" )

The results at this time are as follows.

 I once told a joke to a friend, but it didn't work. He just looked

Web Loader

Web loader - You can load web pages using langchain.

 from langchain . document_loaders import WebBaseLoader
from langchain . indexes import VectorstoreIndexCreator

loader = WebBaseLoader ( "https://lilianweng.github.io/posts/2023-06-23-agent/" )
index = VectorstoreIndexCreator (). from_loaders ([ loader ])

Prompt Template

After defining the template as shown below, you can define LLMChain and run it. See langchain-sagemaker-endpoint-Q&A.ipynb for details.

 from langchain import PromptTemplate , LLMChain

template = "Tell me a {adjective} joke about {content}."
prompt = PromptTemplate . from_template ( template )

llm_chain = LLMChain ( prompt = prompt , llm = llm )

outputText = llm_chain . run ( adjective = "funny" , content = "chickens" )
print ( outputText )

The results at this time are as follows.

 Why did the chicken cross the playground? To get to the other slide!

Question / Answering

Question/Answering for the document is performed using langchain.chains.question_answering. See langchain-sagemaker-endpoint-Q&A.ipynb for details.

Defines the template of the prompt.

 template = """Use the following pieces of context to answer the question at the end.

{context}

Question: {question}
Answer:"""

prompt = PromptTemplate (
    template = template , input_variables = [ "context" , "question" ]
)

Create a document using langchain.docstore.document.

 from langchain . docstore . document import Document
example_doc_1 = """
Peter and Elizabeth took a taxi to attend the night party in the city. While in the party, Elizabeth collapsed and was rushed to the hospital.
Since she was diagnosed with a brain injury, the doctor told Peter to stay besides her until she gets well.
Therefore, Peter stayed with her at the hospital for 3 days without leaving.
"""

docs = [
    Document (
        page_content = example_doc_1 ,
    )
]

Now do Question/Answering.

 from langchain . chains . question_answering import load_qa_chain

question = "How long was Elizabeth hospitalized?"

chain = load_qa_chain ( prompt = prompt , llm = llm )

output = chain ({ "input_documents" : docs , "question" : question }, return_only_outputs = True )
print ( output )

The results at this time are as follows.

 {'output_text': ' 3 days'}

PDF Summary

langchain-sagemaker-endpoint-pdf-summary.ipynb explains how to do PDF Summery with SageMaker Endpoint based on Falcon FM.

First, read the PDF file stored in S3 using PyPDF2 and extract the text.

 import PyPDF2
from io import BytesIO

sess = sagemaker . Session ()
s3_bucket = sess . default_bucket ()
s3_prefix = 'docs'
s3_file_name = '2016-3series.pdf'   # S3의 파일명


s3r = boto3 . resource ( "s3" )
doc = s3r . Object ( s3_bucket , s3_prefix + '/' + s3_file_name )
        
contents = doc . get ()[ 'Body' ]. read ()
reader = PyPDF2 . PdfReader ( BytesIO ( contents ))
        
raw_text = []
for page in reader . pages :
    raw_text . append ( page . extract_text ())
contents = ' n ' . join ( raw_text )

new_contents = str ( contents ). replace ( " n " , " " )

Since the document is large, use RecursiveCharacterTextSplitter to separate it into chunks and save it in Document. Afterwards, summarize using load_summarize_chain.

 from langchain . text_splitter import CharacterTextSplitter
from langchain . text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter ( chunk_size = 1000 , chunk_overlap = 0 )
texts = text_splitter . split_text ( new_contents ) 

from langchain . docstore . document import Document
docs = [
    Document (
        page_content = t
    ) for t in texts [: 3 ]
]

from langchain . chains . summarize import load_summarize_chain
from langchain . prompts import PromptTemplate

prompt_template = """Write a concise summary of the following:


{text}


CONCISE SUMMARY """

PROMPT = PromptTemplate ( template = prompt_template , input_variables = [ "text" ])
chain = load_summarize_chain ( llm , chain_type = "stuff" , prompt = PROMPT )
summary = chain . run ( docs )

chain_type

• stuff puts all the chunks into one prompt. Thus, this would hit the maximum limit of tokens.
• map_reduce summarizes each chunk, combines the summary, and summarizes the combined summary. If the combined summary is too large, it would raise error.
• refine summarizes the first chunk, and then summarizes the second chunk with the first summary. The same process repeats until all chunks are summarized.

ETC

LangChain by Bedrock

 from langchain import Bedrock
from langchain . embeddings import BedrockEmbeddings

llm = Bedrock ()

print ( llm ( "explain GenAI" ))

Reference

LangChain Docs

LangChain - github

SageMaker Endpoints

2-Lab02-RAG-LLM

AWS Kendra Langchain Extensions

QA and Chat over Documents

LangChain - Modules - Language models - LLMs - Integration - SageMakerEndpoint

LangChain - EcoSystem - Integration - SageMaker Endpoint

Ingest knowledge base data ta Vector DB