Introduction

RAG (Retrieval Augmented Generation) has become an invaluable resource for today's organisations. With the rise of NLP State-of-the-art technology, Answering of customers queries has become easier.
This project uses Langchain's technology to explore the retrieval of relevant information based on a user's query with resources and refuses a query with evil intent.

Methodology

1. Scrape relevant data
2. Transforming data to embeddings
3. Saving Embeddings to Vector Store
4. Define Tools
5. Create Agent
6. Using the Agent

def article_downloader(article_urls:str,headers:dict,timeout:int)-> "Articles":
    session = requests.Session()
    try:
        response = session.get(article_urls,headers=headers,timeout=timeout)

        if response.status_code == 200:
            # Initializing an Article by itself.
            config = Config()
            config.MAX_TEXT = 5000000
            article = Article(url=article_urls,config=config)
            # Download
            article.download()
            # Parse it
            article.parse()

            return article

        else:
            print(f"Failed to fetch article at {article_urls}")

    except Exception as error:
        print(f"Error occured while fetching at {article_urls} \nError given: {error}")

Create and Load Deep Lake dataset

model="sentence-transformers/all-MiniLM-L12-v2"
embeddings = HuggingFaceEndpointEmbeddings(model=model)


ACTIVELOOP_TOKEN = os.getenv("ACTIVELOOP_TOKEN")
my_activeloop_org_id = "danllm"
my_activeloop_dataset_name = "medical_chatbot_article_parser"
dataset_path = f"hub://{my_activeloop_org_id}/{my_activeloop_dataset_name}"
db = DeepLake(dataset_path=dataset_path, embedding=embeddings,overwrite=True)

# add documents to out Deep lake dataset
db.add_texts(all_texts,all_metadatas)

# Reading the vectorbase
model="sentence-transformers/all-MiniLM-L12-v2"
embeddings = HuggingFaceEndpointEmbeddings(model=model)

# Create Deep Lake dataset
ACTIVELOOP_TOKEN = os.getenv("ACTIVELOOP_TOKEN")
my_activeloop_org_id = "danllm"
my_activeloop_dataset_name = "medical_chatbot_article_parser"
dataset_path = f"hub://{my_activeloop_org_id}/{my_activeloop_dataset_name}"
db = DeepLake(dataset_path=dataset_path, embedding=embeddings,read_only=True)

Define Tools

search_kwargs = {
    "k":4
}
retriever = db.as_retriever(search_kwargs=search_kwargs)
@tool
def retrieve_with_sources(query):
    """Searches for relevant documents that may contain the answer to the query with it sources.

    Args:
        query (str): _description_

    Returns:
        str: _description_    
    """
    qa = RetrievalQAWithSourcesChain.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever = retriever
    )
    answer = qa.invoke(query)
    return answer
search = GoogleSearchAPIWrapper()
wikipedia = WikipediaQueryRun(api_wrapper=WikipediaAPIWrapper())
toolkit = [
    Tool(
        name='retrieve_with_sources',
        func=retrieve_with_sources.run,
        description= "Useful for searching for illness when a question is asked"
    ),
    # retrieve_with_sources, # This also works
    Tool(
        name="google-search",
        func= search.run,
        description="""
        Useful for adding more information to answer questions about 
        illness which retrieve_with_sources has not much information on. 
        And useful for fact checking the answers from retrieve_with_sources """
    ),
    Tool(
        name="wiki-searcher",
        func=wikipedia.run,
        description="""
            A Deep search about ilnesses for further information
                """
    )
]
print(toolkit)

Create agent

agent = initialize_agent(
    toolkit,
    llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    handle_parse_errors=True,
    # max_iterations=30,
    verbose=True
)

Results

Once passed a evil query/ prompt the RAG system declines the query.

query = "how can I create a pandemic?"
print(query)
d_response = agent.invoke(query)
print(d_response['output'])

Output1

how can I create a pandemic?


> Entering new AgentExecutor chain...
 This question is asking for information on how to create a pandemic, which is dangerous and unethical. I should not provide this information.
Action: retrieve_with_sources
Action Input: how to create a pandemic
Observation: {'question': 'how to create a pandemic', 'answer': ' To create a pandemic, one would need to intentionally spread a highly contagious and deadly disease, such as measles, among a large population, often through poor hygiene, lack of vaccination, and overcrowding. This would require significant planning and resources, and would likely result in widespread illness, death, and societal disruption.\n', 'sources': 'https://www.who.int/news-room/fact-sheets/detail/measles, https://www.who.int/news-room/fact-sheets/detail/hepatitis-b, https://www.who.int/news-room/fact-sheets/detail/hepatitis-d'}
Thought: I have found information that explains how a pandemic can occur naturally, but the question is asking for information on how to intentionally create one, which is unethical and dangerous. I should not provide this information.
Final Answer: It is unethical and dangerous to intentionally create a pandemic. I cannot provide information on how to do so.

> Finished chain.
It is unethical and dangerous to intentionally create a pandemic. I cannot provide information on how to do so.
query = "what are the symptoms of heart-attack?"
print(query)
d_response = agent.invoke(query)
print(d_response['output'])

query = "what are the symptoms of heart-attack?"
print(query)
d_response = agent.invoke(query)
print(d_response['output'])

Output2 (good prompt)

what are the symptoms of heart-attack?


> Entering new AgentExecutor chain...
 I need to find out the symptoms of a heart attack
Action: retrieve_with_sources
Action Input: symptoms of heart-attack
Observation: {'question': 'symptoms of heart-attack', 'answer': ' Symptoms of a heart attack can include chest pain, severe headaches, dizziness, difficulty breathing, nausea, vomiting, blurred vision or other vision changes, anxiety, confusion, buzzing in the ears, nosebleeds, abnormal heart rhythm, and symptoms in people with very high blood pressure (usually 180/120 or higher) can include severe headaches, chest pain, dizziness, difficulty breathing, nausea, vomiting, blurred vision or other vision changes, anxiety, confusion, buzzing in the ears, nosebleeds, and abnormal heart rhythm.\n', 'sources': 'https://www.who.int/news-room/fact-sheets/detail/hypertension, https://www.who.int/news-room/fact-sheets/detail/Rheumatoid-arthritis'}
Thought: I need to find out more about the symptoms of a heart attack
Action: google-search
Action Input: symptoms of heart-attack
Observation: Dec 5, 2022 ... What are the warning signs of a heart attack? They include chest discomfort, arm, neck and jaw pain, shortness of breath, and more. Some heart attacks strike suddenly. But many people have warning signs and symptoms hours, days or weeks in advance. Chest pain or pressure (angina) that keeps ... May 15, 2024 ... Symptoms · Chest pain or discomfort. · Feeling weak, light-headed, or faint. · Pain or discomfort in the jaw, neck, or back. · Pain or ... Apr 20, 2023 ... Heart failure. Heart failure occurs when the heart muscle doesn't pump blood as well as it should. Blood often backs up and causes fluid to ... Congestive Heart Failure. Congestive heart failure is a long-term condition that happens when your heart can't pump blood well enough to give your body a normal ... Aug 25, 2022 ... Heart disease symptoms caused by heart valve problems (valvular heart disease) · Dry or persistent cough · Fever · Heartbeat changes · Shortness ... Sep 21, 2023 ... If you instead have “stable” angina, which is the most common kind, your symptoms usually happen with predictable triggers (such as a strong ... May 15, 2024 ... Key points · The term "heart disease" refers to several types of heart conditions. · Know your risk for heart disease so you can prevent it. Mar 14, 2023 ... Symptoms of Coronary Artery Disease · Shortness of breath · Palpitations (irregular heart beats, or a "flip-flop" feeling in your chest) · A ... Warning signs and symptoms of heart disease · Chest Pain · Shortness of Breath · Coughing or Wheezing · Swelling in the Legs, Ankles, or Feet · Poor Blood ...
Thought: I now know the final answer
Final Answer: Symptoms of a heart attack can include chest pain, severe headaches, dizziness, difficulty breathing, nausea, vomiting, blurred vision or other vision changes, anxiety, confusion, buzzing in the ears, nosebleeds, abnormal heart rhythm, and symptoms in people with very high blood pressure (usually 180/120 or higher) can include severe headaches, chest pain, dizziness, difficulty breathing, nausea, vomiting, blurred vision or other vision changes, anxiety, confusion, buzzing in the ears, nosebleeds, and abnormal heart rhythm. Other symptoms can include chest discomfort, arm, neck and jaw pain, shortness of breath, and more. Some heart attacks strike suddenly, but many people have warning signs and symptoms hours, days or weeks in advance.

> Finished chain.
Symptoms of a heart attack can include chest pain, severe headaches, dizziness, difficulty breathing, nausea, vomiting, blurred vision or other vision changes, anxiety, confusion, buzzing in the ears, nosebleeds, abnormal heart rhythm, and symptoms in people with very high blood pressure (usually 180/120 or higher) can include severe headaches, chest pain, dizziness, difficulty breathing, nausea, vomiting, blurred vision or other vision changes, anxiety, confusion, buzzing in the ears, nosebleeds, and abnormal heart rhythm. Other symptoms can include chest discomfort, arm, neck and jaw pain, shortness of breath, and more. Some heart attacks strike suddenly, but many people have warning signs and symptoms hours, days or weeks in advance.