How to Properly Prompt an LLM?

As AI and Large Language Models (LLMs) transform the way we approach data science projects, it's important to understand how to use them effectively. The first and most important step is learning how to prompt LLMs correctly.

While smaller LLMs lack the capabilities of larger models, they can be very efficient in cost and resources for simpler tasks when prompted appropriately. You can use them locally on your GPU machines without any concerns about data privacy or security.

Based on my experience across various business challenges, here are some tips for crafting effective text-based prompts, especially for smaller LLMs:

1) Know the size of your LLM

Larger models generally handle longer and more complex prompts better than smaller models. If you're working with a small LLM (let's say under 10 billion parameters), it's essential to use shorter (count the number of prompt tokens), simpler, and more precise prompts.

2) Use a breakdown strategy

Think of it as explaining something to a child who knows nothing about the subject. Start with the basic information the model needs, and gradually build upon it, introducing more complexity step by step.

Example:

Suppose you want to explain to the LLM what input you are passing to it:

• "The Input is about the sales performance of Company X over a three-year period.
• The Input might include typos.
• Company X sells Products Y, Z, and G.
• Company X sells Product Z Exclusively in Europe."

This way, the LLM can follow the steps and gradually understand the details of the input more effectively.

3) Structure your prompt into sections

Break your prompt into clearly defined parts, such as :
###Input
###Definitions
###Rules
###Examples
###Format
###Output

This can vary depending on the task but helps the LLM process information more effectively.

4) Know your model's preferred prompt format and use proper punctuation

Although LLMs can generally adapt to different formats, their performance can improve significantly if you use the format they "prefer"!

Example:

In my experience, the Gemini model responds well to highlighted text, while Gemma (a smaller model) works better with < highlighted text > or HIGHLIGHTED TEXT when you want to highlight or emphasize something in the prompt.

Additionally, bullet points and indentations, punctuations like question marks at the end of questions, comma-separated phrases, line spacing, etc. improve clarity and task understanding.
Here are three steps to determine the optimal prompt format for your model:

• a) Review the documentation of the LLM. Most AI developers provide prompting guides for their models. Gemini Prompting Strategies and Mistral Prompting Capabilities are two references for Gemini and Mistral models respectively.
• b) Research the formats and structures others have used. Many helpful resources can be found on platforms like GitHub and elsewhere.
• c) Combine the insights from the first two steps and experiment through trial and error!

Note: Avoid excessive formatting and punctuation, especially with smaller LLMs, as it adds unnecessary complexity to the prompt and diminishes the quality of the outputs. They may provide what is requested in some cases, but you are likely to notice more mistakes when testing a batch of inputs.

5) Maintain consistency

When referring to specific terms, be consistent throughout the entire prompt. For instance, if you use the term "Customer," don’t switch to synonyms like "User" or "Client" later on. Keep the terminology and formatting uniform across all sections of the prompt.

6) Make life easier for your LLM!

While some simple sentences in our human language may look pretty easy and understandable; they might put LLM in a challenge to give you the answer!
The best way to truly understand an LLM’s language capabilities is through experimentation and exploration.

Example:

If your input is a text about countries and your prompt asks something like:
"Give the capital of Germany, the capital of Italy, and the population of the smallest country as the outputs."
A more effective version of this prompt would be:
"Answer the following questions based on the information in the Input:

• What is the Capital of Germany?
• What is the Capital of Italy?
• What is the Population of the Smallest Country mentioned in the Input?

If you do not find the answer to any of the questions, your answer is always [Not Available]."

With this approach, you’ve made several key improvements for the LLM. It now understands that it needs to answer three separate questions based solely on the input provided. Additionally, by allowing it the option to respond with "Not Available," the model avoids struggling to generate an answer when the information doesn’t exist, reducing the chances of producing incorrect responses.
Overall, it's easier for an LLM to answer clear, predefined questions than to extract information on its own.

7) Refine your prompt using a more powerful model from the same family

Before finalizing your prompt, or while drafting its sections, consult a more powerful LLM from the same family to improve clarity and effectiveness. For instance, if you're working with Llama3-8B, you can pass the prompt to Llama3-70B or even Llama3-405B, explaining that it's intended for Llama3-8B, and ask for overall improvements or specific suggestions on any part of the prompt.

Conclusion

All the steps outlined above are crucial for writing a prompt that effectively addresses your task. I strongly recommend not skipping the final step, as it serves as a validation of all the previous steps.
Additionally, you can find more detailed prompting tips, particularly regarding system and user prompts, in the following links:
OpenAI Prompt Guide
Promptingguide.ai