Skip to content
Chat-Lab.AI | Documentation

Tips for Making Better Assistants

Below are some tips and tricks on how to improve your Assistant’s performance based on my experimentating while making CostGPT.

Best practices

These are some of the best practices that I have developed from extant research as well as my own experimentation with working with the Assistants API.

1. Don’t Forget to Hit the Save Button

  • Your Assistant will not be created and/or your changes will not update otherwise.

2. Write Clear Instructions

  • Take some time to think about how users will interact with your chatbot. Then, experiment in the playground with possible user prompts and iterate the instructions until the bot performs the way you expect.It’s ok if the final instructions are long, provided they get the chat assistant to behave as indicated.
  • To help you think about what to include in your instructions, below are the instructions that I used in CostGPT. You can use it as a starting point, and iterate over these instructions until the bot performs to your liking.
INSTRUCTIONS:

You are a friendly TA specialized in managerial accounting who provides concise answers to queries from undergraduate students in an introductory-level managerial accounting class. All of the course materials, including lecture notes and course syllabus, are part of your knowledge base, and you should search these files first when responding to user prompts.  

You use a friendly conversational tone, ask clarifying questions, and employ a Socratic teaching method.  

Your responses should provide brief answers, prioritizing relevant information from the lecture notes in your files over your general accounting knowledge.  

You should do whatever you are trained to do to help students study the material from the lecture notes in your files, such as explain concepts, make flashcards, generate multiple choice quiz questions.  You should also let students know what capabilities you have that might help them study.  

Where appropriate, you should look up real-world examples on the internet, relating these examples to specific concepts and identifying relevant chapters discussed in the lecture notes in your files.  

For queries about accounting topics not covered in the chapters, you can rely on your baseline knowledge along with browsing and Python tools for extra information or calculations. This approach ensures students receive relevant, succinct responses while having the option to request more in-depth information if needed.  

You do not answer questions about topics unrelated to the course material in your files. You should decline to answer unrelated questions and refer users to the ChatGPT website (chat.openai.com) if they want to learn more about any unrelated topics.

3. File management

  • Use smaller files, even if you have to upload more of them. Research shows that these models tend to ignore relevant context from the middle portion of very long documents.
  • Give your files descriptive file names that follow your course format and refers to the course content in a way that you and students would use naturally. For instance, the filenames for my lecture notes followed this template: Chapter N-Chapter Title.docx. This enabled students to receive relevant responses whether they asked specific questions (e.g.,“what is job-order costing?”) or broad questions (e.g., “what is chapter 3 about?”).
  • Using chapter numbers in the filename also enabled a short heuristic to encourage the bot to refer to the files in its knowledge-base—students could ask questions like
    • “According to chapter 14, what is NPV?”
    • “Please provide an example IRR problem like the sample problems in chapter 14 notes.”
  • Referring to the chapter in the question generally led to better responses.
  • The Assistants API accepts PDFs and PowerPoint files, but in my experience, it works best the closer the files are to plain text format—.docx, .rtf, .md, .json, .csv, .txt, etc.

4. Compare model performance

  • The gpt-4-1106-preview model is the most performant model, with the largest context window (128K tokens), however it is also the most expensive. While OpenAI’s models are generally cheap to use, the Assistants API will submit the following with every message sent from your app: the prompt, the conversation history (i.e., thread), the assistant’s instructions, and relevant information retrieved from your files. All of this context for each message can easily increase the token count, which can make this model quite expensive.
  • Therefore, if you plan on uploading your own files for information retrieval, you should experiment with the gpt-3.5-turbo-1106 model and decide whether the more expensive model is worth the additional cost.

5. Enable Code Interpreter

  • If your assistant must perform math calculations, enable code interpreter.