Skip to main content

Overview and Setup

What are we building​

In this tutorial we'll create an LLM application that can extract various metadata from a given article. This tutorial has the following phases:

Phase 1: We will manually copy-paste the content an article and extract various metadata using different OpenAI models.

Phase 2: We will setup a benchmark to continously evaluate the performance of our application as we iterate on it.

Phase 3: We will setup an AI Gateway with portkey such that we can experiment with a lot more LLM models.

Phase 4: Setup a prompt management system using pug that will let us compare more complex prompts.

Phase 5: We will replace the manually copied content with a web scraper that can automatically extract the content of the article given an URL.

What you will learn​

By the end of this tutorial you will know how to:

  • Build an LLM application using Palico
  • Preview your application using the Palico Playground
  • Setup benchmarks and run evaluations to measure performance of your application
  • Analyze and compare different variations of your application
  • Setup a modular application layer that let's you experiment and iterate on your application

Application Setup​

Create a Palico App​

Create a palico app by running the following command:

npx palico init <project-name>

This will create a basic Palico app with a single conversational LLM application. This starter LLM application uses OpenAI so you will need to provide your OpenAI API key in the .env file.

Add your OpenAI API Key​

Add your OpenAI API key to the .env file in the root of your project. You can get your OpenAI API key from https://platform.openai.com/api-keys.

OPENAI_API_KEY=<your-openai-api-key>

Bootstrap your Services​

Your Palico app has additional services associated with it, such as a database and tracing service. To bootstrap these services, run the following command:

npm run palico bootstrap

You will only need to run this command once when you first setup your code repository, or when you update the version of your @palico-ai/app package.

Start your Palico App​

To start your Palico app, run the following command:

npm start

You can modify your chat application in src/agents/chatbot/index.ts. You can also chat with your chatbot in the Palico Studio at http://localhost:5173/chat.