If you get an error because you have previously cloned an older version of the repository, ensure you're in the correct repository directory and update it using:
git pull
This will update your local repository with the latest changes from the remote repository.
Step 3: Navigate to the relevant project directory
Change to the directory where the example is located.
cd examples/pipelines/demo-question-answering
Step 4: Changes in the requirements.txt, app.py and config.yaml files
Now that you’ve cloned the repo, let’s walk through some changes you’ll make to the files in the directory. This hands-on step will help you better understand how everything works under the hood.
Changes in the requirements.txt file:
To ensure you have all necessary dependencies, update the requirements.txt file with the following content:
After cloning the repository and navigating to the correct directory, modify the app.py file with the following content to ensure proper functionality:
import logging
import sys
import click
import pathway as pw
import yaml
from dotenv import load_dotenv
from pathway.udfs import DiskCache, ExponentialBackoffRetryStrategy
from pathway.xpacks.llm import embedders, llms, parsers, splitters
from pathway.xpacks.llm.question_answering import BaseRAGQuestionAnswerer
from pathway.xpacks.llm.vector_store import VectorStoreServer
# Set your Pathway license key here to use advanced features.
# You can obtain a free license key from: https://pathway.com/get-license
# If you're using the Community version, you can comment this out.
pw.set_license_key("demo-license-key-with-telemetry")
# Set up basic logging to capture key events and errors.
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(name)s %(levelname)s %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
# Load environment variables (e.g., API keys) from the .env file.
load_dotenv()
# Function to handle data sources. The example uses a local folder first,
# but you can easily connect to Google Drive or SharePoint as alternative data sources.
def data_sources(source_configs) -> list[pw.Table]:
sources = []
for source_config in source_configs:
if source_config["kind"] == "local":
# Reading data from a local directory (default is the 'data' folder).
# This is the first option used in this example, but it's flexible for other sources.
source = pw.io.fs.read(
**source_config["config"],
format="binary",
with_metadata=True,
)
sources.append(source)
elif source_config["kind"] == "gdrive":
# Reading data from a Google Drive folder.
# Requires OAuth credentials specified in the config.yaml file.
source = pw.io.gdrive.read(
**source_config["config"],
with_metadata=True,
)
sources.append(source)
elif source_config["kind"] == "sharepoint":
try:
# Import the SharePoint connector for reading data from SharePoint.
import pathway.xpacks.connectors.sharepoint as io_sp
# Reading data from a SharePoint folder.
# Note: The SharePoint connector is part of Pathway's commercial offering.
source = io_sp.read(**source_config["config"], with_metadata=True)
sources.append(source)
except ImportError:
# If SharePoint is configured but the connector isn't available, exit the program.
print(
"The Pathway Sharepoint connector is part of the commercial offering. "
"Please contact us for a commercial license."
)
sys.exit(1)
return sources
# Command-line interface (CLI) function to run the app with a specified config file.
@click.command()
@click.option("--config_file", default="config.yaml", help="Config file to be used.")
def run(config_file: str = "config.yaml"):
# Load the configuration from the YAML file.
with open(config_file) as config_f:
configuration = yaml.safe_load(config_f)
# Fetch the GPT model from the YAML configuration.
GPT_MODEL = configuration["llm_config"]["model"]
# Initialize the OpenAI Embedder to handle embeddings with caching enabled.
embedder = embedders.OpenAIEmbedder(
model="text-embedding-ada-002",
cache_strategy=DiskCache(),
)
# Set up OpenAI's GPT model for answering questions with retry and caching strategies.
chat = llms.OpenAIChat(
model=GPT_MODEL,
retry_strategy=ExponentialBackoffRetryStrategy(max_retries=6),
cache_strategy=DiskCache(),
temperature=0.05, # Low temperature for less random responses.
)
# Host and port configuration for running the server.
host_config = configuration["host_config"]
host, port = host_config["host"], host_config["port"]
# Initialize the vector store for storing document embeddings in memory.
# This vector store updates the index dynamically whenever the data source changes
# and can scale to handle over a million documents.
doc_store = VectorStoreServer(
*data_sources(configuration["sources"]),
embedder=embedder,
splitter=splitters.TokenCountSplitter(max_tokens=400), # Split documents by token count.
parser=parsers.ParseUnstructured(), # Parse unstructured data for better handling.
)
# Create a RAG (Retrieve and Generate) question-answering application.
rag_app = BaseRAGQuestionAnswerer(llm=chat, indexer=doc_store)
# Build the server to handle requests at the specified host and port.
rag_app.build_server(host=host, port=port)
# Run the server with caching enabled, and handle errors without shutting down.
rag_app.run_server(with_cache=True, terminate_on_error=False)
# Entry point to execute the app if the script is run directly.
if __name__ == "__main__":
run()
Changes in the config.yaml file:
Rename the existing app.yaml file to config.yaml.update the content of the config.yaml file with the configuration below. It's easy to comprehend if you read the comments.:
llm_config:
model: "gpt-3.5-turbo" # Specify the GPT model to use for question answering.
host_config:
host: "0.0.0.0" # Host for running the app.
port: 8000 # Port for the app.
cache_options:
with_cache: True # Enable caching for better performance.
cache_folder: "./Cache" # Directory to store cached data.
sources:
- local_files: # Data source is local files.
kind: local
config:
path: "data/" # Path to the local data directory.
# Optionally, you can configure Google Drive or SharePoint as additional data sources.
# For Google Drive:
# - google_drive_folder:
# kind: gdrive
# config:
# object_id: "1cULDv2OaViJBmOfG5WB0oWcgayNrGtVs"
# service_user_credentials_file: SERVICE_CREDENTIALS
# refresh_interval: 5
# For SharePoint (commercial offering):
# - sharepoint_folder:
# kind: sharepoint
# config:
# root_path: ROOT_PATH
# url: SHAREPOINT_URL
# tenant: SHAREPOINT_TENANT
# client_id: SHAREPOINT_CLIENT_ID
# cert_path: SHAREPOINT.pem
# thumbprint: SHAREPOINT_THUMBPRINT
# refresh_interval: 5
Make sure to save the file after applying the changes.
Step 5: Update your .env File with your OpenAI API Key
In the .env file already present in the project, replace the placeholder API key (sk-xxxxxx) with the OpenAI API key you retrieved from the website.
OPENAI_API_KEY=<your_openai_api_key>
Make sure to save the file after updating.
Step 6: Build the Docker Image
Now, let’s build the Docker image. This step might take a few minutes depending on your machine. Ensure you have enough space (approximately 8 GB).
docker build -t rag .
The -t rag part is tagging the Docker image with the name ‘rag’. Whereas the . at the end specifies the build context directory, which is the current directory. This tells Docker to look for the Dockerfile in the current directory and include any files/subdirectories in the build context.
Step 7: Run the Docker Container
Run the Docker container, mounting (described below) the data folder, and exposing port 8000.
For Windows:
docker run -v "%cd%/data:/app/data" -p 8000:8000 rag
For Linux/Mac:
docker run -v "$(pwd)/data:/app/data" -p 8000:8000 --env-file .env rag
Note: You will see the logs for parsing & embedding documents in the Docker image logs. Give it a few minutes to finish up on embeddings. You will see 0 entries (x minibatch(es)) have been... message. If there are no more updates, this means the app is ready for use!
Handling Port Conflicts: If port 8000 is already in use and you see an error related to it, you can specify a different port. For example, if you want to use port 8080 instead, modify the command as follows:
For Windows:
docker run -v "${PWD}/data:/app/data" -p 8000:8000 rag
For Linux/Mac:
docker run -v "$(pwd)/data:/app/data" -p 8080:8000 --env-file .env rag
This will map port 8080 on your local machine to port 8000 in the Docker container. Just remember to update the port in the next step as well.
Open up another terminal window and follow the next steps.
Step 8: Check the List of Files
You will see the logs for parsing & embedding documents in the Docker image logs. Give it a few minutes to finish up on embeddings, you will see 0 entries (x minibatch(es)) have been… message. If there are no more updates, this means the app is ready for use!
Now, let's retrieve the list of files from which our LLMs will gather information. To check the available inputs and associated metadata, you can use either the curl command (for Linux/Mac users) or Invoke-WebRequest (for Windows PowerShell users) as described below:
This will also return the list of files with the associated metadata, similar to the example above.
Key Differences:
Use curl for Linux/Mac environments or for users who have installed curl on Windows.
Use Invoke-WebRequest for users working within Windows PowerShell.
This ensures that no matter which environment you're using, you can retrieve the list of documents and associated metadata to confirm that the app is ready for queries.
Step 9: Last Step – Run the RAG Service
You can now run the RAG service. Start by asking a simple question. For example:
For Linux/Mac Users (curl command):
You can use the following curl command to ask a simple question to the RAG service:
curl -X 'POST' \
'http://0.0.0.0:8000/v1/pw_ai_answer' \
-H 'accept: */*' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "What is the start date of the contract?"
}'
Note: If you change the port from 8000 to another value (e.g., 8080), make sure to update the curl command accordingly. For example, replace 8000 with 8080 in the URL.
It should return the following answer:
December 21, 2015
For Windows Users (PowerShell Invoke-WebRequest):
If you're using PowerShell on Windows, use the Invoke-WebRequest command to ask the same question:
Invoke-WebRequest -Uri 'http://0.0.0.0:8000/v1/pw_ai_answer' `
-Method POST `
-Headers @{ "accept" = "*/*"; "Content-Type" = "application/json" } `
-Body '{"prompt": "What is the start date of the contract?"}'
Note: Just like with curl, if you change the port to a different value (e.g., 8080), make sure to update the URL in the Invoke-WebRequest command.
This will return the same response with the answer:
December 21, 2015
But how do you tweak this for your use-case? Let's see that by understanding the contents of the repo which just used.
In the (which you’ve already starred ⭐), you’ll see the demo question-answering template that uses YAML files to help you quickly set up the pipeline. While this makes Dockerized RAG applications 10x easier to productionize, in this bootcamp, we will take a different approach. You’ll skip the shortcuts and dive deeper into the application code, to build a solid foundation to create powerful customizations as you progress in your Gen AI journey.
This will return the list of files e.g. if you start with the provided in the demo, the answer will be as follows: