Ollama (Local Model)

Ollama is for those who prefer open-source models and value running them locally on their own hardware.

This can offer potential benefits in terms of cost savings, data privacy, and customization options compared to using closed-source models from companies like OpenAI or Anthropic, which might have usage restrictions or data sharing concerns.

Additionally, the ability to run models locally may be a pro for users with limited or unreliable internet connectivity.

(You can also read up more on our Ollama local model management interface.)

Requires Ollama Enabled

This plugin requires the PROMPT_OLLAMA_HOST from the quickstart installation.

Setting up your agent

• After setting up your system, navigate to the "Add Panel" section and select your Ollama plugin.
• Enter the Context Size for the model you plan on using.
• Set a System Message, this will help guide your model if you'd like it to behave a certain way or do something specific.
• Create your agent.
• Once your agent is created, you'll be able to select your desired local model in the top-right corner of your message thread.

Once you've added your API key, your agent will be ready to use!

Document upload ( and /append command)

This plugin allows you to upload text-based documents (e.g., .txt, .doc, .md) to enhance your chat experience.

The plugin will:

• Convert your uploaded files into plain text.
• Store these text versions for your AI model to access.

When you want to use your document when chatting:

• Use the /append command in your chat
• Followed by your document(s) filename(s) /file filename.pdf
• And then any additional questions you have to add the documents to your context when chatting.

An example prompt with the command would look like:

/append /file filename.pdf What are the core themes I should be aware of in this document?

Note on longer context

If your document is too large to fit within your chosen model's context window it will be excluded from your message context and you will receive a warning.

For larger documents, consider using our Document Lookup (RAG) agent plugin.

Vision support

This plugin offers "vision" capabilities, meaning it can process uploaded images (e.g., .png, .jpg) alongside your text input.

Example vision usage

After uploading a receipt image, you could ask:

"What are the amounts listed in this invoice? Could you also categorize the purchase for me?"

Review the code

Want to review how this model is made, or use it as the base to creating a new agent plugin?

You can find the code that powers this plugin available here:
https://github.com/promptpanel/promptpanel/tree/main/plugins/ollama