Managing LLM Profiles in Volt Vector

Managing LLM Profiles in Volt Vector

Volt Vector allows you to create custom profiles to configure how the AI assistant interacts with large language models (LLMs) like GPT-3. Profiles give you control over cost, quality, and potential biases when generating AI responses.

Creating a New LLM Profile #

To start, go to the options page and click on “LLM Profiles”. This is where you can create and manage profiles that connect to LLMs.

  • Give the profile a unique name
  • Select the LLM service (e.g. OpenAI)
  • Enter your API key for the service
  • Adjust the settings like max tokens, temperature, etc.

Once created, you can set the profile as the default or choose it when starting a new chat.

Key LLM Profile Settings #

Here are some of the key settings to understand:

Max Tokens #

  • Controls length of LLM responses
  • Default is 2048 tokens (~1500 words)
  • Can increase up to 4096 tokens for longer responses

Temperature #

  • Controls creativity vs. correctness
  • Lower temperature (0.0-0.5) gives more predictable responses
  • Higher temperature (>0.5) produces more creative but possibly incoherent responses

Top-P #

  • Controls response diversity
  • Lower Top-P gives more diverse responses
  • Higher Top-P (1.0) gives more repetitive/predictable responses

Frequency and Presence Penalties #

  • Controls repetition of concepts
  • Positive values reduce repetition, negative values increase it

Tips for Optimization #

  • Start with default settings and tweak gradually
  • Create multiple profiles for different purposes
  • Lower max tokens, temperature, and Top-P can reduce cost
  • Higher settings may improve quality but increase cost
  • Monitor chat sessions and adjust profiles accordingly

Proper LLM profile configuration helps balance cost, quality, and coherence in Volt Vector chats. Take time to experiment with the settings and options to find your ideal setup.