Preset Config Menu

❓ What Are Model Presets?

Model Presets are predefined configurations that bundle commonly used settings for AI models.

⭐ 3 Things You Need to Know

1. Create a Preset for each model and task

   • Every model can require different settings.
   • Every task can benefit from different settings.

2. Is your task creative, or do you need a more focused response?

   • Focus on the Temperature and Top-p settings. For example, the deterministic (focused) preset uses lower Temperature and Top-p values.
   • If you want to learn more about this, see Samplers.

3. (Max) Context Size sets the maximum amount of tokens your model gets fed with

   • This value can be too short for e.g. a very large Char. Make sure to understand what a Context is. It is one of the most important concepts to understand when working with Language Models.
   • Increase it according to what the model supports (see Model Menu), but be aware that larger context sizes can significantly affect token generation speed and system resource usage.

Why So Many Settings?

AI models are highly flexible tools, and small changes in their configuration can dramatically affect how they respond. These settings exist to let you fine-tune the model for:

• Tone and style (e.g. creative vs. factual)
• Performance (e.g. short replies vs. long reasoning)
• System resources (e.g. reducing context size or RAM usage on limited hardware)

While the number of options may seem overwhelming at first, presets and automatic defaults help make the process approachable — while still giving power users the control they need to configure their models manually.

What Are Samplers?

Samplers define how the model selects the next word or token when generating a response. They influence the creativity, diversity, and predictability of the output.

For example, if you're looking for a factual summary or trying to improve code, it's best to avoid samplers designed to reduce repetition or prioritize novelty, since these can introduce unnecessary variation. However, those same samplers can be valuable for tasks like roleplay or storytelling, where variety and unpredictability can enhance the experience.

Important Samplers

Standard Samplers

• Temperature: Think of this as a randomness dial. Higher values (>=1.0) let the AI be more adventurous with its word choices. Lower values (0.2) make it stick to safer, more predictable words. At Temperature = 0, it always chooses exactly the most likely next word (purely greedy).

• Top-p: Imagine sorting all possible next words from most to least likely and then picking just enough words so their combined chances reach a certain percentage p (like 90%). The AI then picks from that smaller set.

• Min-p: It throws out any words that are below a percentage of the top choice's chance. So if the best guess has 50% probability and your min-p is 0.1, words with less than 5% chance get removed. This keeps things on track without strangling diversity.

tip

This site provides a good Visualization🔗

Additional Samplers

• Exclude Top Choices (XTC): A sampler for creative tasks that occasionally (based on the probability setting) removes the highest-probability tokens (top choices), then samples from the remainder to promote creativity and avoid clichés.

• Don't Repeat Yourself (DRY): Detects if continuing with a given token would extend a previously seen sequence in the prompt or generation history and applies an exponentially increasing penalty to discourage repetition.

Setting	What It Does for You
Temperature	Controls how adventurous the text feels
Top-p	Keeps a dynamic pool of plausible next words
Min-p	Filters out unlikely words based on top-choice strength
Exclude Top Choices (XTC)	Occasionally removes top picks to spark creativity
Don't Repeat Yourself (DRY)	Stops the AI from parroting phrases it already used

There are more than these, but these are all you need.

info

Different samplers suit different tasks. Creative writing benefits from more randomness, while technical tasks like summarizations, translations, transformations, ... may prefer more predictable sampling. You can use no sampler, a combination of samplers, or all of them.

Prompt Format

Prompt Format is managed automatically when you use a Text AI Model.

The application automatically selects the appropriate prompt format for the model whenever the required information is available. In most cases, you do not need to configure it yourself.

If you have experience with model configuration, you can still set the Prompt Format manually.

Why does Prompt Format matter? There is no universal standard for how models are fine-tuned to behave in a chat-like manner, each model builder can handle it differently.

Using the appropriate prompt format helps ensure that the model understands and follows a conversational structure, which can be especially important for smaller or less capable models.

Performance - GPU

When using a GPU runtime (see Settings), GPU layers are managed automatically. The app will try to use up to 80% of your GPU's VRAM by default.

If this fails or you have experience with model configuration or need to troubleshoot a specific setup, you can still set the number of GPU layers manually.

Optimizations and RAM Usage

Text Model Loading Optimizations

Text Model Loading Optimizations is enabled by default to improve model loading times.

• Enabled: Faster loading, but higher system RAM usage, especially with GPU layers.
• Disabled: Slower loading, but can reduce system RAM usage by avoiding fully loading the model into RAM when using GPU layers.