After configuring your AI provider, model, instructions, response behavior, language settings, and fallback rules, you should test your assistant before relying on it for community support.
Testing helps you identify configuration problems and improve the quality and consistency of Novus AI responses.
What Should You Test?
A good test should cover more than simply checking whether the bot responds.
You should verify:
AI provider connectivity
AI model availability
AI Instructions
Response behavior
Knowledge Base usage
AI Routing
Multilingual responses
Fallback behavior
Discord channel configuration
Discord permissions
Testing each area separately makes it easier to identify where a problem originates.
Start With a Basic Response Test
Begin with a simple message in a Discord channel configured for Novus AI.
For example:
Hello! Can you help me?
If mention-only mode is enabled, mention Novus AI when sending the message.
A successful response confirms that the basic connection between Discord, Novus AI, and your AI provider is working.
Test Your AI Instructions
Next, test whether your assistant follows the general behavior you configured.
Ask several different questions and review whether Novus AI:
Uses the expected communication style
Provides an appropriate amount of detail
Follows your community-specific rules
Avoids behaviors you explicitly prohibited
Handles unsupported information correctly
Uses the expected language
If the assistant repeatedly ignores a rule, review how that instruction is written.
Clear and direct instructions generally work better than vague or conflicting instructions.
Test Knowledge Base Questions
Ask questions that should have clear answers inside your Knowledge Base.
Use realistic questions similar to those your members are likely to ask.
Verify that:
The correct information is used.
Important details remain accurate.
The assistant does not add unsupported information.
The response follows your AI Instructions.
The answer is understandable to the member.
If Novus AI cannot answer a question that exists in your Knowledge Base, review the content and make sure the information is clear and properly organized.
Test Unknown Questions
You should intentionally ask questions that are not covered by your Knowledge Base.
This helps verify your fallback behavior.
Novus AI should follow your configured rules instead of inventing community-specific information.
Check whether the assistant provides the expected fallback response or directs the member to the appropriate support option.
Test AI Routing
If your server uses AI Routing, create test questions for each route.
For example, if you have separate routes for technical support and general questions, test both types of concerns.
Verify that each question is handled using the appropriate route instructions.
If similar questions are frequently assigned to the wrong route, review the route descriptions and make them easier to distinguish.
Test Different Ways of Asking the Same Question
Members will not always use the exact wording found in your Knowledge Base.
Test multiple versions of the same question.
For example:
How do I register?
Where can I create an account?
Can you give me the registration link?
I need to make an account.
Novus AI should be able to understand that these questions have similar intent when the necessary information is available.
Test Multilingual Questions
If your Discord community uses multiple languages, test questions using the languages commonly used by your members.
Also test mixed-language messages when they are common in your community.
Verify that:
Novus AI understands the question.
The correct information is provided.
The response uses an appropriate language.
Important names, commands, links, and values remain correct.
Routing still works properly.
Test Response Behavior
Make sure Novus AI responds only where and when you expect it to.
Test:
An enabled AI channel.
A channel not configured for AI responses.
A direct mention of Novus AI.
A message without a mention.
Different member roles when relevant.
If the bot responds where it should not, review your response and channel configuration.
If it does not respond where expected, check both Novus AI settings and Discord permissions.
Test Longer Conversations
Do not limit testing to individual questions.
Have a short conversation with the assistant and ask follow-up questions.
For example:
Member: How can I create an account?
Then:
Member: What should I do after that?
This helps you evaluate whether your selected model can appropriately use the available conversation context.
Separate AI Problems From Configuration Problems
An incorrect response does not always mean the AI model is the problem.
When something goes wrong, identify which part of the configuration is responsible.
If the factual information is wrong, check the Knowledge Base.
If the assistant's behavior is wrong, check AI Instructions.
If the wrong specialized instructions are being used, check AI Routing.
If the bot responds in the wrong places, check Response Behavior and Channel Configuration.
If the bot cannot generate any AI response, check the AI Provider, API Key, Model, Quota, and Provider Status.
If the assistant guesses when information is unavailable, check your Fallback Instructions.
Change One Thing at a Time
When troubleshooting your AI configuration, avoid changing many settings simultaneously.
A better approach is:
Identify the unwanted behavior.
Determine which configuration controls that behavior.
Make one targeted change.
Save the configuration.
Test again.
Compare the result.
This makes it easier to understand which change actually solved the problem.
Create Realistic Test Questions
One of the best ways to test Novus AI is to create a collection of questions your real members commonly ask.
Include:
Simple questions
Detailed questions
Poorly worded questions
Different ways of asking the same question
Questions with missing information
Questions for each AI route
Questions in different languages
Questions the AI should not answer
Run these tests again whenever you make major changes to your configuration.
Review Before Going Live
Before allowing Novus AI to handle regular community support, confirm that:
Your AI provider is working.
Your selected model is available.
AI Instructions are being followed.
Knowledge Base information is accurate.
AI Routing behaves correctly.
Fallback responses work.
Multilingual responses are appropriate.
Response behavior is correct.
Discord permissions are properly configured.
Your assistant does not need to answer every possible question.
It is more important that Novus AI provides reliable information when available and handles unknown information appropriately.
Continue Improving Over Time
Testing should not stop after your initial setup.
Your community will introduce new questions, terminology, and situations that may reveal gaps in your configuration.
Use these situations to improve your Knowledge Base, instructions, routes, and fallback behavior.
Regular testing helps Novus AI remain useful as your community grows.
AI Configuration Complete
You have now completed the core AI Configuration setup.
The next major step is building the information Novus AI will use to answer community-specific questions.
Continue to the Knowledge Base documentation.