Abstract

Ensuring content aligns with ethical guidelines is a crucial challenge. Prompt Validator is a Node.js package designed to validate OpenAI prompts for violations such as hate speech, inappropriate content, or harmful material. By leveraging the OpenAI API, this tool helps developers ensure compliance with AI moderation guidelines before deploying AI-generated text.

This project contributes to Agentic AI by preventing harmful AI responses and promoting responsible AI communication.

Introduction

AI systems are becoming increasingly autonomous, producing large volumes of text-based content. However, unmoderated AI-generated responses may pose risks, including bias, misinformation, and unethical outputs. Prompt Validator was developed to mitigate these issues by providing automated validation of OpenAI prompts before they are processed.

The tool offers:

Automated Prompt Validation – Detects high-risk regular and AI-generated content.
Developer-Friendly Integration – Simple API for quick adoption.
Compliance with AI Moderation Guidelines – Prevents unsafe prompts from execution.
This solution aligns with the Agentic AI movement, ensuring AI systems operate responsibly, ethically, and safely.

Methodology

System Design
Prompt Validator is a lightweight, dependency-free Node.js package that integrates with OpenAI's moderation API to validate prompts before they are used.

Implementation
The validation process follows these steps:

Input a prompt (user-generated or AI-generated).
Send the prompt to OpenAI's moderation API for analysis.
Receive a classification result indicating whether the prompt is safe or violates guidelines.
Return a structured response to inform developers of potential risks.

Experiments

const { promptValidator } = require("prompt-validator");

const apiKey = "your-openai-api-key";
const prompt = "Your prompt text goes here";

promptValidator(apiKey, prompt)
.then((response) => console.log("Response:", response))
.catch((error) => console.error("Error:", error));

Results

Violation Detected:
{
"validate": "The message contains a high-level violation as it incites violence and harm towards others."
}
Safe Prompt:
{
"validate": "verified"
}

Conclusion

This project aligns with Agentic AI principles by enhancing AI safety while maintaining developer efficiency. Future improvements may include:

Enhanced ML-based filtering for more nuanced violations.
Support for additional AI moderation APIs beyond OpenAI.
Real-time moderation dashboards for enterprise use.
References
OpenAI API: https://openai.com/api
GitHub Repository: Prompt Validator
Contact: info@buddhakharel.com