Poyo gives teams free access to AI content moderation for text, images, and text with an image. You can try OpenAI's omni-moderation-latest in the Playground, inspect category flags and scores, and use the same Poyo account to bring moderation into your model workflow.
For a community, that can mean checking a post before publication. For an image product, it can mean assessing an upload with its caption. For an AI assistant, it can mean reviewing a message or generated reply before deciding how to present it.
Start with a few representative examples in the Playground. When you are ready to integrate, Poyo provides a task ID, status tracking, and an optional completion callback so your application can connect the result to its publishing and review rules.
What is AI content moderation
AI content moderation uses automated classification to help teams assess user submissions against their platform's rules. It can identify potential risks in text and images, giving reviewers and applications a consistent set of signals to work with.
With Poyo, those signals include an overall risk flag, individual category flags, confidence scores, and the input types used for each category. A discussion site can use them to identify comments that need attention, while a photo product can review images before they enter a public gallery.
Your product can map the result to actions such as publishing, requesting an edit, or assigning human review. This gives the team a defined starting point while preserving control over its own rules.
The purpose here is to assess content for potential harm. Determining whether writing or an image was AI-generated is a separate task.
What Omni Moderation can detect
Poyo's Omni Moderation integration supports multilingual text, one image, or text together with one image. Text covers 13 categories, and six categories also support images. This gives teams a shared set of fields for reviewing comments, captions, and uploaded pictures.
| Risk group | Response categories | Text | Images |
|---|---|---|---|
| Harassment | harassment, harassment/threatening | Supported | Text only |
| Hate | hate, hate/threatening | Supported | Text only |
| Illicit instructions | illicit, illicit/violent | Supported | Text only |
| Self-harm | self-harm, self-harm/intent, self-harm/instructions | Supported | Supported |
| Sexual content | sexual | Supported | Supported |
| Sexual content involving minors | sexual/minors | Supported | Text only |
| Violence | violence, violence/graphic | Supported | Supported |
Use category_applied_input_types to see which inputs each result covers. Categories marked Text only apply to supplied text; a zero score on an unsupported image category means it was not assessed for that image.
You can extend this review with your own marketplace or community rules, such as advertising restrictions and quality checks. When a workflow depends on text embedded in an image, audio transcripts, or selected video frames, prepare that material separately and submit the supported text or image for assessment. The result covers the supplied inputs.
The sexual/minors category is text-only. Known or suspected child sexual abuse material must not be submitted to OpenAI's Moderation API; follow the official child-safety guidance for that situation.
See the Poyo Omni Moderation documentation for request formats and the supported fields.
Where AI moderation fits in your product
The best place to introduce moderation is a specific moment when user content changes visibility or reaches another person.
For a community, that moment might be posting a comment. For an image-sharing service, it might be adding a photo to a public gallery. For a chatbot, it might be displaying an assistant response. In each case, decide whether users must wait for a result or whether the content can appear while review continues.
| Product situation | Content to assess | Decision to define |
|---|---|---|
| Discussion forum | A post or comment | Publish, request an edit, or hold for review |
| Public image gallery | An image and its caption | Show publicly, restrict visibility, or hold |
| AI assistant | A user message or generated reply | Continue the interaction, review, or apply the product's response policy |
| Marketplace | A description and one product image | Allow the listing or request further checks |
Private messages require a deliberate privacy decision as well. Define what your product reviews, who can see the content, and what users are told before introducing a moderation workflow.
Poyo brings the assessment and task tracking into one API workflow. Connect the result to your existing publishing rules and reviewer tools, so each submission follows the process you define.
Try free text and image moderation with Poyo
You can begin on the Omni Moderation model page. The Playground offers a form, example requests, and structured results. Sign in to run a test; you do not need to manually create or paste an API key in the Playground. An API key is needed when your own application calls the API.
Start with a text submission
Sign in, open the Playground, and enter the text you want reviewed in the Prompt field. Here, Prompt means the content being assessed. It is not an instruction asking the model to behave as a moderator.
Use a short, ordinary example first, such as:
I'm planning a hike in the park with friends this weekend. Does anyone have a beginner-friendly route to recommend?
Select Run to request a new assessment. The page initially shows recorded examples; changing the input does not automatically replace an example result. Wait for your new request to complete before interpreting its output.
Read the overall risk flag, the individual categories, the scores, and the applied input types. Then try a few permitted examples from your actual content, including languages and situations your product regularly encounters.
Add an image
For an image-only assessment, clear the text and add one image in Image Urls, using the available image input. You can also keep a caption in Prompt to review text and an image together.
At the API level, Poyo accepts one HTTPS image URL. The Playground's upload control first uploads a selected local file and then supplies its URL to the review request. For several images, submit a separate task for each one and associate the results with the original post.
If you use your own URL, it must be accessible to the moderation service. A link that opens only after you sign into a private dashboard will not provide the image itself.
Move from a trial to an integration
Once the examples are useful, create a Poyo API key and integrate from your server. Keep the key out of public browser code.
A text-and-image request has this shape:
{
"model": "omni-moderation-latest",
"input": {
"prompt": "A view from this morning's hike.",
"image_urls": [
"https://your-image-host.example/photo.jpg"
]
}
}
Replace the example URL with an actual accessible HTTPS image. The request goes to POST https://api.poyo.ai/api/generate/submit with your Poyo API key. It returns a task ID; retrieve the completed result through the status endpoint or configure a completion callback. This is an asynchronous workflow, so receiving a task ID is not the same as receiving a finished assessment.
How to read the result
There are two separate questions: did processing complete, and what did the model detect?
| Field | Meaning | How to use it |
|---|---|---|
status | Whether the task is waiting, running, finished, or failed | Decide whether an assessment is available |
flagged | Whether the model flagged potential risk | Start a review or routing decision |
categories | A true or false flag for each category | Identify the type of potential risk |
category_scores | A confidence signal for each category, between 0 and 1 | Compare cases and evaluate rules on your own examples |
category_applied_input_types | The input types a category result applies to | Check whether text, images, or both contributed |
For the status endpoint, the category results are under data.output.results. A task with status: "finished" and flagged: true completed successfully and identified potential risk. A failed task has not delivered a usable completed assessment.
A score of 0.8 should not be presented as an 80% guarantee that content violates your policy. A high score can indicate strong model confidence in a category, while your reviewers still need to consider the context and your rules.
For instance, an account describing a violent incident may trigger a violence category even if the account is appropriate for a news discussion. A normal-looking promotional post may receive no harmful-content flag and still violate a community's no-advertising rule. Both examples show why category detection and policy enforcement require separate decisions.
Build review rules from your own examples
Use the Playground to build a small evaluation set with ordinary submissions, clear violations, and cases that need context. Include your common languages, image styles, captions, and quoted material. Synthetic or appropriately permitted examples make a useful starting point.
Record the decision your team expects for each item, then compare it with the category signals. This helps you choose which submissions can follow a routine path and which should reach a reviewer.
Test the complete workflow as well: a pending task should have a visible waiting state, a processing problem should have a retry path, and a reviewer should be able to correct a decision. Poyo's task ID and status fields give you a way to follow that process.
Choose category-specific thresholds using those examples, and reassess them when your content patterns or the underlying model change.
Frequently asked questions
What is AI content moderation?
AI content moderation uses machine learning to assess text and images for potentially harmful content. It returns signals that help a platform apply its rules, decide what can be published, and identify submissions that need human review.
Is AI content moderation free on Poyo?
Poyo's Omni Moderation calls cost zero credits for supported text and image inputs. You can test classification without adding a per-call moderation charge, then use the same model in your application. Other models, storage, and human review have their own costs; the free moderation guide explains how to plan them.
How can I try it without writing code?
Sign in to the Poyo Playground, enter text, add one image, or combine both, and select Run. Inspect the new task's category results after it completes. The Playground handles authentication, so you do not need to manually create or enter an API key to try examples.
What results does Poyo return?
Completed assessments include an overall flagged value, individual category flags, category scores, and category_applied_input_types. Together, these fields show the potential risks detected and which supplied input types were assessed. Use them to configure review rules, and interpret scores as model signals rather than guaranteed policy decisions.
How do I connect moderation to my existing product?
Submit supported content using your Poyo API key, save the task ID with the content version, and query status or configure a completion callback. Your application can then connect the result to publishing, a request for changes, or an existing human-review tool. Start with one content surface and test the routing before adding more.
Start with the Poyo Omni Moderation Playground and compare its results with your team's decisions. For a deeper workflow, read the image moderation guide, the free access and cost explanation, or the small-team UGC moderation guide.