ClickCease
AI content moderation

Using AI for community management: How to use AI moderation in Amity Console

Italo Orihuela
Italo Orihuela
Engineer
Android
iOS
Web
Jul 28, 2023

Maintaining a thriving online community requires effective moderation, but the task can be overwhelming without the right tools. Discover the game-changing benefits of AI-Powered Community Management, simplifying the way you manage and foster a safe, engaging environment just as you would.

Through Amity Social Cloud Console you can empower your administrators with advanced AI moderation tools, including profanity filters, links and images moderation. Customize settings effortlessly to create a secure community environment while leveraging the ASC Console, so let’s see through this blog a quick walkthrough of these features!

Pre-requisites

Before you begin, you will need the following:

  • An Amity Social Cloud Portal account
  • An Amity Social Cloud Console Account

Note: If you haven’t already registered for an Amity account, we recommend following our comprehensive step-by-step guide on how to create your new network.

Embrace Controlled Link Sharing with Allow List Moderation

If you decide to enable it, the default setting ensures a safe environment by blocking any links or URLs from being sent. However, we understand the need for flexibility in community interactions. Within the Allow List Moderation, you have the power to whitelist specific websites, granting users the freedom to share links from those domains. Links not whitelisted in the list as enabled will not be able to be shared between users at any time.

Ensuring a Safe and Positive Community Experience with Block List Moderation

Amity’s Chat and Social SDK comes equipped with an essential feature known as Block List Moderation. By utilizing this tool, administrators gain the ability to block specific words, curating the content that appears within the community. When a word is added to the blocklist, any attempts by users to send messages, comments, posts, or poll questions and answers containing that word will be halted, promoting a respectful and welcoming atmosphere.

It’s important to note that certain elements like community names, channel names, user display names, and more are not moderated, preserving user freedom while focusing moderation efforts where they matter most.

Special characters are fully supported, enabling you to effectively tackle potential loopholes. You have the option to enable words to be 'exact,' ensuring that the AI looks for precise matches of blocked words, automatically blocking related content to guarantee that nothing unwanted gets published. Alternatively, by choosing the 'any' setting, the AI's search scope broadens to include variations of blocked words.

For instance, enabling the 'any' setting for the word 'hell' would also block content containing words like 'hello' or 'shell.' This proactive approach safeguards the community from any potential misuse or offensive language variants.

Enhancing Community Safety with Image Moderation

Image moderation is a crucial feature that ensures all uploaded images in posts and messages undergo thorough scanning for inappropriate, offensive, and unwanted content before publication. By partnering with Amazon’s Rekognition AI technology, we’ve integrated an advanced system capable of detecting and moderating images containing violence, nudity, suggestive, or disturbing content. This AI-driven approach creates a safer online community for your users, eliminating the need for manual intervention.

When enabling image moderation, keep in mind that setting confidence levels for each moderation category is essential. By default, all categories are set to a confidence level of “0”. However, it’s important to avoid setting any single category to “0” confidence level, as it may lead to blocking all images, regardless of their content. For more accurate results, consider setting higher confidence thresholds, ensuring effective detection of problematic images. Lower confidence values (below 50) may produce more false positives, so use them only if it aligns with your specific use case and tolerance for lower confidence detection.

In case you decide to disable it, you only need to turn off the ‘Allow Image Moderation button’.

Final Thoughts

AI-Powered Community Management with Amity’s Social Cloud Console provides essential tools for effortless moderation. With features like Allow List Moderation and Block List Moderation, you can easily control link sharing and curate content to maintain a safe and positive community atmosphere. Additionally, Image Moderation powered by Amazon’s Rekognition AI enhances community safety by scanning and detecting inappropriate images. By customizing confidence levels, you ensure accurate results without blocking harmless content. Further details can be found in Amity’s documentation.

Amity Social Cloud Analytics & Moderation Panel