Designed and shipped an AI trust-and-safety moderation layer embedded directly into the existing product (Strapi backend + React Native iOS/Android app), not a bolt-on tool
Trust & SafetyWe taught a player-connection app to catch the bad stuff before anyone sees it — across photos, video, and chat.
Sports / Social Networking (consumer mobile platform)
connectplayers.ca
Moderation gate
Profile photo
Chat message
Feed video
Passedsafe · published
Blockedhate · OCR
Loggedaudit trail
PassBlock before store
The challenge
Connect Players is a player-to-player sports and social app where strangers connect, message, share photos and videos, and form clubs. Open, user-generated connection at scale means real exposure to harmful content — nudity, violence, hate, slurs, scams, and grooming risk — that no small team can police by hand without slowing the product down.
Trust & Safety
Sports / Social Networking (consumer mobile platform)
What we built
The system, in parts.
Image and profile-photo moderation with AWS Rekognition label detection plus OCR text extraction, so harmful images and text-on-image (slurs, scam overlays) are both caught
Video moderation pipeline: FFmpeg frame and scene-change sampling with per-frame analysis, backed by Rekognition's asynchronous content-moderation job for full clips
Text moderation across chat messages and feed posts using the OpenAI Moderation API with custom-tuned category thresholds (sexual/minors, self-harm, hate, harassment, violence)
Custom rule-based detectors layered on top for slurs, racial preference/exclusion, nationality attacks, criminal activity (drugs, weapons, fraud), and scam keywords
Enforced at the point of creation: flagged content is rejected before it is ever stored, offending files are deleted from S3, and a clear reason is returned to the app
ContentModerationLog records every decision — user, surface (feed, chat, chat media, group post), block status, and review state — for auditability and human oversight
Outcomes
What changed for them.
Harmful images, videos, and messages are blocked before they reach another user, keeping connections between strangers safer at scale
Moderation runs automatically across every user-generated surface — profiles, feed, 1:1 chat, group chats, and media — with no manual review queue to staff
Text-on-image abuse (slurs and scam overlays) is caught via OCR, closing a gap that image-only filters miss
Every block is logged with its reason and surface, giving the team an audit trail and a path for human review of edge cases
Cost- and latency-aware by design: 10MB upload caps, parallelized checks, short timeouts, and frame sampling keep moderation affordable and fast as volume grows
How it’s built
The stack.
AWS Rekognition (image moderation, OCR text detection, async video content moderation)OpenAI Moderation APIFFmpeg (fluent-ffmpeg) frame and scene-change extractionStrapi v4 lifecycle hooksAWS S3 + CloudFrontNode.jsReact Native (iOS + Android)Custom rule-based text classifiers
More work
See allTwo ways to start
Want a system like this?
Tell us the workflow you want to run itself. We will scope a focused first project — designed, built, and operated, with humans in control.