For the 2019 LabDay, I build a little Cloud pipeline to create Emoji subtitles for videos.

Project: Crappy Crocodile

Using the Google Cloud Platform, one can upload videos to a Cloud Storage Bucket and they will be processed using Google’s Video Intelligence API to create Emoji subtitles for the scenes contained in the videos.

The project uses various Serverless GCP services (Cloud Functions, Cloud Run & Cloud Pub/Sub) to implement an asynchonous processing pipeline. Mostly, the goal was to gain knowledge about and experience with Serverless Computing and to create a “real-life” example for a Serverless Architecture. But also to create a fun demo for a Cloud Native video processing application.

All the necessary code and setup instructions and scripts can be found on GitHub: https://github.com/wtfc63/crappy-crocodile

GCP Services Used

• Google Cloud Platform
  • Cloud Video Intelligence API
  • Cloud Storage (Object Storage)
  • Cloud Pub/Sub (event bus)
  • Cloud Run (serverless execution of Containers)
  • Cloud Functions (FaSS: serverless execution of simple Functions)

Resources

• Google Video Intelligence API
  • Video Intelligence Client Libraries
  • Google Knowledge Graph Search API (to search for entities)
• Adding captions and subtitles to HTML5 video
• WebVTT, The Web Video Text Tracks Format
  • Specifications
  • MDN Documentation
• Emoji Search
  • Emojidex
  • Unicode Emoji List
• Animated Emoji URLs (unused)

Screenshots

Processing

• Processing Storage Bucket:

Results

• Water Scene
  • Objects:
  • Emoji:
• Landscape Scene
  • Objects:
  • Emoji: