Video Generation from Text Prompt

Our project aims to address two main problems. The first problem is access to video-based educational content. Most students are visual learners. However, most of the content available is text-based. Only a small subset of topics is available in video format online.

The second problem is that whenever the government releases a new policy or a reform, its reach is limited by the population’s literacy level.

Both these problems can be solved by automating the process of video generation.

The last few years have seen exponential advancements in the field of Generative AI. We have tools like ChatGPT for text generation, DALL-E 2 for image generation, etc. Tools like Imagen Video and Make-a-Video only generate videos of a few seconds in length, without multiple scenes or contextual understanding.

We propose the combination & enhancement of the coherence between existing models for generating videos with an associated script, animations, diagrams, and speech. The system will take a text prompt as input and generate a video that accurately represents the prompt.

Recommended citation: V. S., A., Kulkarni, A., Chawla, D., Rawther, A., & Rangareddy, J. Video Generation from Text Prompt. Poster presented at AI Impact Workshop 2023, C-MInDS, IIT Bombay, Mumbai, MH, India.
Download Paper