a method and system for parsing and richly tagging
every movie ever made
Michael Naimark, Research Associate Professor
Steve Anderson, Assistant Professor
Maya Churi, Graduate Student
Perry Hoberman, Research Associate Professor
Andres Kratky, Adjunct Professor and Visiting Scholar
Erik Loyer, Former Adjunct Professor and Consultant
Interactive Media Division
School of Cinematic Arts
"Movie Tagger" is conceived as an easy-to-use online system that allows a community of users to parse and add relevant keywords ("tags") to movies - scene by scene, shot by shot, and frame by frame. The tags can be unlimited in number but entirely searchable. They can be about the performers, action, dialogue, cinematics, mood, environment, or anything else.
The system is based on unique collaborative timeline and custom logging tools. The collaborative timeline tool is an interactive graphical interface for video and any other one-dimensional data sets. For any point or any interval on the timeline, every associated tag can be seen both as a list and as a graphic, displayed as nested brackets of in and out points. Tags can be ranked, searched, and filtered. In a wiki style, tags can be entered (and modified and contested) by the user community, along with attribution and discussion. The custom logging tool allows tags to be easily entered, along with in and out points, while watching a movie DVD on a personal computer. This metadata (without any image or sound) is uploaded to our shared database.
The Movie Tagger system can also be used for video other than feature length movies, for example, for such video databases as the USC Shoah Foundation Institute for Visual History and Education Archive. It can also be used for other one-dimensional data sets, such as historical timelines.
We conjecture that, given that every movie and every scene has its scholars and fans, that a massive online community will embrace Movie Tagger.
For the 2007 Academy Awards,
Apple Computer produced a television ad for their iPhone called
"Hello" in which it "remixed" 30 short clips from popular
movies of well-known performers saying "hello" into a telephone, all
in under 22 seconds. How did they find these clips? The answer is through brute
force and a lot of manual searching, since no database exists for popular
movies with such shot-by-shot detail.
We believe that given the right toolset, a massive text-based descriptive database of all shots in all movies can emerge. Such a toolset would be part of a long, deep education movement based on "learning by doing." This movement (without splitting hairs) includes the Constructionism of Piaget and Papert, the One Laptop per Child initiative of MIT and the UN, the "DIY" (do it yourself) and "Make" communities, as well as bloggers, MySpace members, and YouTube contributors.
1.2 Rich Tags
2. Basic System Disclosure
Our grandest mission is to tap a massive community of Internet users to build and refine a massive database quickly, efficiently, and enjoyably. Examples already exist: Wikipedia, Flickr, Del.icio.us, Google Earth. We intend to do this for cinema.
2.1 Collaborative Timeline Tool
The Collaborative Timeline Tool is conceived as an interactive graphical interface for video and any other one-dimensional data sets. For any point or any interval on the timeline, every associated tag can be seen both as a list and as a graphic, displayed as nested brackets of in and out points. Tags can be ranked, searched, and filtered. In a wiki style, tags can be entered (and modified and contested) by the user community, along with attribution and discussion.
Though several interactive timelines have begun to appear on the Internet, none currently exist which combine open collaborative tagging with a timeline. This is our unique approach, and our preferred embodiment is designed from scratch building on related interfaces and state-of-the-art technologies, using Flash and ShockWave for the front end and PHP MySQL for an Internet server-based back end.
2.2 Logging Tool
While it's entirely possible for an impassioned cineaste to log an entire movie, shot by shot, using a pencil and clipboard while watching a DVD on home television, it's hardly ideal. Our Logging Tool is conceived as an easy to use software tool that's integrated with video playback for entering tags and in / out points. It allows easy "scrubbing" through the movie to find particular in and out points, like a simplified version of a digital video editing logging process. A tag directory is always available to help keep tags in common when desired.
A preferred embodiment relies
on Internet-enabled computers, particularly laptops, with built-in DVD players using internal PC control code to control its DVD Player
application. When a movie DVD is loaded, its current Collaborative
Timeline is downloaded from our server. If no data yet exists for that
particular movie, a new Collaborative Timeline appears. (This is exactly what
happens when a music CD is loaded and iTunes displays its playlist.)
3. Optional Features and Future Applications
Movie Tagger can be used for any video database, not just commercial movies. For example, the USC Shoah Foundation Institute for Visual History and Education Archive consists of nearly 52,000 testimonies of survivors and other witnesses of the Holocaust, almost 120,000 hours of video. It is the largest video database in the world. The Collaborative Timeline Tool can provide an intuitive front-end and the Logging Tool can open up the database to additional annotation.
3.1 Use for other one-dimensional data sets
Finally, we believe that the Movie Tagger system can add unique value to other one-dimensional data sets, such as historical timelines. We can envision a Wikipedia-style "Timeline of World History" may result.
4. Some Conjectures