The Notation Editor provides a visual user interface through which Director Notation scores can be created, edited and visualised. Starting with a film script, it allows users to outline the shots, or ‘set-ups’ that they wish to use to film each scene, then describe in detail the positions and movements of actors and cameras, using Director Notation. Implemented using Adobe Flex, the Notation Editor is a browser-based application and can be run without any locally-installed software. It is configured dynamically, so updates to the Director Notation language are reflected automatically by its user interface.

 

The ANSWER ontology has two parts. By far the largest is the semantic model of Director Notation. This describes in precise detail how each Notation symbol is constructed, and the combinations in which symbols can be combined to convey more complex concepts. It also describes visual placement of the symbols, and updates to the Director Notation language in the ontology are immediately reflected in the Notation Editor, without any further manual configuration. The second part of the ontology contains a selection of higher-level film production concepts that can be used to supplement the Notation, for example to represent the genre of a particular production.

 

The Rule Engine is responsible for semantic checking of each modification the user makes to a score, and translation of the completed, semantically-valid score into a set of Player Markup Language commands that will drive the Instant Reality visualisation. The Rule Engine is internally split in two halves. The core system is language-agnostic, and provides knowledge checking, evaluation, transformation, communication and messaging functions. The other part is language dependent, and consists of the configuration of the Director Notation language itself, special code for the interpretation of specific parts of the language and a collection of rules which are applied to any symbol collection by the core system to achieve the evaluation.

 

The Instant Reality framework is a 3D visualization/ Mixed Reality system that is used in ANSWER visualize Director Notation scores. It processes scene description and animation requests in Player Markup Language format, controlling the movement of 3D models and cameras within a pre-created 3D environment (set). Individual rendered scene frames are returned to the Rule Engine for conversion to a Flash Video file that is then streamed back to the Notation Editor. The animation frames are also made available for post-processing by the Postproduction Synchronization Tool.

 

The ANSWER Postproduction Synchronization Tool (PST) supports post-production operators in synchronizing actual footage and corresponding pre-vis. It is designed as a remote service application used by the BONES postproduction suite. The ANSWER PST automatically reveals the correspondences and differences between Director Notation and recorded footage, supporting the operator in the process of alignment. To implement the event extraction, the PST performs automatic content based analysis of the actual footage and corresponding pre-vis, and returns extracted events (appearances of human face and film slate, characteristic camera motion, video shot boundaries) to the BONES Dailies Postproduction System.

 

Bones Dailies is a commercial post-production platform supporting creation of colour-graded dailies from ingested footage. Bones has been augmented in ANSWER to provide an additional video track and monitor, which are used to view ANSWER-generated pre-vis animations alongside the corresponding actual footage. The PST automatically aligns the two video sequences and drives the placement of timeline markers in Bones which the operator can use to identify pairs of significant visual events in the footage and pre-vis.

 

The ANSWER Dialog Designer component is an off-line tool that allows the conversion of ANSWER generated cinematic content into a proprietary format suitable for visualization by a game engine. Larian Studios designed their original Dialog Designer to produce cinematic content for their computer games. It features dozens of content generation workflows that can be combined to produce a fully localizable cinematic sequence that can be interactive or non-interactive; camera directed, or without camera; all within the concept of a dialog-oriented story script between multiple actors. For ANSWER, the Dialog Designer has been extended to allow import of Player Markup Language data created by the Rule Engine.

 

Larian Studio’s Divinity 2 game engine has been selected to demonstrate that ANSWER can be applied to interactive and non-interactive media projects or machinima as a content creation tool. The game engine has been intensively modified such that the camera system is able to support a growing subset of features from the PML specification. The actor system supports multiple animations and animation blending over time. The scheduler is able to work with multiple actors and actor scripts.  At the same time, the dialog designer is used as a base-layer in the conversion tool, now capable of producing engine specific content.