Welcome to the FaceFX User Guide

Welcome to the FaceFX User Guide

Welcome to the FaceFX User Guide Welcome to the FaceFX User Guide! This document provides a high-level overview of the FaceFX solution and is designed for artists, technical artists, and even the "business types" that are trying to figure out if FaceFX is the right product to begin with.

Other Documentation

Users are encouraged to ask questions and seek additional information at http://support.oc3ent.com.

FaceFX has some documentation specifically for programmers. Depending on where you are in the Evaluation Process you may also have the FaceFX SDK Reference.chm help files created with doxygen. Fully licensed FaceFX customers with source code should also read the HowToBuild.txt file located in the FaceFX Source folder of the installation.

Get Started

What are you waiting for? Time to get started!


What's New in Version 1.7.3.1

What's New in Version 1.7.3.1

What's New in Version 1.7.3.1

Bugfixes

  • Fixed a crash when the user passes an invalid curve index to the key command.
  • Fixed tools installer to output correct XSI native coordinate plugin. 1.7.3 placed FaceFX coordinate plugins in the native coordinate folder
  • Fixed tools installer so that no FaceFX coordinate plugin is created. The MotionBuilder plugin is always in Native Coordinates.
  • Fixed refresh issues with the XSI FaceFX.vbs script.
  • If a prompt-to-save triggered by the application closing fails (ex, a read-only target file), the application no longer ignores the failure and closes anyway. It properly aborts closure to give the user a chance to save his work.
  • Graphviz and libresample DLLs now have correct version information embedded suitable for Games for Windows certification.
  • Added fix for workspace instances closing on save. They now remember where they were and what workspaces they contained. Also, the main workspace widget remembers what workspace was selected across saves now.
  • Improved coarticulation on long vowels (primarily found in singing).
  • Fixed a long standing issue with workspace sliders. If the curve controlling the slider was virtual, and the curve happened to be over or under driving the node, and you pressed load current frame to sliders, the sliders would draw outside of the bounds of the slider 'groove'.
  • Importing an XML file with a refernce pose will now replace the existing reference pose rather than combining the old and new reference pose.

Features

  • Added audio chunking text tags. You can split up long or poorly analyzing files by placing time tags in the text. The time tags are in seconds and deliminated like so: <12.5>
  • Version 1.7.3.1 Unreal Engine 3 and standalone plugins are interchangable
  • Added GCC makefile and tests

Structural Changes

  • Switched to Visual Studio 2009
  • Using new version of Fonix VoiceIn with improved performance on digital silence
  • Rename target must not exist in name table. This resolves issues renaming animation groups into existing animation groups and other problems associated with the rename command.

What's New in Version 1.7.3

Bugfixes

  • Improved speed of batch analysis by removing unnecessary reads from the hard disk.
  • Fixed potential infinite loop in the text tag parser with extraneous open square brackets or open curly brackets.
  • Fixed text tag problem where a tag's beginning and ending tags are between the same words, and there is a silence in between those words. The tag would get inverted.
  • Template command now correctly flushes the undo / redo buffer when the sync is successful, fixing potential crash in Studio.
  • Fixed an issue with the "Key All" button with virtual curves.
  • The actor's mapping is now used if used if no session was found.
  • Improved the Max FXR Exporter's ability to find textures.
  • Fixed a bug in the animation playback sample where the face graph wasn't properly decompiled before use.
  • Curve widget and timeline widget text controls properly respond to enter now.
  • Fixed a crash when a torn-off workspace is in animate mode on new actor.
  • FxStringBase lexographical comparisons now use the FxCharTraits::Compare() without the 'num' to avoid impossible comparison results.
  • The application directory is correctly set on program startup so that you can launch the application from within any directory at the command prompt.
  • Corrected setting of _inUse in FxArchiveStoreMemory.cpp fixing rare crash.
  • Fixed a potential memory leak in FxFaceGraph::Serialize() by _pNodeHash being allocated again if it was already allocated (only a problem in licensee written code).
  • Fixed a crash when creating an FxName from a null pointer.
  • Fixed a bug where the currently selected animation user data would be lost when saving.
  • Max FXA exporter now has horizontal scrollbars for reading items in list boxes with long names.
  • Fixed a problem with the plug-in not fully descending the modifier stack in all cases when looking for morph targets.

Features

  • Added support for XML import and export.
  • Tools added for Max 2009, Maya 2009, Motion Builder 7.5 Ext 2a, MotionBuilder 2009, XSI 6.5, and XSI v7
  • 64-bit FXA exporters added. the FXR Exporters are still 32-bit only.

Structural Changes

  • Changed Max/Maya script functions to accept a float for the time in fxinsertkey to allow for non-whole value frame numbers.
  • For animations generated with 1.7.3, FxAnimUserData's _audioPath variable stores the full audio path as opposed to the audio path relative to the actor.

What's New in Version 1.72

Bugfixes

  • Register animation and normal animation playback can now correctly coexist.
  • Corrected a crash when sorting the curves in the actor widget's curve view when no animation is selected.
  • Fixed a crash when trying to display the phoneme widget options dialog.
  • FaceFX auto-computes slope correctly when inserting new keys in between keys with broken tangents.
  • Maya FXA exporter refreshes Maya after importing a pose.
  • Max 9 FXA exporter no longer sets scale keys at negative time. Max 9 does not support this.
  • Max FXR exporter looks in the FXR export directory for textures in addition to the location specified in the Material Editor.
  • FaceFX Commands now permit more than one space between command elements.
  • Fixed crash loading content with non-joint objects in skeleton exported from Maya FXR Exporter.
  • Fixed hang in Maya FXA exporter caused by cleaning animation timeline with bones that have attributes with connections.
  • Text Tagging now works correctly in the following cases: 1) where tags create overlapping curves 2) when tags are at the very beginning or the very end of the text and 3) when there is no text at all.

Features

  • The Animation Playback sample now contains a register panel for testing the effects of setting FaceFX registers from the FaceFX SDK. (See the FaceFX SDK docs for more information on registers).
  • Improved curve baking.
  • Added the fgcollapse command.
  • Selecting a workspace slider automatically selects the appropriate curves.
  • Hitting F5 or F6 were added as keyboard shortcuts for moving to the previous and next frame (frame rate specified in Tools->Application Options.)
  • fxgetbakedcurvekeyslopein and fxgetbakedcurvekeyslopeout commands added to Max and Maya plugins.

Structural Changes

  • Max 2008 and Maya 2008 FXA Exporters and FXR Exporters included.
  • Updated FXCollada to use FCollada version 3.03J from Feeling Software.
  • A new component-based default mapping was created.
  • Additional phonemes added to mapping. Old content should update automatically.
  • The file format version number is now separate from the FaceFX Studio version meaning that future releases of FaceFX Studio do not necessarily have to change the file format version.
  • Changed sample words for some phonemes.

What's New in Version 1.71

Version 1.71 is primarily a bugfix release over version 1.7. The version 1.7 changelist is included below the 1.71 bugfixes.

Bugfixes

  • Fixed evaluation of Face Graph nodes with "Multiply Inputs" selected as the input operation.
  • FaceFX Studio can be built and run with Visual Studio .NET 2005.
  • The version 1.71 FaceFX SDK builds cleanly on GCC and other GCC-like compilers.
  • Fixed crash in Face Graph widget when a window is torn off and the user attempts to create a link that would create a cycle in the graph.
  • Fixed crash in phoneme bar that occurs when you change the phonemes in an animation that was created with the "Do not generate curves" option.
  • Fixed crash in Face Graph properties window that occurs when you change the value of a custom user property then click outside the box before committing the change by hitting enter or tab. Morph target nodes are the only ones with custom node properties in standalone FaceFX Studio.
  • Slow and Very Slow animation playback fixed in DirectX audio mode.
  • Fixed General Protection Faults in the FxLoadActorFromMemory function when memory pooling is used.

What's New in Version 1.7

Optimizations

Version 1.7 of FaceFX includes a completely new face graph algorithm. It is far more cache friendly, and optimized for next-gen processors. Memory consumption has also been optimized for 1.7 by storing the FaceFX archive version number once as opposed to once per object in the archive.

Features

  • Curve colors are stored and transferred with templates.
  • Curves can be copied from the Curve Manager.
  • Batch analysis is now available from the GUI (in non-evaluation builds).
  • Max 9 and Maya 8.5 are now supported. Support for Maya 6 has been removed.
  • Workspaces have keyboard shortcuts, a configurable tab order, and the notion of a selected slider which influences the functionality of the "next key" and "previous key" buttons in animate mode.
  • Speech gestures can be disabled when generating animations.
  • Node names can be changed from the node properties window. This only changes the node's name, not any associated curves or mapping entries. Consider using the Global Search and Replace Name function in the Edit menu to replace everything, but be careful that you don't create duplicate FaceFX objects like curves or animations.

Structural Changes

  • The face graph now needs to be compiled when nodes or links have been added. If you are performing some advanced face graph operations in your toolset, you may need take this into consideration. Other code changes that are required should be apparent due to compilation errors. Check the SDK FaceFX CHM file for details.
  • Audio analysis has moved to FxAnalysis, a reusable component that takes 16 kHz 16 bit audio and creates a FaceFx animation. Games wishing to include audio analysis inside their toolkit should be able to use the FxAnalysis component without having to use large amounts of FaceFX Studio code.
  • In preparation for Windows Vista, log files for version 9 of 3D Studio Max and all versions of Maya have moved to the user profile directory inside the Max or Maya application folder.
  • The FxCooker and FxUpdateActor tools have been replaced by the cook command, and coordsys command.
  • When performing batch process operations, the relative path from the batch folder is used to construct the animation group name. This ensures that a single batch operation can not produce a duplicate animation.
  • The custom link function has been deprecated.


Unreal Engine 3 Integration

Unreal Engine 3 Integration

Unreal Engine 3 Integration FaceFX is built into Unreal Engine 3, demonstrating that FaceFX is engine-independent and capable of meeting the requirements of next-generation game development. This page of documentation is directed towards Unreal Engine 3 developers using FaceFX.

The below sections provide an overview of how the UE3 integration is different from standalone FaceFX and which parts of this documentation do not apply to Unreal Engine 3. For the most part, the entire FaceFX Studio application is launched from inside UnrealEd, but there are a few minor differences to be aware of.

How is the UE3 Version Different?

  • The Sample Rendering Engine is replaced with an Unreal Engine 3 Viewport. As a result there is no need for FXR files or the FXR exporters. Also, the buttons available on the Preview Tab are specific to the FaceFX Sample Rendering engine and therefore the buttons may not be present in UnrealEd.
  • FXA, FXS, FXE, and FXM files have been integrated into the Unreal package system for a seamless integration. FXA files can exist separately on disk, but only when they are exported from a package to be loaded by the Max and Maya FXA Exporters.
  • External animation sets do not use FXE and FXM files and the interface for using them is not in the Actor Menu. External animation sets are manipulated directly from UnrealEd. Check the tutorials on UDN for more information.
  • Several Unreal-specific nodes have been added to the FXA file format. These are used to drive morph targets or material properties in Unreal. Even with these new types as of version 1.7.3.1, Unreal Developers can use the standalone FXA Exporters.
  • Workspace viewports (where a real-time preview window is located behind the sliders) are not supported in UE3.
  • FaceFX Studio's Command Line Interface works in Unreal, but the files must be located in the Binaries\FaceFX folder. Place any autoexec.fxl, presaveactor.fxl, etc. files in this directory and they will be executed at the appropriate time. Because there are no filenames in UE3, FileNameOfFXA_PreLoad.fxl wont work, but placing ActorNameofFXA_PreLoad.fxl in the Binaries\FaceFX folder will work.

Where Can I Can I Get Additional Information?

There are several FaceFX tutorials up on UDN. These describe all of the Unreal-specific FaceFX functionality and are kept up to date. All support questions for Unreal Engine developers should be sent to the Unreal Engine 3 message lists.


Getting Started

Getting Started

Getting Started

Overview

  1. Set up your character with morph targets or bone poses that define how the character talks, blinks, raises its eyebrows, and moves its head.
  2. Export an FXA and FXR file for your character.
  3. Open your character in FaceFX Studio. You should see it in the Preview Tab and be able to activate targets from the Face Graph section of the Actor Panel.
  4. Define a phoneme to target mapping for your character from the Mapping Tab, or simply use the Default Mapping.
  5. Create an animation by Analyzing Audio.
  6. Fine tune the animation from the Animation Tab or the Workspace Tab.
  7. Optionally go to the Face Graph Tab to modify your character's Face Graph.
  8. Get the animation data out of FaceFX by Integrating FaceFX into your game or application, or using the FXA Exporters to get animation data out into Max or Maya.

Learning FaceFX

FaceFX is different from other animation applications, so read up on the Key Concepts first. After that, take a look at the Tutorials and Face Graph sections. If you are looking to create an automated pipeline, be sure to understand the Command System, and all of the Melscript and MaxScript functions available in the FXA Exporters.


Evaluation Process

Evaluation Process

Evaluation Process

No-Save Version

The no-save version can not save any type of file. So you can use the program normally, but you can't save your work (though recording FaceFX commands in a text file is an option). To bring your own content into the no-save version, you must use the FXCollada tool to save out an FXA and FXR file.

The no-save FXA Exporters can only open up the sample content provided with the no-save version or content exported with the included FXCollada tool. You can experiment with getting data out of FaceFX using the Slade example character's FXA file and the FXA Exporters.

Node Locked

FaceFX is available for sale on a per-seat basis. Node-locked FaceFX Studio is locked to a particular mac address or network card. Contact us to purchase a node-locked license. Batch Processing is disabled in node-locked builds of FaceFX studio.

Time-locked Evaluation

In some circumstances, OC3 Entertainment can provide a time-locked evaluation. It is a binary build of FaceFX Studio that is fully functional except:
  • The evaluation will time out. You can check the timeout date from the Help menu's About box. The timeout date is also printed to the Console Tab when FaceFX Studio opens.
  • Batch Processing is disabled.

Programmer Evaluation

Programmer evaluations let developers integrate FaceFX into their game engines. A Programmer evaluation is identical to an Time-locked Evaluation except for the following:
  • The FaceFX SDK is provided in binary form.
  • The Native Coordinate versions of the FXA Exporters are included.

Full Source Code

Full source code versions of FaceFX are available. Developers get:
  • Full source code to the entire FaceFX solution with the exception of the Fonix speech recognition technology used to generate animations offline.
  • The Batch Processing is enabled.


The Problem FaceFX is Trying to Solve

The Problem FaceFX is Trying to Solve

The Problem FaceFX is Trying to Solve FaceFX is a world class facial animation solution designed to be both powerful and flexible. It achieves the below objectives in an engine-independent way without putting any restrictions on how the artist generates content. It can be used as a batch process tool to generate hundreds of hours of dialogue or as an animation tool for creating a few minutes of heavily tweaked cinematic-quality animation.

High Quality Facial Animation

FaceFX provides the most accurate and realistic facial animation data possible using only an audio file and an optional text file. Curves for lip-synchronization, head and eye rotations, eyebrow movements and blinks combine to create a base layer of animation that can be used to effortlessly bring characters to life.

Facial Animation Complexity

Animating the human face is an inherently difficult task. The face can move in extremely complex ways and even the smallest mistake is obvious to most observers because people are naturally very perceptive to facial movement. OC3 Entertainment’s goal is to provide you, the developer, with the tools you will need to address the problem in a way that is optimal for your game or application. To create facial animation, a 3D artist is responsible for creating a set of bone poses or morph targets that represent the movements of the face. This leaves two questions:
  • What targets need to be created by the artist?
  • How should those targets interact with each other?
The second question has traditionally been very difficult to answer, primarily because there were no tools that could make targets interact. Without any interaction, it is difficult to achieve expressive talking characters because expressions can interfere with lip-synchronization. See below for an example of a smile target interfering with a bump target. (The bump target is used to pronounce the B, M, or P sounds. In order to pronounce these phonemes the lips must be pressed firmly together.)

PBM      +     Smile    =     Junk
 FaceFX solves this problem with the face graph. Problems like the one above can easily be fixed by creating a relationship between the bump and smile nodes. FaceFX does not force the artist to solve the problem in any specific way, but rather gives the artist the tools needed to create the appropriate interactions between targets.

Animation Memory Usage

Bones-based animations take up a lot of memory even when advanced key-frame reduction algorithms are used. Facial animations are no exception, especially on next generation characters which can have upwards of 50 bones in the face alone. FaceFX addresses this problem by procedurally generating bone position, rotation and scale data at runtime. It works like this:
  • The artist creates bone poses to define the facial movements. Bone poses work just like morph targets.
  • FaceFX lets the artist create relationships between the bone poses with the face graph.
  • FaceFX generates animations based on audio dialogue and stores curves that propagate through the face graph and define how the bone poses change over time. The artist can then tweak the curves or add new curves.
  • Based on the bone poses, curves, and face graph setup, FaceFX calculates the final bone positions, rotations and scales of every bone in the face at runtime in the game. FaceFX does not work only with bones-based animations. The technology works equally well with morphs or any combination of morphs, bones and other systems. The point is that if you are using bones, you can achieve massive memory savings on facial animation data.

Efficient Pipeline

Lots of applications allow developers to create animation data from audio dialogue. FaceFX is unique in how it allows developers to tweak and fine tune the animation. FaceFX allows users to tweak animations 3 different ways, all while getting instant visual feedback from your character in your engine.
  • Phoneme Bar - FaceFX lets developers perfect the audio analysis with tweaks to the phoneme bar. Perfecting phoneme timing data makes a big difference to the quality of the lip-synchronization.
  • Sliders - FaceFX allows artists to easily edit animations by playing with one dimensional and two dimensional sliders. The sliders can control bone poses, morph targets, material properties, or any combination thereof.
  • Key-framed Curves – FaceFX includes an advanced key frame editor that lets animators modify curves as they would in any other 3D application. Artists can insert, delete, and move keys with the curve editor as well as modifying key tangents.


Design Philosophy

Design Philosophy

Design Philosophy

Visual Interface

Creating facial animation is an artistic process, and artists need to be able to see what they are doing visually. FaceFX is designed with the artist in mind.

Minimize Work

FaceFX helps save time. Artists will typically only create one face graph setup for all of the characters in a game because characters can share the same graph regardless of the number of bones in the skeleton, the number of polygons attached to the bones, or whether the character is set up with bone poses or morph targets to begin with.

Platform Independent

FaceFX is designed from the ground up to be platform independent. The FaceFX SDK runs on Windows, Linux (32-bit and 64-bit), Mac OS X, Xbox, and Xbox 360. Support for all gaming consoles is a priority, but subject to OC3 Entertainment’s participation in the appropriate tools and middleware programs. FaceFX Studio is also designed to be platform independent, although it is only actively supported on Visual Studio .NET 2003 (starting in 2008, Visual Studio 2005 will be the primary development platform). FaceFX Studio uses multi-platform software like wxWidgets, OpenGL, OpenAL, etc. whenever possible in anticipation of supporting more platforms directly.

Optimized for Speed and Memory

FaceFX is heavily optimized so the in-game portion of the solution uses system resources sparingly.

  • FaceFX saves memory by storing animation data as keyframed target curves rather than position and rotation data for every bone at every frame.
  • The Face Graph short-circuits itself so that it only evaluates relevant portions once. Sections of the face graph that do not need to be evaluationed are autmatically ignored.
  • The FaceFX naming system ensures that strings are stored only once in memory and can be compared rapidly.
  • External Animation Sets are designed to be loaded at runtime from your game allowing the game to only load animations that are needed at a particulat time.
  • Actor Instances allow animation data, bone poses, and face graph data to be shared by multiple instances of the actor.
  • Bone poses only contain data for the bones that move with respect to the reference pose in any given pose. All other bones are pruned.

    Provide Source Code

    After purchase, source code is included for the entire FaceFX solution with the exception of the core analysis engine that converts audio dialogue into animation data. Full source code allows your team to integrate the toolset in your game engine for mod developer use and easily port the runtime component to platforms not currently supported by OC3 Entertainment.


FaceFX File Types

FaceFX File Types

FaceFX File Types

FXA Files (*.FXA)

The purpose of FaceFX is to create animation data for your game. All of the info needed for your game is stored in an FXA file, so this is the most important file. FXA files begin their life-cycle in the Max or Maya plug-ins where the reference pose and bone poses are defined. They then get fleshed out in FaceFX Studio and then finally get loaded by the FaceFX SDK in your game. Alternatively, you can load the FXA files back up in Max or Maya and bring animation data into those packages directly.

FXR files (*.FXR)

These are also created in Max or Maya but they are less important than FXA files because they are not used by your game. FXR files are used simply to store the 3D geometry, textures, and morph target information so that the FaceFX sample rendering engine can display content. If you replace the sample rendering engine with your own game engine, FXR files are removed completely from the facial animation pipeline.

FXS Files (*.FXS)

FaceFX Session files store everything that isn’t already stored in the FXA or FXR files. The FaceFX Studio specific data includes audio data, phoneme data, phoneme to target mapping, face graph node positions, workspaces, cameras, etc. FXS files can get large if you analyze a lot of audio, but they aren't used by your game.

FXL Files (*.FXL)

FaceFX Language files are simple text files that store a list of FaceFX Studio commands.

FaceFX Template Files (*.FXT)

FaceFX Template files allow you to share a common setup across multiple actor files. You can transfer the face graph, workspaces, node groups, mapping information, cameras, and color maps.

External Anim Sets (*.FXE, *.FXM)

External Anim Sets consist of FXE and FXM files. The FXE files store animation data which can be linked up to an actor file (*.FXA) in FaceFX Studio or at runtime in your game. In FaceFX Studio, this process is called “mounting” and “unmounting” animation sets. FXE files are usually accompanied by a matching FaceFX Mini Session file (*.FXM). FXM files store all of the data needed by FaceFX Studio but not your game. This includes audio data and phoneme information.

FaceFX File Diagram

Gesture Configuration Files (*.FXG)

Gesture Configuration Files (*.FXG) are not included in the above diagram because they are not as essential to the application. They specify how gesture curves should be created when audio is analyzed. Several FXG files are included with the application by default.

See Also

Speech Gestures
Templates
External Anim Sets


Key Concepts

Key Concepts

Key Concepts Sample Rendering Engine
Animation Data Only
Mapping
Filenames Must Match
Native Coordinates vs. FaceFX Coordinates
Owned by Analysis vs. Owned by User
Reference Pose vs. Bone Poses
External Anim Sets
Templates
Negative Keyframes
Baked Curves
Getting Data Out of FaceFX


Sample Rendering Engine

Sample Rendering Engine

Sample Rendering Engine FaceFX comes with a sample rendering engine for previewing animations. This is what you need to know:

  • It can be replaced with your engine
  • It is only a sample, so it has a limited feature set
  • You can preview morph and/or bones animations on your Max or Maya content.

Purpose

The sample rendering engine included with FaceFX has 3 purposes:

  1. To provide developers evaluating FaceFX with an immediate way to experiment with FaceFX without doing any coding or integration.
  2. To provide a sample integration for developers attempting to integrate their engine with the FaceFX SDK and FaceFX Studio.
  3. The Sample Rendering Engine is not designed for commercial use, but it is suitable to preview and tweak animations prior to exporting them to a 3D animation package or your game.

Limitations

  • Scaling bones only works when the scale is uniform (X, Y, and Z are all scaled to the same degree.)
  • Only features required for previewing facial animations are supported.

See Also

Preview Tab
Max FXR Exporter
Maya FXR Exporter


Animation Data Only

Animation Data Only

Animation Data Only The primary goal of the FaceFX product is to provide developers with animation data. The animation data comes in two forms:

  • Bone Transformation Data - For outputting final bone transformations to a skeletal system at every frame.
  • Curve Data - For driving all other animation systems supported by the target engine (morph system, materials, normal maps, etc.) FaceFX itself does not provide these systems, only the animation curve data to drive them.

What does this mean?

  • FXA and FXE files are the only ones needed by your game. Notice they take up virtually no memory. FXS, FXM, and FXR files are not used in game. They are used in FaceFX Studio only.
  • FaceFX does not provide an audio system, morph system, or rendering engine that can be used in your game.


Mapping, Curves and Nodes

Mapping, Curves and Nodes

Mapping, Curves and Nodes FaceFX animations are composed of a series of curves. Some of these curves are created automatically based upon the phonemes detected in an audio file. These include Speech Target curves which are named according to the targets specified in the Mapping Tab.

Curve names and Node Names Must match

Curves by themselves do not do anything. They need to drive something. If there is a node in the Face Graph with the same name as the curve, the curve will drive the node. Names are case sensitive.

Nodes Can Drive Other Nodes

Complex relationships can be set up in the face graph so that a single curve can drive many nodes, bone poses, morph targets, material properties, etc. Curve values are modified as they pass through the face graph by link functions, Min/Max values, etc. While the curves can remain simple and easy to create, the resulting values passed along to the target values can be very complex.


Filenames Must Match

Filenames Must Match

Filenames must match

Opening an FXA file

When you open an FXA file, FaceFX Studio searches the directory for FXS, FXR and FXL files with the same name. If it doesn’t find the FXR, no character will be displayed. If it doesn’t find the FXS, a warning message is displayed saying that no mapping information is found. The FXL file is used to list FaceFX Studio commands that will be executed when the FXA file is opened.

Opening an FXE file

Similarly, FXE and FXM files are matched up by name. When you mount an FXE file, FaceFX Studio will look for a FXM file in the same directory with the same name. If it doesn’t find an FXM file, the audio and phoneme information will not be shown.


Native Coordinates vs. FaceFX Coordinates

Native Coordinates vs. FaceFX Coordinates

Native coordinates vs. FaceFX coordinates To drive bones-based animations, FaceFX needs to know what coordinate system to use. More specifically, the reference pose and bone poses must be stored in the same coordinate system as the target game engine. By default, the FXA plug-ins export bone poses and the reference pose using the FaceFX coordinate system, which are designed to be used with FXR files in the FaceFX Studio sample rendering engine. Native Coordinate FXA plug-ins are also available, but the FXA files they generate can not be used with the FaceFX Studio sample rendering engine. These plug-ins output bone poses in Max or Maya native coordinates.

Artists evaluating FaceFX should use the default FXA exporters that use FaceFX coordinates. After programmers have implemented the FaceFX SDK into the game engine, it is possible to use the native coordinate FXA plug-ins, the coordsys command, or a combination of the tools to achieve the desired pipeline.

See Also

FXA plug-ins
coordsys command


Owned by Analysis vs. Owned by User

Owned by Analysis vs. Owned by User

Owned by Analysis vs. Owned by User
  • Curves in FaceFX can either be owned by analysis or owned by the user.
  • To change a curve's ownership, right-click on the curve in the Animations section of the Actor Panel. Select Curve Properties then modify the Owned by Analysis checkbox.
  • Targets contained in the Mapping are initially owned by analysis. Also, gesture curves are initially owned by analysis because they are also recalculated when phonemes are adjusted.
  • Curves owned by analysis are recreated every time the phoneme bar is modified. All existing keys in the curve are deleted and new keys are added.
  • You can not create, alter or delete keys in a curve that is owned by analysis. Trying to do so from the Curve Editor or Workspaces will fail.
  • When cleaning up an animation, leave the curves owned by analysis until you have finished modifying the phoneme timings. If further tweaks are required, change the ownership to Owned by User so you can modify the keys directly. Never change the curve back to Owned by Analysis or you will loose your work.

See Also

Actor Panel
Mapping Tab
Speech Targets


Reference Pose vs. Bone Poses

Reference Pose vs. Bone Poses

Reference Pose vs. Bone Poses
  • The Reference Pose defines what bones FaceFX cares about. It also defines the default transformations for those bones.
  • Bone Poses store transformation data for all bones that have changed with respect to the Reference Pose.
  • If you change your Reference Pose, you should always re-export all bone poses.
  • Bone Poses do not store transformation data that is relative to the reference pose. For example, opening the jaw a bit in the Reference Pose will not open the jaw in all bone poses. On the contrary, bone poses that contain the jaw bone will not be changed, but bone poses that do not contain the jaw bone will inherit the change from the Reference Pose. For this reason, it is always recommended to re-export all bone poses whenever the Reference Pose is changed.
  • Understanding the difference between the Reference Pose and bone poses can turn the FXA plug-ins into powerful animation tools. For example, if you notice that some of your speech targets contain unwanted movement in bones around the eyes, you can export a reference pose that contains only the bones around the eyes, then import the reference pose to all of the frames that contain speech targets. The Eye bones will be corrected, leaving the bones around the mouth unchanged.

See Also

FXA plug-ins


External Anim Sets

External Anim Sets

External Anim Sets FaceFX animation data is stored as curve names and keys. As long as an actor has nodes in its face graph that match the animation curve names, the character can play the animation. Animations can therefore be shared across multiple actors. External anim sets give developers an interface to do this.

External Anim Sets store animation data which can be linked up to an actor file (*.FXA) in FaceFX Studio or at runtime in your game. In FaceFX Studio, this process is called "mounting" and "unmounting" animation sets. FXE files are usually accompanied by a matching FaceFX Mini Session file (*.FXM). FXM files store all of the data needed by FaceFX Studio but not your game. This includes audio data and phoneme information.

To export an animation set from FaceFX Studio, simply select the animation group you want to export, then go to Actor->Export Anim Set... Be aware that you can't export the Default animation group, so you may need to transfer animations around with the Animation manager prior to exporting the animation set. Exporting the Animation Set creates an FXE and matching FXM file with the animation data from the animations in the selected group.

To mount an existing animation set, go to Actor->Mount Animation Set and select the FXE file you want to mount. Once mounted, FaceFX will treat the animation group just like it is part of the loaded FXA file but any changes to the animation group will be saved to the FXE and FXM files.

Animsets are useful for the following things:

  • Reducing Memory Footprint - All of the animations needed for a particular level can be stored in a single file then mounted with the FaceFX SDK when the level is loaded.
  • Transferring Animation Data - Transferring animation data between actors is simple with animation sets.
  • Division of Labor - Now you can have multiple artists working on the same actor's animations. Each person can work on their own unique animations and use anim sets to merge the animations back into the main actor file when they are finished.
  • Sharing animation data - Different actors can load the same FXE file in your game, but each actor will keep a separate copy of the file in memory
  • Copying animation data - After an animset is exported, it can be imported into an actor or mounted to the actor. If it is imported, a copy of the animation data is made and the animation data is stored both in the FXE file and in the FXA file.

See Also

FaceFX File Types
Working with Anim Sets Tutorial


Templates

Templates

Templates FaceFX Template files store all the information you are likely to want to share across multiple actors. The idea is that you get one character set up the way you want and you can easily copy the setup to your other characters. Template files have the following components:
  1. Face Graph
  2. Mapping
  3. Node Groups
  4. Cameras
  5. Workspaces
  6. Color Map
You can export a template with the Actor->Export Template option. All of the above components are exported. You can import a template with the Actor->Sync To Template option. When syncing to a template you have the option of selecting one or more of the above components to sync to.

When using the Template system, keep the following in mind:

  • Conflicting Data - It is possible for a template to have data that conflicts with the currently loaded FXA and FXS files. For example, both could have a workspace called “Speech Targets” or a camera named “Eye Left” etc. When conflicts exist, preference is given to the data in the Template. The only exception is when performing a sync to the face graph which is described in detail below.
  • Face Graph Sync - There is some complex logic that goes on behind the scenes when you sync to a template's face graph. The resulting face graph will contain only the links present in the template, but it will contain all nodes in both the template and the original face graph. If a node exists in both the template and the original face graph, the resulting node type will be the same as the one in the original face graph, but it will get min, max, and other user property values from the template. Bone pose information is not stored in the template so this will be preserved from the original face graph.
  • Cameras are Actor Specific – FXT files by definition store data that can be used across multiple actors. FXT files can store cameras to support workspaces, but cameras created for one actor will not necessarily work for another actor. This is due to the fact that cameras can be linked to bones and bones can be different across different actors.

See Also

FaceFX File Types


Negative Keyframes

Negative Keyframes

Negative Keyframes In order to voice a particular sound, the mouth needs to be in the appropriate position before any sound is made. In some cases, this causes FaceFX to insert keys at negative times (up to around 300 milliseconds for speech curves and more for gestures). Below are some things to keep in mind with negative keyframes:
  • Aggressively Cut Audio - Some games aggressively cut audio so that speech starts right away. Padding audio with some silence can resolve many issues with negative keyframes but this is not always possible so read below for other ways to deal with negative keyframes. On a side note, aggressively cut audio can also cause FaceFX Studio to analyze the audio incorrectly (typically by inserting a silence at the beginning of the file). Some files may produce a warning in the log about "Inadequate leading silence" but they may still analyze correctly. To fix files that are being analyzed incorrectly, use the set command to turn off speech detection prior to analyzing the problematic audio.
  • Use FaceFX to Trigger the Sound - The correct way to deal with negative keyframes is to let FaceFX trigger the sound file at time 0 in the animation. See the FaceFX SDK Documentation for more details. Using FaceFX to trigger sounds can have the side-effect of delaying audio playback slightly. In dialogue sequences this may not be a problem, but action-oriented grunts and groans may require the sound to be played immediately.
  • Gesture Curves - Gesture curves can take longer to ramp in than speech target curves. As a result, they can have keys pushed furthest into negative time. One way to resolve this issue for time-sensitive animations is to turn off speech gesture generation. This can be done from the Analyze Command or the animation creation wizard.

See Also

Integrating FaceFX with your Game Pipeline
Get and Set Command
Analyze Command


Baked Curves

Baked Curves

Baked Curves When a character is displayed in FaceFX, the Face Graph is constantly being evaluated to figure out the value of the bone pose nodes, and morph target nodes that move the vertices of the character. To display an animation outside of FaceFX where the FaceFX SDK is not available, one option is to bake curves out into keyframes which consist of a time, value, input slope and output slope. You can then insert these keys into other software, provided that the software has the ability to drive something equivalent to a bone pose or morph target. In Maya for example, you can use "Set Driven Key" functionality to achieve something identical to a bone pose.

Baked Curves vs. Raw Curves

The FXA exporters can return key information for curves (although not in mod-developer versions of the plug-ins). Curves can be raw or baked. Most of the time, baked curves are more useful. Raw curves return the exact key values, times, and slopes but may not accurately reflect the node's value over time because they do not take into account the node's minimum or maximum values or any input values from other nodes. Baked curves are an accurate approximation of the node's value over time.

Collapsing the Face Graph

Curves are also baked with the fgcollapse command.


Getting Data Out of FaceFX

Getting Data Out of FaceFX

Getting Data Out of FaceFX Once you have created Animation Data inside FaceFX, you can get it out by Integrating FaceFX with your game or application, or by getting the data into another animation program with the FXA Exporters. Getting data out is typically done in one of two ways:

Setting Keys per Bone per Frame

The FXA Exporters can import an animation by setting a key for every bone at every frame. This approach is simple but creates a lot of data that is difficult to manipulate and expensive to store and use.

Exporting curves

Another approach is to export curves from FaceFX into another application. Curves typically need to be baked with the FXA Exporters or the fgcollapse command prior to use in another application. FaceFX curves are stored as 4 floats representing the time, value, slope in, and slope out. FaceFX uses Hermite curve evaluation with tangents lengths equal to one third of the distance between keys.


FaceFX Studio

FaceFX Studio

FaceFX Studio


The Face Graph

The Face Graph

Introducing the Face Graph FaceFX uses a lot of technology to address the problems listed in Chapter 2, but the most import breakthrough is called the face graph. Artists may recognize the face graph as a Directed Acyclic Graph (DAG). In its simplest form, the face graph can be seen as a black box that sits between animation curves and the end values for bone transformations and morph targets, or any other in-game output.

Notice that the face graph is not limited to moving bones or morph targets based on animation curves. The face graph can be used to drive any conceivable system including material properties and normal map transitions, and can accept information from any conceivable source. For example, you could input the character’s “emotional state” into the face graph at runtime in order to change the way the character speaks. If you wanted to, you could even compute the characters “emotional state” in the face graph itself!

One of the benefits of the face graph is that it breaks the 1-to-1 relationship between curves and targets. In the past, curves could only drive a single target. If you wanted a character to be able to raise her right and left eyebrows independently, you had to model them as separate targets. But then if you wanted to drive both eyebrows together, you had to drive two separate curves. The more powerful you made your targets, the more cumbersome they were to drive. The face graph allows you to have both powerful targets and powerful curves.

FaceFX has three ways to add intelligence to a facial animation system:

  • Intelligence Layer 1: The curves – Curves are traditionally what drive facial animation. In FaceFX they provide the first layer of intelligence in a three tiered system.
  • Intelligence Layer 2: Face Graph Connections – The face graph can manage the mundane relationships between curves and targets. It can also fix problems associated with driving up two curves simultaneously that conflict with each other like the “bump + smile = junk” example above. The curves are therefore less intelligent than the face graph connections. Having unintelligent curves is a good thing because they are easier to create, modify and generate on the fly.
  • Intelligence Layer 3: Min Max Values – Min and Max values have the final say in the face graph. They modify values as they flow through the face graph, ensuring that a node is never driven higher or lower than it is supposed to be. Min and Max values can be used as a final layer of intelligence to ensure that any combination of curves and in-game events will never drive the face outside of realistic parameters.

See Also

Face Graph Tab
Face Graph Keyboard Shortcuts


Link Functions

Link Functions

Link Functions Link functions connect two nodes in the face graph. Values start at the input node and flow through the link function to the output node. The input node’s value can be thought of as x, and the link function can be thought of as some function of x: f(x).

The graph you see when you modify a link function demonstrates the range of possible values that link function can input and output. The X-axis is determined by the input node's min and max values, while the y-axis is determined by the output node's min and max values. The link function's graph represents the function's traversal of this 2-dimensional space. By default, red indicates negative values while green indicates positive values.


Linear

Linear

Linear

Most functions you will create in the face graph will be linear with b=0. As you drive up one node's value, the other node's value changes at a rate depending on the slope described by the variable m.


Clamped Linear

Clamped Linear

Clamped Linear

The clamped linear function joins a linear link function with a constant link function. Depending on the value of x, the output value is determined by either the linear link function or the constant. Clamped Linear functions are used in the sample content's gaze direction setup.

Variables

m - The slope of the linear link function

clampx - The x-value where the linear link function becomes constant.

clampy - The y-value where the linear link function becomes constant.

clampdir - At the point (clampx, clampy), the clamped linear function switches between a linear link function and a constant link function. The clampdir variable determines which function is evaluate to the right of the point and which is evaluated to the left. Clampdir values greater than zero indicate that the linear link function should be evaluated to the right of the point and constant link functions should be evaluated to the left. Values for clampdir equal to or less than zero do the opposite.


Constant

Constant

Constant

Constant link functions do not depend on the input node's value. They always output the same value regardless of the input. As a result, it is irrelevant what node is the input node.


Corrective

Corrective

Corrective

When two targets A and B conflict with each other but target A is more important, you can use a corrective link function from A to B. When A and B are both active, A forces B's value down. The amount of correction is determined by the "Correction Factor"


Cubic

Cubic

Cubic

A cubic link function.


Inverse

Inverse

Inverse

In the Face Graph, the inverse function's output is 0 when x = 0.


Negate

Negate

Negate

Negate is simply a linear link function with m = -1 and b=0.


One Clamp

One Clamp

One Clamp

One clamp is a link function required to implement our legacy bone blending algorithm that blends between a series of bone pose sets. Although bone pose sets are not a recommended way to implement talking characters with expressions, the One Clamp function is still supported. It remains a good example of a link function created to implement a particular equation in the Face Graph.


Quadratic

Quadratic

Quadratic


Square Root

Square Root

Square Root


Node Types

Node Types

Node Types

Node Evalutaion

Most nodes, including FxCombinerNodes, FxBonePoseNodes, and FxMorphTargetNodes based on the node's input values and the node's input operation g(y1, y2, y3, …). The input operation can be one of the following:
  1. Sum inputs - g(y1, y2, y3) = y1 + y2 + y3
  2. Multiply inputs - g(y1, y2, y3) = y1 * y2 * y3
  3. Select min - g(y1, y2, y3) = min(y1, y2, y3)
  4. Select max - g(y1, y2, y3) = max(y1, y2, y3)

Purpose of Nodes

  • Modify values – Every node has a node function g that determines how the Node will modify or use its inputs (y1, y2, y3, …). Most nodes in the face graph will simply add their inputs so g(y1, y2, y3) = y1+y2+y3. Multiplication is another supported operation for g and any function g is possible provided that a programmer adds the node type to the FaceFX SDK.
  • Receive values – Nodes can receive values from other nodes, but they also have the job of receiving inputs from animation curves and other in-game systems. Nodes that have the same name as an animation curve will receive the animation curve’s value at the beginning of the face graph evaluation.
  • Store Data – The final purpose of a node is to store data. FxBonePoseNodes store the bone position, rotation, and scale information so that the bones can be blended to the appropriate final transformation. Not all types of nodes hold data.


FxCombinerNode

FxCombinerNode

FxCombinerNode

FxCombinerNodes are the most common type of node. They can have any number of inputs and any number of outputs. Combiner nodes do not do anything with the data; they simply pass on and modify values as they flow through the face graph.


FxBonePoseNode

FxBonePoseNode

FxBonePoseNode

Bone pose nodes work just like combiner nodes, because they can have multiple inputs and multiple outputs. The nodes also store bone position, rotation, and scale information for a group of bones. Bone pose nodes are created by the FaceFX Max and Maya plug-ins.

Note: Bone pose nodes that contain rotations should always have min and max values of 0 and 1 respectively. The FaceFX SDK uses an optimized linear interpolation to evaluate quaternions that is only mathematically correct when evaluated between 0 and 1. When you load an actor with a bone pose that has rotations different from the reference pose and the minimum is less than 0 or the maximum is greater than 1, you will get a warning:
Found face graph bone pose node "POSE_NAME" containing
bones with rotations but with min(MIN_VALUE) or max(MAX_VALUE)
out of range (min < 0.0 or max > 1.0)

In order to fix the problem, make sure all bone poses that contain rotations have minimum and maximum values of 0 and 1 respectively. You may have to increase the number of bone poses in your actor in order to model both positive and negative rotations.


FxMorphTargetNode

FxMorphTargetNode

FxMorphTargetNode

FxMorphTargetNodes are used to drive morph targets. They have an additional parameter called Morph Target Name that is responsible for matching the node up to the appropriate blend shape in Maya or morph target in Max. In Maya, make sure this parameter matches exactly the name underneath the slider used to drive the target. All strings in FaceFX are case sensitive.

FaceFX Studio will automatically add FxMorphTargetNodes to your FXA file if the corresponding FXR file contains morph targets. This normally happens the first time you open an an actor in FaceFX Studio. The automatic process will only work if the new nodes do not have the same name as existing nodes.

FxMorphTargetNodes are a good example of how to derive off of existing node classes in the FaceFX SDK. Driving morph targets or material property transitions in your engine may require additional node parameters like the Morph Target Name parameter. Check the programmer documentation for more information on creating your own FaceFX node types.


FxDeltaNode

FxDeltaNode

FxDeltaNode

FxDeltaNodes output the difference between the node's value in the prior frame versus the node's value in the current frame. They can be used to calculate derivatives and find the change in time since the last frame. FxDeltaNodes do not use the Min/Max value even though a very large Min/Max is shown when you create the node.


FxCurrentTimeNode

FxCurrentTimeNode

FxCurrentTimeNode

FxCurrentTimeNodes output the current time. During in-game scenarios time is ever increasing. In FaceFX Studio however, the user is free to click the timeline and jump to a specific time. For this reason, FxCurrentTimeNodes do not work in FaceFX Studio. They always output a value of zero. To test a face graph setup that uses FxTimeNodes, you will need to open the actor with the Animation Playback sample included in the installation. Once you start up the animation playback sample, be sure to turn on Allow Non-Animation Tick Events from the options menu. FxCurrentTimeNodes can not have any inputs.


Analyzing Audio

Analyzing Audio

Analyzing Audio

Input Files

To generate an animation select Animation Manager from the Actor Menu at the top of the FaceFX Studio application. Then click on the Create Animation button to display the create animation dialog. Here you can browse for WAV or AIFF and text file to analyze and automatically generate an animation. Animations can also be created with the analyze command which is described in detail in the following chapter.

Audio Specifications

Make sure the audio being analyzed conforms to the following guidelines:
  • The audio file must be an uncompressed 16 bit PCM WAV or AIFF file.
  • The audio must be recorded at 16 kHz or greater.
  • The audio should be mono, though FaceFX will reduce the sound to one channel automatically.
  • The audio file is between 0.5 and 90 seconds long. FaceFX will attempt to analyze longer WAV files by breaking the sound into chunks that are less than 90 seconds long and analyzing the chunks without text. FaceFX will create placeholder animations (a simple mouth open and mouth closed animation) for audio files less than 0.5 seconds. These default min and max values can be changed with the set command using the a_audiomin and a_audiomax variables.
  • For best results, speech should not start within the first 100 milliseconds of the audio file. This allows the speech detection algorithms to get initialized properly and it also reduces the importance of how negative keyframes are dealt with in the FaceFX integration. If audio must be used that has speech starting immediately, it is possible that the beginning of some animations will be incorrectly recognized as silence. This can be fixed by setting the a_detectspeech variable to false in the the set command.

Text Specifications

FaceFX will give the best results when WAV files are accompanied by a text file that includes a transcription of the WAV file. Follow these guidelines when using text:

  • If a text file is found in the same directory and with the same name as the WAV file, the default behavior is to use the text in the analysis.
  • When using Japanese text, Katakana and Hiragana must be used because Kanji is not supported.
  • When using Korean text, Hangul characters are supported and a Romanized phonetic spelling of the Hangul characters can also be used. Make sure you have the Arial Unicode MS (TrueType) font installed (ARIALUNI.TTF) in your Windows\Fonts folder so you can see the words in the phoneme bar.
  • Do not use abbreviations in your text because most punctuation (including periods) is removed from the text prior to analysis.
  • Fonix VoiceIn technology will try to make an educated guess at the pronunciation of any digit strings in the text, but for best results, spell out all numbers in the text.


Speech Targets

Speech Targets

Speech Targets Speech targets are used to form the mouth shapes that occur when a character is talking. To create speech target curves, audio is segmented into phonemes, the phonemes are mapped to the speech targets, and coarticulation rules are used to create the speech target curves from the phoneme boundaries and mapping values.

Phoneme to Target Mapping

During analysis, audio files are first segmented into phonemes before animation curves are created. The relationship between the phonemes and animations is determined with phoneme to target mapping. Each phoneme is “mapped” to one or more animation curves so that the animation curve becomes active when the phoneme is detected. The Default Mapping can be used, or the mapping can be defined from the Mapping Tab.

To create the default mapping for a new actor, simply open the Mapping Tab of the application and click on the “New Default Mapping” button. To create your own mapping, you can add one target at a time from the Mapping Tab, or use Workspaces in Mapping mode for a more visual process. Please refer to the chapter entitled The FaceFX Studio Application for more information on these two tabs.

Coarticulation

To generate animation curves from the phoneme segmentation results, FaceFX uses coarticulation rules. These rules take into account how the mouth moves into position prior to any sound being made and how the mouth transitions between phonemes. Only one coarticulation model is currently supported.

See Also

Character Setup
Owned by Analysis vs. Owned by User


Speech Gestures

Speech Gestures

Speech Gestures Speech Gestures are automatically calculated when you generate an animation from audio. Some of the gestures are randomly generated while others are based upon audio stress detection. The generated curves control blinks, eyebrow raises, and head and eye rotations. The final head and eye rotations are a combination of the orientation and emphasis curves, so some setup is required in the Face Graph to use the curves correctly. To create the required Face Graph setup, simply follow the Setting Up Gestures Tutorial.

What gestures are created and how they are driven is determined by a Gesture configuration file (*.fxg). These can be specified from the Generate Animations dialog or from the analyze command with the -gestureconfig flag as follows:
analyze -audio "C:\audio.wav" -gestureconfig "Slow"

Notice that the full path to the gesture configuration file is not included and the name is enclosed in quotes. The following are 5 FXG files that ship with FaceFX Studio:

  • Default.fxg - The default speech gesture configuration. Broad gestures with a stress threshold suitable for a wide range of speakers and utterance intonations.
  • Fast.fxg - A gesture configuration with faster, and slightly more muted gestured than default. A lower stress threshold produces more gestures than the default configuration.
  • Slow.fxg - A very broad and sweeping gesture configuration. Long, slow, and relatively infrequent gestures makes this an ideal configuration for slower speech or speakers.
  • Backchannel.fxg - The backchanneling config is used for generating gestures appropriate for the listener of a conversation, rather than the speaker. The gestures are slightly faster and more muted than the default, as well as time-delayed.
  • Idle.fxg - A configuration suited for producing idle animations with very slow gestures and orientation drifts.

Orientation Head Rotations

Periodic shifts in the orientation of the head add considerably to the realism of an animation. To achieve this effect, the following orientation curves are created: Orientation Head Pitch, Orientation Head Roll, Orientation Head Yaw. These curves are designed to drive the rotation of the head in degrees by a very small amount, typically less than 5 degrees. This type of head rotation should not change the direction of the gaze, so the curves should be negated for the eye rotation as is done in the default Face Graph gesture setup. Orientation curves are generated completely randomly and are not affected by changes to the phoneme alignment.

Emphasis Actions

Emphasis actions used are a small subset of the gestures people use during speech. They are chosen from dozens, arguably hundreds of common movements to satisfy some tight criteria.
  • They can be linked to a single stressed syllable.
  • They are brief and discrete, returning to neutral when completed.
  • They do not convey (much) meaning, so their use is not catastrophically wrong in most contexts.
  • They are easily parameterized and modified for variation.
  • No one is so distinct that its repetition becomes obvious.

The following Emphasis Actions are created, all of which are combined into the Emphasis Head Pitch, Emphasis Head Yaw, Emphasis Heat Roll curves: Strong Head Nod, Inverted Head Nod, Quick Head Nod, Normal Head Nod, Head Tilt, Head Turn. The Emphasis Actions are defined in a Gesture Configuration file (*.fxg). Because Emphasis Actions are linked to a single stressed syllable, the curves are recalculated when you modify an animation with a change to the Phoneme Bar. Like the Orientation Head Rotations, the Emphasis Action curves are not intended to change the gaze direction, so eye rotations must be negated with these curves. The final head rotations are a combination of the Emphasis Actions and the Orientation Hear Rotations.

Blinks and Eyebrow Raises

Blinks and Eyebrow Raises also qualify as Emphasis Actions, but they are special cases because they do not rotate the head. Blinks are assigned randomly, with stress-based calculations, and with extra rules to ensure that the character always blinks at least once for a given duration of time. Eyebrow raises are generated based upon audio stress calculations. They are recalculated when you modify the animation from the Phoneme bar.

Curve Descriptions

The following table describes all of the gesture curves created by the the included FXG files. The min and max values are those used in Default.fxg.

Curve Name | Description | Min | Max
Orientation Head Pitch | Nodding the head "yes". Negative is up, positive is down. | 0 | 3.5
Orientation Head Roll | Twisting the head as if to crack the neck. | -2.5 | 2.5
Orientation Head Yaw | Shaking the head "no" | -4 | 4
Gaze Eye Pitch | Looking up and down. Positive is down, negative is up. | 0 | 1
Gaze Eye Yaw | Shaking the head "no" | -.6 | .6
Emphasis Head Pitch | Nodding the head "yes". Negative is up, positive is down. Same as Orientation Head Pitch, but used for emphasis actions. | -3 | 5
Emphasis Head Roll | Twisting the head as if to crack the neck. Same as Orientation Head Roll, but used for emphasis actions. | -2 | 2
Emphasis Head Yaw | Shaking the head "no". Same as Orientation Head Yaw, but used for emphasis actions. | -2 | 2
Blink | Drives the left and right eyelids. Blinks are rapid movements that always drive completely to 1. | 0 | 1
Eyebrow Raise | Raises the left and right eyebrows in unison to suggest emphasis. | 0 | 1

See Also

Setting Up Gestures
Character Setup


Text Tagging

Text Tagging

Text Tagging Text tagging is a way to insert expression curves or signal in-game events in synchronization with the words spoken in an audio file. FaceFX technology figures out the timing of every word in the text file then uses the information in the tags to create curves based upon the position of the tags in the text. You can also break up the analysis of a larger WAV file into smaller files using audio chunking text tags.

Curve Text Tags

Curve Types

FaceFX text tagging supports four types of curves: Leading Triplets, Centered Triplets, Trailing Triplets, and Quadruplets. Below is an example of all four curve types drawn on top of each other. Notice that the type= tag is not required for the Quadruplet below because quads are the default curve type. Also notice that the quotes around the name are only necessary when there are spaces in the name.
Text
["Yellow curve" type=lt]
[Orange type=ct]
[Green type=tt]
[Black]
tagging
[/Black][/Green][/Orange] [/"Yellow curve"]
is fun!

Parameter List

The following parameters come after the name of the curve to specify additional attributes: type, v1, v2, v3, v4, TimeShift, Easein, Easeout, and Duration. The parameters are case insensitive, so you can type “timeshift”, “TIMESHIFT”, or “TiMeShIfT” and it will have the same effect.

v1, v2, v3, v4

These parameters are used to change the value of the curve at the specified points. The default values are 1.0 at the apex of the curve and 0.0 at the bottom of the curve.

Timeshift

This function shifts the entire curve by a fixed amount of time in seconds. Negative values are permitted.

Easein

This parameter specifies the length of time before the starting word boundary that the curve begins. The first point in a curve is located before the starting word boundary because the curve needs time to ramp up. The exact amount of time is specified with the Easein parameter. The default value is 0.2 seconds.

Easeout

This parameter works exactly like Easein except that it modifies the speed at which the curve tapers off. It does this by adjusting the distance between the final point and the ending word boundary.

Duration

The duration parameter is only valid for quadruplet curves. The parameter specifies the time in between the V2 and V3 values. The duration parameter has the effect of shifting the V3 and V4 values and overriding the placement dictated by the closing tag.

Default Values for Curve Parameters

type | type | v1 | v2 | v3 | v4 | timeshift | easein | easeout | duration
 |  | values |  |  |  | seconds
quadruplet | quad | 0 | 1 | 1 | 0 | 0 | 0.2 | 0.2 | (from text)
leading triplet | lt | 0 | 1 | 0 | N/A | 0 | 0.2 | 0.2 | N/A
trailing triplet | tt | 0 | 1 | 0 | N/A | 0 | 0.2 | 0.2 | N/A
center triplet | ct | 0 | 1 | 0 | N/A | 0 | 0.2 | 0.2 | N/A

Notes

  • Text tags can be used even without text. In this case, all key times will be relative to time zero. Use of the timeshift parameter will be important in this case.
  • Make sure there are no spaces between the parameters and the arguments. For example timeshift = 1 is incorrect and timeshift=1 is correct.
  • Text tags can create multiple keys with different values at exactly the same time. This will be evaluated by FaceFX in an unpredictable way, so this scenario should be avoided.

Audio Chunking Text Tags

Syntax

  • Tags are encapsulated with angle brackets: <>
  • Tags contain only one number: the time in seconds of the marker: <5.4>

Usage

Imagine a single audio file used in a cinematic. The audio file is 120 seconds long, but the character only speaks two small phrases. "Yes, I'm here", starting at 5.6 seconds and going to 6.8 seconds, and "Who? Sawyer? I haven't seen him in three days.", starting at 100.2 seconds and going to 104.3 seconds. Mark up the text as such:

<5>Yes, I'm here<7.5> <99.5>Who? Sawyer? I haven't seen him in three days.<105>

Notice the padding given on either end of the speech. If you place the marker too close to the onset of the speech, the recognizer won't give its best results. Use between 0.5 and 1.0 seconds of padding on either end of the speech for best results.

Now imagine another audio file, one that has a lot of silence at the beginning, and the character's speech "Whoa! You scared me", right at the end, starting at about 15 seconds into the audio. Mark up the text as such:

<14.5>Whoa! You scared me

Notice that you don't need to mark the end of the chunk if it's near the end of the audio.

Imagine one last audio file, where the character immedately says "Duck!", just a half second into the audio file, and stops talking around 1 second, but there is a large amount of trailing silence with no further speech. Chances are this doesn't need to be specially marked up, but if you find a problem with a similar audio file, mark it up as such:

Duck!<1.5>

Notes

  • Tags should always increase in value. The only caveat is the following legal markup: <1.0>Phrase one<10.0> <10.0>Phrase Two<14.2>. Even though the two middle tags have the same time, because they're tagging different phrases, that's fine.
  • You should never overlap chunks.
  • Don't place chunk markers with negative times.
  • Don't place chunk markers with times greater than the audio duration.


Languages

Languages

Languages FaceFX supports 8 languages: American English, UK English, French, German, Italian, Spanish, Korean, and Japanese. By default, FaceFX only ships with US English. Please contact us if your game needs to support multiple languages.

The language specifies both the neural nets used to analyze the audio as well as the dictionary used to decipher the text. If text is not being used, you can get good results analyzing any language using the default US English selection. A slight improvement in quality can be achieved using the appropriate neural nets in the no-text case. For optimal results however, text must be used and the appropriate language must be selected when analyzing the audio.

When using Japanese text, Katakana and Hiragana must be used because Kanji is not supported.

To analyze an audio file with a particular language, simply select the language from the Language listbox in the Create Animation Wizard. If you are using the analyze command, you can specify the language with the -language flag.


Batch Processing

Batch Processing

How To

Animation Manager

Creating new animations is done from the Animation Manager. Open it from the application's toolbar or from Actor->Animation Manager. To create a new animation, simply click the Create Animation button and follow the wizard.

Batch Processing

Batch processing is done with the analyze command. When batch processing a large number of files, be aware that FaceFX Studio will keep all of the audio data in memory until you save the actor. When batch processing lots of audio via the command system, you may want to break-up the analysis with the saveActor command to avoid high memory usage.

Be aware that the analyze command is disabled in evaluation builds of FaceFX. In order to use it, you must purchase a commercial license for FaceFX.

See Also

Analyze Command
Evaluation Process


Command Interface

Command Interface

Command Interface FaceFX Studio uses a command system internally to implement undo /redo and to log information about the FaceFX Studio session. Users can take advantage of the command system to perform batch operations. The most recent commands are printed to the Console Tab and an archive of all commands are printed to the FaceFxStudioLog.html file in the Logs directory of the FaceFX Studio installation. Some of the most useful commands are documentd in the below pages for easy reference.

DOS Prompt Functionality

The FaceFX Command Interface can be used to turn FaceFX Studio into a powerful command-line tool. Simply open a DOS Prompt and navigate to the FaceFX Installation directory (Note: the current directory must be the FaceFX installation directory for everything to work). Then type something like:
fxstudio -exec "C:\my-batch-file.fxl"

You can also execute a batch file from the FaceFX Studio GUI. Simply type the following from the FaceFX Studio CommandLine:
exec -file "C:\my-batch-file.fxl";

The DOS promt can also be used to launch FaceFX Studio in safe mode. Safe mode does not remember window positions and should be used if you have unplugged a second monitor and can no longer access FaceFX Studio from a single monitor.

Automatically Executed Commands

FaceFX will automatically execute commands found in the following files when the appropriate events occur. The Autoexec.fxl, preloadactor.fxl, postloadactor.fxl, presaveactor.fxl. postsaveactor.fxl files must be located in the FaceFX install directory, while the files that use the name of the FXA file must be located in the same directory as the FXA file. The global files in the installation path are executed before the actor-specific files. In Unreal Engine 3, all files should be located in the executable directory.
  • Autoexec.fxl – Commands in this file are executed when FaceFX Studio opens.
  • preloadactor.fxl – Commands in this file are executed before any actor is loaded.
  • postloadactor.fxl – Commands in this file are executed after any actor is loaded.
  • presaveactor.fxl – Commands in this file are executed before any actor is saved.
  • postsaveactor.fxl – Commands in this file are executed after any actor is saved.
  • FileNameOfFXA_PreLoad.fxl – Commands in this file are executed before the specific actor is loaded but after the commands in the preloadactor.fxl file.
  • FileNameOfFXA_PostLoad.fxl – Commands in this file are executed after the specific actor is loaded and after the commands in the postloadactor.fxl file.
  • FileNameOfFXA_PreSave.fxl – Commands in this file are executed before the specific actor is saved but after the commands in the presaveactor.fxl file.
  • FileNameOfFXA_PostSave.fxl – Commands in this file are executed after the specific actor is saved and after the commands in the postsaveactor.fxl file.
  • FileNameOfFXA.fxl – For backwards compatibility, this script is searched for and executed if FileNameOfFXA_PostLoad.fxl cannot be found.

Syntax

  • Semicolon – Every FaceFX Studio command should end with a semicolon (;).
  • Comments – The following characters are used to start a comment line: “//”
  • Spaces - Multiple spaces will be reduced to a single space unless the spaces are encapsulated in quotes. no spaces are allowed in between a Flag and the Flag's argument.
  • Case sensitive – Unlike text tags, FaceFX Studio commands are case sensitive.
  • Flags – Most commands can accept one or more flags. Flags are preceded by the dash character: “-“. Flags are used to set options in the command or pass variables to the command. Some flags are optional while others are mandatory. The following example of the select command uses three flags: select -type "node" -names "Happy" -nozoom;
  • Strings Separator – The pipe character “|” is used to separate strings in commands. Names of nodes, animation curves, and other objects used by FaceFX Studio should not contain the pipe character to avoid complications with the script interface.

See Also

Console Tab
ScrubBar - Timeline - CommandLine


analyze

analyze

analyze Command Notice: The analyze command is not available in all versions of FaceFX.

The analyze command generates animations from audio files. The analyze command is the interface for batch processing animations. Like all commands, it can be issued from the FaceFX command line or the DOS prompt via the exec command. When batch processing lots of audio via the command system, you may want to break-up the analysis with the saveActor command because FaceFX will keep all analyzed audio in memory until the actor is saved.

If a text file is present in the WAV file directory with the same name as the WAV file, the text will be used in the analysis by default.

Command Syntax

Flag | Long Flag | Arg | Description
-au | -audio | Y | Specifies a single audio file to analyze
-dir | -directory | Y | Specifies an entire directory to analyze. The group name is determined by the directory name. Can be used with the -recurse flag.
-g | -group | Y | Specifies the group that the animation should be imported to. Used only when analyzing a single file with the -audio flag. By default or when analyzing one or more directories, the group is specified by the folder name.
-an | -anim | Y | Specifies the animation name. By default, the animation name is taken from the WAV file name. Within a group, all animation names must be unique. As a result, the analysis command will fail if an animation with the same name already exists in the group unless the -overwrite flag is used.
-l | -language | Y | Specifies the language to use. Accepts "USEnglish", "UKEnglish", "French", "German", "Spanish", "Italian", "Korean", "Japanese".
-cc | -coarticulationconfig | Y | Allows you to select the coarticulation config file to use. At the current time, only one coarticulation model is supported, so this flag is useless.
-gc | -gestureconfig | Y | Allows you to select the gesture config file to use. Accepts “Default”, “Fast”, “Slow”, “Idle” and “Backchannel”.
-r | -recurse | N | Used with the -dir flag to recurse to all subdirectories when analyzing audio files.
-o | -overwrite | N | Used to specify that the analyze command should overwrite existing animations if they exist with the same name. Default is false.
-nt | -notext | N | Audio file(s) will be analyzed without text.
-ng | -nogestures | N | Speech gesture curves will not be generated.

Examples

analyze -dir "C:\Sounds" -recurse -notext
analyze -audio "C:\test.wav" -group "first level" -anim "test2" -notext
analyze -dir "C:\Sounds" -overwrite -language "French" -gc "Fast"

See Also

Evaluation Process
Batch Processing


rename

rename

rename Command The rename command directly modifies the FaceFX nametable. All objects that share the same name (nodes, curves, animations, etc.) reference one entry in the name table. Using the rename command will therefore change all objects with a particular name. Names for nodes, curves, animations, and animation groups are stored in the actor file (*.FXA).

Command Syntax

Flag | Long Flag | Arg | Description
-n | -name | Y | The original name
-t | -to | Y | Specifies an entire directory to analyze

Examples

rename -name "Old Name" -to "New Name"


exec

exec

exec Command The exec command is used to execute a batch file containing one or more FaceFX Studio commands. Batch files are simply text files with a list of FaceFX commands in them. The (*.FXL) extension is common for batch files and it stands for FaceFX Language file.

Command Syntax

Flag | Long Flag | Arg | Description
-f | -file | Y | Specifies the full path to a text file that contains a list of commands that should be executed. The *.fxl extension is commonly used for this type of file.

Examples

exec -file "C:\my-batch-file.fxl";

Dos Prompt

The exec command can also be run from the Dos command prompt with the following syntax:

fxstudio -exec "C:\my-batch-file.fxl";

Notice that you do not include the “.exe” file extension when calling fxstudio. The -exec parameter serves as a command-line interface to FaceFX. Using batch files, FaceFX can easily be used as part of an automated build script for your game. Remember that you will have to call the loadActor and saveActor commands in your batch file to do anything meaningful.


batch and execBatch

batch and execBatch

batch and execBatch Commands The batch and execbatch commands are used to tell FaceFX not to update the user interface while performing a group of commands. This can speed up batch file execution a great deal. The execBatch command is followed by one or more of the following flags which specify what elements of the GUI should not be updated: addednodes, removednodes, editednodes, addedcurves, removedcurves, changedmapping, editedcurves, changedanimation.

Be aware that you can not have nested batch and execbatch commands. In other words, every batch command must be followed by an execbatch command before another batch command is issued. If you are trying to run a script from inside another script this can be one common cause of script failure.

batch Command Syntax

There are no flags for the batch command.

execBatch Command Syntax

Flag | Long Flag | Arg | Description
-an | -addednodes | N | Specifies that no GUI updates should occur for added nodes.
-rn | -removednodes | N | Specifies that no GUI updates should occur for removed nodes.
-en | -editednodes | N | Specifies that no GUI updates should occur for edited nodes.
-ac | -addedcurves | N | Specifies that no GUI updates should occur for added curves.
-rc | -removedcurves | N | Specifies that no GUI updates should occur for removed curves.
-cm | -changedmapping | N | Specifies that no GUI updates should occur for changed mapping.
-ec | -editedcurves | N | Specifies that no GUI updates should occur for edited curves.
-ca | -changedanimation | N | Specifies that no GUI updates should occur for changed animations.

Examples

The batch command is issued at the start of the group of commands and the execBatch command is issued at the end as follows:

batch;
graph -addnode -nodetype "FxCombinerNode" -name "Node A";
graph -addnode -nodetype "FxCombinerNode" -name "Node B";
graph -link -from "Node A" -to "Node B" -linkfn "linear";
execBatch -editednodes;


get and set

get and set

get and set Commands The get and set commands are used to get and set internal variables used by FaceFX Studio. For many of these options there is no way to set

Be aware that you can not have nested batch and execbatch commands. In other words, every batch command must be followed by an execbatch command before another batch command is issued. If you are trying to run a script from inside another script this can be one common cause of script failure.

get Command Syntax

Flag | Long Flag | Arg | Description
-n | -name | Y | Specifies the name of the variable to get. "rw_showbindpose" "a_audiomin", "a_audiomax" are examples.

Get Examples

get -name "a_audiomin";
get -n "rw_showbindpose";

set Command Syntax

Flag | Long Flag | Arg | Description
-n | -name | Y | Specifies the name of the variable to set. "rw_showbindpose" "a_audiomin", "a_audiomax" are examples.
-v | -value | Y | Specifies the value of the variable to set. All variables accept a string. Numerical values are converted from the string into a float or integer, etc.

Set Examples

set -name "rw_showbindpose" -value "1";
set -n "rw_scalefactor" -v "10.0";
set -n "a_audiomin" -v "0.1";
set -name "a_detectspeech" -value "false";
set -name "g_audiosystem" -v "directsound"

Accepted Variables

  • rw_showbindpose - Specifies if the render widget should display the bind pose or allow the bones to be driven by the FXA file. This is useful for debugging content issues and finding out if the FXA file or FXR file is to blame for a particular problem. Remember to set the value to "0" when you want to start using FaceFX normally again.
  • a_audiomin - Extremely short segments of audio can not always be successfully analyzed by FaceFX Studio. This variable sets the minimum audio duration that can be analyzed without returning an error. The default is 0.5 seconds. Reducing this value to 0.1 may allow more files to analyze correctly, but no guarantee can be made that analysis will be successful for all short files.
  • a_audiomax - When using text, extremely long segments of audio can not always be successfully analyzed by FaceFX Studio. Without text, FaceFX can automatically chop up the audio into smaller segments. This variable sets the maximum audio duration for text-based analysis. Files longer than this duration will automatically be chopped up into smaller segments and analyzed without text. The default is 90 seconds.
  • cw_historylength - Changes the maximum number of lines visable from the Console Tab prior to the results being flushed. A full log is kept in the FaceFxStudioLog.html file. The Default value is 250.
  • a_detectspeech - Determines if speech detection should be used when analyzing audio files. In most cases, speech detection should be on. If audio files are edited such that speech starts at the very beginning of the audio file however, speech detection can cause the beginning of the animation to be incorrectly classified as silence. In these cases, setting a_detectspeech to false will fix the problem.
  • rw_scalefactor - The scale factor is used by the FaceFX Sample Rendering Engine to determine the scale that objects are rendered at. This was added to prevent scenarios where small objects could not be seen because the camera had to be so close that the object was in front of the clipping plane. In this scenario, increasing the rw_scalefactor to "10.0" should resolve the issue.
  • g_audiosystem - By default FaceFX uses OpenAL to playback sounds. Not all sound drivers implement the OpenAL specification completely or correctly, so the sound system can be changed to Direct Sound with this variable. Acceptable values include "openal" and "directsound".


print

print

print Command The print command simply prints a message to the log file.

Command Syntax

Flag | Long Flag | Arg | Description
-m | -message | Y | Specifies the message to be printed to the log file.

Examples

print -m "This will be printed to the log file.";


saveActor

saveActor

loadActor Command Saves the currently loaded FXA file. The corresponding FXR and FXS files will also be saved in the same directory. Any External Anim sets that are mounted will also be saved as FXE and FXM files. If a file already exists, the existing file will be overwritten.

Warning: If any of the saved files are marked as read only, the files will not be saved and no warning will be given.

Command Syntax

Flag | Long Flag | Arg | Description
-f | -file | Y | Specifies the full path of the FXA file to save.

Examples

saveActor -f "C:\Slade-Maya.fxa"


developer

developer

developer Command A command used to help debug FaceFX Studio. Some of the flags are also useful for debugging content problems and are included below.

Command Syntax

Flag | Long Flag | Arg | Description
-dbp | -dumpbonepose | Y | Pass the name of a bone pose after this flag to get a list of the bones contained in the pose and their transformations.
-dmas | -dumpmountedanimationsets | N | Include this flag to dump the names of all mounted anim sets to the log.
-drp | -dumprefpose | N | Include this flag to dump the transformations of all the bones in the reference pose.

Examples

developer -dumpbonepose "Eat"
developer -dumpmountedanimationsets;


loadActor

loadActor

loadActor Command Loads the specified FXA file. The directory is searched for FXR and FXS files to load with the same name. The directory is also searched for an FXL file with the same name. If an FXL file is found, the commands contained in the FXL file are automatically executed

Command Syntax

Flag | Long Flag | Arg | Description
-f | -file | Y | Specifies the full path to the FXA file to load.

Examples

loadActor -f "C:\Slade-Maya.fxa"


coordsys

coordsys

coordsys Command The coordsys command swaps the coordinate system of the currently loaded FXA file. See Native coordinates vs. FaceFX coordinates for more information on coordinate systems. Be sure to check out how Automatically Executed Commands work to understand how you can automatically swap the coordinate system whenever an actor is saved.

Command Syntax

Flag | Long Flag | Arg | Description
-sp | -sourcepackage | Y | Specifies the 3D animation package this file is from ("MAX" or "MAYA"). The appropriate coordinates will be flipped.

Examples

coordsys -sp "MAYA"

See Also

Native coordinates vs. FaceFX coordinates
Automatically Executed Commands


cook

cook

cook Command The cook command ensures that the byte order of an FXA or FXE file matches the byte order of a specific platform. If the byte order is incorrect when you load the file in your game, the FaceFX SDK will automatically swap the byte order at the expense of some unnecessary processing and increased load times.

The cook command will save out a new file in the same directory as the loaded file. The new file name will be in the following format:
OLD_FILENAME + "-Cooked_" + PLATFORM_NAME + EXTENSION

For example the below command:
cook -p "XBOX360" -f "C:\Slade-Maya.fxa";

Would result in the following file:
C:\Slade-Maya-Cooked_xbox360.fxa

Command Syntax

Flag | Long Flag | Arg | Description
-p | -platform | Y | Specifies the platform you are cooking to. Acceptable values are: "XBOX360", "PC", "BIGENDIAN", "LITTLEENDIAN"
-f | -file | Y | Specifies the full path to the FXA or FXE file to cook.

Examples

cook -p "XBOX360" -f "C:\Slade-Maya.fxa";


fgcollapse

fgcollapse

fgcollapse Command The fgcollapse command is used to create a simplified version of an existing FXA file. The simplified version only contains essential nodes like bone pose nodes and morph target nodes. The simplified face graph has zero Face Graph links. Each node contains keys representing the node's value for each animation.

The fgcollapse command is most useful as a final step before taking FaceFX data out of FaceFX. Without access to the FaceFX SDK, most applications can only use pre-calculated animation curves. Alternatively, this command can be used to simplify a complex cinematic Face Graph with hundreds of nodes into an optimized actor file for in-game use.

Warning - This command is slow and it doesn't have a progress bar!

Specifically, the fgcollapse command does the following:
  1. Open the specified actor file for processing (does not open the file in FaceFX Studio).
  2. Create a list of all important nodes including bone pose nodes and morph target nodes.
  3. Add the specified "forced nodes" to the list of important nodes.
  4. For each animation stored in the actor, evaluate the animation at a very high frame rate in order to calculate key times, values, input slopes, and output slopes for each of the important nodes.
  5. Delete all Face Graph links and all non-important face graph nodes.
  6. Save a copy of the collapsed actor to a new file. Append -FG_Collapsed to the existing filename.

Command Syntax

Flag | Long Flag | Arg | Description
-f | -file | Y | Specifies the full path to the FXA or FXE file to collapse.
-fn | -forcenodes | Y | Force these nodes to be included in the collapsed file. Combiner nodes whose values are important to you should be included here.

Examples

fgcollapse -file "C:\Slade-Maya.fxa" -forcenodes "ImportantCombinerNodeName1|ImportantCombinerNodeName2";


Graphical User Interface

Graphical User Interface

Graphical User interface

UI Concepts

  • Left-click or use the middle mouse button in a control to give it focus
  • Double click on a tab header to tear it off the application. This lets you position it or move it to another monitor
  • Drag a tab header to change the tab order.
  • Hit F11 to make a tab go in full screen mode.
  • Hit F12 to toggle the "Always On Top" feature for torn-off tabs
  • Ctrl + P to access the properties/options for a particular widget
  • Ctrl + O to open a new actor.
  • Ctrl + R to reset the view in a particular widget if you get "lost"
  • Use the middle mouse button (MMB) to pan in the 2D and 3D tabs
  • Use the mouse wheel to zoom in the 2D and 3D tabs
  • Some tabs contain multiple controls delimitated by horizontal or vertical splitter bars. Double-click on a bar or drag it beyond the bounds of the window to separate the controls into distinct windows.


Actor Panel

Actor Panel

Actor Panel

Simply click on one of the expandable windows to view the panel. With any of the panels, you can sort the lists alphebetically by clicking on the top.

Actor Info

The Actor Info Panel currently has no use other than to list the bones included with the actor's reference pose and to list the name of the actor. Like all of the list boxes in the Actor Panel you can sort the contents alphabetically by clicking on the column header.

Face Graph

The Face Graph Panel lists nodes in the face graph. Click on the top of the list to sort the list alphabetically.
  • Node Groups - You can select a Node Group from the Face Graph Panel so that only nodes from that group are displayed. Node Groups are created from the Node Group Manager which is available from: Actor-> Node Group Manager.
  • Quick Preview Slider - The Quick Preview Slider at the bottom of the Face Graph Panel becomes active when a single node is selected. You can't set keys from this slider, but it is useful for setting up the Face Graph and previewing poses. Right-click in the slider to reset it to its default value. When the Workspace Tab is in Create Combiner mode, you can create a complicated pose for your character by setting the values for multiple nodes with the slider, then clicking on the Create Combiner Node from Sliders button from the workspace tab. To clear all values from the Face Graph, you can click on the Reset all sliders to their default values button. The Quick Preview Slider prevents artists from having to create a workspace for simple operations.

Animations

The Animation Panel selects the active animation and the active curve(s) within the animation. Selecting curves makes them visible in the Animation Tab. Right clicking on a curve allows you to change the curve properties or open up the curve manager.
  • Curve Properties - You can change the color of a curve or specify if the curve is owned by the analysis or owned by the user. Notice the icons to the left of each curve specifying these two variables. Curves owned by the analysis are usually included in the mapping and are regenerated every time you move a phoneme boundary. They do not have keys because you would loose your edits by making a change to the phoneme bar.
  • Curve Manager - The curve manager lets you add curves to an animation, an animation group or all animations in the actor. You can access by right-clicking a curve or selecting Curve Manager from the Actor menu at the top of the application.


Actor Menu

Actor Menu

Actor Panel
The following is a list of operations available from the FaceFX Studio Actor Menu:

  • Rename Actor - Issues the rename command to change the actor's name.
  • Node Group Manager - Launches the Node Group Manager which can be used to create, modify, or delete node groups. Node groups let you find nodes in a complex face graph easily.
  • Animation Manager - Launches the Animation Manager which is used to create animations and transfer them in between groups.
  • Curve Manager - Launches the Curve Manager which is used to add or remove curves from an animation. Before you can insert keys via the Curve Editor the curve must already exist in the animation
  • Export Template - Saves a template file (*.FXT) for the current actor.
  • Sync to Template - Syncs the current Actor to a template file (*.FXT)
  • Mount Animation Set - Mounts an external anim set (*.FXE and *.FXM) to the current actor. Once mounted, the animation group appears to be part of the actor, but the files are kept separate when saved.
  • Unmount Animation Set - When the selected animation group is an external anim set, you can select this option to unmount it.
  • Import Animation Set - Merges an external anim set (*.FXE and *.FXM) with the current actor. Unlike the mounting process, this operation copies the data from the external anim set into the FXA and FXS files.
  • Export Animation Set - Creates an external animation set from the selected animation group. You can't create an external anim set from the Default group.
  • Edit Default Bone Weights - Bone weights store the amount that FaceFX controls a particular bone. By default, FaceFX completely controls all bones in the reference pose. The Default Bone Weights are stored on a per actor basis and affect all animations.
  • Edit Anim's Bone Weights - The Default Bone weights mentioned above can be overridden on a per-animation basis. Select the animation, then choose this option to modify an animation's bone weights.

See Also

FaceFX File Types
Templates
External Anim Sets


Face Graph Tab

Face Graph Tab

Face Graph Tab

The face graph is the heart of the FaceFX Studio application. This section deals primarily with the mechanics of working with the face graph. For more general information, check the See Also section below.

Work Area

  • Selection - Select a node by left-clicking on it. The node properties are displayed in the Properties Section. Select multiple nodes by dragging out a selection box, or holding Ctrl while selecting multiple nodes.
  • Moving Nodes - Left-click on a selected node and drag to move the node or a group of selected nodes. Hold Shift while dragging to move the selection without having to click on it first. To layout the graph, right-click in the face graph and select Relayout Graph.
  • Zooming - Use the mouse wheel to zoom in and out. Holding Ctrl + Shift while dragging out a selection will zoom to the selection. Hit Ctrl + R to reset the view. Selecting a node in the Face Graph Panel on the left will zoom to the selected node.
  • Creating links - Hold Ctrl + Shift while dragging the left mouse button from one node to another to create a link between the two nodes. Make sure you don't try linking two node inputs or two node outputs together. You can also click on one of the node nubs to drag out a link without hitting Ctrl + Shift. All links start out as linear. You can change the link type by selecting the node and double-clicking on the link in the Properties section.

Right-Click & Toolbar Options

  • Add node - Adds a node to the graph. FxCombiner nodes are the most common.
  • Bezier links - Toggles if the links are drawn as straight lines or Bezier curves.
  • Node value visualization - Toggles if the nodes change colors as values flow through the face graph. The node values are also printed on the node.
  • Draw only selected node links - Only shows links for the selected nodes.
  • Animated transitions - Toggles if the face graph view instantantly zooms to nodes or if it slowly zooms out and back in.
  • Auto-find selected nodes - Toggles if changing the selection in the Face Graph Panel changes the view in the Face Graph Tab.
  • Relayout Graph - Uses AT&T's GraphViz technology to arrange the nodes in a logical fashion.
  • Reset View - Zooms out so that all nodes in the Face Graph are visible.
  • Find Selected Node(s) - Zooms to the selected nodes.
  • Create Speech Gesture Setup - FaceFX generates curves for speech gestures such as Orientation Head Pitch, Emphasis Head Pitch, Eye Pitch In most applications the curves used to rotate the head should also drive the eyes in an inverse manner so that the character remains looking straight ahead. The Create Speech Gesture Setup operation creates nodes in the Face Graph to receive the speech gesture curves and combine the curves into nodes that can drive head and eye rotations directly.

Properties Section

The Properties section of the face graph lets you modify the node properties and link functions. You can select multiple nodes and modify properties like the Min and Max values. Double click on one of the link functions displayed in the Properties Section to modify the link function. The Link Function Editor appears, and you can change the link function type or modify the parameters of the link function.

See Also

Face Graph Keyboard Shortcuts
Introducing the Face Graph


Animation Tab

Animation Tab

Animation Tab

Curve Editor

Controls

  • Select - To see a curve and its associated keys, you have to make sure the curve is selected in the Animation Panel. Click on a key, or use the left mouse button to drag out a selection of multiple keys.
  • Move - Click on a selection and drag the left mouse button to move the keys. If you hold down the Shift key when keys are selected you do not need to click on the keys prior to moving them. If you hold Ctrl+Shift when moving a key, you will constrain the movement to the horizontal or vertical axis.
  • Insert Key - Hold the insert key while left-clicking in the curve view to add keys to all of the selected curves.

Buttons (Left to right)

  • Cut, Copy, Paste - You can cut, copy, and paste keys from one curve to another using the right-click menu, the keyboard shortcuts Ctrl+X, Ctrl+C, and Ctrl+V, or the X, C, and V buttons at the top of the Animation Tab. The keys will be pasted starting at the currently selected time.
  • Lock/Unlock Tangents - select a key to display the key's tangents. Click on the blue manipulator to modify the tangent. You can lock or unlock tangents with the LT and UT (Lock Tangents, Unlock Tangents) buttons.
  • Snap to Horizontal/Vertical grid - You can turn on and off snapping to the horizontal and vertical grid lines with the SV and ST (Snap Value, Snap Time) buttons.
  • Fit Value/Time - Use these buttons to fit the selected curves to the value or time axis. The FV button fits the curves to the value axis, while FT fits the curve to the time axis.
  • Display All Active - Displays all curves with non-zero values at the current time.
  • Next Key / Previous Key - Sets the current time to the next/previous key in the timeline. Only selected curves are included.

Virtual and Offset Curves

The FaceFX Studio Curve Editor is unlike most curve editors because the animation curves are not solely responsible for driving the nodes. Nodes can also have inputs from other nodes which are added to the animation curve values. As a result, the keys in a node's curve do not determine the final value of the node. Instead, the keys create an offset curve which is added to the value calculated from the node inputs. The sum of these two curves is called the virtual curve. Virtual curves are represented by dotted lines in the curve editor and they exist only when a node has inputs from other nodes.

When a node does not have any inputs, you can not see the dotted virtual curve, only the solid line connecting the keys. When a dotted line exists, it represents how the node will be evaluated over time.

Phoneme Bar

The phoneme bar displays the phonemes detected in the audio WAV file. The timing of the phonemes determines the curves that are created to drive the lip-synchronization. Use Tools->Application Options to change the phoneme notation to the one you are most comfortable with. FaceFX supports SAMPA, IPA, and the FaceFX internal phoneme notation.

Controls

  • Click and drag on a phoneme or word boundary to adjust the boundary time.
  • Hold Ctrl while clicking and dragging a phoneme boundary to shift all subsequent phoneme boundaries by the same amount. This has the effect of lengthening or shortening the total length of the lip-synch animation.
  • Hold Shift to select a range of phonemes.
  • Double-click on a phoneme to hear the audio associated with it
  • Use the right-click menu to :
    • Delete the selection
    • Insert a phoneme
    • Group/ungroup a bunch of phonemes to a word
    • Change the selected phoneme
    • Rename the selected word
    • Reanalyze a range of phonemes (text tags will be ignored)

Audio View

The audio view shows the audio used to generate the animation. You can view the audio in spectral view (as shown in the screenshot above) or as a more traditional waveform plot. Select the Audio widget and hit Ctrl + T to switch between the plot types. The spectral view is the default because phoneme boundaries can be seen clearly with it.

See Also

Phoneme Bar Keyboard Shortcuts
Curve Editor Keyboard Shortcuts


Mapping Tab

Mapping Tab

Mapping Tab

The Mapping Tab is a spreadsheet that sets up the relationships between the phonemes FaceFX recognizes and the targets in your character. Any set of targets can be used provided that some combination of those targets can make the character look like he or she is speaking each of the phonemes. When a phoneme is detected, keys are added to the targets. Values in the spreadsheet are usually between 0 and 1 but any value can be used.

To create the default mapping for your character, click on the Make Default Mapping button at the top of the Mapping tab. Alternatively, you can create a mapping from scratch by inserting one target at a time with the Add Target To Mapping button.

The Sync Current Animation to Mapping button will add or remove curves from the currently selected animation so that the animation has only the speech target curves specified in the mapping. It will only delete a curve if it is owned by analysis.


Preview Tab

Preview Tab

Preview Tab

The Preview Tab lets you view your character while you work with FaceFX. Ideally, the Preview Tab displays the game engine being used in your game but by default it displays the FaceFX sample rendering engine.

The Preview tab has buttons on the top to activate various functions and features available in the sample rendering engine. From the left you can reset the view, toggle wireframe mode, turn on/off lighting, toggle displaying the mesh, toggle normals, toggle influences, and toggle skeleton rendering.

On the right, you can create cameras. Simply use the left, right, and middle mouse buttons to position the camera how you want it with respect to your mesh. Then click on the Create New Camera button. This will prompt you for a camera name and let you bind the camera to a bone.

Binding the camera to a bone lets you use the camera in a workspace and guarantees that the viewport will be pointed at the area of the face the sliders are working with. Not all content is compatible with the camera binding system. Specifically, the character's reference pose must not have any rotations, and the bone which the camera is attached to should not be rotated at the time the camera is bound. To find out if your Reference Pose has any rotations, you can use the developer command with the -dumprefpose flag. When creating cameras for workspaces, it is also a good idea to leave the Use Fixed Aspect Ratio checkbox checked. This only effects workspace cameras and keeps the proportions of the head constant.

See Also

Preview Tab Keyboard Shortcuts
FaceFX Sample Rendering Engine


Workspace Tab

Workspace Tab

Workspace Tab The workspace tab allows animators to create an optimized pipeline for hand animation. Several features enable this:
  • Sliders can be placed, rotated and enlarged to create a logical arrangement that makes sliders easy to find and animate.
  • 2D sliders allow the animator to control two nodes in the Face Graph simultaneously.
  • Viewports can be incorporated into the workspaces. Viewports provide instant visual feedback underneath the sliders so that no additional screen real-estate is needed for the Preview Tab.
  • Viewports use a camera to view the character. The camera can be linked to a bone. Linking a camera to a bone in the skeleton (usually the head bone) ensures that the viewport is always pointed at the region of the face that is being animated.

The Workspace Tab can be in one of two modes: Edit Mode or Normal mode. Edit mode is used to create the workspace while Normal mode is used to animate the sliders. The button that looks like a pencil is responsible for switching between the two modes.

Edit Mode

Before using the Workspace Tab, a workspace needs to be created. The first task is therefore to click on the icon that looks like a pencil. This puts the Workspace Tab into "Edit mode". Here you can add sliders, viewports, and background images to create an optimized workspace for hand-animation.

Controls and Buttons (Left to right)

  • Workspace Selection Dropdown - Selects the workspace you want to edit. If you haven't saved a workspace, the default "Workspace 1" is shown.
  • Exit Edit Mode - Puts the currently selected workspace into Normal mode so you can work with the created sliders.
  • New Workspace - Creates a blank canvas for you to work on.
  • Workspace Creation Wizard - If you just want to arrange your sliders in a horizontal or vertical row, use the Workspace Creation Wizard.
  • Save Workspace - Saves the current workspace
  • Rename Workspace - Renames the current workspace.
  • Delete Workspace - Deletes the current Workspace.
  • Add Background Bitmap- This button allows you to add a background bitmap to your workspaces. Background bitmaps can be useful for helping you organize sliders.
  • Add Viewport - Viewports allow you to see your character animate underneath your sliders. You can use the Default camera or create a new camera from the Preview tab. Cameras can be bound to the head bone or other bones in the skeleton so they are always pointing at what you want to animate.
  • Delete Workspace Element - Deletes the currently selected Workspace element
  • Change the Workspace Text Color - This button is useful when your viewport or background image prevents the workspace text from being visible.
  • Set workspace Tab order - You can set the tab order of the sliders by clicking this button. Once the tab order is set, you can change the currently selected slider by hitting the tab button and use the keyboard shortcuts to animate with sliders without a mouse.

Working with Sliders

To create a slider, drag your mouse from one of the nodes listed on the right to one of the two white boxes below labeled "Drop x-axis" or "Drop y-axis". This creates a control below labeled "Drag To Create". Use the left mouse button to drag the "Drag To Create" box onto the workspace area. This will create a slider in the workspace.

To create a 2D slider, simply drag two different nodes into the "Drop x-axis" and "Drop y-axis" boxes before dragging the slider onto the workspace.

Once you have a slider in the workspace, you can change its position, rotation, and scale with the green control dots. All workspace elements (sliders, viewports, and background images) have green control dots that let you position and scale the elements. Only sliders have a rotate control.

If you want to create a bunch of sliders that are lined up perfectly, you may want to try using the workspace wizard. Alternatively, try holding down Ctrl when moving sliders around to restrict the movement to a grid.

Normal Mode

Once a workspace is created in Edit mode, you can use Normal mode to animate the sliders by left-clicking in the slider box and dragging your mouse around. Use the right-mouse button to set the slider's value to the default value set by the node. When in animate mode, you can use the middle mouse button to set a key at the current slider value.

From Normal Mode, the controls at the top of the tab have the following functionality (from left to right):
  1. Select workspace drop-down
  2. Enter Edit-mode button
  3. Select mode drop-down
  4. Auto-select curve for selected slider button
  5. Toggle slider label button
  6. Custom controls depending on mode (see below)
  7. Create workspace instance button (on far right)
If you want to have multiple workspaces visible at once, you can flick on the button at the far right to "peel off" an instance of the workspace.

When a work space is in Normal mode it is also in one of three sub-modes that determines what the sliders are being used for. The three modes are listed below:

Create Combiner

Create Combiner mode allows animators to combine multiple targets into a single combiner node by ramping up sliders. Essentially, it is a visual interface to the Face Graph.

There are three buttons available in Create Combiner mode:
  • Reset all Sliders - This button clears the Face Graph of all values.
  • Load Current Frame - If you want to create a combiner node using a frame of an animation, you need to first load the current frame into the sliders. Scrub to the frame in your animation you want to use and click this button. Any sliders present in your workspace will have their values set by the current frame.
  • Create Combiner Node From Sliders - This button creates a new combiner node that drives all of the sliders in your workspace to their current values. Driving the new node has the same effect as driving all of the individual sliders.

Animate

Animate mode lets you set keys in the current animation using the sliders in your workspace. Use the middle-mouse button to set a key for the slider. The value lines inside the slider are color coded when in Animate mode:

Green: The slider's value is in sync with the current animation. Adding a key at this location will therefore not change the value of the curve at the current frame.
Yellow: The slider has been modified so it is not in sync with the animation. Adding a key will change the value of the curve at the current frame.
Red: The slider's curve has a key at the current time. Adding a new key here will overwrite the existing key because two keys cannot exist at the same time.

There are four buttons available in Animate mode:
  • Auto Key - Turns Autokey on. Any changes to sliders will automatically set keys.
  • Key All - Keys all sliders in the current workspace.
  • Previous Key - Set the current time equal to the previous key's time value. Only curves belonging to the selected slider will be considered. If no sliders are selected, all sliders in the current workspace will be considered.
  • Next Key- Set the current time equal to the next key's time value. Only curves belonging to the selected slider will be considered. If no sliders are selected, all sliders in the current workspace will be considered.

Mapping Mode

Mapping Mode is used as a visual alternative to the Mapping Tab's spreadsheet format. Both systems allow you map the 42 phonemes FaceFX recognizes to your character's targets. To use Mapping Mode, create a workspace with all of the sliders you want to use in your mapping. Select the phoneme you want to map to at the top. Change the values in the sliders until the character looks like he is saying the phoneme. Then click on the Apply to Mapping button.


Console Tab

Console Tab

Console Tab

The Console Tab writes out the log file as formatted HTML. Warnings are printed in yellow. Errors are printed in red. Comments are in green. The rest of the log file consists of FaceFX commands which can be used to create FaceFX batch files.

A maximum of 250 lines of output are viewable from the Console Tab before the results are flushed. A full record of the session can be found in the FaceFxStudioLog.html file. The maximum number of lines can be changed using the Get and Set Commands but to prevent system instability a reasonable maximum needs to be in place.

Copying and Pasting

Copying and pasting commands from the log file can be useful for creating FXL files quickly. Getting the Copy to work correctly can be tricky however because of the way wxWidgets works. Make sure the mouse is over the window when trying to copy. Also make certain you hold the Ctrl key for the entire duration that the C key is down.

See Also

Command Interface


Scrub Bar, Timeline, Command Line

Scrub Bar, Timeline, Command Line

Scrub Bar, Timeline, Command Line

At the bottom of all tabs, you can see the following controls from top to bottom: the scrub bar, the timeline and the command line. All of these controls can be minimized with the controls on the left.

Scrub Bar

The Scrub Bar is used to control and display the portion of the animation that is viewable in the Animation Tab and the timeline. The scrub bar itself is located in the middle of the control and can be panned within the boundaries of the timeline's min and max values. The Timeline's min and max values can be changed with the outer edit boxes, while the inner edit boxes provide a numerical input device for the scrub bar's left and right edge.

Timeline

The timeline controls the position of the play cursor which in turn determines the audio that is heard and the frame that is displayed in the Preview Tab. The timeline only displays values inside the current position of the Scrub Bar. You can use the buttons in the timeline to loop the animation, reset the play cursor, and play the animation. Be aware that you can play animations at different speeds with the play button. The edit box provides a numerical input device for the exact position of the play cursor.

Command Line

The Command Line allows you to execute FaceFX commands and see the most recent output of the commands. Look at the Console Tab for a complete list of FaceFX commands. Read the Command Interface chapter for more information on FaceFX commands.


Keyboard Shortcuts

Keyboard Shortcuts

Keyboard Shortcuts Application Keyboard Shortcuts
Preview Tab Keyboard Shortcuts
Phoneme Bar Keyboard Shortcuts
Curve Editor Keyboard Shortcuts
Face Graph Keyboard Shortcuts
Workspace Keyboard Shortcuts


Application Keyboard Shortcuts

Application Keyboard Shortcuts

Application Keyboard Shortcuts
Ctrl + X | Cut
Ctrl + C | Copy
Ctrl + V | Paste
Ctrl + O | Open Actor
Ctrl + N | New Actor
Ctrl + Z | Undo
Ctrl + Z | Redo
Ctrl + Shift + Z | Redo
Ctrl + S | Save
Ctrl + R | Reset View
F5 | Previous Frame
F6 | Next Frame
F10 | Tear-off Tab
F11 | Full Screen
F12 | Always on Top


Preview Tab Keyboard Shortcuts

Preview Tab Keyboard Shortcuts

Preview Tab Keyboard Shortcuts
LMB Drag | Rotate
MMB Drag | Pan
MMB Scroll | Zoom
Ctrl + LMB | Smooth Zoom
W | Wireframe
I | Influences
N | Normals
S+ Z | Skeleton
L | Lighting
M | Mesh
F10 | Tear-off Tab
F11 | Full Screen
F12 | Always on Top


Phoneme Bar Keyboard Shortcuts

Phoneme Bar Keyboard Shortcuts

Phoneme Bar Keyboard Shortcuts
LMB Click | Select
LMB Dbl.Click | Play Selected
Ctrl + LMB Drag | Shift Phoneme Boundary All
RMB + Click | Phoneme Options
Shift + LMB Click | Select Multiple
Shift + LMB Dbl. Click | Play Multiple
Ctrl + W | Zoom to selection
Ctrl + Q | Re-analyze range
F10 | Tear-off Tab
F11 | Full Screen
F12 | Always on Top


Curve Editor Keyboard Shortcuts

Curve Editor Keyboard Shortcuts

Curve Editor Keyboard Shortcuts
LMB Click | Select
LMB Drag | Marquee Select or Move
MMB Drag | Pan Time
MMB Scroll | Zoom Time
Shift + MMB Scroll | Zoom Value
Shift + LMB Drag | No-Hit-Test Move Selected
Ctrl + LMB Drag | Toggle Select
Ins + LMB Click | Add key
Ctrl + Shift + LMB Drag | Horizontal/Vertical move key
Ctrl + Shift +MMB Drag | Zoom to selection
Ctrl + R | Auto Resize
F10 | Tear-off Tab
F11 | Full Screen
F12 | Always on Top


Face Graph Keyboard Shortcuts

Face Graph Keyboard Shortcuts

Face Graph Keyboard Shortcuts
LMB Click | Select
LMB Drag | Marquee Select or Move
MMB Drag | Pan
MMB Scroll | Zoom
Shift + LMB Drag | No-Hit-Test Move Selected
Ctrl + Shift + LMB Drag from nub | Quick Link Nodes
Ctrl + Shift + LMB Drag | Zoom to Selection
Ctrl + G | Relayout Graph
Ctrl + B | Bézier Links
Ctrl + D | Node Value Visualization
Ctrl + L | Draw only selected node links
F10 | Tear-off Tab
F11 | Full Screen
F12 | Always on Top


Workspace Keyboard Shortcuts

Workspace Keyboard Shortcuts

Workspace Keyboard Shortcuts
TAB | Select next slider in the tab order
Shift + TAB | Select previous slider in the tab order
Arrow Keys | Control the value of the selected slider
Shift + Arrow Keys | Large changes to the value of the selected slider
Ctrl + Arrow Keys | Set current slider to the slider's edge in the direction of the arrow (min or max for the node).
Space | Reset current slider value
Enter | Set key for current slider.
[ | Previous key (only in main workspace widget)
] | Next key (only in main workspace widget)

Note: To type a numerical value for the selected slider, just start typing. By default, the value will go into the X-axis of a 2D slider. To enter a value into the Y-axis, simply type "y" then begin typing the numerical value.


External Tools and Plug-ins

External Tools and Plug-ins

FXA Exporter Plug-ins

Introduction
Max FXA Exporter
Maya FXA Exporter
MAXScript and MELScript functions

FXR Exporter Plug-ins

Max FXR Exporter
Maya FXR Exporter

External Utilities

FxCollada


FXA Exporters

FXA Exporters

FXA Exporter Plug-ins The FXA plug-ins are Max and Maya utilities that can create, modify, and get data out of FaceFX Actor Files (*.fxa), which contain everything needed by a game or 3D application to play bones or morph-based animations. Actor files are used to store the reference pose, bone poses, animation data, and the face graph. The reference pose and bone poses are created with Max or Maya, but the face graph and animation data are created inside of FaceFX Studio. The FXA plug-ins can also be used to get animation data created in FaceFX Studio into Max or Maya.

Batch Import and Export Functionality

Both the Max and Maya FXA plug-ins support "Batch Import" and "Batch Export" functionality. This feature reads a text file to get a list of pose names and frames where those poses can be found. The text file has the following format:
// Comment lines are ignored
// the format is:
// pose_name  frame number
// Make sure there is only one space.
// The semi-colon is optional.
// For pose names with spaces, make sure the name is enclosed in quotes.
// For pose names without spaces, the quotes are optional.
Eat 10;
Earth 20;
If 30;
Ox 40;
"Oat" 50;
"Pose with spaces in name" 60;
// And so on...

Baked Curves vs. Raw Curves

The plug-ins can return key information for curves (although not in mod-developer versions of the plug-ins). Curves can be raw or baked. Raw curves return the exact key values, times, and slopes. Raw curves may not accurately reflect the node's value over time because they do not take into account the node's minimum or maximum values or any input values from other nodes. Baked curves are an approximation of the node's value over time.

See Also

Native coordinates vs. FaceFX coordinates
Max FXA Exporter
Maya FXA Exporter
FxCollada


Max FXA Exporter

Max FXA Exporter

Max FXA Exporter To launch the FaceFX plug-in from 3D Studio Max, go to the Utilities Panel on the right, click on the "More" button, and select FaceFX from the list.

FaceFX Panel

This panel lets you create, load and save actor (*.fxa) files.

The Normalize Scale checkbox influences how the reference pose and bone poses are exported. Read below for information regarding how to use the Normalize Scale option.

The Reference Pose Panel

The Reference Pose Panel is used to import and export the reference pose. You should include all bones that you want FaceFX to control. For best results, don't include end-effectors or nubs in the reference pose. Only include objects that are actually weighted to the mesh or have children weighted to the mesh. The bones in the reference pose should must have unique names in the scene.

After export, the bones contained in the reference pose are listed, and you are now able to create bone poses from the Bone Pose Panel. If you change your reference pose at any point, you should also re-export all of your bone poses because different bones may be pruned out of the poses now that the reference pose is different.

The Bone Pose Panel

The Bone Pose Panel is used to import and export bone poses. The list of poses in the actor is displayed. Simply go to the frame you want to export or import and click on the appropriate button. The Batch Import and Batch Export buttons will ask for a text file. See the beginning of this chapter for a description of the expected format.

Bone poses are stored based upon how they differ from the reference pose. For this reason, if you change your reference pose, you have to re-export all of your bone poses.

Animations

This window lists the animations in the actor. You can switch between the various animation groups in the actor from the combo box. To import an animation onto your bones or morph-based character, just select the animation and click on the Import button.

Using Scale in Max

3D Studio Max uses scale differently than the FaceFX Sample Rendering Engine. For example, FaceFX does not support non-uniform scale which is commonly used in Max to modify the position of bones in the skeleton. Even uniform scale can make Max content appear distorted in FaceFX. To resolve this issue, the Max plug-in includes the Normalize Scale option. Checking this option affects how the reference pose and bone poses are exported from the plug-in. When checked, the reference pose and bone poses are exported with the scale baked into the position and rotation of the bones. The scale is then set at [1, 1, 1].

If your Max rig uses scale to position bones correctly in the reference pose, you may need to export your reference pose and bone poses with this option checked. This will allow you to preview your content in FaceFX while generating animations and tweaking them. To view those animations in Max, you will have to load the FXA file into the plug-in, and overwrite the reference pose and bone poses with the Normalize Scale button unchecked. This will not affect your animation data or face graph set-up, but it will allow the FaceFX Max plug-in to insert keys that are compatible with Max's use of scale.

Changing key controllers in Max

When importing an animation, a bone pose, or the reference pose, FaceFX attempts to convert all bone position, rotation, and scale controllers to Linear. In some cases this automatic process fails (when Euler controllers are used for example). You can change controllers manually by selecting the bones you want FaceFX to control and typing the following three max-script commands:
for obj in selection do obj.position.controller = Linear_Position()
for obj in selection do obj.rotation.controller = Linear_Rotation()
for obj in selection do obj.scale.controller = Linear_Scale()

Logging information

The Max FXA plug-in outputs debug information to the FaceFX_Log.txt file. In Max versions before Max 9, this is located in the Max installation folder. In Max 9 and beyond, this is located in the APP_MAXDATA_DIR directory, which looks something like the following:

C:\Documents and Settings\SomeUser\Local Settings\Application Data\Autodesk\3dsmax\9 - 32bit\enu

Check the log file to view details on the number of bones pruned out of each pose. The FXR exporter outputs debug information to the FxMaxExporter_Log.txt file.


Maya FXA Exporter

Maya FXA Exporter

Maya FXA Exporter To view the FXA Exporter panel, first install the plug-ins with the FaceFX installer. Then from Maya 6, 6.5, or 7 go to Window->Settings/Preferences->Plug-in Manager… and check the loaded and auto load checkboxes next to FaceFX.mll. Then open the FaceFX.mel script in the Maya Script Editor and selecting Script->Execute. Once opened, the FaceFX Panel can be used like any other panel in Maya and incorporated into saved layouts.

From the File menu, you can load an existing actor file (*.FXA), save the current actor file, or create a new actor file. The Panel menu duplicates the functionality that Maya users are familiar with.

The FaceFX Panel consists of three collapsible windows: Reference Pose, Bone Poses, and Animations.

Reference Pose

Lists the bones in the reference pose of the currently loaded actor. You can Import or Export the reference pose to/from the current frame. The bones in the reference pose should must have unique names in the scene.

Bone Poses

Lists the bone poses in the currently loaded actor. The Import and Export buttons facilitate importing and exporting a bone pose to/from the current frame. You can also perform Batch operations using the text file format described at the beginning of the chapter or with a MEL script. Bone poses are stored based upon how they differ from the reference pose. For this reason, if you change your reference pose, you have to re-export all of your bone poses.

Animations

This window lists the animations in the actor. You can switch between the various animation groups in the actor from the combo box. To import an animation onto your bones or morph-based character, just select the animation and click on the Import button.

Logging information

The Maya FXA plug-in outputs debug information to the FaceFX_Log.txt file in the MAYA_APP_DIR directory. Check here to view details on the number of bones pruned out of each pose. The FXR exporter outputs debug information to the FxMayaExporter_Log.txt file.


MAXScript and MELScript Functions

MAXScript and MELScript Functions

MAXScript And MELScript

MAXScript

To use the MAXScript commands, you need to create a FaceFX object. This can be done as follows from a Max Script window:
obj = FaceFX();

Once created, all of the MAXScript functions described can be called by referencing the FaceFX object.

MEL Script

To use the MEL commands, simply make sure the FaceFX.mll plug-in is loaded and call the functions as described below.


fxgetactorname

fxgetactorname

fxgetactorname Returns the actor name

Example

Maya | Max
fxgetactorname; | obj.fxgetactorname();


fxcreateactor

fxcreateactor

fxcreateactor Creates an actor. This is the first step to getting your character talking in FaceFX. The name is not the name of the file, but rather the name that is used internally by FaceFX.

Example

Maya | Max
fxcreateactor -name "Slade"; | obj.fxcreateactor "Slade";


fxloadactor

fxloadactor

fxloadactor Loads an existing actor file (*.fxa) from disk. Only one actor can be loaded at a time.

Example

Maya | Max
fxloadactor -file "C:\\Slade.fxa"; | obj.fxloadactor "C:\Slade.fxa";
  • Notice that the when specifying a file from MEL Script, you must use two backslash characters as the folder deliminator. This is optional in 3D Studio Max.


fxsaveactor

fxsaveactor

fxsaveactor Saves the current actor file to disk.

Example

Maya | Max
fxsaveactor -file "C:\\Slade.fxa"; | obj.fxsaveactor "C:\Slade.fxa";
  • Notice that the when specifying a file from MEL Script, you must use two backslash characters as the folder deliminator. This is optional in 3D Studio Max.


fximportrefpose

fximportrefpose

fximportrefpose Imports the reference pose to the specified frame.

Example

Maya | Max
fximportrefpose -frame 0; | obj.fximportrefpose 0;


fxexportrefpose

fxexportrefpose

fxexportrefpose Exports the reference pose for the actor. The bones contained in the reference pose are specified along with the frame to export from. The positions, rotations, and scales of these bones are recorded at the specified frame and used to create the reference pose. If the actor already has a reference pose, the existing pose is overwritten.

The reference pose is important for the following reasons:
  • To save memory, FaceFX only stores the difference between a bone pose and the reference pose. Any bones that do not move in the pose are pruned. For this reason, you may want to re-export your existing bone poses if you change the reference pose.
  • The reference pose determines what bones are included in the FaceFX system. If the bone isn't in the reference pose, FaceFX will not influence it.

Example

Maya | Max
fxexportrefpose -frame 0 -bones "BN_Root BN_Head"; | obj.fxexportrefpose 0 #("BN_Root", "BN_Head");


fximportbonepose

fximportbonepose

fximportbonepose Imports the specified bone pose to the specified frame, inserting keys for all bones in the reference pose at that frame.

Example

Maya | Max
fximportbonepose -frame 10 -name "Eat"; | obj.fximportbonepose 10 "Eat";


fxexportbonepose

fxexportbonepose

fxexportbonepose Exports the specified bone pose from the specified frame. Any bones that have identical position, rotation, and scale values with respect to the reference pose are pruned out from the exported pose.

Example

Maya | Max
fxexportbonepose -frame 10 -name "Eat"; | obj.fxexportbonepose 10 "Eat";


fxbatchimportboneposes

fxbatchimportboneposes

fxbatchimportboneposes Imports all of the poses in the batch export text file from the FXA file into Max or Maya.

Example

Maya | Max
fxbatchimportboneposes -file "C:\\batch-file.txt"; | obj.fxbatchimportboneposes "C:\batch-file.txt";
  • Notice that the when specifying a file from MEL Script, you must use two backslash characters as the folder deliminator. This is optional in 3D Studio Max.


fxbatchexportboneposes

fxbatchexportboneposes

fxbatchexportboneposes Exports all of the poses in the batch export text file to the actor.

Example

Maya | Max
fxbatchexportboneposes -file "C:\\batch-file.txt"; | obj.fxbatchexportboneposes "C:\batch-file.txt";
  • Notice that the when specifying a file from MEL Script, you must use two backslash characters as the folder deliminator. This is optional in 3D Studio Max.


fximportanim

fximportanim

fximportanim Imports the specified animation into the timeline starting at frame 0. All bones in the reference pose will get position, rotation, and scale keys at every frame (similar to raw motion capture data). Any morph targets in the scene that that have the same name as a node in the face graph will be driven by a curve that represents that node's value over time.

Example

Maya | Max
fximportanim -group "Default" -anim "welcome"; | obj.fximportanim "Default" "welcome";
  • This function is turned off in Mod developer versions of FaceFX.


fxclean

fxclean

fxclean Imports the reference pose to frame 0 and deletes all other animation data on the bones specified in the Reference Pose. This function also sets a key with time=0 and value=0 for all morph targets that have the same name as a node in the face graph.

Example

Maya | Max
fxclean; | obj.fxclean();


fxgetbones

fxgetbones

fxgetbones Returns an array of strings representing the names of all the bones contained in a given pose. If a bone is present in a pose, it means that there is a difference between the bone's translation in the reference pose and the bone's translation in the specified pose.

Example

Maya | Max
fxgetbones -name "Eat"; | obj.fxgetbones "Eat";


fxgetanimgroups

fxgetanimgroups

fxgetanimgroups Returns an array of strings representing the animation groups present in the actor.

Example

Maya | Max
fxgetanimgroups; | obj.fxgetanimgroups();


fxgetanims

fxgetanims

fxgetanims Returns an array of strings representing the animations present in the specified animation group.

Example

Maya | Max
fxgetanims -group "Default"; | obj.fxgetanims "Default";


fxgetcurves

fxgetcurves

fxgetcurves Returns an array of strings representing the curves present in the specified animation.

Example

Maya | Max
fxgetcurves -group "Default" -anim "welcome"; | obj.fxgetcurves "Default" "welcome";


fxgetnodes

fxgetnodes

fxgetnodes Returns an array of strings representing the names of all the nodes of the specified type in the actor.

Example

Maya | Max
fxgetnodes -type "FxBonePoseNode"; | obj.fxgetnodes "FxBonePoseNode";


fxgetrefbones

fxgetrefbones

fxgetrefbones Returns an array of strings representing the names of all bones in the reference pose in the current actor.

Example

Maya | Max
fxgetrefbones; | obj.fxgetrefbones();


fxsetnormalizescale

fxsetnormalizescale

fxsetnormalizescale Sets the normalize scale option. When Normalize scale is on, all scale values will be neutralized when exporting the reference pose or bone poses. To turn it on, set the toggle flag to "on". To turn it off, set the toggle flag to anything else.

Example

Maya | Max
N/A | obj.fxsetnormalizescale "on";


fxsetdisplaywarningdialogs

fxsetdisplaywarningdialogs

fxsetdisplaywarningdialogs Toggles if warning dialog boxes are displayed. If you want to perform an operation from script, this function can be useful to make sure that the script does not require user intervention. Warning dialog boxes are on by default because they alert the user of common problems and mistakes. To turn it on, set the toggle flag to "on". To turn it off, set the toggle flag to anything else.

Example

Maya | Max
fxsetdisplaywarningdialogs -toggle "on"; | obj.fxsetdisplaywarningdialogs "on";


fxgetanimduration

fxgetanimduration

fxgetanimduration Returns the duration in seconds of the specified animation in the specified group.

Example

Maya | Max
fxgetanimduration -group "Default" -anim "welcome"; | obj.fxgetanimduration "Default" "welcome";


fxgetbakedcurvekeytimes

fxgetbakedcurvekeytimes

fxgetbakedcurvekeytimes Returns an array of key times (in seconds) for the specified node in the specified group and animation. The returned curve is baked. The function returns an empty array if the node or animation can't be found.

Example

Maya | Max
fxgetbakedcurvekeytimes -group "Default" -anim "welcome" -curve "Eat"; | obj.fxgetbakedcurvekeytimes "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxgetbakedcurvekeyvalues

fxgetbakedcurvekeyvalues

fxgetbakedcurvekeyvalues Returns an array of key values for the specified node in the specified group and animation. The returned curve is baked. The function returns an empty array if the node or animation can't be found.

Example

Maya | Max
fxgetbakedcurvekeyvalues -group "Default" -anim "welcome" -curve "Eat"; | obj.fxgetbakedcurvekeyvalues "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxgetbakedcurvekeyslopein

fxgetbakedcurvekeyslopein

fxgetbakedcurvekeyslopein Returns an array of input slopes for the specified curve in the specified group and animation. The returned curve is baked. The function returns an empty array if the curve can't be found.

Example

Maya | Max
fxgetbakedcurvekeyslopein -group "Default" -anim "welcome" -curve "Eat"; | obj.fxgetbakedcurvekeyslopein "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxgetbakedcurvekeyslopeout

fxgetbakedcurvekeyslopeout

fxgetbakedcurvekeyslopeout Returns an array of output slopes for the specified curve in the specified group and animation. The returned curve is baked. The function returns an empty array if the curve can't be found.

Example

Maya | Max
fxgetbakedcurvekeyslopeout -group "Default" -anim "welcome" -curve "Eat"; | obj.fxgetbakedcurvekeyslopeout "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxgetrawcurvekeyslopeout

fxgetrawcurvekeyslopeout

fxgetrawcurvekeyslopeout Returns an array of output slopes for the specified curve in the specified group and animation. The returned curve is raw. The function returns an empty array if the curve can't be found.

Example

Maya | Max
fxgetrawcurvekeyslopeout -group "Default" -anim "welcome" -curve "Eat"; | obj.fxgetrawcurvekeyslopeout "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxgetrawcurvekeyslopein

fxgetrawcurvekeyslopein

fxgetrawcurvekeyslopein Returns an array of input slopes for the specified curve in the specified group and animation. The returned curve is raw. The function returns an empty array if the curve can't be found.

Example

Maya | Max
fxgetrawcurvekeyslopein -group "Default" -anim "welcome" -curve "Eat"; | obj.fxgetrawcurvekeyslopein "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxgetrawcurvekeyvalues

fxgetrawcurvekeyvalues

fxgetrawcurvekeyvalues Returns an array of key values for the specified curve in the specified group and animation. The returned curve is raw. The function returns an empty array if the curve can't be found.

Example

Maya | Max
fxgetrawcurvekeyvalues -group "Default" -anim "welcome" -curve "Eat" | obj.fxgetrawcurvekeyvalues "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxgetrawcurvekeytimes

fxgetrawcurvekeytimes

fxgetrawcurvekeytimes Returns an array of key times for the specified curve in the specified group and animation. The returned curve is raw. The function returns an empty array if the curve can't be found.

Example

Maya | Max
fxgetrawcurvekeytimes -group "Default" -anim "welcome" -curve "Eat" | obj.fxgetrawcurvekeytimes "Default" "welcome" "Eat";
  • This function is turned off in Mod developer versions of FaceFX.


fxoutputwindowlog

fxoutputwindowlog

fxoutputwindowlog Toggles if FaceFX logs to the Output Window. To turn it on, set the toggle flag to "on". To turn it off, set the toggle flag to anything else. Output always goes to the FaceFX_Log.txt file in the main Maya install directory.

Example

Maya | Max
fxoutputwindowlog -toggle "on"; | N/A


fxinsertkey

fxinsertkey

fxinsertkey Inserts a key into the specified curve at the specified frame. The function will create the curve if it doesn't exist, but the group and animation must be present or the function will fail. On success, the index of inserted key is returned. On failure, -1 is returned.

Please be aware that any changes made to curves that are owned by analysis will be lost if the FXA is loaded back up into FaceFX Studio.

Example

Maya | Max
fxinsertkey -group "Default" -anim "welcome" -curve "Eat" -frame 10 -value 0.82 -inslope 0.0 -outslope 0.0; | obj.fxinsertkey "Default" "welcome" "Eat" 10 0.82 0.0 0.0;


fxdeleteallkeys

fxdeleteallkeys

fxdeleteallkeys Deletes all keys in the specified curve.

Please be aware that any changes made to curves that are owned by analysis will be lost if the FXA is loaded back up into FaceFX Studio.

Example

Maya | Max
fxdeleteallkeys -group "Default" -anim "welcome" -curve "Eat"; | obj.fxdeleteallkeys "Default" "welcome" "Eat";


fxisnosave

fxisnosave

fxisnosave Returns if the plugin has been built with saving capabilities turned off. No-save versions of FaceFX Studio and the tools allow OC3 Entertainment to distribute free trials of the software.

Example

Maya | Max
fxisnosave; | obj.fxisnosave;


fxismoddeveloper

fxismoddeveloper

fxismoddeveloper Returns if the plugin has been built with for mod developers. Mod developer plugins can not import animation data using the tools. They can only be used to define now bone poses.

Example

Maya | Max
fxismoddeveloper; | obj.fxismoddeveloper;


FXR Exporters

FXR Exporters

FXR Exporter Plug-ins Max FXR Exporter
Maya FXR Exporter
FxCollada


Max FXR Exporter

Max FXR Exporter

Max FXR Exporter

Export Process

  1. Install the FXR plug-ins by running the FaceFX Studio installer. Install the plugins for 32-bit Max version 8, 9, 2008, or 2009. 64-bit FXR exporters are not supported. Try using FxCollada or consider upgrading to newer FaceFX versions that use Ogre.
  2. Open the file you want to export and go to File->Export
  3. Change the Save as type to FaceFX Studio (*.FXR)
  4. Browse to the directory where your actor's FXA file is located.
  5. Type a name for your FXR file. Make sure it has the same name as the FXA file so that FaceFX Studio will find it when loading the actor.
  6. Click on the Save button

Content Guidelines

  • All meshes you want exported should be triangulated
  • The vertices should be weighted to the bones.
  • The skeleton's bind pose should be located at frame 0 of the animation.
  • Make sure the texture file referenced in the Material Editor is present on your system. If the full path to the file is not included, you have to export the FXR file to the same directory as the textures in order for the exporter to find the textures correctly.

Logging information

If you experience any problems exporting an FXR file, the first place to look is the FxMaxExporter_Log.txt file in the APP_MAXDATA_DIR directory.

See Also

Preview Tab
Sample Rendering Engine


Maya FXR Exporter

Maya FXR Exporter

Maya FXR Exporter

Export Process

  1. Install the FXR plug-in by running the FaceFX Studio installer. Install into 32-bit Maya version 8, 8.5, 2008, or 2009. 64-bit FXR exporters are not supported. Try using FxCollada or consider upgrading to newer FaceFX versions that use Ogre.
  2. Go to Window->Settings/Preferences->Plug-in Manager and check the loaded and auto load checkboxes next to FxMayaExporter.mll
  3. Open the file you want to export in Maya.
  4. Turn off any custom shaders or custom plugins as these can cause a crash when opening the FXR file in FaceFX Studio.
  5. Go to File->Export All
  6. Change the Save as type to FaceFX Rendering Archive (*.FXR)
  7. Browse to the directory where your actor's FXA file is located.
  8. Type a name for your FXR file. Make sure it has the same name as the FXA file so that FaceFX Studio will find it when loading the actor.
  9. Click on the Export button.
  10. Close the Maya file without saving. The FXR export process puts the skelton into the bind pose for export and does not return the skeleton to its original state. Closing the file prevents this change from being saved. Note that if the skeleton deforms incorrectly during export, the bind pose will be ecxported incorrectly. Make sure bone attributes are not locked, IK is turned off with Modify -> Evaluate Nodes ->Ignore All, and there is only one bind pose.

Content Guidelines

  • All meshes you want exported should be triangulated
  • All the meshes you want to export should be visible
  • The vertices should be weighted to the bones.
  • The history should be deleted immediately prior to weighting the vertices to the bones. This is not always required, but it is recommended.
  • The bind pose should not contain any bones that are scaled.
  • If you have IK or other constraints in your file, you may have to turn them off with the Modify -> Evaluate Nodes ->Ignore All option. Make sure no errors occur after performing this operation.
  • If you have any locked channels that will prevent your skeleton from going into the bind pose, you will have to unlock the channels before exporting the FXR file. The FXR exporter will put the skeleton into the bind pose upon export.
  • If you have mirrored any meshes, the FaceFX exporter may invert the normals on the mirrored mesh. To get the content displaying correctly in FaceFX Studio you may have to invert the normals on the mirrored mesh (so that they are reversed in Maya).
  • If you are using blendshapes, the blendshape meshes must not be deleted even though Maya allows you to delete the meshes and still use the blendshapes.
  • Make sure there is only one bind pose for your skeleton.
  • Make sure you have turned off and unloaded any custom shaders prior to export as these can cause a crash when opening the FXR file in FaceFX Studio.

Logging information

If you experience any problems exporting an FXR file, the first place to look is the FxMayaExporter_Log.txt file located in the MAYA_APP_DIR directory.

See Also

Preview Tab
Sample Rendering Engine


FxCollada

FxCollada

FxCollada FxCollada can export both FXA and FXR files using a COLLADA file generated from any 3D application that supports the COLLADA format. FxCollada is in Beta because the COLLADA format is still in its infancy and many kinks are being worked out of the importers and exporters availabale for this format.

The left side of the FxCollada application mirrors the functionality of the Max FXA exporter and Maya FXA exporter. The right side of the application contains a FaceFX Preview Window to display content.

Tutorial

  1. Go to File-> Open DAE and select a COLLADA file (*.DAE) to open.
  2. The buttons at the top of the Preview Window should be enabled. If you can't see your content, try clicking on the wireframe and other buttons to diagnose the problem.
  3. Go to Options->FXR options to hide any meshes that are in the way.
  4. Use the scrub bar at the bottom of the application to scrub to the frame that contains your Reference Pose.
  5. Click on the Export button in the Reference Pose section on the left. Select the bones you want FaceFX to control.
  6. Use the scrub bar to go to a frame that contains a bone pose you want to export. Click on the Export button in the Bone Poses section to export the pose. Give the pose a name using the input dialog.
  7. If you have a lot of poses to export, consider creating a batch export script like the one described for the FXA Exporters. Then you can use the Batch Imp. and Batch Exp. buttons to rapidly import/export bone poses.

Troubleshooting

  1. If your content does not show up correctly, the first step is to try to import the COLLADA file back into the 3D application that exported it. If the COLLADA file doesn't show up correctly there, try adjusting some options until you can get a COLLADA file that can be exported and imported by the same application.
  2. To find out if the problem is with FxCollada itself, it is useful to try to import the file with the Max or Maya importers from Feeling Software. These importers also use the FCollada library so if the probem is not reproducible in Max or Maya then it is more likeley an issue with FxCollada.
  3. Check the FaceFX_Log.txt file that resides in the same directory as the FxCollada.exe file. Look for warnings or information that can help you find the problem.
  4. Go to Options->Clean Animation Timeline to remove all animation data. This will display the content in the bind pose which can be useful for diagnosing the problem.
  5. If it is possible to send along a COLLADA file that can reproduce the problem, please send it along to support@oc3ent.com


Character Setup

Character Setup

Character Setup Introduction FaceFX lets you set up your characters any way you want. You determine what targets you want to create. Make sure your targets can do the following:
  1. Move the mouth into all of the shapes required for speech.
  2. Blink the eyelids
  3. Raise the eyebrows
  4. Rotate the head (pitch, roll, and yaw).

Default Mapping

To speed up the character setup process, FaceFX provides a Default Mapping which is applied from the Mapping Tab. It is only a default, and can be replaced with your own mapping.

Gesture Targets

In addition to lip-synchronization curves, FaceFX generates gesture curves. These include curves for blinks, eyebrow raises , and head and eye rotations. You can learn more about the gestures FaceFX creates here, or learn how to set up your Face Graph to handle these curves in the Setting Up Gestures tutorial.

The Meaning of the Arrows

The pictures of the targets linked to above have arrows describing how the mesh moves for each of the targets.

An orange arrow signifies how a bone should move with respect to its parent. The lower lip bones are parented to the jaw bone so the arrow might be pointed up even if the bone moves down with respect to the rest of the face.

The yellow arrow signifies that you may have to rotate some of the bones to achieve the desired effect in the pose.

See Also

Setting Up Gestures
Speech Gestures


Neutral

Neutral

Neutral
Reference Pose - Neutral - Silent

The reference pose is the relaxed and neutral position of the face, and the reference for all morph targets or bone poses. The Neutral pose should be expressionless with at most a slight upturning of the lips to give the character a pleasant expression. The Neutral pose is used when creating a reference pose for your character with the FXA Exporters. You do not need to create a bone pose or morph target with the Neutral pose.


Default Mapping

Default Mapping

Default Mapping FaceFX Studio uses a default mapping that relies on the targets shown below. If you want to use the Default Mapping, simply create the targets as described and use the exact same names (capitalization included).
You do not have to use the Default Mapping!

Default Mapping Benefits

  • The tongue is separated out. You can get your character talking initially then add a tongue with very little hassle.
  • Only a minimal set of targets is used. They take very little time to set up.
  • Editing curves by hand is easy since many targets share the same components.
  • Creating advanced effects with the Face Graph is easy. To make your character yell, simply amplify the open node.
  • The targets roughly follow a component model. They can be used with advanced coarticulation models.

    Default Mapping Targets

    Mouth Targets
    Tongue Targets

    Modifying the Default Mapping

    Feel free to change the default mapping. Some artists prefer to have a more 1-to-1 relationship between phonemes and targets. From the Workspace Tab in "Mapping Mode", if you see a phoneme that doesn't look correct, create a new target for it. R, ER, T, N, TH, UW, EE, S, K, IH, AH, and Y are phonemes that frequently have their own targets in alternative mappings.


Mouth Targets

Mouth Targets

Mouth Targets
open
Open is the jaw-open meta-target. The jaw is opened and the corners of the mouth drop slightly. Open the jaw wide since the Default Mapping never drives this pose much more than 50%.
W
The W target makes the mouth look like it is whistling. The jaw is opened slightly. The upper lips are pulled down to cover the teeth. In the Default Mapping, this pose is important because it is used to round the mouth in vowels like when you say "bOOt".
ShCh
The ShCh target is used to make the mouth look like it is saying "Shhh". The upper and lower teeth should just touch and the lips are pursed to show the teeth. In the Default Mapping, this pose is important because it is also used to make the mouth form the "ER" sound like in "fevER".
PBM
The PBM target pushes the lips together completely to form a seal. If desired, the upper and lower lips can curl inwards (in a bone rig, some rotation may be used). The jaw can be opened just slightly behind the lips.
FV
The FV target tucks the lower lip behind the upper teeth. The jaw is opened slightly. The upper teeth are exposed by moving the upper lip slightly.
wide
The wide target brings the upper lip up slightly and the corners of the mouth out. No jaw movement is required. When combined with a little bit of the open target, the wide target creates the "EE" sound in "tree" or "beet". Moving the tongue to the roof of the mouth creates the "S" and "Z" sounds. Wide is also used as a component in many of the vowel targets to expose the upper teeth and widen the mouth.


Tongue Targets

Tongue Targets

Tongue Targets The below tongue targets can be used with the Default Mapping. Not all mappings require tongue targets. You can ignore the tongue or include it with your mouth targets. To create the below poses, only move the tongue, not the jaw or the mouth. Notice that the pictures below show an open jaw which is incorrect, but necessary for displaying the targets. Creating the targets can be tricky since only the tongue should move.

One technique is to create the below poses using the FXA Exporters is as follows:
  1. Create a new actor in the FXA exporters and export a reference pose with only the tongue bones in the neutral position.
  2. Open the jaw the appropriate amount (depending on how open the mouth is in the mapping that is used), then create the pose for the tongue.
  3. Export the poses you have created. Because the reference pose only contains tongue bones, that is all that is stored in the pose.
  4. Save the tongue-only FXA file.
  5. Load up your normal FXA file that contains the reference pose with all the bones. Import the reference pose on the frame where you want the tongue pose stored. Now open the tongue-only FXA file and import the appropriate tongue pose. All the bones are now in the correct position.
  6. Load up your normal FXA file with all of the bones and export the tongue pose.
  7. Open FaceFX Studio. From the Workspace Tab, go into "Mapping Mode", select one of the phonemes using your tongue pose and make sure the tongue is in the correct position with respect to the mouth.

tBack
The tBack target moves the back of the tongue to the back of the mouth for making the "K" or "G" sound. Remember that the target should not include any jaw movement (unlike the picture shown to the left).
tRoof
The tRoof target moves the tip of the tongue to the roof of the mouth for making the "T", "D", "N", and "L" sounds. Remember that the target should not include any jaw movement (unlike the picture shown to the left).
tTeeth
The tTeeth target brings the tip of the tongue in front of the upper teeth for making the "Th" sound. Remember that the target should not include any jaw movement (unlike the picture shown to the left).


Eyebrow Raise

Eyebrow Raise

Eyebrow Raise Left and Eyebrow Raise Right

Eyebrow Raise Left and Eyebrow Raise Right

  • Moves the eyebrow up to its maximum position.
  • Make sure the mouth bones are identical to the reference pose so they get pruned out.
  • You can optionally move the cheek up a bit in towards the eye to exaggerate the blink.
  • A curve for the eyebrow targets are generated when you analyze audio. The curve is named "Eyebrow Raise" This curve should be used to drive both the left and right eyebrow up targets using a simple linear link in the Face Graph.


Blink

Blink

Blink Left and Blink Right

Blink Left and Blink Right

  • Blink targets will move the eyelid down over the eyeball.
  • The mouth bones should be identical to the neutral pose.
  • You can optionally move the cheek up a bit in towards the eye to exaggerate the blink.
  • A curve for the blink targets is generated when you analyze audio. The curve is named "Blink" This curve should be used to drive both the left and right blink targets using a simple linear link in the Face Graph.


FaceFX Phonemes

FaceFX Phonemes

FaceFX Phonemes All of the phonemes that FaceFX supports are listed below. Due to the large number of supported phonemes, creating a new mapping from the Mapping Tab can be tedious. The good news is that it only needs to be done once and other characters can copy the mapping using a Template. It also can be useful to organize phonemes into phoneme groups. For example in the default mapping, many phonemes share the same target values.

Consonants

IPA | FaceFX | X-SAMPA | Manner | Place | Voicing | Examples
 | P | p | Plosive | Bilabial | Voiceless | /p/rint, s/p/in, ti/p/
 | B | b | Plosive | Bilabial | Voiced | /b/oot, a/b/out
 | T | t | Plosive | Alveolar | Voiceless | /t/ry, /t/oday
 | D | d | Plosive | Alveolar | Voiceled | /d/og, bombe/d/
 | K | k | Plosive | Velar | Voiceless | /c/at, s/k/in
 | G | g | Plosive | Velar | Voiced | /g/um, ba/g/
 | M | m | Nasal | Bilabial | - | /m/ore, botto/m/
 | N | n | Nasal | Alveolar | - | /n/i/n/e, pla/n/
 | NG | N | Nasal | Velar | - | si/ng/, ba/n/k
 | RA | r | Trill | Alveolar | - | Spanish pe/rr/o
 | RU | R | Trill | Uvular | - | French /r/ue, German /R/echt
 | FLAP | 4 | Flap | Alveolar | Voiceless | wa/t/er
 | PH | p/ | Fricative | Bilabial | Voiceless | Japanese /f/uhai.
 | F | f | Fricative | Labiodental | Voiceless | /f/arm, ta/ff/y
 | V | v | Fricative | Labiodental | Voiced | /v/isit, ra/v/e
 | TH | T | Fricative | Dental | Voiceless | /th/ing, ba/th/
 | DH | D | Fricative | Dental | Voiced | /th/is, /th/e
 | S | s | Fricative | Alveolar | Voiceless | /s/it, pa/ss/
 | Z | z | Fricative | Alveolar | Voiced | /z/oo, ro/s/es
 | SH | S | Fricative | Postalveolar | Voiceless | /sh/oe, pa/ssi/on, dona/ti/on
 | ZH | Z | Fricative | Postalveolar | Voiced | trea/s/ure, mira/ge/
 | CX | C | Fricative | Palatal | Voiceless | /h/uman (some English dialects). German i/ch/
 | X | x | Fricative | Velar | Voiceless | /Ch/anukah.
 | GH | G | Fricative | Velar | Voiced | Japanese ha/g/e
 | HH | h | Fricative | Glottal | Voiceless | /h/appy, a/h/ead (murmured)
 | R | r | Approximant | Alveolar | - | /r/ed
 | Y | j | Approximant | Palatal | - | /y/ou, /y/esterday.
 | L | l | Lateral Approximant | Alveolar | - | /l/ip, p/l/ease
 | W | w | Approximant | Labial-velar | - | /w/ater, /w/alk
 | H | H | Approximant | Labial-palatal | - | Not in English. French /hu/it, Mandarin Chinese yue
 | TS | ts | Affricate | Alveolar | Voiceless | pi/zz/a, German /z/ehn, Schni/tz/el
 | CH | tS | Affricate | Postalveolar | Voiceless | /ch/ip.
 | JH | dZ | Affricate | Postalveolar | Voiced | /g/irrafe, /j/ump

Vowels

IPA | FaceFX | X-SAMPA | Height | Backness | Roundedness | Diacritic | Examples
 | IY | i | Close | Front | Unrounded | - | b/ee/t, French f/i/n/i/
 | E | e | Close-mid | Front | Unrounded | - | b/e/d (AuE and NzE), French beaut/é/, German S/ee/le
 | EN | e~ | Close-mid | Front | Unrounded | - | French /ain/si
 | EH | E | Open-mid | Front | Unrounded | - | b/e/d (GA), French b/ê/te, German b/e/tt
 | A | a | Open | Front | Unrounded | - | c/u/t (AuE and NzE), French r/a/t, German r/a/tte, Spanish r/a/t/a/
 | AA | A | Open | Back | Unrounded | - | sp/a/ (RP & GA), h/o/t (GA), French p/â/te
 | AAN | A~ | Open | Back | Unrounded | Nasalized | French qu/and/, French /un/
 | AO | O | Open-mid | Back | Rounded | - | b/a/ll (RP), h/o/t (AuE & NzE), French s/o/rt, German v/o/ll
 | AON | O~ | Open-mid | Back | Rounded | Nasalized | French s/on/
 | O | o | Close-mid | Back | Rounded | - | b/a/ll (AuE and NzE), French rés/eau/, German K/oh/l
 | ON | o~ | Close-mid | Back | Rounded | Nasalized | French m/an/teau
 | UW | u | Close | Back | Rounded | - | b/oo/t (RP and GA). French f/ou/
 | UY | y | Close | Front | Rounded | - | Not in English. French ch/u/te.
 | EU | 2 | Close-mid | Front | Rounded | - | Not in English. French p/eu/, German sch/ö/n
 | OE | 9 | Open-mid | Front | Rounded | - | German H/ö/lle
 | OEN | 9~ | Open-mid | Front | Rounded | Nasalized | French j/eu/ne
 | AH | V | Open-mid | Back | Unrounded | - | n/u/t (RP & GA), Korean b/eo/l
 | IH | I | Near-close | Near-front | Unrounded | - | b/i/t (RP, GA, AuE)
 | UU | Y | Near-close | Near-front | Rounded | - | Not in English. German h/ü/bsch
 | UH | U | Near-close | Near-back | Rounded | - | h/oo/k. German Sch/u/tz, Swedish.
 | AX | @ | Mid | Central | - | - | /a/bout, syn/o/nym, /uh/.
 | UX | 6 | Near-open | Central | - | - | c/u/t (CaE), German de/r/.
 | AE | { | Near-open | Front | Unrounded | - | f/a/t (RP, GA, AuE)
 | ER | 3 | Open-mid | Central | Unrounded | - | English p/er/fect (GA), f/ur/ (RP and AuE)
 | AXR | @' | Mid | Central | - | Rhotacized | butt/er/
 | EXR | 3' | Open-mid | Central | - | Rhotacized | f/ur/

Dipthongs

Note: Dipthongs are split into component vowels, so mappings for dipthongs is irrelevant.
IPA | FaceFX | X-SAMPA | First | Second | Examples
 | EY | EI | EH | IH | pl/ay/
 | AW | aU | A | UH | ab/ou/t
 | AY | aI | A | IH | b/i/te
 | OY | OI | O | IH | b/oy/
 | OW | @U | AX | UH | b/oa/t


Tutorials

Tutorials

Tutorials

Slade

Introduction to Slade
Talking Like the Godfather
Working with an Animation
Working with Animation Sets
Creating Slade

Workspace Tutorials

Creating a Workspace
Create Combiner Mode
Animate Mode
Visual Mapping Mode

Getting Your Character Talking

Maya Morph Tutorial
Maya Bones Tutorial
Max Morph Tutorial
Max Bones Tutorial
Setting up Gestures


Slade Tutorials

Slade Tutorials

Slade Tutorials Introduction to Slade
Talking Like the Godfather
Working with an Animation
Working with Animation Sets
Creating Slade


Introduction to Slade

Introduction to Slade

Introduction to Slade
  1. Launch FaceFX Studio, and open the Slade-Maya.fxa file found in the Samples folder of the FaceFX Studio install directory.
  2. From the Actor Panel on the left, click on the Animations section. Notice that the Default group and welcome animation are selected. Play the welcome animation from the green play button at the bottom of the application.
  3. The Preview Tab should be selected. At the top left of the tab there are icons for the various modes of operation available in the Sample Rendering Engine. Click on the button on the far right to turn on the skeleton. You can see the 51 bones used to form all of Slade's poses.
  4. From the top right of the Preview Tab, you can select a pre-existing camera or create a new one. Change the camera to Eye Left.
  5. Click on the Face Graph section of the Actor Panel to expand it. Click on the Node Name button at the top of the list to sort the list alphabetically. Change the Node Group at the top to Lighting Interactions. Click on the Face Light Intensity node then play with the quick preview slider at the bottom. Notice that Slade's pupils dilate and contract with the slider. When the slider is far to the right Slade will squint his eyes, grit his teeth, and turn his head away from the (imaginary) bright light.
  6. As you can tell the Face Light Intensity slider controls multiple targets that depend on the value of the slider. Click over to the Face Graph Tab to see how this is done.
  7. The Face Light Intensity node should take up most of the Face Graph Tab because it was selected from the Actor Panel. You can hover over the nubs on the right of the node to see the link functions. The Orange Link functions are all Custom Link Functions that output a value of zero until the value of the Face Light Intensity slider is sufficiently high.
  8. Right-click on one of the nubs and select Edit Link Properties
  9. From the Link Function Editor Dialog, you can change the link function type from the drop-down at the top and modify the link function properties from the fields at the bottom. For Custom link functions, you can hit the insert key and click in the graph to ad new points to the link function.
  10. Close the link function Editor Dialog. Right-click in the Face Graph and turn on Node Value Visualization. Make sure the Face Light Intensity node is the only node selected. Zoom out in the Face Graph with the middle mouse scroll or by right-clicking in the face graph and selecting Reset View. Play with the quick preview slider in the Face Graph Section of the Actor Panel. When the slider is near the max, watch as the nodes in the face graph change colors as the values propagate through the face graph.
  11. Most of the complexity of Slade's face graph comes from the 6 bones around each eye. Each bone has its own bone pose to move it up and down and any change in Slade's gaze direction results in the eyelid bones forming a realistic shape around Slade's eyes. There are easier ways to get realistic eyelids (for example using a single bone pose for Look Left and Look Right, but Slade's gaze direction is a good example of the power of the face graph.


Talking Like the Godfather

Talking Like the Godfather

Talking Like the Godfather The face graph is a powerful tool for creating expressive talking characters. Using the face graph, you can fix problematic target interactions so that animating an expression is very easy. This tutorial illustrates the Godfather expression in the Slade sample content.

  1. Launch FaceFX Studio, and open the Slade-Maya.fxa file found in the Samples folder of the FaceFX Studio install directory.
  2. From the Animation section of the Actor Panel change the Group to Movie Quotes. From the Animation dropdown, select the peace animation.
  3. Play the peace animation. Watch Slade talk like the Godfather.
  4. In the Animations section of the Actor Panel, select the Godfather curve. Go to the Animation Tab. Notice that the Godfather curve is comprised of only a single key with value = 1.
  5. Now select the Godfather Mouth curve from the Animations section of the Actor Panel. There are no keys in this curve, but you can see from the dotted line (the virtual curve) that the Godfather Mouth node exhibits some complicated behavior based upon the nodes in the Face Graph that feed into it.
  6. Hold the Ctrl button while selecting the following curves from the Actor Panel: Godfather Mouth, FV, PBM. Notice that whenever one of the speech target curves goes up, the Godfather Mouth curve goes down. What is happening is that the Godfather curve is driving Godfather Mouth to 1 except when the PBM or FV target curves are active because these curves interfere with each other.
  7. To demonstrate how the Godfather pose interferes with some speech targets, select the Godfather Speech Correction node in the Face Graph. On the right of the properties section of the face graph, notice the Output To link functions. There should be one output link function to Godfather Mouth. Select the it and hit delete
  8. Now go to the Preview Tab. In the Face Graph section of the Actor Panel use the quick preview slider to ramp up the Godfather node to 1. Now select the FV node from the Face Graph section of the Actor Panel and ramp it's value up to 1. Notice that Slade's teeth are popping through his lower lip.
  9. Hold down Ctrl + Z to undo your changes to the face graph and reinsert the corrective link between Godfather Speech Correction and Godfather Mouth.
  10. Now use the quick preview slider to again ramp up Godfather and FV at the same time. Notice that the corrective link drives down the Godfather Mouth node so you don't see Slade's teeth pop through his lip.
  11. The moral of the story is that the corrective node between Godfather Speech Correction and Godfather Mouth fixes what is broken with the Godfather target. The Godfather pose is split out into several components, but the Godfather Mouth node is the only one that interferes with the speech targets. So when a problematic speech target occurs, it drives down Godfather Mouth leaving the eyes and head-tilt alone. The Godfather setup is a good example of how to create expressions that don't interfere with speech targets.


Working with an Animation

Working with an Animation

Working with an Animation
  1. Launch FaceFX Studio, and open the Slade-Maya.fxa file found in the Samples folder of the FaceFX Studio install directory.
  2. To generate a new animation, go to Actor->Animation Manager.
  3. From the Animation Manager, make sure the Default Animation group is selected from the drop down, then click on the welcome animation. Click on the Delete animation(s) button at the bottom. Now we can add the welcome animation back (pointless I know, but you want to learn how to do this right?).
  4. Click on the Create Animation... button at the bottom of the Animation Manager. This launches the Create Animation Wizard.
  5. On the first page of the Wizard, leave Automatically generate speech and gesture curves selected. The other option is for hand animating from scratch. Click next.
  6. Browse for the welcome.wav file in the Samples/Src folder in the FaceFX Studio installation directory. Click next.
  7. The text for the welcome animation should be loaded into the Analysis Text box. Notice the text tag for the Gaze Left and Right curve. Click next.
  8. Leave the defaults for the Language, Gesture Config, and Coarticulation Config. Click next.
  9. Leave the name of the animation as welcome. Click Finish.
  10. The welcome animation has been added. Close the Animation Manager dialog unless you want to transfer the animation into a different group.
  11. From the Animations section of the Actor Panel, select the Default group and the welcome animation you just generated. Play the animation. Notice that Slade looks towards the Actor Panel because of the text tag.
  12. Go to the Animation Tab. Select all of the speech target curves (Eath Earth, If, Ox, Oat, etc.) from the Animations section of the Actor Panel. You should see curves in the Curve Editor without any keys. You can't edit the keys because the curves are still Owned by the analysis.
  13. Try moving some phoneme and word boundaries. You should see the speech target curves update in the curve editor. Any curves owned by analysis are regenerated every time you move a phoneme boundary.
  14. Double click on a phoneme or word to hear the associated audio. If the phoneme segmentation is off you can adjust the boundaries to get perfect segmentation. If you need to perform some major surgery on an animation, keep in mind you can cut and paste phonemes and words, shift multiple phonemes by holding CTRL while dragging a phoneme boundary, and reanalyze a specific section of audio with or without text (text tags are ignored during reanalyze range).
  15. When you have finished perfecting the phoneme timings, if the animation isn't exactly the way you want you can modify the keys in the curves directly. Keep in mind that once you change a curve to be Owned by the User you will loose any work you have done if you change it back to Owned by Analysis
  16. Select some curves that you want to modify from the Animations section of the Actor Panel. Right click on the curves, go to Curve Properties and uncheck the Owned by Analysis checkbox. Now you can grab keys to move them or modify tangents.
  17. Be aware that you can scrub time from the curve editor by holding CTRL and scrubbing the middle mouse button. This will snap to any keys in the selected curves. Getting the time curser to match a key's time can also be done from the curve editor's previous and next key buttons in the toolbar. This is an important operation when trying to modify a key's value with a slider.
  18. For doing serious animating, you will probably want to use workspaces. Once you Create a Workspace with all of the sliders you want to animate with, you can use the workspace in animate mode to get the animation looking exactly the way you want.

See Also

Owned by Analysis vs. Owned by User
Animation Tab


Working With Animation Sets

Working With Animation Sets

Working with Animation Sets
  1. Launch FaceFX Studio, and open the Slade-Maya.fxa file found in the Samples folder of the FaceFX Studio install directory.
  2. From the Animations section of the Actor Panel, change the Animation Group to No Text - No tweaking. Change the animation to Borah. Play the Borah animation.
  3. Both of the animations in the No Text - No tweaking group are stored in an external animation set. From the FaceFX Studio interface, there are very few indications that these animations and their associated phoneme and audio data are being stored in external files. In fact the only way to tell is that the Actor->Unmount Animation Set is enabled and the Actor->Export Animation Set is disabled.
  4. When you loaded the Slade-Maya.fxa file, the Slade-Maya.fxl file used the command system to load the ExternalAnimSet_noText.fxe file. The external anim set contains the No Text - No tweaking animation group. Loading external animation sets this way is recommended to avoid the repetitive operation of manually loading animation sets whenever you open an actor.
  5. To create a new External animation set, select the Movie Quotes animation group. Then go to Actor->Export Animation Set. Click Yes on the waning message that tells you the operation is undoable. Save the file in the Samples folder of the FaceFX Studio installation directory. Call the External Animation set ExternalAnimSet_Movie.fxe.
  6. After exporting the animation set, browse to the Samples folder to see the files. Notice that the ExternalAnimSet_Movie.fxe file is extremely small. This file only contains the animation keys and curve names required to play the animation. All of the audio data, phoneme times, and other FaceFX Studio specific data is stored in the much larger ExternalAnimSet_Movie.fxm file.
  7. From the Animations section of the Actor Panel, notice that the Movie Quotes animation Group is no longer available. Go to Actor->Mount Animation Set. Select the ExternalAnimSet_Movie.fxe file.
  8. Because the Movie Quote group is mounted, any changes you make to the animations will be saved out to the external file. Just save from FaceFX Studio like you normally do to update all of the mounted animation sets.

See Also

External Anim Sets


Creating Slade

Creating Slade

Creating Slade
  1. Start Maya and open the Slade-Maya.mb file found in the FaceFX Studio installation folder.
  2. Follow the Maya Bones Tutorial for more details on the this step. Export an FXR file. Go to frame 0 and export the Reference Pose selecting all of the bones in Slade's skeleton.
  3. From the Maya FXA exporter, click on Batch Export. Browse to the Samples/Src folder in the FaceFX installation directory. Select Slade-Batch-Export.txt. Save the FXA file to the same directory as the FXR file you saved.
  4. Use FaceFX Studio to open up the FXA file you created for Slade. The FXR should load as well. You will get a warning about not having a mapping specified. Go to the Mapping Tab and click on the Make Default Mapping Button.
  5. At this point Slade's Face Graph has the bone poses exported from Maya, but nothing else. To set up Slade's Face Graph and create some node groups, run the Slade-Face-Graph-Setup.fxl file found in the Samples/Src folder on the FaceFX installation directory. Run the exec command as follows (but change the directory to your FaceFX Studio installation folder):
    exec -f "C:\Slade-Face-Graph-Setup.fxl"
  6. Save your work. At this point the Slade file you have created is exactly like the one that ships with FaceFX except that it doesn't have any animations, workspaces, or cameras.
  7. Open the Slade-Maya.fxa that ships with FaceFX. Go to Actor->Export Template and save an FXT file.
  8. Now open the version of Slade you exported from Maya. Go to Actor->Sync to Template. Click Yes on the warning message. When asked which components of the template to sync to, make sure Workspaces and Cameras are selected. The other components should already be identical.
  9. Save your work. At this point, your version of Slade should be exactly the same as the one that ships with FaceFX except that it doesn't contain any animations. If you want, you can follow the Working with Animation Sets Tutorial to transfer animation data into your version of Slade. Alternatively you can generate your own animations. What Fun!


Workspace Tutorials

Workspace Tutorials

Workspace Tutorials Creating a Workspace
Create Combiner Mode
Animate Mode
Visual Mapping Mode


Creating a Workspace

Creating a Workspace

Workspace Tutorials: Creating a Workspace
  1. Go to the workspace tab and go into "Edit mode" by clicking on the icon that looks like a pencil.
  2. Click on the New Workspace icon to get a new workspace.
  3. You can create a few 1D sliders by dragging the node names listed on the right onto the work area on the left.
  4. To create 2D sliders, drag two different node names onto the boxes below labeled Drop x-axis and Drop y-axis. Then you drag the Drag to create box onto the workspace. This will create a 2D slider.
  5. You can move, rotate and scale the 1D and 2D sliders by manipulating the green control points.
  6. To create a viewport, click on the Add a viewport button at the top of the mapping tab.
  7. Save the workspace. This will mark your actor as dirty so that the next time you save your actor, your changes to the workspace will also be saved. Because there is no Undo functionality when editing a workspace, you need to confirm your changes by saving the workspace.
  8. Leave edit mode by clicking on the icon of the pencil with the line through it.
  9. Select the animation you want to work with from the Animation section of the Actor panel.
  10. If you want to animate the curves in your mapping or the gesture curves generated by FaceFX Studio analysis, these curves will initially be “owned by analysis”. You can’t animate them until you change them to “Owned by User” Select the curves, right click on them, go to “Curve Properties” and uncheck “Owned by Analysis” Notice that the icons next to the curves have changed.
  11. Now you are ready to animate from your workspace. Go to the workspace tab, select your workspace, and change the mode from “Create Combiner” to “Animate”
  12. Scrub the timeline to where you want to insert a key from one of your sliders. Move the slider value with the left mouse button. When the value is where you want it, use the middle mouse button to set a key.


Create Combiner Mode

Create Combiner Mode

Workspace Tutorials: Create A Combiner Node Visually The Face Graph is a very powerful tool that allows the technical artist to achieve many kinds of target interactions. To support all of the different options, the face graph user interface is not suited for artistic experimentation. For creating simple target interactions that merely drive multiple nodes with a single slider in the workspace tab, try following this tutorial to create a combiner node visually.
  1. Open FaceFX Studio and go to the workspace tab.
  2. Create a new workspace. Add sliders for all of the nodes that you want to play around with. Follow the Creating a Workspace tutorial for more information on how to do this. Note: To create a combiner node visually, you do not need to create a workspace. You can set values with the quick preview slider then create the node as described below.
  3. When you have finished modifying your workspace, save it then leave Edit Mode. Make sure you are in the Workspace Tab'sCreate Combiner mode. The Workspace modes are in a dropdown box at the top of the tab. Available modes are Create Combiner, Animate, and Visual Mapping.
  4. From the Create Combiner mode, you can play with the sliders until you get the character looking the way you want. Then click on the Create combiner node from sliders button (the green one at the top of the tab).
  5. A node properties dialog will pop up. Give the new node a name then click on the Create button.
  6. The new node is added to the Face Graph. You can drive it from the quick preview slider or create a new workspace that includes the slider.


Animate Mode

Animate Mode

Workspace Tutorials: Animate Mode
  1. Open FaceFX Studio and go to the workspace tab.
  2. Create a new workspace. Add sliders for all of the nodes that you want to animate. Follow the Creating a Workspace tutorial for more information on how to do this.
  3. When you have finished modifying your workspace, save it then leave Edit Mode. Make sure you are in the Workspace Tab's Animate mode. The Workspace modes are in a dropdown box at the top of the tab. Available modes are Create Combiner, Animate, and Visual Mapping.
  4. When in Animate mode, you need to have an animation selected to do anything meaningful. Select the animation you want to work with from the Actor Panel on the left.
  5. If you want to animate with multiple workspaces simultaneously, you can detach the workspaces with the button on the upper-right hand corner. Then select a different workspace from the dropdown on the upper-left.
  6. Make sure all of the curves you want to modify are Owned by the user. Scrub to a time in the animation where you want to insert a key.
  7. Move modify a sliders value. Notice that the lines which were once green are now yellow, meaning that inserting a key will modify the curve's value at the current time. Click on the middle mouse button to insert the key. The lines now turn red meaning that there is a key for the slider at the current time.
  8. To animate more quickly, turn on the auto-key feature from the button at the top of the workspace tab. Any slider moved will automatically insert a key.
  9. If you want to modify an existing key as opposed to adding a new key, you will need to get the current time to match the key's time exactly. This is nearly impossible to do by scrubbing the mouse through the timeline, so use the previous key and next key buttons at the top of the workspace. This will change the time to the next key among all of the visible sliders.


Visual Mapping Mode

Visual Mapping Mode

Workspace Tutorials: Create Mapping Visually If you don't want to set up your character with our default mapping as described in Character Setup, you will need to define a new mapping. This involves defining the how your character forms the mouth shapes for the 42 phonemes FaceFX recognizes. This can be done from the Mapping Tab or by following this tutorial for a more visual pipeline.

  1. Open FaceFX Studio and go to the workspace tab.
  2. Create a new workspace. Add sliders for all of the nodes that you want to use to create your speech targets. Follow the Creating a Workspace tutorial for more information on how to do this.
  3. When you have finished modifying your workspace, save it then leave Edit Mode. Make sure you are in the Workspace Tab's Visual Mapping mode. The Workspace modes are in a dropdown box at the top of the tab. Available modes are Create Combiner, Animate, and Visual Mapping.
  4. From the Visual Mapping mode, you can select a phoneme from the dropdown menu at the top of the tab. Play with the sliders until you get the character looking like he/she is pronouncing the phoneme you selected (There should be a sample word listed next to the phoneme). Then click on the Apply to Mapping button (the green one at the top of the tab).
  5. Values are automatically added to the Mapping Tab.


Getting Your Character Talking

Getting Your Character Talking

Getting Your Character Talking Maya Morph Tutorial
Maya Bones Tutorial
Max Morph Tutorial
Max Bones Tutorial
Setting up Gestures


Maya Morph Tutorial

Maya Morph Tutorial

Maya Morph Tutorial
  1. Install the FXR exporters from the FaceFX installer. You won't need the FXA exporters for a strictly morph rig. If your rig uses both morph targets and bones, you will need to follow both this tutorial and the Maya Bones Tutorial.
  2. Open Maya and make sure the FxMayaExporter.mll plug-in is loaded from the Window -> Settings/Preferences -> Plug-in Manager dialog.
  3. Open the character you want to export. Make sure the mesh is triangulated. Go to Modify->Evaluate Nodes->Ignore All. Make sure the blendshape meshes are still in the scene (you can hide them if you want) exactly the way they were when you created the blendShape target. Also make sure that the name of the blenshape meshes matches the name of the slider used to drive the mesh in Maya (by default this is the case).
  4. Go to File->Export All. Select FaceFX Rendering Archive (*.FXR) from the Files of Type dropdown. Save the FXR file.
  5. Open FaceFX Studio. Go to File -> New Actor. Give your Actor a name.
  6. Go to the Mapping tab and click on the Make Default Mapping button. This assumes you are setting up your character as described in Character Setup.
  7. Save the Actor file with the same name and in the same directory as the FXR file you exported. An FXS file with your mapping should be saved automatically.
  8. Now go to File->Open Actor, and select the FXA file you just saved. When you open the file, it should find the FXR file in the same directory with the same name and open that file displaying your character. When an FXR file with morph targets is opened, morph target nodes are automatically added to the FXA file (assuming they do not already exist).
  9. Make sure the mesh imports correctly and your targets deform the mesh correctly by selecting each target from the Face Graph section of the Actor panel and driving the target with the quick preview slider. If the mesh or targets are incorrect, check the FxMayaExporter_Log.txt log file in the Maya install directory for any problems that occurred when you exported your FXR file. Also read through the documentation listed in the See Also section below.
  10. Make sure that your morph target nodes in the Face Graph have the same name as the targets listed in the mapping. Curves will be created from the targets in the mapping, and the curves will drive nodes in the face graph if the names match up.
  11. Go to Actor->Animation Manager. Navigate to the Animation Group you want to create an animation in (you can always transfer the animation around later). Click on the Create Animation button and follow the wizard.
  12. You should be able to play the animation and watch the character lip-synch to the audio file. From tha Animation section of the Actor Panel, notice that curves are created for all of the targets in the mapping as well as gesture targets.
  13. When you save, the animation will be saved in the FXA file. The audio and phoneme information will be stored in the FXS file, which is only used by FaceFX Studio.

See Also

FxMorphTargetNode


Maya Bones Tutorial

Maya Bones Tutorial

Maya Bones Tutorial
  1. Install the FXR and FXA exporters from the FaceFX installer.
  2. Open Maya and make sure the FxMayaExporter.mll and FaceFX.mll plug-ins are loaded from the Window -> Settings/Preferences -> Plug-in Manager dialog.
  3. Open the character you want to export. Make sure the mesh is triangulated. Go to Modify->Evaluate Nodes->Ignore All.
  4. Go to File->Export All. Select FaceFX Rendering Archive (*.FXR) from the Files of Type dropdown. Save the FXR file.
  5. From the Maya Script Editor, go to File->Open Script. Select the FaceFX.mel script located in the Tools/Maya directory of the FaceFX installation (you may want to copy this script file into your default scripts directory for easy access). Once the script has been opened in the Script Editor, go to Script->Execute.
  6. The FaceFX Panel will open up in Maya. Go to File->New to give your FXA file a name. Type in a name and click OK.
  7. The first step is to define a Reference pose. This pose determines what bones you want FaceFX to control, and each bone's default position, rotation, and scale. The Reference Pose should have the mouth closed in a neutral state, the eyelids open, the head facing forward, and the eyebrows relaxed. Go to a frame in the Maya timeline where the bones are in this configuration and click on the Export... button in the Reference Pose section of the FaceFX Panel.
  8. A dialog will pop up that allows you to select the bones you want FaceFX to control. If you want FaceFX to move the bone, select it here. If you never want FaceFX altering the bone in question, don't select it here.
  9. Now that you have defined a Reference Pose, you can define one or more Bone Poses. When you export a Bone Pose, the FaceFX plug-in looks at all of the bones defined in the Reference Pose. The plug-in compares the positions, rotations, and scales of these bones to the translations stored in the Reference Pose. If any bones have changed with respect the Reference Pose, those bones will be included in the Bone Pose. Go to a frame where the bones form a pose you want to export. In the Bone Poses section, click on the Export... button.
  10. Give your new bone pose a name. Check the Maya Output Log window for information on how many bones were removed via pruning. This information is also stored in the FaceFX_Log.txt file. If all bones are pruned, then the pose is identical to the Reference Pose, and you will get a warning.
  11. Export as many bone poses as you like, following the steps above. For a more rapid process, you can look into MEL Script commands or creating a Batch Export text file that defines the pose names and frame numbers.
  12. From the FaceFX Panel in Maya, go to File->Save. Save the FXA file in the same directory and with the same name as the FXR file you saved above. You can close Maya at this point if desired.
  13. Open FaceFX Studio. Go to File->Open Actor, and select the FXA file you just saved. An warning message will tell you that the FXA file does not have a mapping defined. To define a mapping, go to the Mapping tab and click on the Make Default Mapping button. This assumes you are setting up your character as described in Character Setup.
  14. Go to File->Save Actor. This will create an FXS file with your mapping so you do not get a warning message when opening your actor in the future.
  15. Make sure the mesh imports correctly and your targets deform the mesh correctly by selecting each target from the Face Graph section of the Actor panel and driving the target with the quick preview slider. If the mesh or targets are incorrect, check the FxMayaExporter_Log.txt and FaceFX_Log.txt log files in the Maya install directory for any problems that occurred when you exported your FXR or FXA files. Also read through the documentation listed in the See Also section below.
  16. Go to Actor->Animation Manager. Navigate to the Animation Group you want to create an animation in (you can always transfer the animation around later). Click on the Create Animation button and follow the wizard.
  17. You should be able to play the animation and watch the character lip-synch to the audio file. From the Animation section of the Actor Panel, notice that curves are created for all of the targets in the mapping as well as gesture targets. To take advantage of the Gesture targets, check out the Setting up Gestures Tutorial.
  18. When you save, the animation will be saved in the FXA file. The audio and phoneme information will be stored in the FXS file, which is only used by FaceFX Studio.


Max Morph Tutorial

Max Morph Tutorial

Max Morph Tutorial
  1. Install the FXR exporters from the FaceFX installer. You won't need the FXA exporters for a strictly morph rig. If your rig uses both morph targets and bones, you will need to follow both this tutorial and the Maya Bones Tutorial.
  2. Open the character you want to export in Max. Make sure the mesh is triangulated. Make sure all of the textures reference files that exist on your system. Make sure the character is in its bind pose at frame 0.
  3. Go to File->Export. Select FaceFX Studio (*.FXR) from the Files of Type dropdown. Save the FXR file. To ensure that textures are found, save the FXR file in the same directory as the source texture files. You can optionally close Max at this point.
  4. Open FaceFX Studio. Go to File -> New Actor. Give your Actor a name.
  5. Go to the Mapping tab and click on the Make Default Mapping button. This assumes you are setting up your character as described in Character Setup.
  6. Save the Actor file with the same name and in the same directory as the FXR file you exported. An FXS file with your mapping should be saved automatically.
  7. Now go to File->Open Actor, and select the FXA file you just saved. When you open the file, it should find the FXR file in the same directory with the same name and open that file displaying your character. When an FXR file with morph targets is opened, morph target nodes are automatically added to the FXA file (assuming they do not already exist).
  8. Make sure the mesh imports correctly and your targets deform the mesh correctly by selecting each target from the Face Graph section of the Actor panel and driving the target with the quick preview slider. If the mesh or targets are incorrect, check the log files in the Maya install directory for any problems that occurred when you exported your FXR or FXA files. Also read through the documentation listed in the See Also section below.
  9. Make sure that your morph target nodes in the Face Graph have the same name as the targets listed in the mapping. Curves will be created from the targets in the mapping, and the curves will drive nodes in the face graph if the names match up.
  10. Go to Actor->Animation Manager. Navigate to the Animation Group you want to create an animation in (you can always transfer the animation around later). Click on the Create Animation button and follow the wizard.
  11. You should be able to play the animation and watch the character lip-synch to the audio file. From the Animation section of the Actor Panel, notice that curves are created for all of the targets in the mapping as well as gesture targets.
  12. When you save, the animation will be saved in the FXA file. The audio and phoneme information will be stored in the FXS file, which is only used by FaceFX Studio.

See Also

FxMorphTargetNode


Max Bones Tutorial

Max Bones Tutorial

Max Bones Tutorial
  1. Install the FXR and FXA exporters from the FaceFX installer.
  2. Open the character you want to export in Max. Make sure the mesh is triangulated. Make sure all of the textures reference files that exist on your system. Make sure the the character is in its bind pose at frame 0.
  3. Go to File->Export. Select FaceFX Studio (*.FXR) from the Files of Type dropdown. Save the FXR file. To ensure that textures are found, save the FXR file in the same directory as the source texture files.
  4. Go to the Utilities Panel on the right, click on the More button select FaceFX from the list, and click OK.
  5. The FaceFX Panel will open up in Max. Click on the Create Actor button to give your actor a name. Type in a name and click OK.
  6. By default, the Normalize Scale checkbox is checked. It is generally a good idea to keep this checked when you follow this tutorial. The only downside is that you can not have poses that animate bone scale values (for example, the pupil dilation poses in the Slade sample content).
  7. The first step is to define a Reference pose. This pose determines what bones you want FaceFX to control, and each bone's default position, rotation, and scale. The Reference Pose should have the mouth closed in a neutral state, the eyelids open, the head facing forward, and the eyebrows relaxed. Frame 0 should have the bind pose, which is normally a good pose to use for the Reference Pose. Go to frame 0 in the timeline and click on the Export... button in the Reference Pose section of the FaceFX Panel.
  8. A dialog will pop up that allows you to select the bones you want FaceFX to control. If you want FaceFX to move the bone, select it here. If you never want FaceFX altering the bone in question, don't select it here. You should only include an objects if the skin is weighted to it or it's children.
  9. Now that you have defined a Reference Pose, you can define one or more Bone Poses. When you export a Bone Pose, the FaceFX plug-in looks at all of the bones defined in the Reference Pose. The plug-in compares the positions, rotations, and scales of these bones to the translations stored in the Reference Pose. If any bones have changed with respect the Reference Pose, those bones will be included in the Bone Pose. Go to a frame where the bones form a pose you want to export. In the Bone Poses section, click on the Export... button.
  10. Give your new bone pose a name. You can check the FaceFX_Log.txt file found in the 3D Studio Max executable directory to see what bones were pruned from the pose. If all bones are pruned, then the pose is identical to the Reference Pose, and you will get a warning.
  11. Export as many bone poses as you like, following the steps above. For a more rapid process, you can look into MAXScript commands or creating a Batch Export text file that defines the pose names and frame numbers.
  12. Click on the Save Actor button near the top of the FaceFX Panel. Save the FXA file in the same directory and with the same name as the FXR file you saved above. You can close Max at this point if desired.
  13. Open FaceFX Studio. Go to File->Open Actor, and select the FXA file you just saved. An warning message will tell you that the FXA file does not have a mapping defined. To define a mapping, go to the Mapping tab and click on the Make Default Mapping button. This assumes you are setting up your character as described in Character Setup.
  14. Go to File->Save Actor. This will create an FXS file with your mapping so you do not get a warning message when opening your actor in the future.
  15. Make sure the mesh imports correctly and your targets deform the mesh correctly by selecting each target from the Face Graph section of the Actor panel and driving the target with the quick preview slider. If the mesh or targets are incorrect, check the log files in the Max install directory for any problems that occurred when you exported your FXR or FXA files. Also read through the documentation listed in the See Also section below.
  16. Go to Actor->Animation Manager. Navigate to the Animation Group you want to create an animation in (you can always transfer the animation around later). Click on the Create Animation button and follow the wizard.
  17. You should be able to play the animation and watch the character lip-synch to the audio file. From the Animation section of the Actor Panel, notice that curves are created for all of the targets in the mapping as well as gesture targets. To take advantage of the Gesture targets, check out the Setting up Gestures Tutorial.
  18. When you save, the animation will be saved in the FXA file. The audio and phoneme information will be stored in the FXS file, which is only used by FaceFX Studio.


Setting up Gestures

Setting up Gestures

Setting Up Gestures
  1. In addition to the curves specified in your mapping, FaceFX generates gesture curves based on human behavioral rules, a random number generator, and stress detection in the audio. To use these curves, start by creating bone poses or morph targets for blinks and eyebrow raises.
  2. To control head and eye rotations, you will need to create bone poses that rotate the appropriate bones. The poses in the Slade sample content have names like the following: BN_Head_RotPos_X, BN_Head_RotNeg_X, BN_Eyeball_R_RotPos_Y, etc. Your naming scheme may be different. Use the Max or Maya plug-in to create these poses. Each pose should rotate a single bone in a single direction by the maximum amount possible for the character. The min/max values for these bone pose nodes should be 0 and 1 respectively.
  3. Open your actor in FaceFX Studio. For each pair of bone poses that rotate a particular bone in a particular axis, a single combiner node is created that can control both the positive and negative components of the rotation. For example, the BN_Head_RotPos_X and BN_Head_RotNeg_X bone pose nodes are controled by the BN_Head_Rot_X combiner node. To create the combiner node, right-click in the face graph and go to Add Node->FxCombinerNode. Name the node BN_Head_Rot_X.
  4. Make sure the BN_Head_Rot_X node is selected, then change the Minimum Value of the node to -1. The Minimum Value field is located at the bottom of the face graph tab and it controls the properties of the selected nodes.
  5. Create links from the BN_Head_Rot_X node to the BN_Head_RotPos_X and BN_Head_RotNeg_X bone pose nodes. To create a link, click on the first node's right-nub, and drag it to the second node's left-nub. Once a link is created, you can modiify the link by selecting one of the nodes then double clicking on the picture of the link function in the Node properties section. Change the slope (m) of the link function linking BN_Head_Rot_X to BN_Head_RotNeg_X to -1. Now you have a single combiner node with min/max of -1/1 that drive two bone pose nodes with min/max of 0/1.
  6. Repeat the above 4 steps for each of the poses used to rotate the head and eyes in the positive and negative directions.
  7. Now you need to create FxCombinerNodes with the appropriate names to receive values from the speech gesture curves that FaceFX creates. Right click in the face graph, and select Create Speech Gestures Setup.
  8. Now you need to hook up the nodes that receive the curve values to the nodes that drive the bone poses. You should now create linear links in your Face Graph as follows (The below names assume Maya coordinates. Max has the Y and Z axis swapped and the X axis negated.):
    Head Pitch ->  BN_Head_Rot_X
Head Yaw -> BN_Head_Rot_Y
Head Roll -> BN_Head_Rot_Z

Eye Yaw -> BN_Eyeball_L_Rot_Y
Eye Yaw -> BN_Eyeball_R_Rot_Y

Eye Pitch -> BN_Eyeball_L_Rot_X
Eye Pitch -> BN_Eyeball_R_Rot_X

  9. The slope of the linear link functions described above is determined by the amount the bone is actually rotated in the bone poses you created. If you rotated the head bone in the X axis by 20 degrees in both the positive and negative direction, you would want the slope of the Head Pitch -> BN_Head_Rot_X link function to be 1/20 or 0.05. The curves generated by the FaceFX gesture system are meant to drive the bone's rotation in degrees.
  10. Right click in the face graph and select Node Value Visualization. You should be able to troubleshoot any problems with node minimums, maximums, or link functions using this feature. You can scrub through an animation and see the values and colors of the nodes change with time. Another test is to go to the Preview tab, select the Combiner Edit Mode, and find the slider for one of the Gesture nodes. When you move this slider, the character should move accordingly.

See Also

Character Setup
Speech Gestures


Integrating FaceFX

Integrating FaceFX

Integrating FaceFX with your Game Pipeline There are many different ways to implement talking characters in your game including integrating FaceFX and using FaceFX as a tool. Some FaceFX features are only available if the FaceFX SDK is integrated with your engine. The following section discusses some of the potential ways FaceFX can be used in your game.

1: Using a Commercial Game Engine that Supports FaceFX

FaceFX is currently integrated with Unreal Engine 3 and may support other commercial game engines in the future. If your game is using one of the engines we integrate with, you are in luck! The FaceFX sample rendering engine will be replaced by these engines and artists will be able to create and tweak animations while looking at the character in the target engine.

2: Integrating FaceFX with Your Engine

If you are using an internal game engine or an engine that is not currently supported by FaceFX, you will probably want to integrate the FaceFX SDK in your game. Rest assured, the FaceFX SDK is designed to be as engine-independent as possible as this allows the OC3 Entertainment team to integrate the solution with several commercial game engines. Below are some common integration tasks and issues associated with integrating the FaceFX SDK with your engine. For additional information, check the FaceFX SDK documentation.

FaceFX vs. Native coordinates

When integrating the FaceFX SDK into your engine, one important consideration is the coordinate system your engine supports. By default, all actor files (*.FXA) contain bone poses that are exported into FaceFX’s internal coordinate system which is also used by the sample rendering engine to display content. These can be converted to your engine's coordinate system when the files are loaded by the game or earlier in the pipeline. For additional information, see the Native Coordinates vs. FaceFX Coordinates documentation and the coordsys command documentation. Also check the FaceFX SDK documentation for instructions on how to load FXA files in your game.

Audio Integration

The best way to integrate FaceFX with an audio system is to query the audio system for the exact position in the audio buffer at each frame. Once FaceFX knows the current time in the audio, it is easy to get the correct bone transformations and node values. If the audio system does not support querying the audio buffer, the only other option is to “play and pray” (start the FaceFX animation and the audio at the same time and use CPU timers to estimate the current position of the audio at every frame ).

Bones Integration

Integrating FaceFX with a skeletal system is easy. At every frame you get the time in the animation from the audio, then you ask the FaceFX SDK for the final bone transformations for all the bones in the reference pose. Then you update your skeleton with the bone positions, rotations and scales retrieved from FaceFX.

Morph & Normal Map Integration

Integrating the FaceFX SDK with your engine’s morphing system or normal map system involves getting a node’s value at every frame and feeding that value into your engine’s morphing or normal map system. FaceFX doesn’t animate the vertices for you, it simply provides values from the Face Graph that you can use to drive whatever you want.

Blending FaceFX Animations

One common integration task is to blend into and out of FaceFX animations. FaceFX does not do this operation automatically because it has no knowledge of the other animations playing in your game. FaceFX exposes a Bone Weight interface to support using FaceFX as part of a larger animation system. Every bone in the reference pose of an actor has a weight assigned to it. The default value is 1.0 meaning that FaceFX should completely control the bone’s transform. To modify the actor’s default bone weights, go to Actor->Edit Default bone Weights. Individual animations can override the actor’s bone weight. To modify those weights, select the animation and go to Actor->Edit Anim’s Bone Weights. Using Your Engine in FaceFX Studio Once the FaceFX SDK is integrated with your game, you have the option of replacing the sample rendering engine in FaceFX Studio with your own engine. Some advantages to replacing the sample engine include:

  • The sample engine only has exporters that work with Max and Maya. If you are using any other 3D software you will have to use your own engine to preview content in FaceFX.
  • Using your own engine allows artists to tweak animations while seeing exactly how the character will look in the game.
  • The face graph can be used to drive anything your engine supports (e.g. material properties and normal map transitions). The sample engine supports only a limited feature set, so viewing these properties on your character requires you to replace the sample rending engine.

See Also

FXA Exporters


Performance Considerations

Performance Considerations

Performance Considerations For more detailed information on performance, please consult the FaceFX SDK Documentation. The below information is designed to provide a high-level overview of what can be expected of FaceFX from a performance standpoint.

To describe how FaceFX uses memory, it is necessary to introduce the concept of an actor instance. Every object you can see in your game that is controlled by FaceFX is an actor instance. Actor Instances can share Face Graph information and animation data and can therefore be used to optimize memory usage. Actor instances must share the same face graph, bone poses, and skeleton. See the FaceFX SDK documentation for more information on actor instances.

Memory Usage

FaceFX uses significantly less memory than systems which "bake out" animation data. To further optimize for memory, follow the guidelines below.

Animation Keys

Most memory usage will come from animation keys. Each key has 16 bytes, and a typical animation will have somewhere around 40 keys per second. So a hundred 5-second animations will take up approximately 320 KB of memory.

To minimize memory usage, external animation sets should be used so that only animations that can be potentially played at any given time are loaded into memory. External animation sets can also be shared across multiple actors while only being loaded once in memory.

Nodes

Nodes and links in the Face Graph also use memory. The links are only stored once at the actor level, but every actor instance in your game needs to store a register for each node in the face graph. As a result, the number of nodes in the graph can have a large impact on memory usage, especially if there are a lot of characters on screen at once.

For a ball-park calculation, every register uses roughly 50 bytes of memory. So if your game plans on having 50 characters on screen at once, and your face graph has 50 nodes in it, you will be using approximately 125,000 bytes of memory on Face Graph registers. Reducing the number of nodes in the Face Graph will not only reduce memory usage, but it will also reduce processing requirements as described below.

Processing

At every frame of your game, every FaceFX actor instance needs to calculate a value for every node in the Face Graph and update the bones in the Reference Pose. The amount of processing power FaceFX uses on a next-generation console will typically be negligable unless FaceFX is driving many characters simultaneously.

Bone Blending

While the bone-blending calculations are significant, the only way to speed up bone-blending is to reduce the number of bones in the reference pose and bone poses. In most cases, this is not the best way to optimize for speed.

Nodes

In games that are driving many characters with FaceFX simultaneously, reducing the number of nodes in the Face Graph can significantly improve speed. Be sure to remove unused bone poses and combiner nodes and optimize Face Graph algorithms to use a minimal number of nodes. Keep in mind that the clamped linear link function can often be used in place of a more complicated node setup.

Links

The square root link function is the slowest and should be avoided.

Morph Systems

Using FaceFX to drive an external morph system is no different than driving a skeleton except that FaceFX does not need to do any bone blending or store bone poses in memory. Although FaceFX is doing less work and using less memory, as a whole morph systems typically require more memory and processing than bone systems. The work and memory is simply offloaded to an external morph system.


License

License

License FaceFX is governed by a EULA that is displayed on installation and license agreements executed by OC3 Entertainment. Please see below for license and copright information on the 3rd party software used by FaceFX Studio. Also, please refer to the gpl-3.0.txt and lgpl-3.0.txt files in the FaceFX installation folder for copies of the GNU LESSER GENERAL PUBLIC LICENSE and the GNU GENERAL PUBLIC LICENSE as required by the terms of the GNU LESSER GENERAL PUBLIC LICENSE.

Graphviz

Use: FaceFX includes FxGraphLayout.dll, which includes AT&T's Graphviz, which is used to layout the nodes in the Face Graph.
License: Common Public License Version 1.0
Copyright: © AT&T Corp
Additional Terms: In compliance with this software's license terms, please contact support@oc3ent.com for the source code to this program.

wxWidgets

Use: wxWidgets is used by FaceFX Studio to develop all graphical user interfaces
License: wxWindows Library Licence, Version 3.1
Copyright: © 1998-2005 Julian Smart, Robert Roebling et al

libresample

Use: libresample is used by FaceFX Studio to resample audio to 16 kHz prior to analysis.
License: GNU LESSER GENERAL PUBLIC LICENSE
Copyright: © 1994-2002 by Julius O. Smith III
Copyright: © 2003 by Dominic Mazzoni
Additional Terms: In compliance with this software's license terms, please contact support@oc3ent.com for the source code to this program.

libsndfile

Use: libsndfile is used by FaceFX Studio to load audio files.
License: GNU LESSER GENERAL PUBLIC LICENSE
Copyright: © 1999-2006 by Erik de Castro Lopo
Additional Terms: In compliance with this software's license terms, please contact support@oc3ent.com for the source code to this program.

DevIL

Use: DevIL is used by FaceFX Studio's sample rendering engine to load image files.
License: GNU LESSER GENERAL PUBLIC LICENSE
Copyright: © Denton Woods, Nicolas Weber, Meloni Dario
Additional Terms: In compliance with this software's license terms, please contact support@oc3ent.com for the source code to this program.

OpenAL

Use: OpenAL is used by FaceFX Studio to play audio.
License: GNU LESSER GENERAL PUBLIC LICENSE
Copyright: © OpenAL
Additional Terms: In compliance with this software's license terms, please contact support@oc3ent.com for the source code to this program.

Zlib

Use: Zlib is used by FaceFX Studio to compress audio.
License: ZLIB LICENSE
Copyright: © 1995-2005 Jean-loup Gailly and Mark Adler

FCOLLADA

Use: FxCollada uses FCOLLADA to load and parse COLLADA files.
License: MIT LICENSE
Copyright: © 2006 Feeling Software.

LibXML2

Use: LibXML2 is used by FCOLLADA.
Copyright:Except where otherwise noted in the source code (e.g. the files hash.c, list.c and the trio files, which are covered by a similar licence but with different Copyright notices) all the files are: © 1998-2003 Daniel Veillard. All Rights Reserved. License:Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is fur-nished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FIT-NESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE DANIEL VEILLARD BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CON-NECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Except as contained in this notice, the name of Daniel Veillard shall not be used in advertising or otherwise to promote the sale, use or other deal-ings in this Software without prior written authorization from him.

Fonix

Use: Fonix VoiceIn software is used to extract phoneme times from audio.
License: Fonix VoiceIn software is licensed to OC3 Entertainment under the terms of a confidential license agreement between OC3 Entertainment, Inc. and Fonix Corporation.
Copyright: © Fonix Corporation


Troubleshooting

Troubleshooting

Troubleshooting Having some trouble with FaceFX? Here is a quick checklist of things to try:

Troubleshooting Tips

  • Update your sound drivers! Most crashes that occur when audio is playing or when the application closes are the result of outdated audio drivers or not having OpenAL installed. You can download the latest OpenAL installer here. Even new development machines may require a driver update because of FaceFX's use of OpenAL.
  • Check the logs. FaceFX Studio outputs to the FaceFxStudioLog.html file in the Logs directory. Look for any errors (red) or warnings (yellow). Other FaceFX tools and plug-ins have their own log files located in the installation folder of the parent application.
  • Check the autoexec.fxl file. It is located in the FaceFX Studio install directory. Open it with Notepad or a similar text editor and read the comments to see if there is a common global setting that can be applied to address your issue.
  • Crash reports. When FaceFX crashes, it attempts to send a crash report containing information that tells us the exact line of code that the application crashed on. This information is very useful for helping us improve the product. FaceFX sends this report via email. After a crash report has been generated, please allow your email client to send the report to OC3 Entertainment.
  • Visit our support site. You can see what questions other developers have asked and ask new questions that will benefit the entire FaceFX community.

Known Issues

Crashes

  • Crash on Start - pBuffer - Viewports in FaceFX Studio workspaces rely on the graphics card's pbuffer, but not all graphics cards support this feature. Worse still, some graphics card drivers report that they support the pbuffer, then crash when FaceFX Studio tries to use it. There is no way to detect or avoid this problem, so FaceFX Studio crashes on these machines. To identify this issue, open the FaceFX Studio log. The last entries warn that FaceFX Studio is allocating space for the pbuffer.
  • Crash on Open File - Maya Shader Issue - If FaceFX Studio crashes when you open a file, and you exported the FXR file from the Maya FXR Exporter, it is most likely due to custom shaders present during export. Try deleting the FXR file and opening only the FXA and FXS files to confirm that the FXR file is causing the crash. If the FXA and FXS file open up without a problem, then try turning off all custom shaders and re-exporting the FXR file.

Other Issues

  • Maya FXR Exporter goes to Bind Pose - When exporting an FXR file from Maya, the skeleton is put into the bind position for export and left there. Close and reopen the file to undo the change. If the character is deformed incorrectly during export, then there was a problem putting the skeleton into the bind position. Make sure bone attributes are not locked, IK is turned off with Modify -> Evaluate Nodes ->Ignore All, and there is only one bind pose.
  • Command Line Mode Always returns -1 - When using FaceFX Studio as a command line tool with the exec command, the operation always returns -1 as opposed to returning 0 on successful completion.
  • Out of Sync FXS/FXA files - Failure to keep the FXA and FXS files in sync (by deleting the FXS file, renaming one of the files, or checking in or checking out the FXA and FXS files separately from a version control system ) can lead to a number of problems including getting an error when adding curves. To avoid these issues, keep the FXA and FXS files in sync.


Contact Us

Contact Us

Contact Us OC3 Entertainment is very proud of the FaceFX product, but we can’t make it better without brutally honest feedback from our users. Most of the new features in FaceFX are the results of a handful of developers (you know who you are) who told us what they would like to see in a facial animation product. Imagine how much better the product could be if all of our users gave us feedback. So if something in FaceFX is driving you nuts or you could really use a new feature, send an email to support@oc3ent.com or let us know at http://support.oc3ent.com. Support inquiries are equally important to OC3 Entertainment, and supporting your game or application is our top priority. If you find a bug in FaceFX, let us know. If you are just confused about something, please send an email. We will be able to save you time and gain valuable feedback about where users encounter problems with our application.

OC3 Entertainment, Inc.
support@oc3ent.com
http://support.oc3ent.com
2 Davis Drive
PO Box 13169
Research Triangle Park, NC 27709
Tel. (919) 767-0626
Fax. (919) 287-2392