Advanced Unity Serialization

TL;DR
This is a quite long post about issues and solutions regarding Unity’s serialization system. If you would like to skip reading this, please go and vote here: Vote for Polymorphic Serialization. There’s a quick summary of the issue there.

Introduction
I have been playing around with the new PropertyDrawers in Unity 4.0. If you do any sort of editor customization (or would like to), you should definitely look into them. They expose new possibilities in terms of complex setup and user interaction, and finally give me a chance to revisit a framework design I had wanted to do for ages but couldn’t due to limitations in the old CustomEditor system. I was about two weeks into the design of my new awesome framework when I finally noticed a bug which would criple its usage. Unity still doesn’t serialize polymorphic types properly. I spent hours upon hours figuring out solutions until I finally solved my issue and I wanted to share my findings. In this blog post, we’re going pretty deep into Unity serialization and customization. Let’s get started!

The problem I encountered and wanted to solve was that Unity does not serialize polymorphic types properly. There’s a lot of big words there, so let’s look at an example:

[Serializable] class Base { public int myInt; }
[Serializable] class Derived : Base { public string myString; }
 
class MyBehaviour : MonoBehaviour
{
   public Base  _myBase = new Derived(); // Error... This will NOT work, only myInt will be saved.
}

The issue is that Unity’s serializer looks at the field type (in this case Base), then serializes only what’s in Base. The result is that the Derived instance gets ‘sliced’, losing the functionality that makes it special, and saving only the Base functionality. Don’t believe me? Here’s what it looks like:

Derived Gets Sliced to Base

Now you’re probably thinking to yourself “Aha! You didn’t read the Unity Serialization Blog, they have an answer to this!” Well that’s true, the blog post is available here: http://blogs.unity3d.com/2012/10/25/unity-serialization/. Now, near the bottom of it they say that simply deriving from ScriptableObject will solve this slicing issue, and that’s true… in fact I had gone two weeks without realizing the subtle issue it causes.

Issues With ScriptableObject

In the example above, switching to ScriptableObject looks like this:

class Base : ScriptableObject { public int myInt; }
class Derived : Base { public string myString; }
class MyBehaviour : MonoBehaviour
{
  public Base _myBase;
   
  // This gets called in the Editor when you first create this MonoBehaviour.
  void Reset()
  {
    _myBase = ScriptableObject.CreateInstance<derived>();
  }
}

Now, this will actually work, you can now get a proper instance saved and loaded. For reference, here’s what it looks like. Note: since I’m doing a really simple example, you’ll have to double-click the object field to see the values are working and you actually have a Derived reference instead of Base.

ScriptableObject Does Not Get Sliced!

The issue now becomes when you start trying to make prefabs. You see, ScriptableObjects can only live in the scene or in the project as an asset. In this design, you expect the MyBehaviour to “own” the ScriptableObject, but it doesn’t, the scene does. When you drag a GameObject over to create a prefab from it, you don’t get the ScriptableObject with it (it lives in the scene). Furthermore, this code won’t create a ScriptableObject for you if you “Reset” your MyBehaviour through the inspector on your prefab, because they just can’t live in a prefab. Here’s what it looks like in the prefab:

ScriptableObject Cannot Exist in Prefabs

This means any configuration you do with ScriptableObjects must remain in the scene. Also, since they’re references, when you duplicate any object that references a ScriptableObject you get a REFERENCE to the original one, not a new one. So forget about keeping ‘prefab’ objects around in your scene to get this design to work, you’re hooped.

Frustration

I looked everywhere for a solution, and vented on the Unity IRC Channel (a great resource). MattMaker was kind enough to hear me out and explain exactly what was going on. Searching on google only resulted in others with the same issue, and particularly explained well here: http://helmutduregger.wordpress.com/2012/05/18/unity-serialization-with-scriptableobject-does-not-support-polymorphisminheritance/.

This is a huge issue for my design where I want things to behave polymorphically depending on how they are configured. I saw no way around this: The only way to avoid slicing is to use a MonoBehaviour or a ScriptableObject, and ScriptableObjects do not survive into prefabs. This left me with just MonoBehaviours, but they need to be attached to GameObjects and can really confuse a design. Especially when you have multiple instances on the same object; it’s impossible to tell which MonoBehaviour belongs to which reference. Then I remembered: you can hide MonoBehaviours.

Solution Summary

The solution, although very complex, would be to add a MonoBehaviour that just acts as a normal class (no special Unity calls like Update). This sounds really easy to do, but it turns out it’s extremely difficult to work around the edge cases. So let’s do that now:

class Base : MonoBehaviour { public int myInt; }
class Derived : Base { public string myString; }
class MyBehaviour : MonoBehaviour
{
  public Base _myBase;
  void Reset()
  {
    _myBase = gameObject.AddComponent<derived>();
    _myBase.hideFlags = HideFlags.HideInInspector;
  }
}

This looks simple enough, right? Well there’s already a ton of issues this causes. I’ll list them first then go through solving each one:

1. When you copy this component using Unity 4.0′s new feature of copy & pasting components, both MyBehaviours will point to the same Derived object, not what you want (you want a sepearate instance so you can configure the two separately). This is related to:

2. Reset() is called whenever an object is added to the scene, but before Serialization. Say you add a MyBehaviour and _myBase is referencing Derived1. When you copy & paste a MyBehaviour (new feature in Unity 4.0), you will create another Derived component on the gameObject, let’s call this Derived2. Then serialization of the copy occurs, and _myBase gets assigned to Derived1. You now have a Derived2 on the object that nothing is using.

3. When you remove this MyBehaviour, the Derived component will sit on your object until you remove it.

Solving Issues #1 & 2 – Instancing

Since Unity has no instancing flag, we’re going to need to write instancing ourselves. Here’s some code:

void DoInstancing()
{
  var oldObject = _myBase;
 
  // Create the new monobehaviour of the correct type
  System.Type polymorphicType = oldObject != null ? oldObject.GetType() : typeof(Derived);
  var newObject = gameObject.AddComponent( polymorphicType ) as Base;
 
  // Copy over the data from the old object to the new one
  if ( oldObject != null )
    EditorUtility.CopySerialized( oldObject, newObject );
   
  _myBase = newObject;
  _myBase.hideFlags = HideFlags.HideInInspector;
}

So when do we call this function? The issue is that we get no notification from Unity that an object is copied, we’re stuck with just the Reset() call on object creation. We can solve this by delaying our call from Reset() using some editor trickery.

void Reset()
{
#if UNITY_EDITOR
  // Delay this call so it will happen after copy serialization
  UnityEditor.EditorApplication.delayCall += DoInstancing;
#endif
}

Solving Issue #3 – Manual Reference Counting / GC

So now we’ve solved the instancing issue, how do we deal with all of the old invalid data laying around? If your instincts are anything like mine, you would want to Destroy() the Base/Derived data whenever MyBehaviour is destroyed. The issue with this is that you do not get callbacks from Unity whenever an object is destroyed … unless you tag the object with [ExecuteInEditMode]. Unfortunately, you do not get those callbacks for objects in Prefabs, causing your Prefab edits to leak objects. This is not a solution I could live with, so I needed to improvise yet again. The solution I came up with was to manually reference count

You stick a reference to the owning MonoBehaviour inside of “class Base” (called Owner). Then upon saving any asset or scene, you search for all instances of Base and DestroyImmediate() any that do not have a reference to an Owner. Again, this is not ideal, but will ensure you do not have leaked objects in your scenes/prefabs. Some quick code:

public class BaseDataProcessor : UnityEditor.AssetModificationProcessor
{
    /**
    * When we're about to save an asset, make sure that asset does not contain bad data
    */
    public static void OnWillSaveAssets( string[] assets )
    {
        foreach( string asset in assets )
        {
            // Check prefabs
            if ( asset.EndsWith(".prefab") )
            {               
                Object[] allObjs = AssetDatabase.LoadAllAssetsAtPath( asset );
                CheckList( allObjs.OfType<base>() );
            }
            else if ( asset.EndsWith(".unity") )
            {
                // If we're saving a level file, assume it's the current one... so process the current scene.
                Base[] allObjs = GameObject.FindSceneObjectsOfType( typeof(Base) ) as Base[];
                CheckList( allObjs );
            }
        }
    }
 
    /**
     * Check an object list for bad data
     */
    private static void CheckList( IEnumerable<base>    dataObjs )
    {
        foreach( var obj in dataObjs )
        {
            if ( obj.Owner == null )
                GameObject.DestroyImmediate( obj, true );
        }
    }
}

Conclusion

So there you have it! Proper serialization of instanced polymorphic types that can live in prefabs. There are some other minor issues you will have to deal with on your own, such as the hideFlags not surviving the prefab process. If you’ve made it to the end, you can see this is quite a complex process for something that should really be stock-engine functionality. Agree with me? PLEASE vote for solving this issue here: Vote for Polymorphic Serialization. I know Unity must be rewriting their serializer to support nested prefabs, so having them solve this issue may not be quite as far fetched as one might think.

Leave a Comment