Transaction-Based Pipeline Programming – [C#]

posted in: Uncategorized | 0

Overview

Today we’ll be exploring the transaction programming pattern, and its relevance to developing an asset pipeline. A transaction can be thought of as a specific subset of the command design pattern, but with specific characteristics. In this post, I’ll be defining transactions in terms of three specific components: an executable action, a means to handle failure or error, and a post-action or “clean-up” step.


Case Example

Over time, I’ve encountered the need for this type of behavior all over the place. In particular, during the vast majority of asset export and import processes, I’ve made use of this pattern to prevent disasters. Before we dive deeper into the technical implementation of our transaction code, let’s think in terms of a practical example. Start out by imagining how a typical implementation of an export process might behave:

    Acquire all data to export
    Determine asset dependencies
    Process the asset  # (i.e. clean-up, meta-data, etc.)
    if (the export path doesn't exist):
       Create the output directory
    else:
        Delete the previous asset
    Copy/save the asset to the output path



This works all fine and well until, for whatever reason, some part of the export process fails. In a worst possible case, this might happen: you’re handling an object whose scene name contains a character not valid in Windows file paths. You are smart and determine the export path based on the object’s name but forget to perform path validation logic before trying to copy it and your program throws an exception. At this point, you’ve already deleted the previous working asset and haven’t even replaced it with the newly exported version.. what a mess!

In another case, the exporter might fail during a processing or conditioning step. Now you’ve left the artist’s scene in some unstable state. You’d better make sure the artist is immediately made aware that the exporter failed and hope they don’t save the half-exported asset before closing it. Have fun explaining how to fix the asset for export when the original file has been saved over.


Refactor Attempt #1

As you can see, this type of asset export processing is very dangerous the instant something occurs unexpectedly. Thankfully, there are some defensive programming techniques in our toolbox to help handle the worst of surprises. Exception handling seems like just the right fit for solving these issues! We can wrap up our export logic into a try clause and catch any exceptions that get thrown! Let’s look at how this new and improved exporter might function:

    try:
        Acquire all data to export
        Determine asset dependencies
        Process the asset (i.e. clean-up, meta-data, etc.)  
        if (The export path doesn't exist):
           Create the output directory
        else:
            Delete the previous asset
        Copy/save the asset to the output path     
    catch Errors:
        # Uh oh something went wrong!
        Close the current operating file without saving
        Display a pop-up window to the user with some error log

We’ve now safeguarded against any unexpected failures in the export process. Furthermore, we can cleanly abort the process and even display an informative log to the user! What’s cooler yet, is how we to take this a step further.

A Tangent on Sandboxing

Even though we’re failing beautifully, we’re still leaving the artist frustrated with a closed scene and likely some highly technical gibberish about what went wrong. To clean things up more, we’ll employ a sandboxing technique. Rather than operating directly on asset in question, we’ll insert an additional step at the beginning of our export process that saves a copy of the asset off to a temporary disk location so that no matter how bad things get, we won’t be touching the artist’s original work. At the end of our export process, we can even clean up this sandbox file to prevent hard drive clutter. We can ensure this clean-up step happens no matter what by tacking on a finally clause to our process. The code within this block will execute regardless of whether or not the export completes successfully!

    try:
        Create sandbox copy of original file
        Do all of the exciting export things...
    catch Errors:
        # Uh oh something went wrong!
        Close the current operating file without saving
        Display a pop-up window to the user with some error log
    finally:
        Delete sandbox copy

We’ve come a way now from our original implementation of the exporter and I bet you’ve been waiting out for some real code this whole time.


The Transaction Class

Remembering to implement all this try, catch, finally bulk around your code can clutter things up and makes re-organizing your actual logic dependent on fitting into these blocks. To abstract this process and genericize it into an easy-to-implement interface, I created the Transaction class designed with the intent that one could refactor old code into the transaction pattern and change logic out on the fly without worrying about the syntax blocks. Here’s the implementation in C#:

using System;

class Transaction : ITransaction
{
    readonly private Action doThis;
    readonly private Action handling;
    readonly private Action cleanUp;

    public Transaction(Action inDoThis, Action inHandling, Action inCleanup)
    {
        if (inDoThis == null)
        {
            throw new System.Exception("Can not assign a null function to the primary action of a transaction.");
        }

        this.doThis = inDoThis;
        this.handling = inHandling ?? this.nullMethod;
        this.cleanUp = inCleanup ?? this.nullMethod;
    }

    private void nullMethod() {}

    public void Execute()
    {
        try
        {
            doThis();
        }
        catch (System.Exception e)
        {
            handling();
        }
        finally
        {
            cleanUp();
        }
    }
}

The Transaction object requires three methods that don’t have return values. You’ll notice I do some additional null guard checking and override null values passed in for the second and third parameters with an empty function. Without these C# will throw a null reference when you try to call a null Action! For the first Action, I throw an exception if the user tries to pass in null since it doesn’t make much sense for a transaction to exist without a core action to execute.

I made the decisions to use C#’s System.Action delegate arguments for this implementation and an Execute method that returns void to constrain how the class is used. Rather than expecting some returned value from the transaction object, the user should typically be passing in methods that operate on existing objects and their internal data members. This contract is set in place by this brief ITransaction interface:

interface ITransaction
{
    void Execute();
}



With our Transaction class in hand, we’re ready to write some awesome pipeline tools! Let’s take a look at how we might use it in action. First we define our asset exporter class. I’ve nealty defined three class methods that wrap all the fancy busywork of exporting the asset. We can directly plug these into our Transaction class.

    class AssetExporter
    {
        public void DoExport()
        {
            // Create a working copy of the asset, clean it up, etc...
        }

        public void OnExportFailed()
        {
            // Display a pop-up window to the user...
        }

        public void CleanUp()
        {
            // Delete temporary assets and return to the existing state...
        }
    }

To execute the exporter, we build the Transaction with AssetExporter’s methods and call its main Execute method!

class Program
{
    public void Program()
    {
        AssetExporter exporter = new AssetExporter();
        Transaction exportTransaction = new Transaction(exporter.DoExport, exporter.OnExportFailed, exporter.CleanUp);
        exportTransaction.Execute();
    }
}

The logic to implement transaction-style processing is contained to two lines of code, making it simple to inject this pattern into existing work and to experiment in new code.


Outcome

The Transaction pattern provides a contract with which to operate and recover when things to unexpectedly. The use of language exception handling utilities simplify our work and can easily be contained in a Transaction Object that allows us to add and remove transaction-based logic wherever it is needed. This type of code flow is especially useful in asset processing or file authoring of any sorts.


Next time, we’ll be using this transaction style of programming to build a slick object-oriented Unity import pipeline to precisely manipulate import behavior!

Leave a Reply