In-Depth: Builders And Syncing And Artists Oh My!

[In this reprinted #altdevblogaday in-depth piece, Uber Entertainment software engineer Forrest Smith explains how to implement a "magic sync button" to solve mismatched syncing issues.] Let me tell you a story. You're at the office when someone yells "Build's broken!" D'oh! After poking around for a few minutes, you figure it out. The artist synced to latest, but the build machine is still running so they have mismatched content and executables. Everyone wasted their time, and when the builder finishes in a few minutes, things magically work. Familiar tale? Fortunately there's a simple and slick way to handle it. I'll describe the process in terms of Perforce, but the concept applies to any source control system. Proper Syncing The problem is that our poor artist hero synced new content and script before the matching executable was checked in. For anyone who builds their own EXEs, there isn't an issue. Assume that the builder takes an hour to run. The "good" build is what the builder started with plus what the builder outputs. Any check-ins made during that hour build process should be ignored until the next build. If the artist can sync to builder_input + builder_output, then they will NEVER have code, script, data, or EXEs that don't work properly together¹. We're going to create a way for a user to easily sync to exactly that. Magic Sync Button At my last job, we took things a step further and made a magic sync button. It was a giant glowing button at the top of our one-stop-shop game and tool launcher utility. The magic sync button did a few things. First, clicking it would sync to our latest builder result. Second, it turned yellow if there was a new builder result to sync. Third, it would automatically save progress in any open editor, close the editor, sync the new editor EXE, relaunch, and reopen files/levels/etc. It worked so well that designers and artists could sync almost exclusively through the magic sync button². All tools had checkout/submit integration with Perforce so p4v was needed only to resolve conflicts. The Dirty Details Sounds great. How do you do it? Short answer, via p4 label and p4 tag. Longer answer:

That's not so bad! It's honestly just a small handful of safe, non-destructive p4 calls. Let's dig deeper. 1 – Create New Label Command: p4 label magic_sync_label 2a – Builder Get Latest Changelist Get the single most recent submitted changelist. Your builder will need to capture the output and parse out the changelist which is stored as builder_input_changelist. Command: p4 changelists -s submitted -m 1 Example Output: p4 change 7238 on 2011/12/05 by some_user@some_workspace 'words words submit notes words' Note: Store 7238 in builder_input_changelist. 2b – Builder Sync Latest Changelist Sync the builder to builder_input_changelist. Command: p4 sync //depot…@7238 3 – Builder Builds and Submits The builder can build and take as long as it wants. When it's done and submitted find the changelist and store it as builder_output_changelist. Command: p4 changelists -s submited -u builder_user -m 1. Example Output: p4 change 7251 on 2011/12/05 by builder_user@builder_workspace 'words words words' Note: Store 7251 in builder_output_changelist. 4 – Builder Tags Label This is the step that enables the magic. Assume the following directories for a simplified example:

We want to associate Source/RawData with builder_input_changelist (7238) and CookedData/Exes with builder_output_changelist (7251). Critical note, this is done with a single atomic tag command all in one line. Command: p4 tag -l magic_sync_label //depot/Game/Source…@7238 //depot/Game/RawData…@7238 //depot/Game/CookedData…@7251 //depot/Game/Exes…@7251 5a – Press the Magic Button You can have a magic button, a batch file, or simply sync to label through p4v. The double label in the command is peculiar p4 syntax that preserves workspace files not part of the label. Command: p4 sync @magic_sync_label,@magic_sync_label 5b – Checking for Available Update How can the magic button check to see if an update is available? The -n symbol means preview only. If text returned by the following command ends with "- file(s) up-to-date." then the user is fully synced to the label. Any other text means there are files to update. Command: p4 sync -n @magic_sync_label,@magic_sync_label Bonus Thoughts No operation listed is destructive. If users have local changes to a file within the magic_sync_label set a standard conflict will occur that can be resolved. You can use -n on all commands for preview testing purposes. Conclusion Syncing to a builder produced label is awesome and I highly recommend it. It's easy to do and in highly volatile in-development environments it can make a world of difference. I'd like to end this post with an open question, how do users sync in your office? Have any cool tricks or setups to share? Discuss in the comments below! Footnotes

[This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]