Terracotta: Git to Perforce Seamlessly Bidirectionally

I coded Terracotta because I was tired of watching engineering teams waste days on manual Git-to-Perforce migrations that should take hours. After migrating my third monorepo by hand and dealing with the inevitable file type corruption issues, I realized the existing tools weren't solving the real problem: they treated migration as a one-time event rather than an ongoing bidirectional workflow. Terracotta emerged from the need for a system that could handle continuous synchronization between Git and Perforce without requiring teams to choose one or the other.

This document describes the technical implementation of converting Git repository structures to Perforce depot architectures. Git operates on a commit-based distributed version control model with branch-based workflows. Perforce implements a centralized changelist-based system with stream-oriented repository structures. These architectural differences require systematic translation mechanisms.

Implementation Tools: P4Python and GitP4Transfer

The migration implementation utilizes two primary components. P4Python serves as the official Python API for Perforce server interaction, providing programmatic access to all Perforce operations with structured exception handling and typed data structures. GitP4Transfer functions as the migration utility that handles commit-to-changelist conversion while preserving repository history and metadata.

GitP4Transfer handles the mapping between Git's SHA-1 commit identifiers and Perforce changelist numbers, manages file mode translation, and performs repository structure conversion. P4Python enables custom validation and migration scripts with direct Perforce server communication without command-line parsing overhead.

Bidirectional Workflow

Terracotta enables continuous synchronization between Git and Perforce repositories. The following terminal output demonstrates a typical bidirectional workflow where changes flow seamlessly between both systems:

The workflow shows submitted changes to the main branch, pending changes from the dev branch using p4 interchanges, automatic merging with p4 merge, and conflict resolution with p4 resolve. This demonstrates how Terracotta maintains synchronization between Git commits and Perforce changelists while preserving branch relationships and merge history.

Perforce Typemap Configuration

Perforce requires explicit file type declarations through typemap configuration. The typemap determines storage format, line-ending normalization, keyword expansion, and merge behavior based on file paths and extensions.

Technical Requirements

Git implements content-agnostic storage with minimal file type differentiation. Perforce requires explicit type declarations to prevent data corruption during storage operations. Incorrect type assignment results in binary files marked as text experiencing line-ending conversion that corrupts binary data, while text files marked as binary lose merge operation availability.

Repository Analysis Results

Repository analysis identified multiple file type categories requiring distinct handling. Source code files require standard text processing. Binary executables require exclusive locking. Shell scripts with inconsistent line endings present cross-platform compatibility issues. Data files with extension-based ambiguity require context-dependent type assignment. Image assets with non-standard extensions and vendor libraries with mixed binary and configuration files add additional complexity.

Typemap Implementation

Typemap:
    binary+l //depot/....exe
    binary+l //depot/....dll
    binary+l //depot/....so
    binary+l //depot/....a
    binary+l //depot/....lib
    binary+l //depot/....dylib
    binary //depot/....png
    binary //depot/....jpg
    binary //depot/....gif
    binary //depot/....ico
    binary //depot/....pdf
    binary //depot/....zip
    binary //depot/....tar.gz
    text+l //depot/....c
    text+l //depot/....cpp
    text+l //depot/....h
    text+l //depot/....py
    text+l //depot/....sh
    text+l //depot/....xml
    text+l //depot/....json
    text+l //depot/....config
    binary+F //depot/.../data/....dat
    text //depot/.../configs/....dat

Type modifiers include +l for exclusive lock on checkout to prevent concurrent binary modifications, and +F for full revision storage to disable delta compression.

Implementation Constraints

Perforce processes typemap rules in reverse order with bottom-to-top evaluation. Perforce wildcard patterns using three dots represent arbitrary directory depth, distinct from regex or glob syntax. Type changes do not apply retroactively to existing revisions and require p4 reopen -t for each affected file. GitP4Transfer implements conservative binary classification that requires manual override for text files. Extension-based ambiguity necessitates path-based rules for files with identical extensions but different content types.

Validation Implementation

File type verification script using P4Python API:

from P4 import P4

p4 = P4()
p4.connect()

problem_files = []
for file_info in p4.run_files("//depot/..."):
    file_type = file_info['type']
    file_path = file_info['depotFile']
    
    # Validate type assignments
    if file_path.endswith('.exe') and 'binary' not in file_type:
        problem_files.append(f"{file_path} is {file_type}, should be binary")
    elif file_path.endswith('.txt') and 'text' not in file_type:
        problem_files.append(f"{file_path} is {file_type}, should be text")

for problem in problem_files:
    print(f"Warning: {problem}")

This validation identifies type mismatches before production deployment.

Verification Script Implementation

Post-migration verification requires systematic comparison of Git repository state against Perforce stream state. The verification script validates file existence, revision number consistency, and depot path mapping.

set -euo pipefail

export P4PORT="ssl::1667"
export P4USER="assembla"

git_repo_path="/opt/perforce/servers/git/SA-Zim-ROSAHip"
p4_stream="//ossoart/stream_test_1"

total_files_checked=0
files_matched=0
files_not_matched=0

# Iterate through files and verify against Perforce
while IFS= read -r -d '' local_file; do
    p4_stream_path="$p4_stream/$local_file"
    p4_output=$(p4 files "$p4_stream_path" 2>&1 || true)
    
    if [[ $p4_output == *"no such file"* ]]; then
        ((files_not_matched++))
    else:
        ((files_matched++))
    fi
    
    ((total_files_checked++))
done < <(find "$git_repo_path" -type f -print0)

# Generate statistics
echo "Files checked: $total_files_checked"
echo "Files matched: $files_matched"
echo "Match rate: $((files_matched * 100 / total_files_checked))%"

The verification algorithm iterates through all files in the Git repository using null-terminated file lists for filename safety. For each file, it extracts revision numbers from filename metadata, constructs corresponding Perforce depot paths, and queries the Perforce server for file existence and revision data. Files are classified into matched, mismatched, or missing categories, with associative arrays tracking mismatches by file extension for targeted remediation.

As you can see, it's across the board checking for diffs, file sizes, etc. The horizontal bar chart shows the distribution of mismatched files by extension type, providing immediate visibility into which file categories require typemap adjustments or manual review.

Technical Conclusions

Git and Perforce implement fundamentally different version control models requiring systematic translation. P4Python and GitP4Transfer provide necessary abstraction layers for migration implementation. File type configuration directly impacts data integrity and requires comprehensive testing. Automated validation scripts are essential for migration success verification. File extension patterns in mismatch reports enable targeted typemap refinement. Null-terminated file lists prevent parsing errors with whitespace-containing filenames. Typemap validation on repository subsets reduces risk of large-scale data corruption.