Fully Homomorphic Encryption: Google Open-Sources HEIR Compiler for Private AI

For more than a decade, confidential cloud computing faced an unyielding mathematical compromise: data could be secured during transit and at rest, but evaluating machine learning models on a remote server invariably required decrypting that data in memory. On August 14, 2026, Google altered this paradigm by open-sourcing HEIR (Homomorphic Encryption Intermediate Representation) under the Apache-2.0 license as a core pillar of its Private Computing Toolkit. By providing a unified, multi-layered compiler toolchain, HEIR turns the promise of fully homomorphic encryption (FHE) into an automated software engineering reality, allowing pre-trained artificial intelligence models to execute inference directly over encrypted data without the host server ever decrypting the underlying inputs.

Demystifying Fully Homomorphic Encryption: Computation Over Ciphertext

Traditional confidential computing models rely heavily on hardware-based Trusted Execution Environments (TEEs) or complex multi-party computation (MPC) protocols. While enclaves isolate memory at the silicon level, they remain susceptible to side-channel exploits, hardware bugs, and sovereign access subpoenas. In contrast, fully homomorphic encryption relies strictly on mathematical hardness assumptions—predominantly lattice-based cryptography, such as Ring Learning With Errors (RLWE).

FHE enables arbitrary arithmetic operations to be performed directly on ciphertexts. When an encrypted input is processed through an FHE-compiled model, the output is another ciphertext that, when decrypted by the client with their private key, yields the exact result of the computation. The server acts as a blind executor: it performs millions of polynomial multiplications and additions without ever gaining access to the plaintext values.

Historically, building software on top of FHE required dedicated teams of PhD-level cryptographers. Developers had to manually manage complex mathematical artifacts, including:

• Noise Budgets: Every homomorphic operation injects mathematical noise into ciphertexts; exceeding the noise threshold results in corrupted, undecryptable plaintext.
• Parameter Selection: Finding secure polynomial modulus degrees ($N$) and coefficient moduli ($Q$) that balance 128-bit quantum security against computation latency.
• Vector Packing and Rotations: Mapping multi-dimensional neural network tensors into cyclotomic polynomial slots (SIMD-style execution) without triggering catastrophic rotation latency.
• Scheme-Specific Dialects: Hand-tailoring code to fragmented libraries with disparate APIs.

HEIR eliminates these manual barriers by treating cryptographic transforms as standard compiler optimization problems.

The Architecture: MLIR Dialects and Progressive Lowering

At the center of HEIR’s design is the Multi-Level Intermediate Representation (MLIR) framework, an extensible compiler infrastructure originating from the LLVM project. Rather than forcing AI models into a monolithic cryptographic transpiler, HEIR introduces a hierarchy of domain-specific MLIR dialects that progressively lower high-level tensor operations down to raw algebraic instructions.

Multi-Scheme Cryptographic Coverage

Modern cryptographic inference does not rely on a single algorithm; different operations require distinct mathematical schemes. HEIR natively manages multiple primary FHE scheme families:

• CKKS (Cheon-Kim-Kim-Song): Tailored for floating-point arithmetic and continuous deep learning models, enabling approximate evaluation of matrix multiplications, convolutions, and polynomial activation functions.
• BGV (Brakerski-Gentry-Vaikuntanathan) and BFV (Brakerski-Fan-Vercauteren): Optimized for exact integer arithmetic, making them ideal for tabular data, discrete lookups, and exact logical rule engines.
• TFHE (Torus FHE) and FHEW: Designed for gate-by-gate programmable bootstrapping, allowing fast evaluations of arbitrary boolean circuits and non-linear step activations.

Compiler Passes and Arithmetization

HEIR includes dozens of custom optimization passes that bridge high-level model structures with low-level ring arithmetic. When a neural network graph enters the compiler, HEIR carries out automated arithmetization: it replaces non-polynomial activation functions (like standard ReLU or Softmax) with calibrated Chebyshev polynomial approximations or table-lookups. From there, the compiler optimizes the layout of ciphertext vectors using Halevi-Shoup matrix-vector multiplication strategies, drastically minimizing expensive cyclic rotations.

Once high-level tensor operations are mapped to homomorphic primitive dialects, HEIR performs Residue Number System (RNS) decomposition. It breaks arithmetic over multi-hundred-bit moduli into parallel channels of native 64-bit integer operations, unlocking maximum CPU and GPU throughput before generating code for downstream runtimes.

Developer Workflow: Democratizing FHE with Python

One of HEIR’s most significant contributions is lowering the developer barrier to entry. With the release of heir_py via PyPI, machine learning engineers do not need to rewrite their codebases in low-level C++ or write manual cryptographic circuits. Instead, developers can write idiomatic Python functions, applying lightweight type annotations to designate confidential data paths.

By declaring arguments with types such as secret[T], developers trigger HEIR’s automated pipeline. The compiler analyzes the dataflow graph, identifies which operators interact with ciphertexts, calculates the maximum multiplicative depth, automatically configures the appropriate noise management strategies (such as Modulus Switching or Bootstrapping), selects compliant 128-bit security parameters, and outputs an executable binary.

Interoperability: Backends and Hardware Acceleration

HEIR does not force developers into a proprietary runtime. Instead, it acts as a universal compiler hub that targets widely adopted open-source FHE backends:

1. OpenFHE: The multi-institution C++ library that implements high-performance CKKS, BGV, and BFV schemes.
2. Lattigo: A Go-based lattice cryptography library known for efficient multi-party extensions and high concurrency.
3. tfhe-rs: Zama’s Rust-based implementation of fast Torus FHE for boolean and short-integer computation.
4. Jaxite: Google’s open-source, JAX-native FHE library designed to accelerate homomorphic primitives across GPU and TPU clusters.

Crucially, HEIR provides an abstraction layer specifically engineered for custom silicon. As hardware accelerators for homomorphic encryption mature—including specialized ASICs, FPGAs, and optical processors designed by startups such as Niobium, Belfort, Cornami, and Optalysys—HEIR’s intermediate representation enables hardware vendors to write a single code-generation backend rather than building custom compilers from scratch.

Reference Deployments: Proving Practical Utility

To demonstrate that encrypted inference is ready for real-world tasks, Google published four reference implementations compiled entirely through HEIR:

• Privacy-Preserving Financial Fraud Scoring: Financial institutions can send encrypted transaction features to a centralized anti-fraud engine without exposing proprietary customer balances, merchant IDs, or locations.
• Zero-Visibility Network Intrusion Detection: Enterprise security appliances evaluate incoming encrypted packet signatures using trained classification trees without inspecting payload plaintexts.
• Client-Side Voice Hotword Recognition: Edge devices encrypt voice embeddings and send them to cloud models for trigger-word verification without broadcasting cleartext acoustic data.
• Confidential Content Recommendation: Streaming platforms deliver personalized recommendations by running matrix factorization over encrypted user preference vectors, preventing the service from logging individual user interests.

Challenges, Trade-Offs, and the Future of Confidential AI

While HEIR marks a massive leap forward, homomorphic AI inference still requires deliberate architectural discipline. Ciphertext operations entail severe memory and computational overheads compared to standard floating-point operations. A ciphertext vector can expand data size by factors of $10\times$ to $50\times$, and homomorphic multiplications remain computationally heavy.

Furthermore, because FHE arithmetic cannot branch conditionally on encrypted values without evaluating every possible code branch simultaneously, models must be adapted to branchless, fixed-depth architectures. HEIR handles these structural transformations automatically, but inference models must still be tuned for multiplicative depth to avoid excessive latency.

Despite these trade-offs, HEIR fundamentally redefines the privacy-utility balance. By shifting cryptographic orchestration out of the hands of specialized researchers and into an automated MLIR compiler, Google has established an open standard for privacy-first AI. As hardware acceleration bridges the remaining latency gap, HEIR stands poised to serve as the foundational toolchain powering the next generation of zero-trust cloud intelligence.