Inside the Compiler

Compilation normally runs inside ANECompilerService.xpc, a sandboxed helper process. Core ML and the _ANEClient API reach it over XPC. The service holds separate compiler classes for different model types (MIL, MLIR, Core ML, ANECIR, CVAIR, and an Espresso translator) and picks one based on the input format. Every trace in this article follows the MIL route.

There is also a direct path. The same ANECompiler.framework can be loaded into any process and called through its exported ANECCompile function. That is how this investigation ran: in-process, with LLDB attached, so every symbol in the framework was reachable.

ANECompiler.framework links MIL, Accelerate, CoreFoundation, CoreAnalytics, libc++, libSystem, and ncurses. No LLVM or MLIR dynamic library is in that list. The framework has a large body of upstream LLVM and MLIR symbols, but they are all statically linked into the binary.

One more library loads on demand: libORTools.dylib, Google's operations-research solver. The framework resolves three entry points from it, all constraint-programming (CP) solvers: an allocator, a transposer, and a memory-cache allocator.

When does libORTools get pulled in? A plain convolution and serial convolution chains did not load it. A graph with two independent convolution branches did, and called the external transposer and CP allocator. Serial chains of every tested size stayed on the internal solver, so the trigger appears to be graph topology or placement pressure rather than graph size.

The bundle's DTCompiler = com.apple.compilers.llvm.clang.1_0 records the toolchain Apple used to build the framework. That is just the build tool. The framework has none of the Clang front-end classes (no ASTs, no source management, no code generation).

The framework does contain a complete MLIR stack: seven MLIRContext constructor sites, mlir::parseSourceFile, four mlir::PassManager::run sites, the Arith dialect, bufferization interfaces, and parser diagnostics. The XPC service has a dedicated _ANEMLIRCompiler class that uses them.

The question is whether the MIL route (the one this article traces) touches any of that MLIR code. We tested with auto-continuing breakpoints on the context constructors, parser, and pass manager. A convolution compiled to a 65,536-byte HWX and none of those breakpoints fired. The Zin controller functions ran normally.

An explicit .mlir input told a different story: it entered MLIRContext, parseSourceFile, and PassManager::run, accepting Apple's MPS dialect (mps.* and mpsx.*). The two routes share a binary but not a code path.

MIL.framework parses the program text and its weight blob into its own graph representation: contexts, functions, operations, tensors, and validation objects. The first Zin function reached under LLDB is BuildLayerGraph. Everything after that call happens inside the hardware compiler.

MIR (machine IR) preparation takes the recognized graph and makes it runnable on hardware. This means splitting tensors that exceed machine limits (along batch, channel, kernel, or spatial dimensions), choosing memory layouts, hoisting compatible work together, and folding weights with their scales.

Interleaved with that work, the compiler runs fusion twice. Fusion is the process of combining separate graph operations into one hardware step. For example, a convolution followed by ReLU can execute as a single Task Descriptor instead of two, with no intermediate write to memory.

Why two rounds? Each sees a different graph. ZinMirPrepareLayers constructs two instances of ZinMirLayerFusion. The first (at offset +2176, mode byte clear) works on graph-level layers, before they are committed to specific hardware engines. The second (at +3916, mode byte set) runs after engine lowering and sees layers that are already bound to hardware.

Between the two rounds, several passes reshape the graph: channel-last conversion, GOC (general-operation-controller) engine reassignment, forward engine lowering, and wrap-axis validation. After the second round, dead-code elimination and post-fusion transpose hoisting clean up.

The broad round's registry covers the ANE's three engine types. NE (Neural Engine) handles convolutions and matmuls. PE (Planar Engine) handles elementwise and pooling work. SNE handles scale-and-bias operations. Each engine type has its own set of fusion patterns.

The narrow round registers only two patterns. NEConv looks for a lowered convolution (opcode 0x5d, a ZinNEConvLayer) or bypass (opcode 0x64, a ZinNEBypassLayer) whose boundary holds an eligible GOC or activation layer. If the predecessor is the captured convolution and the tensor formats and DMA symbols are compatible, it fuses them. TdBranching captures an ane_layer followed by a condition_layer and attaches a hardware condition to the task descriptor.

One methodological note: the framework runs the same fusion machinery during internal validation, so a raw breakpoint count overstates the main path. Every fusion count in this section is filtered to accept a commit only when the call stack contains ZinMirPrepareLayers.

Here is what fusion looks like on a live convolution followed by ReLU. The broad round's ZinNEPatterns::Conv pattern found two matches: the convolution layer (key main) and the ReLU layer (key activation). Conv::Fuse took both pointers and built a single ZinNEConvLayer that combines them. The commit callback then rewired the graph: it added the new fused object, redirected incoming edges, moved outgoing edges, and removed both original objects.

In the plain-convolution control (no ReLU), the activation match was null at both visits and nothing was fused.

The attention probe follows a standard attention pattern: Q, K, and V slices, layout transposes, a QK matmul, scalar scaling, softmax, a PV matmul, and an output transpose. A second version adds a residual connection. Both compile to valid H16G HWX.

The broad round committed nine rewrites on the main attention path: two MatMul (for QK and PV), three Bypass (for view and layout paths), two UnaryElementWise (for the softmax sequence), and two ElementWise (for scaling and graph elementwise work). The residual version added one more PE ElementWise.

The four-layer W8A8 chain committed four Conv rewrites. Its explicit quantize/dequantize edges were already gone by this point, because CollapseQuantDequant runs earlier at +1368, before the first fusion round.

None of the probes produced a commit in the second (narrow) round. This includes every graph we tested: plain and fused convolutions, branch, residual, affine, activation, and layout graphs, the attention block, and the W8A8 chain.

The registry explains why. NEConv needs a very specific setup: a lowered convolution or bypass whose boundary holds an eligible GOC or activation, in the right tensor format with compatible DMA symbols. TdBranching needs a condition layer that none of these MIL graphs produce.

One source of the pattern that NEConv looks for is ZinMirGOCEngineReassignment, which runs between the two rounds. It visits eligible PE elementwise work, checks for a single compatible predecessor with unused output scale/bias state, creates constant scale-and-bias GOC data, inserts a ZinNEBypassLayer, and rewires the graph. That inserted bypass is the kind of node the narrow round's NEConv pattern is designed to find.

The compiler accepts --O0 and --O1 optimization flags. We compiled conv + ReLU under the default policy and under both levels. All three produced a 65,536-byte container with a 472-byte Task Descriptor stream.

--O0 and --O1 were byte-identical. The default stream differed from them in 159 of 472 bytes. Hardware lowering and fusion of the conv/ReLU pair into one TD program stayed active at every level. The optimization level changes the register image (the specific values written into descriptors), but lowering and fusion happen regardless.

We built a separate compiler in Objective-C and Objective-C++ from the recovered pipeline. It reads textual MIL, builds typed SSA and operation graphs, and runs graph passes that normalize and decompose operations. A fusion pass joins adjacent operations when their dependencies and measured task forms allow them to share one HWX program.

The planner creates legal H16G tasks using capability tables derived from measurements on one M4 and one macOS build. When compatible adjacent tasks can share one HWX program, they do. Otherwise they remain separate programs connected by IOSurfaces described in a binding manifest. Unsupported operations, shapes, data types, and reduction axes are rejected before object emission.

The object writer generates HWX files from scratch: Mach-O headers, sections, symbols, relocations, program records, tensor records, and payloads, all built from compiler data structures. It does not read or patch an Apple-compiled HWX container.

Compiler coverage is limited to the configurations in the table above. For convolution, matmul, ALU, and layout operations, the encoders calculate Task Descriptor fields from the graph and selected schedule. Some unary and reduction encoders use a simpler approach: they copy Task Descriptor words measured for one specific geometry. Anything outside the recorded configurations is rejected before HWX emission.

The table reports the final M4 runs on a Mac16,10 with macOS 26.3 build 25D125. The attention comparison used 50 warmups and five alternating batches of 5,000 synchronous evaluations for each compiler. The matmul-GELU figures come from the earlier two-run comparison.

FA2 runs as three HWX programs. Program 0 computes Q × transpose(K), with the scale stored in the matmul output field. Program 1 performs the softmax normalization: reduce_max, subtract, exp, reduce_sum, reciprocal, and multiply. These six tasks share one program because their intermediate values can stay in SRAM. Program 2 multiplies the probabilities by V.

At S128/D128, the complete chain takes 369.458 microseconds, about 3x the Apple compiler's 127.208 microseconds. Matmul-GELU composes into a single program and takes 66.94 microseconds, compared with 76.60 for the Apple compiler.

The 58-program Chunked DeltaNet graph measured 22.73 milliseconds. Its MIL uses ordinary operations (matmul, add, multiply, exp) with regular inputs (normalized Q and K tensors, transposed K layout, triangular matrices). There is no dedicated DeltaNet operation.

Logical rows shorter than 64 bytes are padded to a 64-byte physical row in their tensor records. All higher-level sizes (plane, batch, allocation) derive from that physical stride. For example, a C32/S64/B4 space-to-depth object has a 32-byte logical output row but a 64-byte physical row. Apple's compiler produces the same layout.

The graph keeps the semantic shape. Memory planning assigns the physical strides and storage sizes. The HWX writer records both views, and the runtime fills and reads IOSurfaces through the physical strides. After adding this rule, all 24 reduction forms and all 25 standalone or fused layout forms passed twice on the M4.