The Complete Machine

What the M4 Neural Engine is, how it is built, what it can do, and where it stops

Parts 1, 2, and 3 treated the ANE as a tool: compiling models, measuring throughput, and training a transformer. This part describes the M4 ANE itself, from the host interface and compiled program format to SRAM, DMA, compute, and measured limits.

MIL was the right level for those experiments. It let us describe a graph, ask Apple's compiler to place it on the ANE, and measure the result. It did not expose the decisions made after the graph crossed the compiler boundary. Fusion, tiling, SRAM placement, DMA scheduling, tensor layout, and numeric mode were chosen inside ANECompiler.framework. A MIL benchmark could show that two graphs behaved differently; it could not show which hardware fields made them different.

For this part we moved one layer down and treated the compiled HWX program as the object of study. We compiled matched MIL graphs that differed in one property, compared their HWX binaries, changed candidate fields, and loaded the modified programs directly. A field was assigned a hardware role only after the edited program loaded and its output or timing changed in the expected way. Programs emitted from scratch supplied a second check for the formats we had mapped. HWX still passes through Apple's loader and firmware; this method removes the graph compiler from the experiment, not the runtime.

This account comes mainly from reverse engineering an M4. We traced the private API below CoreML, recovered the in-memory compile and evaluation paths, decoded the 0xBEEFFACE Mach-O program format, mapped Task Descriptor fields, derived the tiling formulas, and recovered the activation lookup-table format. Programs emitted from scratch match compiler output byte for byte and run at 98.8% of measured peak throughput. Separate experiments cover int8 execution, tensor geometry, dispatch latency, Task Descriptor counts, and compute/DMA overlap. The compiler's 2,360-byte hardware parameter table was extracted from a live process with LLDB.

Bryngelson's reference for A11 through M5 supplies names for the compiler passes and fields in that parameter table. The Orion paper documents MIL-level constraints relevant to LLM workloads. Measurements and claims taken from those sources are identified where they appear.

The Pipeline, End to End

The diagram follows one program from MIL text through compilation, loading, dispatch, and execution on the M4 ANE.

What the ANE Is

The Apple Neural Engine is a fixed-function matrix accelerator included in Apple systems-on-chip since the A11 in 2017. It shares unified DRAM and the SoC power budget with the CPU and GPU. Apple exposes it through CoreML as one target behind the framework's scheduler. The instruction format, driver interface, and execution counters are private.

The ANE executes complete compiled graphs as atomic operations. Its program model exposes no branch instructions, loops, or dynamic control flow, and execution cannot be paused to inspect intermediate values. Inputs and outputs use IOSurfaces, shared-memory regions that the CPU, GPU, and ANE can map without copying.

The M4 ANE has the internal codename H16G. IORegistry identifies it as AppleH16ANEInterface with architecture h16g, version 192, and 16 registered cores. The meaning of the "G" suffix has not been confirmed.

Inside the Engine

Most of the internal model came from byte-level comparisons of compiled programs, followed by runs of binaries emitted from scratch. Those comparisons exposed Task Descriptor fields, tiling rules, the weight-streaming stride, and the lookup-table format. We also called ZinIrHalH16g::GetParams() inside the live M4 compiler process and extracted its 2,360-byte Hardware Abstraction Table (HAT). Bryngelson's compiler analysis supplies names for several internal passes and HAT fields. The values below were read from the M4.

SRAM

The on-chip memory is a 64-bank scratchpad with a 16-byte interleave granule. An address maps to bank floor(addr / 16) mod 64. The compiler manages it as scratch space with no runtime cache replacement policy. When a tensor's data fits in SRAM, it stays there for the duration of the computation. When it does not fit, the compiler tiles the work and streams data through in chunks.

The streaming threshold is 2 MiB per operand (not per operation, and not a sum of all operands). This value comes from the HAT at offset 0x1b8. Below 2 MiB, an operand can be fully resident before compute starts. Above it, the compiler splits the operand into tiles and double-buffers them: while tile N computes, tile N+1 loads.

The compiler switches each operand from resident to streamed mode at 2 MiB. The throughput change previously observed around 24–32 MB reflects the behavior of tiled, streamed operands rather than a single total-SRAM capacity boundary. Performance beyond the threshold depends on overlap between tile loads and compute.

Weights have their own budget. Up to 64 KiB of weights stay resident on-chip (the "kernel-coefficient store"). Weights up to 16 MiB can be streamed through in 16 DMA chunks. The compiler divides the weight matrix into 16 pieces regardless of total size and loads them sequentially.

Compute

The M4 compute array contains 8 compute sets, each with 8 accumulators. FP16 is the default path: products use fp16 inputs, accumulation is wider than fp16 (the exact width has not been measured), and the output is rounded to fp16. W8A8 chains use a packed compute mode in their middle blocks and reach a 38 TOPS ceiling. A compute-configuration word in the Task Descriptor stream selects the mode.

The HAT reports 8 compute sets at offset 0x238, while IORegistry reports 16 cores for the device. The mapping between those two units is not exposed in the interfaces we examined. The ANE clock frequency has also not been measured directly.

The compiler controls the output-channel-group (OCG) size, meaning how many output channels are computed in one pass through the accumulator. The OCG is the smaller of a power-of-two accumulator budget and a format-specific byte cap (32, 16, or 8 bytes depending on weight format). The formula and its constants come straight from the HAT (byte caps at 0x388–0x398, per-accumulator tiers at 0x3a8–0x3c0).

DMA

Data moves between DRAM and SRAM through DMA engines. Compiled programs reference three tile-DMA engines and one kernel-DMA engine. The physical engine count and queue depth behind those references have not been measured directly. In our tests, sweeping 2 through 8 independent input planes produced a linear slowdown of ~995 µs per additional plane. No concurrency knee appeared, suggesting the DMA path serializes tile loads in the configurations we tested.

Fused operations keep intermediate values in SRAM. When one operation feeds the next, the compiler emits DMA_INTER instead of DMA_STORE followed by DMA_LOAD. A fused matmul + bias + relu runs 5.7× faster than three separate dispatches because it avoids the intermediate DRAM transfers and two additional dispatches.

What the ANE Computes

The ANECompiler.framework validator exports (_ANECValidate*Layer) identify approximately 50 hardware layer types. The operations that compile and execute fall into five groups.

Operations not supported on the hardware: acos, asin, tan, log, cumsum, mod, and boolean logical ops. These fall back to the CPU (via BNNS) or GPU (via Metal). CoreML handles the segmentation automatically.

Activations like sigmoid and gelu do not have dedicated hardware. Instead, the compiler stores 33 fp16 samples in a segment called __KERN_0 and the hardware linearly interpolates between them at evaluation time. For sigmoid with samples from -8 to +8, the max error is 2.4e-4. We verified that replacing the samples with an arbitrary function (x³, not a MIL op) produces correct output on hardware. Any pointwise function approximable by 33 piecewise-linear segments over a fixed domain can run on the ANE.

How Programs Reach the Hardware

The path from a neural network description to hardware execution has five layers:

MIL text (typed SSA graph)
  → ANECCompile() via XPC to ANECompilerService
    → 0xBEEFFACE Mach-O binary (Task Descriptor stream)
      → aned daemon cache → IOKit DMA → ANE SRAM
        → atomic graph execution → IOSurface output

MIL (Machine Learning Intermediate Language) is the input format. It is a text-based SSA language with typed tensors, named operations, and explicit shapes. The ANE compiler is a C function (ANECCompile) in the private ANECompiler.framework. It can also be reached through the ObjC API (_ANEInMemoryModel.compileWithQoS:).

The compiler produces a custom Mach-O binary with magic number 0xBEEFFACE, CPU type 0x80 (128, for ANE), and CPU subtype 0x07 (H16G). The binary contains a stream of Task Descriptors (TDs) that configure DMA engines and the compute array. The program is not a sequence of instructions in the CPU sense. It is a register-write stream that configures the hardware for one graph execution.

Dispatch from the host goes through the aned daemon via XPC for compile and load, but evaluation takes a direct path through ANEServicesProgramProcessRequestDirect without the daemon. The host posts a mailbox command, the engine runs the entire graph, and the output appears in the designated IOSurface.

Each evaluation adds approximately 90 µs of host time for the XPC round trip and IOKit setup. The floor is CPU-bound and does not change with SoC power state. For operations that complete in under ~1 ms, dispatch occupies a large fraction of wall-clock time. Fusion reduces that cost by placing more work in each submitted program.

Performance

Throughput

Peak throughput requires deep operation chains of 32–64 layers and a channel/spatial balance around 512–1024 channels at 64²–128² spatial dimensions. A single matmul reaches roughly 30% of peak. A 64-layer conv1x1 chain keeps the compute pipeline occupied and reaches 98.8%.

FP16 throughput saturates near 19 TFLOPS. Two packed int8 products per lane give a corresponding rate of 38 TOPS; the highest measured W8A8 result is 36.01 TOPS.

INT8: the W8A8 path

The M4 ANE runs W8A8 convolution chains in a packed mode. Both weights and the intermediate activations must be int8. Int8 weights with fp16 activations stay on the fp16 compute path; keeping the edges between convolutions in int8 selects the packed path.

At MIL level, the compiler is given int8 weights through constexpr_affine_dequantize and an int8 quantize/dequantize pair between consecutive convolutions. The pair describes the precision of the connecting edge. It does not add an intermediate DRAM store: the compiled program carries the quantized value directly into the next block.

The benchmark counts the same convolution operations in both modes: 2 × channels² × kernel² × spatial² × depth. Deep fp16 chains reach 18.77 TFLOPS. Deep W8A8 chains reach 36.01 TOPS. The measured rates correspond to one fp16 product or two int8 products per compute lane per cycle.

The compiled chain has an fp16 boundary at each end and int8 edges inside it. The entry block accepts fp16 input, performs the first convolution, and writes an int8 intermediate. Middle blocks consume and produce int8. The final block consumes int8 and writes the requested fp16 output.

Three parts of the HWX record change together. The compute mode becomes 0xb1418005 in a middle block. The inter-layer DMA word becomes 0x80049240, selecting a one-byte edge instead of the fp16 form 0x80041240. The block grows by four configuration words, including quantization control and scale. Changing only the mode or only the DMA word does not produce a valid W8A8 block.

The mode-only edit also changes the three-layer run from 371.4 µs to 703.9 µs. The program still loads and executes, so the word is active hardware configuration rather than descriptive metadata. Reverting either quantized DMA edge stops evaluation before an output is produced.

A direct HWX emitter must update the complete block: compute mode, both edge-DMA records, quantization words, scales, and the depth-dependent command metadata in the container. The field mutations above establish direct access to the packed path. A geometry-independent W8A8 constructor still requires the remaining container metadata to be mapped.

A single convolution has no quantized internal edge. At 1024 channels and 32² spatial dimensions, the int8-weight version takes 260.4 µs and the fp16 version takes 260.5 µs. Packed execution starts inside a chain, after one convolution produces an int8 activation for the next. W8-only programs therefore measure 1.00× while deep W8A8 chains approach 2×.

Power and efficiency

The ANE draws exactly 0 mW at idle and powers on for each submission. W8A8 provides 1.95× the throughput while drawing 2.7× the power. FP16 is more efficient per counted operation; W8A8 completes more operations per second and uses narrower intermediate tensors. These measurements do not isolate how much of the additional power belongs to the compute array or the surrounding memory system.

Geometry matters

A shallow-channel tensor leaves much of the compute array idle. SpaceToDepth can move the computation into the 512–1024 channel range; DepthToSpace restores the original layout afterward:

S2D/D2S overhead is 0.3-1.4% of the pipeline. Avoid block_size=2: the compiler lowers it as a convolution with one-hot weights (2.8-3.1% overhead) rather than a DMA reshuffle.

ANE vs GPU vs CPU

The CPU, GPU, and ANE occupy different operating ranges on the M4.

For supported operations large enough to amortize the 90 µs dispatch, the ANE measures 5× faster than the GPU and 8–13× more power-efficient than the CPU. The GPU supports a broader operation set and can add work to an active command stream. The CPU runs arbitrary code without the ANE submission cost.

For LLM inference, prefill provides the large matrix operations that suit the ANE. Token-by-token decode contains smaller operations for which the 90 µs submission cost can dominate, making CPU/SME execution more suitable. In our on-device training path, the ANE runs forward and backward matmuls at 2.8–4.6 W; the CPU performs weight updates, masked attention, and the Adam optimizer.

The M6: Dual ANE

Apple announced the 2 nm M6 on August 25, 2026. It contains two 16-core Neural Engines, and Apple states that system frameworks can use both simultaneously. The published claim is twice the peak compute of the preceding configuration.

Apple did not publish a TOPS figure for the M6 ANE. If each engine matches the M4's 19 TFLOPS fp16 rate, the pair would provide 38 TFLOPS fp16 or 76 TOPS in packed W8A8 mode. This is an extrapolation, not an M6 measurement. Apple has not disclosed whether the engines share SRAM, coordinate DMA, or operate independently. DRAM bandwidth rises from 120 GB/s on M4 to 170 GB/s on M6, an increase of 42%.

The unresolved architectural questions concern graph placement, memory, and dispatch. A graph may span both engines or be partitioned into independent subgraphs. Each engine may have its own 2 MiB operand threshold and 64 KiB kernel store, or those resources may be shared. The dispatch path may submit both engines together or pay the host-side cost separately.

Apple also added "Neural Accelerators" to the M6 GPU cores, claiming a 30% improvement over M5 for GPU-side AI work. These are separate from the ANE and appear to be small fixed-function units within the GPU pipeline, similar to NVIDIA's tensor cores. Whether they share the same ISA as the ANE or use their own encoding is unknown.

No M6 Pro or Max variant was announced with the M6. A dual-engine base chip leaves open whether later variants will scale ANE count with die area. Counts beyond the announced pair are speculative.

We do not have an M6 to test. The numerical estimates above extrapolate from M4 measurements and Apple's published specifications. Hardware inspection would begin with the HAT, followed by single- and dual-engine dispatch measurements and an overlap test.

Limits and Open Questions

Measured boundaries

Still unknown

Maximum hardware program size. The compiler stops first at 384 convolution blocks, leaving the physical HWX limit above 383 unmeasured.

Internal overlap. A combined matmul and independent ReLU saved 19.5 µs after subtracting one dispatch, 1.1% of the shorter operation. Eight independent adds were 1.0083× faster than eight chained adds. These are graph-level timings; the overlap between MAC, planar, and DMA engines inside a Task Descriptor is still unresolved.

DMA topology. Tests expose three tile-DMA references and one kernel-DMA reference in compiled programs. The number of physical engines and their queue depth are unknown. A sweep from two to eight input planes serialized into additional Task Descriptors and therefore measured the compiler's schedule rather than the hardware's DMA width.

Lane count and clock. The aggregate rates are 19 TFLOPS for fp16 and 38 TOPS for packed W8A8. The HAT's 8/4/2/1 values describe accumulator tiling. The active clock and the number of output channels produced per cycle have not been isolated.

Synchronization scope. WAIT and WAIT_EXT are required by dependent programs. Their exact targets, scope, and latency are not yet mapped.

The M4 ANE is a fixed-function matrix accelerator with 8 compute sets, 64 banks of on-chip SRAM, and a compiler-managed streaming model built around a 2 MiB per-operand threshold. It reaches 19 TFLOPS in fp16 and 38 TOPS in packed W8A8 mode. Peak fp16 power is 4.57 W, and the engine draws 0 mW when idle. Programs use a fixed operation set with no branching, and every submission includes an approximately 90 µs host-side dispatch cost. Apple keeps the compiler, program format, and driver interface private.

The M6 doubles the engine count. Whether that is two independent accelerators or a coordinated pair remains to be measured.

References

1. Spencer H. Bryngelson. Apple Neural Engine: Architecture, Programming, and Performance. arXiv:2606.22283, June 2026. arxiv.org/abs/2606.22283
2. Ramchand Kumaresan. Orion: Characterizing and Programming Apple's Neural Engine for LLM Training and Inference. arXiv:2603.06728, March 2026. arxiv.org/abs/2603.06728
3. Spencer H. Bryngelson. ANEForge: Python for direct computation on the Apple Neural Engine. arXiv:2606.17090, June 2026. arxiv.org/abs/2606.17090
4. Manjeet Singh. Inside the M4 Apple Neural Engine, Parts 1-3. maderix.substack.com, 2026. Part 1, Part 2, Part 3
5. Dougall Johnson. Apple G13 GPU Architecture. github.com/dougallj/applegpu, 2021-present.
6. Apple Inc. Apple introduces M6 and M5 Ultra for a big leap in performance and AI compute. August 25, 2026. apple.com/newsroom