Cross-compiling Haskell: running Template Haskell on the build machine

Cross-compiling a Haskell library becomes more complicated when compiling it also means executing Haskell. A small deriveJSON splice can turn an otherwise ordinary cross build into a job that needs a target interpreter, an emulator, or access to a device.

The experiment in template-haskell-cross explores another arrangement: execute Template Haskell with a native compiler on the build machine, then pass its results to the cross compiler. The implementation combines a GHC plugin, a native worker process, a binary protocol, and Nix helpers. It has been used to build a Haskell library with generated FFI exports for an Android application.

This article follows that experiment: why the usual execution model makes cross builds difficult, how the alternative works, what broke along the way, and which parts could inform GHC itself.

1. Template Haskell and the challenges of cross-compilation

Consider a familiar use of aeson:

{-# LANGUAGE TemplateHaskell #-}

module Payload where

import Data.Aeson.TH (defaultOptions, deriveJSON)

data Payload = Payload { message :: String }

$(deriveJSON defaultOptions ''Payload)

The final program needs the generated JSON instances. During compilation, however, GHC must execute deriveJSON, inspect Payload, and obtain the declarations to insert. A splice is a computation that GHC executes during compilation. It can use reify to inspect declarations already available in the compiler’s environment and return new syntax for GHC to compile. Template Haskell therefore introduces a second kind of program into the build: the generator that runs while compiling the application.

Here, build machine means the machine running the compiler, and target means the platform on which the compiled application will run. For example, an x86_64-linux machine might produce a library for aarch64-android. The cross compiler runs on the former while generating code for the latter. Native refers to execution on the build machine.

GHC is a single-target compiler: a cross-GHC can generate code only for its configured target. It cannot also compile the generator for the build machine; that requires a second, native GHC. In the usual cross-compilation path, the cross compiler therefore compiles the executable part of a splice for the target, using dependencies built for that target. Although the generator runs at compile time, its compiled code still needs a target-compatible execution environment. An x86_64-linux build machine cannot simply load and execute a generator built for aarch64-android.

This is why a target-side interpreter is needed. With -fexternal-interpreter, GHC delegates execution to a separate interpreter process. In the documented remote arrangement, remote-iserv is built with the cross compiler and runs on the target; iserv-proxy runs on the build machine. The interpreter executes the target-compatible splice code and returns the generated syntax to the cross compiler. GHC starts iserv-proxy using the path supplied with -pgmi; -opti passes the target interpreter’s address and port to the proxy. The guide also warns that file and process IO then happen on the target, which matters for generators that read build files. See Running the interpreter on a different host.

a07454f16c72c7a6bfa54f572666d383b.target-interpreter.png

This is the practical meaning of needing a “target builder” for TH: the build needs somewhere to execute target-compatible code. That can be a device, a remote machine, or a local emulator. It does not necessarily require running the entire compilation on target hardware.

Nix can set up an emulator for the target interpreter, but the build still depends on a working target execution environment.

For a Linux target this may be manageable. For mobile targets, providing the execution environment and its libraries can become a substantial part of the build setup. The question behind this experiment was whether the generator could execute natively on the build machine instead.

2. The idea: native evaluation, followed by target compilation

Many generators inspect Haskell declarations and construct more Haskell declarations. Their useful output is syntax, which the cross compiler can subsequently typecheck and compile for its own target.

The experiment separates those jobs as shown below. A component unit ID identifies a particular build of a package component; native and target builds can have different IDs.

af1274d797d290434eb961de53e15b3d1.native-th-worker.png

Both the plugin and the worker execute on the build machine, using compiler builds patched for their respective roles (described in Section 3). The target libraries remain target libraries; the worker receives a separate native dependency environment. No target interpreter is used for this TH path.

Intercepting a splice

GHC lets plugins replace selected compilation steps through hooks stored in the compiler session (HscEnv), in its hsc_hooks field. One of these, runMetaHook, receives a typechecked splice expression and a request describing the required result. Expression, pattern, type, and declaration requests have different result types. The hook also receives MetaAW requests for serialized annotation values, including annotations added through addTopDecls. The callback carried by each request constructs the corresponding result; the result constructors themselves are hidden. This interface is visible in GHC’s metaprogramming types.

The cross-side plugin installs its hook in the compiler session:

let hooks0 = hsc_hooks hscEnv
    hooks1 = hooks0
      { runMetaHook = Just (workerMetaHook workerState config) }
pure hscEnv { hsc_hooks = hooks1 }

For an expression request, the hook obtains a worker result, converts it to local GHC syntax, and applies the request’s wrapper. The other splice categories follow the same pattern. The plugin implementation also replays the supported effects before returning.

The worker creates a native GHC session and loads the source module, giving each splice access to its module context. During a normal load, GHC evaluates a splice and converts the resulting TH value into its own syntax for compilation. The worker needs the original TH value to send it to the cross compiler, so it installs a separate hook in the native session. That hook captures the value before conversion, then converts it for native GHC so the module load can continue.

For example, the worker’s raw expression runner is:

runMetaExpTH :: LHsExpr GhcTc -> TcM TH.Exp
runMetaExpTH =
  runMeta'
    True
    (text . TH.pprint)
    (\_srcSpan hval -> Right <$> runTHExp hval)

Here TH.pprint serves diagnostic rendering. The response contains the raw TH.Exp, encoded with Data.Binary. It is not transported as pretty-printed Haskell. The same mechanism handles TH.Pat, TH.Type, and [TH.Dec]. Annotation wrappers use a separate, GHC-specific binary payload when replaying annotations added through addTopDecls. See the worker helpers and protocol documentation.

The build system supplies two dependency environments: native packages for executing generators and target packages for compiling their output. Nix builds and connects both; Section 4 shows the consumer setup. This trades target execution during TH expansion for native compilation of the generator environment, while still requiring a cross toolchain.

3. Implementation challenges

The initial idea is small. Preserving Template Haskell’s behavior across two compiler sessions is where most of the work appears.

Making the plugin and worker work with GHC

The plugin must run inside the cross compiler on the build machine: it intercepts splice requests and sends them to the worker. So, although the compiler produces target code, the plugin itself must be built as a native library. The usual -fplugin option loads plugins through GHC’s package machinery, which is configured for target packages in a cross compiler. Instead, this experiment builds the plugin separately with a matching native GHC and passes its library directly to -fplugin-library. GHC’s external-plugin source describes this cross-compiler use.

Getting that library into the compiler required two build changes. First, GHC only compiles the external-plugin loader when HAVE_INTERNAL_INTERPRETER is defined. The cross compiler is built without that definition, so the experiment forces it on to make the loader available. This enables a code path; it does not provide a target interpreter. Second, in the tested Linux build, the compiler executable is statically linked. The plugin needs symbols from the Haskell runtime inside that executable, so the executable is linked with --export-dynamic to make those symbols visible to the plugin.

A separate issue appeared after a splice returned. Template Haskell can register module finalizers: actions to run later during compilation. GHC recorded an empty list of finalizers for a splice, but later tried to obtain an interpreter before checking whether the list contained any actions. With no target interpreter, even the empty list caused a failure. The cross compiler is patched to skip recording that list when it is empty:

addModFinalizersWithLclEnv (ThModFinalizers []) = pure ()

The native worker needs its own compiler changes. It must send the original Template Haskell values back to the cross compiler, but GHC normally converts those values into its internal syntax during module loading. The worker’s GHC exposes the functions needed to capture the original values and records the original arguments to addTopDecls, so the worker can forward those effects too.

Some unsuccessful attempts involved the compiler build rather than Template Haskell itself: Hadrian overrode a flag passed through Cabal options, nixpkgs used a prebuilt Hadrian despite changes to its source, and a linker setting was applied at the wrong build stage. The final setup patches the compiler package description and applies the linker flag at the effective stage. The compiler-patch notes have the details.

Android also needed packaging fixes: dependencies had to provide shared-library interfaces such as .dyn_hi, and some libraries contained references to temporary build directories. The target profile enables those interfaces and removes the temporary paths.

A splice needs its module context

A splice may need information from the module being compiled. For example, a generator can inspect a type declared earlier in the same module:

data User = User String

... -- end of the preceding declaration group

foo = $(do
  info <- reify ''User
  ... -- generate an expression using info
  )

The ellipses stand for omitted code. To evaluate this splice, GHC must know about User and the declarations visible at the splice’s position. Running the generator on its own does not provide that context.

The worker loads the whole module with native GHC, which processes declarations and splices in order. This gives each splice access to its module context. This also lets splices share getQ/putQ state during that module load. See the worker-evaluation notes.

Loading the module may run several splices. The worker captures their results and uses each splice’s location to find the one requested by the cross compiler. It caches the results in memory to avoid loading the module again for every request, and forces recompilation when loading so GHC actually executes the splices. The cache is limited to the worker process; it is not a persistent build cache. Generated or nested splices need further coverage. See Capture and SpliceCache.

Transferring Template Haskell syntax

The worker sends raw Template Haskell syntax to the cross compiler. This preserves information that source text alone cannot represent, such as references to names that have already been resolved by GHC, including non-exported definitions. The plugin then converts the received syntax into GHC’s internal representation and also replays declarations added through addTopDecls.

This approach requires matching compiler versions, as described in Limitations.

Mapping native package identities to target packages

Keeping raw names solves the text problem, but introduces another boundary. A global TH name includes a package identity, and the native and target instances of a package can have different unit IDs.

Before converting the AST, the plugin rewrites global package references using a native-to-target mapping. This excerpt shows the name-level operation; a generic traversal applies it throughout the payload:

rewriteName (TH.Syntax.Name occ
              (TH.Syntax.NameG namespace packageName nameModule)) =
  TH.Syntax.Name occ
    (TH.Syntax.NameG namespace
      (rewritePackageName packageName) nameModule)
rewriteName name = name

The Nix helper queries the native package database, the target component package database, and the target compiler’s global package database to generate unit-id-map entries; the plugin also maps the worker’s main identity to the consuming component. Without this translation, generated code can refer to interfaces that exist only in the worker environment. See the config generator.

This mapping assumes corresponding packages expose compatible declarations. Renaming a unit ID cannot make platform-dependent APIs equivalent.

Splices can have effects beyond generated syntax

A splice can also add declarations, register file dependencies, generate foreign source files, or report diagnostics. The worker captures these effects and the plugin replays the supported ones in the cross compiler. Foreign files must remain accessible until the cross compiler uses them.

Some effects are not fully supported: module finalizers and requests to add Core plugins cannot reliably be replayed in the cross compiler. Loading a whole module can also run later splices earlier than in the cross compilation. These limitations matter for generators that depend on the timing of effects. See the Q-effects analysis for details.

The native compiler must see the right declarations

The worker must see declarations compatible with those seen by the cross compiler. Consider a module where a CPP flag selects a constructor:

#ifdef TH_CROSS_TARGET_CONTEXT
data Platform = TargetPlatform
#else
data Platform = HostPlatform
#endif

$(pure [])

platformValue :: Platform
platformValue =
  $(do
      TyConI (DataD _ _ _ _ [NormalC constructorName []] _) <- reify ''Platform
      conE constructorName)

The empty splice makes Platform available to reify. If only the cross compiler receives the flag, the worker sees HostPlatform and generates a reference to a constructor that does not exist in the target build.

The worker accepts GHC options, and the config generator forwards source directories, language settings, and unconditional Cabal cpp-options. It does not reconstruct every Cabal conditional, generated include path, or target-specific macro, so matching configurations is not guaranteed. buildPackage also checks derived dependency names against Cabal’s configured component dependencies, but this check is not enabled for automatically wrapped TH dependencies and cannot prove the environments equivalent. See the target-context analysis.

The entire module must also be loadable by native GHC. Target-only imports, headers, or FFI symbols can prevent the worker from loading it, even if the generator itself is in another module. Supporting arbitrary target-only modules would require a more selective evaluation pipeline.

4. Results of the experiment

The repository’s version matrix records successful cross builds of its aeson, microlens-th, executable, dependency-with-TH, and target-context examples.

The application-level PoC is tic-tac-toe-hs: a pure Haskell game core, a Haskell FFI adapter, and a Flutter UI. Its Haskell code was cross-compiled with GHC 9.10.3 into a library for aarch64-android. The FFI module generates the exported C API with a real TH splice:

$( deriveForeignExports
    [ foreignExport "tic_tac_toe_new_game" 'Api.newGame
    , foreignExport "tic_tac_toe_free_game" 'Api.freeGame
    , foreignExport "tic_tac_toe_current_player" 'Api.currentPlayer
    , foreignExport "tic_tac_toe_cell_at" 'Api.cellAt
    , foreignExport "tic_tac_toe_game_state" 'Api.gameState
    , foreignExport "tic_tac_toe_make_move" 'Api.makeMove
    , foreignExport "tic_tac_toe_best_move" 'Api.bestMove
    ]
 )

haskell-ffi-th supplies the generator, template-haskell-cross evaluates it during cross compilation, and flutter-haskell-bridge packages the library and generates Dart bindings. Flutter calls the resulting exports through dart:ffi.

Build and installation instructions are available in the application README.

This demonstrates the intended use: generated Haskell declarations become callable functions in a target library packaged into an application.

Nix connects the native and target dependency environments, configures the worker, and injects the plugin into the target build. Building both environments can require substantial dependency closures. The integration guide describes how to use the project with another Cabal package.

5. Limitations

Compiler versions and compatibility

The project lists GHC 9.8.x, 9.10.x, and 9.12.x as supported series, with 9.8.4, 9.10.3, and 9.12.4 as the validated patch releases. Each validated release has an explicit compiler profile, patched compiler builds, and recorded integration and cross-example checks. Other patch releases are expected to retain the relevant API shape but still need a profile and validation, as explained in the version-support notes.

The worker and cross compiler must use the same selected GHC/template-haskell version, and the native plugin must match the compiler into which it is loaded. There are three reasons:

  • The wire format contains compiler-version-dependent values. Raw TH ASTs use GHC’s GHCi.TH.Binary instances, and annotation wrappers contain GHC’s internal serialized representation. The protocol defines no version-independent schema or translation between different TH versions.
  • The plugin uses GHC’s internal API and binary interface. It runs inside the cross-GHC process and manipulates compiler syntax and typechecker state. Matching the TH payload format alone would not make an incompatible plugin safe to load.
  • The worker depends on compiler patches. Its raw TH runners and addTopDecls capture must be available in the selected compiler build. An ordinary unpatched native compiler of the same numeric version is not a substitute for that worker compiler.

The Cabal packages th-cross-plugin, th-cross-protocol, and th-host-worker declare the following bounds on their ghc and ghc-boot dependencies, reflecting the implementation’s current API baseline and validation range:

ghc >= 9.8 && < 9.13
ghc-boot >= 9.8 && < 9.13

The lower bound means earlier GHC releases need API adaptation and validation; it is not a fundamental restriction of native TH evaluation. At the upper boundary, the repository’s GHC 9.14.1 investigation found that the raw-declaration capture patch needs adaptation to changes in GHC.Tc.Types.TH. Worker, plugin, and cross-example builds were not validated for that release.

Build machines and target platforms

The flake exposes build-machine outputs for x86_64-linux and aarch64-darwin. Cross-build validation covers the compiler and target combinations listed below.

Target Status in this experiment Scope or remaining blocker
aarch64-linux Validated with GHC 9.8.4, 9.10.3, and 9.12.4 The repository records successful builds of all examples.
aarch64-android Validated with GHC 9.10.3 Uses the nixpkgs Android NDK profile and Android-specific shared-library fixes. Other GHC versions have not been validated for this target.
iphone64 Experimental, unvalidated Blocked: requires a full Xcode.app available to Nix on Apple Silicon macOS.

These boundaries come from the target profiles and Nix integration notes.

Other build machines and targets require additional toolchain and package-set wiring, platform-specific plugin loading and linking checks, and validation. Executing TH natively removes the target interpreter requirement, but it cannot supply a missing SDK, code generator, linker, or target runtime.

Template Haskell coverage

  • The source module and its relevant dependencies must be loadable by native GHC; target-only imports can prevent evaluation.
  • Native reification must agree with target declarations; CPP and resolved Cabal conditionals are only partly reproduced.
  • Module finalizers and TH-requested core plugins lack complete cross-side handling and are not always rejected.
  • runIO executes on the build machine, so machine-dependent results may be unsuitable for the target.

6. Further development

The worker’s main architectural limitation is how it obtains the context for evaluating splices: it loads the source module with native GHC. This duplicates compilation and can fail or expose different declarations when the source depends on target-specific imports or CPP. A possible alternative is to let a native executor run the generator while the target compiler answers reify and manages compilation effects. GHC’s remote TH protocol already supports requests back to the compiler, but this experiment has not demonstrated that arrangement with a native executor.

That alternative would need to identify and compile the generator’s dependencies for the build machine without loading the entire target module graph. This is where GHC’s Explicit Level Imports, available since GHC 9.14.1, could help: import splice explicitly marks imports used during splice execution. The current prototype does not use this feature. It could inform dependency selection in a future design, but does not itself provide the native compilation and execution pipeline.

A next prototype would also need a mapping between native and target names, then tests involving CPP-dependent declarations, target-only consumer imports, exact names, and module finalizers. Those tests would establish whether the current module-loading restriction can be relaxed while preserving TH semantics. The smaller compiler changes needed by the existing prototype are documented separately in the compiler-patch notes.

Banner that links to Serokell Shop. You can buy cool FP T-shirts there!
More from Serokell
haskell to core thumbnailhaskell to core thumbnail
Algebraic Data Types in Haskell ImageAlgebraic Data Types in Haskell Image
haskell dependency analysis imagehaskell dependency analysis image

Paris, France52 boulevard de Sébastopol, 75003

Tallinn, EstoniaPille tn 7/5-13, Kesklinna linnaosa,
Harju maakond, 10135

2015–2026 SerokellLegalPrivacy Policy