ESPnet3 Demo Guide

This page explains how to package and run an interactive Gradio demo from a published ESPnet3 model.

A key advantage is that demos reuse your packed model through InferenceModel, so the demo does not re-implement inference; it only wires UI values to the same inference call used by pack_model.

1. Run

pack_demo's default model.dir_or_tag: exp/${exp_tag}/model_pack needs exp_tag, which is only resolved when --training_config is passed alongside --demo_config:

python run.py --stages pack_demo \
  --training_config conf/tuning/training_e_branchformer.yaml \
  --demo_config conf/demo.yaml

Important

Running pack_demo/upload_demo with --demo_config alone crashes with omegaconf.errors.InterpolationKeyError: Interpolation key 'exp_tag' not found, because demo.yaml's default model.dir_or_tag references ${exp_tag} and nothing else supplies it. Always pass --training_config (or set model.dir_or_tag to an explicit path/tag in demo.yaml).

Add upload_demo to --stages (with the same flags) to push the packed demo to a Hugging Face Space:

python run.py --stages pack_demo upload_demo \
  --training_config conf/tuning/training_e_branchformer.yaml \
  --demo_config conf/demo.yaml

Important

Re-running pack_demo against the same pack.out_dir can fail with FileExistsError if the packed model directory lives outside the demo directory (the default TEMPLATE layout, where model_pack sits under exp/${exp_tag}/ and demo sits at the recipe root): pack_demo creates demo_dir/model_pack as a symlink to the external model directory, but does not remove a stale symlink left over from a previous run (packing.py's _link_local_model_into_bundle). If a re-run fails this way, delete pack.out_dir (or just demo_dir/model_pack) before running pack_demo again.

Important

Stage logs for pack_demo/upload_demo are written directly into demo_config.pack.out_dir — the same directory upload_demo uploads to the Hugging Face Space. Passing --write_requirements writes a full pip freeze snapshot into that same directory as requirements.txt, overwriting the curated requirements.txt that pack_demo already wrote from pack.requirements (the pack_demo/upload_demo StageSpec declares log_dir="pack.out_dir", resolved by resolve_stage_log_dir, and logging_utils.py's _write_requirements_snapshot writes requirements.txt into that same resolved log directory). Avoid --write_requirements with pack_demo/upload_demo, and check requirements.txt in the packed directory before uploading.

2. Outputs

After pack_demo, ESPnet3 writes a runnable Gradio app into the output directory (demo.yaml's pack.out_dir; the TEMPLATE default is the recipe-relative demo directory):

cd demo
python app.py

This starts a local Gradio server. Open the printed URL in your browser.

[!IMPORTANT] gradio is required for local demo execution. Install it with pip install gradio.

A typical packed demo directory looks like:

demo/
├── app.py            # copied from ui.app_script
├── demo.yaml         # resolved config, paths rewritten to be relative
├── model_pack -> ../model_pack/   # symlink, when the model lives outside demo_dir
├── README.md
└── requirements.txt  # written from pack.requirements

After changing app.py or any UI asset, run pack_demo again — the packed directory is a snapshot copied at pack time, so edits to the source app.py/assets are not picked up until the next pack_demo.

3. The three-layer demo design

The demo has three layers with separate responsibilities:

1. demo.yaml declares the model reference (model.dir_or_tag), model call-time kwargs (model.call_args), the UI input/output specs (ui.inputs/ui.outputs), and the app script to copy (ui.app_script). By default, the input/output specs are generated from the contract section of the packed inference.yaml; ui.inputs/ui.outputs override or extend this default.
2. load_demo_session() loads that config and builds a DemoSession (espnet3.publication.demo.session). The session resolves the packed model through InferenceModel, resolves UI assets from the registry, and builds the inference callable via DemoSession.create_inference_fn().
3. The packed app.py owns the Gradio layout. The default app builds components from session.input_specs/session.output_specs, passes their values positionally to session.create_inference_fn(), and displays the returned values.

The runtime inference path is:

Gradio values -> input spec keys -> InferenceModel(model_input, **call_args)
              -> result keys -> output spec values -> Gradio components

ESPnet3 owns model loading, packed-config resolution, input-key mapping, call-time arguments, and the generic session callable. The recipe owns the UI layout and any presentation-specific logic.

4. Configuring demo.yaml

For the full list of options, see Demo configuration.

Section	Description
model.dir_or_tag	packed model directory (relative to the demo dir) or HF Hub tag
model.call_args	kwargs passed to the model at call time, e.g. beam_size
ui.app_script	Gradio launcher script copied into the pack as app.py
ui.title	app title
ui.description	markdown/text file path, or inline text
ui.inputs/ui.outputs	list of {key, type, label} UI component specs
pack.out_dir	packed demo output directory
pack.include, exclude	extra files/dirs to copy, and exclusion patterns
pack.requirements	pip specifiers written to the packed requirements.txt
upload_demo.hf_repo	target Hugging Face Space

For ordinary UI changes, edit ui.inputs/ui.outputs — each entry's key must match the packed model's input/output contract. By default, the Gradio UI widgets are generated from the contract section of the packed inference.yaml. The ui.inputs and ui.outputs entries override or extend this default mapping. If your model uses a custom kind not built into ESPnet, you must register a corresponding UI class in the demo's session.registry. Example:

ui:
  app_script: src/app.py
  title: ASR demo
  inputs:
    - key: speech
      type: audio
      label: "Input Audio"
  outputs:
    - key: hyp
      type: text
      label: "Transcription"

For a new reusable component type, subclass UIAsset (espnet3.publication.demo.assets) and register it in DEFAULT_UI_ASSETS. For a one-off layout, edit or replace ui.app_script and call load_demo_session()/DemoSession directly, keeping the positional order of Gradio inputs/outputs aligned with session.input_specs/output_specs.

Related pages

Stage API: pack_demo, upload_demo, DemoSession, and UIAsset. Build a model bundle with publication stages before packing a demo.

Demo configuration

All options for configuring the demo stage

Publication stages

Pack and upload a model before packaging a demo around it.