Training a SIREN emitter¶
A SIREN emitter is trained on a lookup table built from a PhotonSim energy scan, then promoted
into data/<material>/<particle>/<cherenkov|dedx>/ with the emission-time (t0) fit and the
emission band the simulator needs. The stages, in order:
| # | stage | command | inputs | outputs |
|---|---|---|---|---|
| 0 | fit s_max(E) |
smax.generate_jobs, merge.sh, smax/analyze.sh |
smax/configs/<m>_<p>.json |
$PHOTONSIM_DEV_PATH/data/<m>/<p>/smax_fit.csv |
| 1 | training-input scan | siren_inputs.generate_jobs, merge.sh |
configs/<m>_<p>*.json, smax_fit.csv |
training_inputs/<m>/<p>/<E>MeV/photonsim.root |
| 2 | build the tables | lucid-build-photon-table, lucid-build-dedx-table |
stage 1, smax_fit.csv |
training_inputs/<m>/lookup_tables/<particle>/{photon,dedx}_lookup_table.h5 |
| 3 | fit t0 | t0.generate_jobs (PhotonSim's calculate_t0.py) |
stage 1, smax_fit.csv |
t0_parametrization/<m>/<p>/timing_parameters.json |
| 4 | train | train_siren.generate_jobs, rank_seeds.py |
a table, train_siren/configs/*.json |
training_tests/<scan>/seed<N>/{emitter.json, weights.npz} |
| 5 | promote | lucid-siren-promote |
a run, smax_fit.csv; Cherenkov: the t0 fit and band |
data/<m>/<p>/<kind>/{emitter.json, weights.npz} |
| 6 | validate | python -m lucid.siren.training.validate |
the emitter; integral: its table |
output/siren/<m>/<p>/ |
<m>/<p> is PhotonSim's material and particle (water/mu-, water/e-); <particle> and the
data/ directories use LUCiD's names (muon, electron). Stage 3 needs only stages 0 and 1,
and can run beside stages 2 and 4. Output paths are under SIREN_OUTPUT_BASE_PATH.
Setup¶
The cluster stages read the site from lucid/production/platform/user_paths.sh, copied from a
template in lucid/production/platform/templates/; a setting a stage needs and does not find is
an error that names it.
LUCID_IMAGE_PATH,SLURM_ACCOUNT,SLURM_PARTITION(CPU jobs),SIREN_OUTPUT_BASE_PATH.PHOTONSIM_DEV_PATH: a PhotonSim checkout (main, May 2026 or later), built in the image (container). Stages 0 and 1 run itsbuild/PhotonSim, which must fillPhotonHist_TimeDistanceNormfor stage 3; stage 0 writes and stages 1 to 3 read itsdata/<m>/<p>/smax_fit.csv; stage 3 runs itsPhotonSim/tools/t0_correction/calculate_t0.py.SIREN_TRAIN_PARTITION, a GPU partition (S3DF:ampere,ada,hopper,turing), andLUCID_ENV_BASE, the Python user site holding the GPU jaxlib the image lacks (the template says how to make one), for stage 4.LUCID_DEV_PATH(optional): the jobs run this checkout instead of the image's LUCiD.SIREN_INPUTS_BASE_PATH(optional): where stages 3 and 4 read earlier stages' outputs; defaultSIREN_OUTPUT_BASE_PATH. See Existing inputs on S3DF.
The generators run in the container through run_in_container.sh, which submits from the host
with -s:
cd lucid/production/siren_inputs
source ../platform/user_paths.sh # the commands below use its settings
RUN=../platform/run_in_container.sh
Stages 0 and 1: the PhotonSim scans¶
Both scans fire one particle along +z from the origin, monoenergetic, one cell per energy, with
decays off and no per-photon output (smax_planning.py, siren_planning.py). The energies,
events and particle are in the configs; the commands are in
smax/README.md
and
siren_inputs/README.md.
$RUN lucid.production.siren_inputs.smax.generate_jobs -c smax/configs/water_mu.json -s
./merge.sh $SIREN_OUTPUT_BASE_PATH/smax_parametrization && smax/analyze.sh $SIREN_OUTPUT_BASE_PATH/smax_parametrization
$RUN lucid.production.siren_inputs.generate_jobs -c configs/water_mu.json -s
$RUN lucid.production.siren_inputs.generate_jobs -c configs/water_mu_lowE.json -s # 150-490 MeV
./merge.sh $SIREN_OUTPUT_BASE_PATH/training_inputs
| particle | stage 0 config | stage 1 configs | stage 1 cells |
|---|---|---|---|
| mu- | smax/configs/water_mu.json |
configs/water_mu.json, configs/water_mu_lowE.json |
307, 150 MeV to 100 GeV, 10k events |
| e- | smax/configs/water_el.json, water_el_lowE.json |
configs/water_el.json |
339, 1 MeV to 100 GeV, 10k to 100k events |
s_max(E) is the track range, fitted per material and particle in one of three forms (form in
the CSV): A*E^B, smooth_two_power (a, b1, b2, E0) or piecewise (two smooth_two_power
pieces joined at e_join_mev). Stage 1 macros set /output/smax <smax_mm> mm, so PhotonSim
fills, against s/s_max in [0, 1] (500 × 500 bins, beyond s_max to the overflow):
PhotonHist_AngleDistanceNorm (opening angle), dEdxHist_DistanceNorm (dE/dx) and
PhotonHist_TimeDistanceNorm (photon delay t − d/c). The first two make the tables, the third
the t0 fit. The fit travels in the tables' metadata into the emitter, so later stages never read
the CSV again, except promotion, which checks the run against it.
Stage 2: build the tables¶
Each table holds every energy of the scan in memory; build it in a batch job. The commands here
run this checkout's modules (python3 -m, with the checkout bound at /opt/LUCiD and first on
PYTHONPATH): an image's console scripts are as old as the image.
T=$SIREN_OUTPUT_BASE_PATH/training_inputs
srun --partition milano --account "$SLURM_ACCOUNT" --qos "${SLURM_QOS:-normal}" --mem 32G --time 02:00:00 \
apptainer exec -B "$APPTAINER_BINDS" -B "$(git rev-parse --show-toplevel):/opt/LUCiD" \
--env PYTHONPATH=/opt/LUCiD "$LUCID_IMAGE_PATH" \
/opt/conda/bin/python3 -m lucid.siren.training.photonsim_data.build_tables photon \
--data-dir "$T" --material water --particle mu- \
--photonsim-dir "$PHOTONSIM_DEV_PATH" --output "$T/water/lookup_tables/muon/photon_lookup_table.h5"
and build_tables dedx likewise into dedx_lookup_table.h5 (lucid-build-photon-table and
lucid-build-dedx-table are the same commands). A Cherenkov table from an older builder, with an
N_photons(E) fit not in logpoly form, is refused by the trainer: rebuild it. Every <E>MeV directory under
<data-dir>/<material>/<particle>/ is used. Flags:
command-line tools.
After writing, the builder re-reads the table and checks the required keys, that the distance
edges span [0, 1], the normalisation, one random energy against its photonsim.root, the
smax_* attributes against smax_fit.csv, and the photon table's nphot_* attributes. It exits
non-zero on a failure.
Stage 3: fit t0¶
The emission time is t(d, E) = d/c + A(E)(exp((d/λ(E))^β(E)) − 1), fitted by PhotonSim's
calculate_t0.py from PhotonHist_TimeDistanceNorm of the stage 1 cells: A, λ, β per
energy, then log10 A, log10 λ and β each a cubic in log10 E. The stage links the cells'
photonsim.root into one directory and writes one container job that runs it:
$RUN lucid.production.siren_inputs.t0.generate_jobs --material water --particle mu- -s
It writes timing_parameters.json and three plots (trends.png shows the cubics against the
per-energy fits). Details:
t0/README.md.
Stage 4: train¶
Not every seed of the nominal recipe converges, so a production emitter comes from a seed scan
ranked by validation loss. The tracked scans, train_siren/configs/water_{mu,el}_{cherenkov,dedx}_seeds.json,
are the scans the shipped emitters were chosen from; each names its table as
${SIREN_INPUTS_BASE_PATH}/training_inputs/....
$RUN lucid.production.siren_inputs.train_siren.generate_jobs -c train_siren/configs/water_mu_cherenkov_seeds.json -s
apptainer exec -B "$APPTAINER_BINDS" "$LUCID_IMAGE_PATH" /opt/conda/bin/python3 \
train_siren/rank_seeds.py $SIREN_OUTPUT_BASE_PATH/training_tests/water_mu_cherenkov_seeds
One run is lucid-train-siren --h5-path T --output-dir D --seed N; the trainer reads the kind,
material and particle from the table. Flags, outputs and how to judge convergence:
command-line tools. Scan configs:
train_siren/README.md.
Stage 5: promote¶
A training run carries the net, its domain and decoding, and the table's s_max(E) and
N_photons(E) fits. lucid-siren-promote adds the t0 fit and the emission band (the band, in
nm, over which PhotonSim counted the photons behind N_photons(E); for water 274.91 to 673.83, the
1.84 to 4.51 eV range of PhotonSim's optical table), checks the run's s_max_mm against
smax_fit.csv, records the provenance, and writes the emitter. It refuses a Cherenkov emitter
without --t0 or --emission-band-nm.
lucid-siren-promote --run $SIREN_OUTPUT_BASE_PATH/training_tests/water_mu_cherenkov_seeds/seed1 \
--smax-fit $PHOTONSIM_DEV_PATH/data/water/mu-/smax_fit.csv \
--t0 $SIREN_OUTPUT_BASE_PATH/t0_parametrization/water/mu-/timing_parameters.json \
--emission-band-nm 274.91 673.83 \
--selection "water_mu_cherenkov_seeds/seed1: lowest final val_loss of 10 seeds"
A dE/dx run takes only --run, --smax-fit and --selection. The destination is
data/<material>/<particle>/<kind>/ of the checkout running it (--dest for another); an emitter
already there is replaced only with --force. In the image, run it as
python3 -m lucid.siren.training.promote with the checkout bound over /opt/LUCiD.
load_emitter reads only promoted emitters.
A material without emitters of its own runs another set: "siren": "<material>" in
config/materials/<material>.json. To ship new weights, they go into the data asset
(tools/make_asset.sh, data/MANIFEST.sha256).
Stage 6: validate¶
srun --partition milano --account "$SLURM_ACCOUNT" --qos "${SLURM_QOS:-normal}" --mem 32G --time 01:00:00 \
apptainer exec -B "$APPTAINER_BINDS" -B "$(git rev-parse --show-toplevel):/opt/LUCiD" \
--env PYTHONPATH=/opt/LUCiD "$LUCID_IMAGE_PATH" \
/opt/conda/bin/python3 -m lucid.siren.training.validate all --material water --particle muon \
--h5-path $SIREN_OUTPUT_BASE_PATH/training_inputs/water/lookup_tables/muon/photon_lookup_table.h5 --save
The checks are in
lucid/siren/README.md.
Existing inputs on S3DF¶
The inputs behind the shipped water emitters are on S3DF under
/sdf/data/neutrino/cjesus/CIDER/SIREN_files (another user's: read them, never write there). The
S3DF template lists them; set SIREN_INPUTS_BASE_PATH to that path to rerun stage 3 or 4 on
them, or pass the paths to stage 2 and 5. Each shipped emitter.json names its run
(provenance.run), table (provenance.table) and t0 scan (t0.source.scan_dir).
| stage | path under it |
|---|---|
| 0 | smax_parametrization/water/{mu-,e-}/<E>MeV/ |
| 1 | training_inputs/water/{mu-,e-}/<E>MeV/photonsim.root |
| 2 | training_inputs/water/lookup_tables/{muon,electron}/{photon,dedx}_lookup_table.h5 (rebuilt 2026-07-01, after the shipped emitters were trained; their N_photons(E) fit predates the logpoly form, so rebuild them before training) |
| 3 | t0_parametrization/water/<p>/timing_parameters.json (for e- it sits in inputs/; the inputs are links into stage 1) |
| 4 | training_tests/<scan>/seed<N>/ |
The s_max fits are PhotonSim's own data/water/{mu-,e-}/smax_fit.csv.
Table layout (format_version = "2.0")¶
photon_lookup_table.h5
├── data/
│ ├── photon_table_raw (n_energies, 500, 500)
│ └── photon_table_average (n_energies, 500, 500) raw / events per energy
├── coordinates/
│ ├── energy_values, energy_centers (n_energies,) MeV, non-uniform
│ ├── energy_edges (n_energies+1,)
│ ├── angle_edges (501,), angle_centers (500,) 0 to π rad
│ └── distance_edges (501,), distance_centers (500,) s/s_max, 0 to 1
└── metadata/
├── attrs: material, particle, distance_axis="s_over_smax", format_version, photonsim_version,
│ normalization="average", average_units="photons/event", bin counts and ranges,
│ table_shape, total_photons,
│ smax_form, smax_<param>..., smax_{fit_min_mev, fit_max_mev, quantile,
│ quantile_multiplier, generated_at_utc},
│ nphot_{form, coeffs, u0, du, fit_min_mev, fit_max_mev, rms_frac, max_frac}
│ (photon table only: the N_photons(E) fit that sets the emitter's yield)
└── events_per_file structured (energy: i4, events: i4)
The dE/dx table uses dedx_table_* for photon_table_*, a dedx_* axis (0 to 1000 keV/mm)
for the angle, total_edep for total_photons, average_units = "MeV/event" (entries are
weighted by deposited energy) and data_type = "dedx".
Troubleshooting¶
PhotonHist_AngleDistanceNormmissing in an<E>MeVfile, ordistance_edges are not [0, 1]: stage 1 ran without/output/smax. Rerun that energy with a config whose generator setssmax_mm. The t0 stage refuses such inputs too (has no PhotonHist_TimeDistanceNorm).--require-listreports missing energies: stage 1 is incomplete. Resubmit withlucid/production/platform/run_in_container.sh lucid.production.siren_inputs.resubmit_failed <scan_dir> --list -s, thenlucid/production/siren_inputs/merge.sh, until all are present.No s_max fit at PhotonSim/data/<m>/<p>/smax_fit.csv: run stage 0 for that material and particle.trained on a table whose s_max fit is not ...(promotion): the table was built from anothersmax_fit.csv; promote against that one, or rebuild the table and retrain.invalid emitter record: ... (install an emitter with lucid-siren-promote): a run was copied intodata/by hand; promote it instead.