API Reference¶

BayesWave Pipeline¶

class asimov_bayeswave.BayesWave(production, category=None)[source]¶

Bases: Pipeline

The BayesWave Pipeline integration for Asimov.

This class provides an interface between Asimov and the BayesWave parameter estimation pipeline, handling DAG file creation, job submission, and PSD collection.

Parameters:

• production (asimov.Production) – The production object.

• category (str, optional) – The category of the job. Defaults to “analyses”.

name¶

The name of the pipeline (“BayesWave”)

Type:

str

STATUS¶

Set of possible job statuses

Type:

set

Examples

>>> from asimov_bayeswave import BayesWave
>>> pipeline = BayesWave(production)
>>> pipeline.build_dag()
>>> pipeline.submit_dag()

Attributes:

bayesline_enabled

Whether BayesLine spectral-line modelling (--bayesLine) should be enabled, from likelihood.components.noise.lines (default True, matching today’s unconditional default).

config_template

The path to the bundled Liquid ini template for this pipeline.

flow

Calculate the lower frequency for the bayeswave job.

model_flags

The BayesWave model-selection flags to set (as bare, value-less keys) in [bayeswave_options] for this production’s run_mode.

run_mode

The BayesWave run mode implied by likelihood.components.

scheduler

Get the configured scheduler instance for this pipeline.

Methods

after_completion()	Perform post-processing after the BayesWave job completes.
after_processing()	Run the after processing jobs.
before_build([dryrun])	Define a hook to be run before the DAG is built.
before_config([dryrun])	Define a hook to run before the config file for the pipeline is generated.
before_submit()	Modify submission files before submitting the DAG.
build_dag([user, dryrun])	Construct a DAG file in order to submit a production to the condor scheduler using bayeswave_pipe.
build_report([reportformat])	Build an entire report on this pipeline, including logs and configs.
check_progress()	Report sampling progress from the BayesWave job's output.
clean([dryrun])	Remove all of the artefacts from a job from the working directory.
collect_assets()	Collect the assets for this job and commit them to the event repository.
collect_logs()	Collect all of the log files which have been produced by this production.
collect_pages()	Collect the HTML output of the pipeline.
detect_completion()	Detect if the BayesWave job has completed.
detect_completion_processing()	Detect that PESummary post-processing outputs exist and are valid.
eject_job()	Remove a job from the cluster.
get_prior_interface()	Get the prior interface for this pipeline.
get_state_handlers()	Get pipeline-specific state handlers.
html()	Return the HTML representation of this pipeline.
read_ini(filepath)	Read and parse a pipeline configuration file.
result_pages()	Link to the copied megaplot output for the report modal.
resurrect()	Attempt to resurrect a failed job.
store_assets()	Add the assets to the Asimov store.
store_reconstructions()	Add signal/glitch/full waveform reconstructions to the event repository and the Asimov store.
store_results()	Store the PE Summary results
submit_dag([dryrun])	Submit a DAG file to the scheduler.
supress_psd(ifo, fmin, fmax)	Suppress portions of a PSD.
upload_assets()	Upload the PSDs from this job to the event repository.
while_running()	Define a hook to run while the job is running.

build

name = 'BayesWave'¶

STATUS = {'finished', 'running', 'stopped', 'stuck', 'wait'}¶

available_outputs = ['psd']¶

log_patterns = ['logs/*.out', 'logs/*.err', 'logs/*.log', '*.err']¶

__init__(production, category=None)[source]¶

property config_template¶

The path to the bundled Liquid ini template for this pipeline.

asimov’s manage build step falls back to this property to render a production’s ini when one doesn’t already exist in the event repository (see Analysis.make_config). Without this, asimov would fall through to looking for asimov/configs/bayeswave.ini inside asimov’s own package, which no longer exists now that BayesWave support has been extracted into this standalone plugin.

build_dag(user=None, dryrun=False)[source]¶

Construct a DAG file in order to submit a production to the condor scheduler using bayeswave_pipe.

Parameters:

• production (str) – The production name.

• user (str) – The user accounting tag which should be used to run the job.

• dryrun (bool, optional) – If True, print the command without executing it.

Raises:

PipelineException – Raised if the construction of the DAG fails.

detect_completion()[source]¶

Detect if the BayesWave job has completed.

Returns:

True if the outputs required by this production’s run_mode have been generated, False otherwise.

Return type:

bool

Note

For a PSD-only run (the default) this only checks for the PSDs, exactly as before. For a run which also requests a signal and/or glitch model ("signal", "glitch", "full" or "coherence" mode), checking for the PSDs alone is not enough: the “clean” PSD-estimation phase finishes well before the signal/glitch post-processing does (they are independent models within the same BayesWave/BayesWavePost run), so that would report completion far too early. In that case this instead requires the reconstruction for every requested model and interferometer, plus the final megaplot summary page. For “coherence” mode this means both post/signal/ and post/glitch/ (BayesWave’s own default model set writes each model to its own post/<model>/ directory, same as “signal”-only/”glitch”-only mode); for “full” mode it means the combined post/full/ directory, which holds both a signal- and a glitch-prefixed reconstruction file. Either way, collect_assets()’s reconstruction glob searches across every post/*/ directory so this doesn’t need to special-case which directory name is used.

after_completion()[source]¶

Perform post-processing after the BayesWave job completes.

This includes converting PSDs to XML format, collecting output pages, storing assets, and applying PSD suppressions if configured.

property flow¶

Calculate the lower frequency for the bayeswave job.

This is required as bayeswave needs to be passed the lowest minimum frequency from the list of interferometer lower frequencies.

Returns:

The minimum frequency across all interferometers.

Return type:

float

Raises:

ValueError – If likelihood.minimum frequency isn’t present as a non-empty per-interferometer dictionary in the production’s metadata.

Note

This is computed fresh from production.meta on every access rather than cached, and deliberately not called during __init__: pipeline construction happens inside Analysis.__init__, before GravitationalWaveTransient’s own quality -> likelihood migration for a deprecated quality.minimum frequency blueprint runs. An eager read here would see the pre-migration state and could raise even for a blueprint asimov itself will happily migrate and accept.

property run_mode¶

The BayesWave run mode implied by likelihood.components.

Returns:

One of "psd" (today’s default: on-source PSD estimation only), "signal", "glitch", "full" (the combined signal+glitch model, no separate evidences) or "coherence" (BayesWave’s own default model set: signal, glitch and noise each run as genuinely separate phases, so their evidences can be compared – what likelihood.coherence test: true needs).

Return type:

str

property model_flags¶

The BayesWave model-selection flags to set (as bare, value-less keys) in [bayeswave_options] for this production’s run_mode.

Returns:

E.g. ["cleanOnly"] or ["fullOnly", "chirplets"], or [] (or ["chirplets"]) for run_mode "coherence", which deliberately sets no model-restriction flag at all so BayesWave’s own defaults apply.

Return type:

list of str

property bayesline_enabled¶

Whether BayesLine spectral-line modelling (--bayesLine) should be enabled, from likelihood.components.noise.lines (default True, matching today’s unconditional default).

before_submit()[source]¶

Modify submission files before submitting the DAG.

This method adds request_disk directives to submission files and fixes Python shebangs to use the correct environment.

submit_dag(dryrun=False)[source]¶

Submit a DAG file to the scheduler.

Parameters:

dryrun (bool) – If True then the DAG will not be submitted but all of the commands will be printed to stdout.

Returns:

The cluster ID assigned to the running DAG file.

Return type:

int

Raises:

PipelineException – This will be raised if the pipeline fails to submit the job.

upload_assets()[source]¶

Upload the PSDs from this job to the event repository.

store_assets()[source]¶

Add the assets to the Asimov store.

This stores PSDs in the central Asimov storage location for use by other productions.

store_reconstructions()[source]¶

Add signal/glitch/full waveform reconstructions to the event repository and the Asimov store.

These are the small, text-format median reconstruction files BayesWavePost produces (see collect_assets()); they are stored the same way as the PSDs, guarded per-file the same way, so one failure doesn’t stop the others.

collect_logs()[source]¶

Collect all of the log files which have been produced by this production.

Returns:

Dictionary mapping log file names to their contents. The asimov log for the production is under "production"; the rest come from log_patterns in the run directory, each capped to its last megabyte.

Return type:

dict

collect_assets()[source]¶

Collect the assets for this job and commit them to the event repository.

Since this job also generates the PSDs these should be added to the production ledger.

Returns:

Dictionary containing ‘psds’ and ‘xml psds’ keys with paths to the generated power spectral density files.

Return type:

dict

supress_psd(ifo, fmin, fmax)[source]¶

Suppress portions of a PSD.

Author: Carl-Johan Haster - August 2020 (Updated for asimov by Daniel Williams - November 2020)

Parameters:

• ifo (str) – Interferometer name (e.g., ‘H1’, ‘L1’, ‘V1’)

• fmin (float) – Lower frequency bound for suppression region in Hz

• fmax (float) – Upper frequency bound for suppression region in Hz

resurrect()[source]¶

Attempt to resurrect a failed job.

This method will resubmit the DAG using rescue files if fewer than 5 rescue attempts have been made.

result_pages()[source]¶

Link to the copied megaplot output for the report modal.

check_progress()[source]¶

Report sampling progress from the BayesWave job’s output.

BayesWave prints RJMCMC: <iteration>/<total> ... and a following logL=... line every Niter/10 iterations. After a checkpoint restart there is one log per submission, so the newest one is used.

Returns:

{"BayesWave": (iteration, logL)}, e.g. ("75000/250000", "logL=-59482.29"), or {} if the job has not started sampling.

Return type:

dict

html()[source]¶

Return the HTML representation of this pipeline.

Returns:

HTML string for displaying pipeline results in web interfaces.

Return type:

str

collect_pages()[source]¶

Collect the HTML output of the pipeline.

This copies the megaplot output to the web directory for visualization.

Module Contents¶

BayesWave Pipeline integration for Asimov.

class asimov_bayeswave.BayesWave(production, category=None)[source]¶

Bases: Pipeline

The BayesWave Pipeline integration for Asimov.

This class provides an interface between Asimov and the BayesWave parameter estimation pipeline, handling DAG file creation, job submission, and PSD collection.

Parameters:

• production (asimov.Production) – The production object.

• category (str, optional) – The category of the job. Defaults to “analyses”.

name¶

The name of the pipeline (“BayesWave”)

Type:

str

STATUS¶

Set of possible job statuses

Type:

set

Examples

>>> from asimov_bayeswave import BayesWave
>>> pipeline = BayesWave(production)
>>> pipeline.build_dag()
>>> pipeline.submit_dag()

Attributes:

bayesline_enabled

Whether BayesLine spectral-line modelling (--bayesLine) should be enabled, from likelihood.components.noise.lines (default True, matching today’s unconditional default).

config_template

The path to the bundled Liquid ini template for this pipeline.

flow

Calculate the lower frequency for the bayeswave job.

model_flags

The BayesWave model-selection flags to set (as bare, value-less keys) in [bayeswave_options] for this production’s run_mode.

run_mode

The BayesWave run mode implied by likelihood.components.

scheduler

Get the configured scheduler instance for this pipeline.

Methods

after_completion()	Perform post-processing after the BayesWave job completes.
after_processing()	Run the after processing jobs.
before_build([dryrun])	Define a hook to be run before the DAG is built.
before_config([dryrun])	Define a hook to run before the config file for the pipeline is generated.
before_submit()	Modify submission files before submitting the DAG.
build_dag([user, dryrun])	Construct a DAG file in order to submit a production to the condor scheduler using bayeswave_pipe.
build_report([reportformat])	Build an entire report on this pipeline, including logs and configs.
check_progress()	Report sampling progress from the BayesWave job's output.
clean([dryrun])	Remove all of the artefacts from a job from the working directory.
collect_assets()	Collect the assets for this job and commit them to the event repository.
collect_logs()	Collect all of the log files which have been produced by this production.
collect_pages()	Collect the HTML output of the pipeline.
detect_completion()	Detect if the BayesWave job has completed.
detect_completion_processing()	Detect that PESummary post-processing outputs exist and are valid.
eject_job()	Remove a job from the cluster.
get_prior_interface()	Get the prior interface for this pipeline.
get_state_handlers()	Get pipeline-specific state handlers.
html()	Return the HTML representation of this pipeline.
read_ini(filepath)	Read and parse a pipeline configuration file.
result_pages()	Link to the copied megaplot output for the report modal.
resurrect()	Attempt to resurrect a failed job.
store_assets()	Add the assets to the Asimov store.
store_reconstructions()	Add signal/glitch/full waveform reconstructions to the event repository and the Asimov store.
store_results()	Store the PE Summary results
submit_dag([dryrun])	Submit a DAG file to the scheduler.
supress_psd(ifo, fmin, fmax)	Suppress portions of a PSD.
upload_assets()	Upload the PSDs from this job to the event repository.
while_running()	Define a hook to run while the job is running.

build

name = 'BayesWave'¶

STATUS = {'finished', 'running', 'stopped', 'stuck', 'wait'}¶

available_outputs = ['psd']¶

log_patterns = ['logs/*.out', 'logs/*.err', 'logs/*.log', '*.err']¶

__init__(production, category=None)[source]¶

property config_template¶

The path to the bundled Liquid ini template for this pipeline.

asimov’s manage build step falls back to this property to render a production’s ini when one doesn’t already exist in the event repository (see Analysis.make_config). Without this, asimov would fall through to looking for asimov/configs/bayeswave.ini inside asimov’s own package, which no longer exists now that BayesWave support has been extracted into this standalone plugin.

build_dag(user=None, dryrun=False)[source]¶

Construct a DAG file in order to submit a production to the condor scheduler using bayeswave_pipe.

Parameters:

• production (str) – The production name.

• user (str) – The user accounting tag which should be used to run the job.

• dryrun (bool, optional) – If True, print the command without executing it.

Raises:

PipelineException – Raised if the construction of the DAG fails.

detect_completion()[source]¶

Detect if the BayesWave job has completed.

Returns:

True if the outputs required by this production’s run_mode have been generated, False otherwise.

Return type:

bool

Note

For a PSD-only run (the default) this only checks for the PSDs, exactly as before. For a run which also requests a signal and/or glitch model ("signal", "glitch", "full" or "coherence" mode), checking for the PSDs alone is not enough: the “clean” PSD-estimation phase finishes well before the signal/glitch post-processing does (they are independent models within the same BayesWave/BayesWavePost run), so that would report completion far too early. In that case this instead requires the reconstruction for every requested model and interferometer, plus the final megaplot summary page. For “coherence” mode this means both post/signal/ and post/glitch/ (BayesWave’s own default model set writes each model to its own post/<model>/ directory, same as “signal”-only/”glitch”-only mode); for “full” mode it means the combined post/full/ directory, which holds both a signal- and a glitch-prefixed reconstruction file. Either way, collect_assets()’s reconstruction glob searches across every post/*/ directory so this doesn’t need to special-case which directory name is used.

after_completion()[source]¶

Perform post-processing after the BayesWave job completes.

This includes converting PSDs to XML format, collecting output pages, storing assets, and applying PSD suppressions if configured.

property flow¶

Calculate the lower frequency for the bayeswave job.

This is required as bayeswave needs to be passed the lowest minimum frequency from the list of interferometer lower frequencies.

Returns:

The minimum frequency across all interferometers.

Return type:

float

Raises:

ValueError – If likelihood.minimum frequency isn’t present as a non-empty per-interferometer dictionary in the production’s metadata.

Note

This is computed fresh from production.meta on every access rather than cached, and deliberately not called during __init__: pipeline construction happens inside Analysis.__init__, before GravitationalWaveTransient’s own quality -> likelihood migration for a deprecated quality.minimum frequency blueprint runs. An eager read here would see the pre-migration state and could raise even for a blueprint asimov itself will happily migrate and accept.

property run_mode¶

The BayesWave run mode implied by likelihood.components.

Returns:

One of "psd" (today’s default: on-source PSD estimation only), "signal", "glitch", "full" (the combined signal+glitch model, no separate evidences) or "coherence" (BayesWave’s own default model set: signal, glitch and noise each run as genuinely separate phases, so their evidences can be compared – what likelihood.coherence test: true needs).

Return type:

str

property model_flags¶

The BayesWave model-selection flags to set (as bare, value-less keys) in [bayeswave_options] for this production’s run_mode.

Returns:

E.g. ["cleanOnly"] or ["fullOnly", "chirplets"], or [] (or ["chirplets"]) for run_mode "coherence", which deliberately sets no model-restriction flag at all so BayesWave’s own defaults apply.

Return type:

list of str

property bayesline_enabled¶

Whether BayesLine spectral-line modelling (--bayesLine) should be enabled, from likelihood.components.noise.lines (default True, matching today’s unconditional default).

before_submit()[source]¶

Modify submission files before submitting the DAG.

This method adds request_disk directives to submission files and fixes Python shebangs to use the correct environment.

submit_dag(dryrun=False)[source]¶

Submit a DAG file to the scheduler.

Parameters:

dryrun (bool) – If True then the DAG will not be submitted but all of the commands will be printed to stdout.

Returns:

The cluster ID assigned to the running DAG file.

Return type:

int

Raises:

PipelineException – This will be raised if the pipeline fails to submit the job.

upload_assets()[source]¶

Upload the PSDs from this job to the event repository.

store_assets()[source]¶

Add the assets to the Asimov store.

This stores PSDs in the central Asimov storage location for use by other productions.

store_reconstructions()[source]¶

Add signal/glitch/full waveform reconstructions to the event repository and the Asimov store.

These are the small, text-format median reconstruction files BayesWavePost produces (see collect_assets()); they are stored the same way as the PSDs, guarded per-file the same way, so one failure doesn’t stop the others.

collect_logs()[source]¶

Collect all of the log files which have been produced by this production.

Returns:

Dictionary mapping log file names to their contents. The asimov log for the production is under "production"; the rest come from log_patterns in the run directory, each capped to its last megabyte.

Return type:

dict

collect_assets()[source]¶

Collect the assets for this job and commit them to the event repository.

Since this job also generates the PSDs these should be added to the production ledger.

Returns:

Dictionary containing ‘psds’ and ‘xml psds’ keys with paths to the generated power spectral density files.

Return type:

dict

supress_psd(ifo, fmin, fmax)[source]¶

Suppress portions of a PSD.

Author: Carl-Johan Haster - August 2020 (Updated for asimov by Daniel Williams - November 2020)

Parameters:

• ifo (str) – Interferometer name (e.g., ‘H1’, ‘L1’, ‘V1’)

• fmin (float) – Lower frequency bound for suppression region in Hz

• fmax (float) – Upper frequency bound for suppression region in Hz

resurrect()[source]¶

Attempt to resurrect a failed job.

This method will resubmit the DAG using rescue files if fewer than 5 rescue attempts have been made.

result_pages()[source]¶

Link to the copied megaplot output for the report modal.

check_progress()[source]¶

Report sampling progress from the BayesWave job’s output.

BayesWave prints RJMCMC: <iteration>/<total> ... and a following logL=... line every Niter/10 iterations. After a checkpoint restart there is one log per submission, so the newest one is used.

Returns:

{"BayesWave": (iteration, logL)}, e.g. ("75000/250000", "logL=-59482.29"), or {} if the job has not started sampling.

Return type:

dict

html()[source]¶

Return the HTML representation of this pipeline.

Returns:

HTML string for displaying pipeline results in web interfaces.

Return type:

str

collect_pages()[source]¶

Collect the HTML output of the pipeline.

This copies the megaplot output to the web directory for visualization.