Usage

Quickstart

To quickstart the benchmark model we can simply run ‘python tests/test_articubench.py’.

If we want to further specify model, dataset size and tasks to evaluate, we can specify by using parser arguments:

• --model

  choices=[‘all’, ‘baseline’, ‘segment’, ‘paule_fast’, ‘semvec’, ‘inverse’], specifies the model to benchmark

• --size

  choices=[‘tiny’, ‘small’], specifies either a “tiny” size with 4 data points used for testing purposes or “small” with 2000 datapoints used for a small first statistical analysis

• --tasks

  choices=[‘all’], specifies the tasks to evaluate on, currently only gives us “all” but specifying single tasks is WIP.

Minimal example

After installing the model a minimal working example can be found in ‘articubench/examples/minimal_example.py’.

from articubench.score import score
from articubench.control_models import synth_paule_fast, synth_baseline_schwa, synth_paule_acoustic_semvec, synth_baseline_segment


results_segment = score(synth_baseline_segment, tasks='all', subscores='all', return_individual_subscores=True)
results_paule = score(synth_paule_fast, tasks='all', subscores='all', return_individual_subscores=True)
results_baseline = score(synth_baseline_schwa, tasks='all', subscores='all', return_individual_subscores=True)
results_paule_full = score(synth_paule_acoustic_semvec, tasks='all', subscores='all', return_individual_subscores=True)

Here we have two parts, first the control model which generates our cp-trajectories and the scoring function which evaluates them.

The control models which come from the PAULE package expect at least a sequence length and depending on the task also target audio or target semantic embeddings.

Loading your own control model

Loading PAULE

To use one of the pre-configured control models seen above, we need to have the PAULE package installed, afterwards we simply load the PAULE class using one of PAULE’s or our own forward, inverse and embedder models.

Then we use the .plan_resynth() function to generate our CPs, as seen in the below example which is specified for the ‘acoustic-semvec’ task.

In-detail explanations on PAULE are given the respective documentation, therefore we will skip them here.

from paule import paule

PAULE_MODEL = paule.Paule(pred_model = FORWARD_MODEL, inv_model = INVERSE_MODEL, embedder = EMBEDDING_MODEL, device=DEVICE)

def synth_paule(seq_length, *, target_semantic_vector=None):

    results = PAULE_MODEL.plan_resynth(target_semvec=target_semantic_vector,
            target_seq_length=int(seq_length // 2))
    cps = results.planned_cp.copy()
    assert cps.shape[0] == seq_length
    return util.inv_normalize_cp(cps)

This example also showcases two of the pitfalls of the PAULE package, first the need to normalize the CPs before returning them and secondly the produced CPs are half the length of the target audio. Meaning they have half the time resolution of the target audio.