Perform a sheet-music file

ferraus score perform reads written music and plays it: it turns the notation into a MIDI performance (repeats unrolled, dynamics and articulations applied) and renders that with an instrument. This walkthrough uses ABC and MusicXML files. Outputs are trimmed copies of what ferraus 0.1.0 printed.

You need: a notation file: MusicXML (.musicxml, .xml, .mxl), ABC (.abc), MEI, LilyPond (.ly), MIDI, or ferraus score JSON. Most notation apps export MusicXML, and thousands of folk tunes are published as ABC.

Printed sheet music (a PDF or a page image) works too: score perform reads it with optical music recognition first. Section 5 shows that path and what to check.

1. Write or get a score

Save this as tune.abc (the first lines of Beethoven's Ode to Joy):

X:1
T:Ode to Joy (excerpt)
C:Beethoven
M:4/4
L:1/4
Q:1/4=96
K:D
!mf! F F G A | A G F E | D D E F | F3/2 E/ E2 |
F F G A | A G F E | D D E F | !p! E3/2 D/ D2 |]

Check that ferraus reads it the way you meant:

ferraus score check tune.abc

{"ok": true, "errors": 0, "warnings": 0, "parts": 1, "measures": 8, "notes": 30, "issues": [], "failing_measures": []}

check finds measures whose durations do not add up, voices that overlap, and constructs ferraus cannot play, with the measure number for each.

2. Perform it

ferraus score perform tune.abc --instrument piano -o tune.wav --json

{"ok": true, "command": "score perform",
 "outputs": [{"path": "tune.musicxml", "role": "musicxml"}, {"path": "tune.mid", "role": "midi"},
             {"path": "tune.wav", "role": "audio", "duration_s": 22.0}],
 "params": {"tempo": "auto", "repeats": "on", "dynamics": "on", "articulation": "on", "humanize": "0%", ...},
 "result": {"engine": "none (abc input)",
            "performance": {"duration_s": 20.0, "notes": 30, "measures_played": 8, "order": [1, 2, 3, 4, 5, 6, 7, 8], "tempo_bpm": 96.0},
            "render": {"instrument": "builtin:piano", ...}}}

Beside the audio you get the MusicXML and the MIDI performance, so you can open either in another program. order lists the measures in the order they were played, which is where you check that repeats and first/second endings were followed.

Listen for: the tempo marking (96 here; --tempo 120 overrides it), the !mf! to !p! dynamic change in the last line, and the dotted rhythm in bars 4 and 8.

3. Shape the performance

ferraus score perform tune.abc --tempo 120 --swing 50% --humanize 30% --seed 7 --instrument strings -o tune-strings.wav
  • --humanize 30% adds small random timing and velocity changes (seeded, so the same seed gives the same performance).
  • --swing 50% delays the off-beat eighths halfway to a triplet feel.
  • --instrument-map "P1=piano,P2=strings" chooses an instrument per part in a multi-part score.
  • --repeats off, --dynamics off, and --articulation off switch each interpretation step off.
  • --click on adds a metronome track.

For better instruments than the built-in synth, install the General MIDI sound bank once (31 MB) and use General MIDI names:

ferraus sounds install gm-basic -y
ferraus score perform melody.musicxml --instrument "Acoustic Grand Piano" -o melody.wav

4. Just the MIDI, or another format

ferraus score play tune.abc -o tune-perf.mid --click on      # the performance as MIDI, no audio
ferraus score convert tune.abc -o tune.musicxml              # ABC to MusicXML
ferraus score to-json tune.abc -o tune.score.json            # the editable JSON model

The JSON model is what an agent edits: change a note's pitch or duration, then ferraus score from-json tune.score.json -o fixed.musicxml and perform it again. Small corrections can also be applied as an edit list with ferraus score read tune.musicxml --fix edits.json -o fixed.musicxml.

5. From printed sheet music

score read and score perform accept a PDF or a page image (PNG, JPEG, TIFF, ...) and read it with the built-in recognizer: oemer's two segmentation networks (MIT license, about 110 MB, downloaded once) find staff lines, noteheads, stems, rests, clefs, and accidentals, and ferraus rebuilds the measures from them. Allow the download with -y the first time (or run ferraus models pull omr/oemer omr/oemer-seg):

ferraus score perform minuet.pdf --instrument piano -o minuet.wav -y --json

{"ok": true, "command": "score perform",
 "outputs": [{"path": "minuet.musicxml", "role": "musicxml"}, {"path": "minuet.mid", "role": "midi"},
             {"path": "minuet.wav", "role": "audio", "duration_s": 26.0}],
 "models": [{"name": "omr/oemer", ...}, {"name": "omr/oemer-seg", ...}],
 "result": {"engine": "builtin",
            "performance": {"duration_s": 24.0, "notes": 73, "measures_played": 16, ...}}}

Reading takes 15–60 seconds per page on a laptop CPU. PDFs are turned into images with pdftoppm (poppler) or mutool when one is installed; on macOS without them, sips reads the first page only. --pages 2-3 picks pages and --dpi the resolution.

Check before you listen. Recognition is not perfect, so read first and look at the review:

ferraus score read minuet.pdf -o minuet.musicxml --review

The result says whether the score passes score check (check_ok), lists measures whose durations do not add up (failing_measures), and gives every measure a confidence with the reason when it is low (for example dot added to fill the measure or uncertain notehead). Fix what is wrong with an edit list (--fix edits.json, see the cookbook) or in a notation app, then perform the corrected MusicXML.

What it reads: clean printed (engraved) music: treble, bass, and C clefs; key and time signatures; notes from whole to 32nd with beams, flags, and dots; chords; rests; sharps, flats, and naturals; one or more staves per system, including piano grand staves. On the project's engraved test set (23 pieces, 975 notes, read both as PNG and as PDF) every pitch and duration came out right; scans and other engravers' fonts will do worse, which is why the review matters.

What it does not read: lyrics and other text, tuplets, grace notes, ties, slurs, dynamics, articulations, repeats and endings, and two voices on one staff (they come out as chords). Handwritten music and phone photos of pages are best-effort. For those, the optional Audiveris bridge (ferraus doctor --install-audiveris, then --engine audiveris) is often better.

Reference

score perform · score play · score check · score read · score convert · score to-json · render