Formats: reading and writing files

A format plugin turns a file into a score, a score into a file, or both. OxiGuitar has no format of its own: what it opens is what the installed plugins read.

Declaring a format

"formats": [
  {
    "id": "extab",
    "name": "Example tab",
    "extensions": ["extab"],
    "media_types": ["text/x-example-tab"],
    "read": true,
    "write": true,
    "signatures": [{ "at": 0, "text": "EXAMPLE TAB" }]
  }
]

Signatures

OxiGuitar recognises a file by its contents, from the manifests, before any plugin is loaded. Extensions are never used for that: they lie (four of the Guitar Pro 3 files in our test corpus are Guitar Pro 5). A file is of your format when any of its signatures matches, and a signature matches when every test it gives holds:

Make a signature as narrow as the format allows: when two plugins’ signatures match one file, the first plugin in folder order reads it. A plugin that declares an extension but doesn’t recognise a file with it makes OxiGuitar ask the user for a bug report; a file nobody recognises gets a feature request instead. Both point at OxiGuitar’s issue tracker, so tell users in your README where to report your plugin’s bugs.

Reading

pub trait FormatReader: Send + Sync {
    fn read(&self, data: &[u8]) -> Result<Read, Error>;
}

pub struct Read {
    pub score: score::Score,
    pub log: Vec<LogEntry>,
}

data is the whole file. Treat it as untrusted: any file a user has can be damaged or made on purpose to break readers. Every problem is an Error that says where (its offset in bytes, and a context such as “bar 12, track 2”), never a panic, an endless loop or an allocation of whatever size the file claims. The contract suite feeds every reader an empty file, a kilobyte of zeros and its own signature alone; fuzz yours too if it reads a binary format (Testing).

The log is what the file holds that the score can’t carry, or carries only roughly: an effect OxiGuitar doesn’t have, a value out of range that was clamped, a bar that didn’t add up. Report it rather than dropping it silently. Each entry has a level (Info when nothing was lost, Warning when something was), a message, the first place it happened, and a count: one entry per kind of problem, counted, not one per bar. oxiguitar-cli info FILE prints the log; the window doesn’t show it yet.

The template’s reader:

impl FormatReader for ExampleTab {
    fn read(&self, data: &[u8]) -> Result<Read, Error> {
        // The file is untrusted: anything wrong is an error saying where,
        // never a panic.
        let text = std::str::from_utf8(data)
            .map_err(|e| invalid("the file is not UTF-8 text", e.valid_up_to(), ""))?;
        let mut lines = lines(text);
        if lines.next().map(|(_, line)| line.trim_end()) != Some(MAGIC) {
            return Err(invalid("this is not an Example tab file", 0, ""));
        }
        let mut info = ScoreInfo::default();
        let mut tempo = 120.0;
        let mut bars = Vec::new();
        let mut log = Vec::new();
        for (at, line) in lines {
            let line = line.trim();
            if line.is_empty() || line.starts_with('#') {
                continue;
            }
            if let Some((key, value)) = header(line) {
                match key {
                    "title" => info.title = value.into(),
                    "artist" => info.artist = value.into(),
                    "tempo" => {
                        tempo = value
                            .parse::<f64>()
                            .ok()
                            .filter(|t| (20.0..=400.0).contains(t))
                            .ok_or_else(|| invalid("the tempo is not 20 to 400", at, "tempo"))?;
                    }
                    _ => add(
                        &mut log,
                        Level::Info,
                        "lines the format doesn't know were ignored",
                        format!("the line \"{key}: …\""),
                    ),
                }
                continue;
            }
            bars.push(bar(line, at, bars.len() + 1, &mut log)?);
        }
        if bars.is_empty() {
            return Err(Error {
                offset: Some(data.len() as u64),
                ..Error::new(
                    ErrorKind::UnexpectedEnd,
                    "the file ends before its first bar",
                )
            });
        }
        Ok(Read {
            score: score(info, tempo, bars),
            log,
        })
    }
}

The score

The score crosses the interface as its own types, oxiguitar_plugin::score (version 1 of the wire score). They mirror OxiGuitar’s model field for field, but belong to the interface: the model can change as the editor grows, and these don’t.

A plugin built for interface 1.2 can read and write version 2 instead (oxiguitar_plugin::score2), which holds what OxiGuitar added since, such as bars in free time, as lists of extensions (Interface versions). Implement FormatReader2 and FormatWriter2 and hand them out from reader2 and writer2; with both versions, the SDK uses version 2. A plugin with only a version-1 writer still gets every score: what version 1 can’t hold is added to its losses.

Score
├── info: ScoreInfo          title, subtitle, artist, album, words, music, …
├── initial_tempo            quarter notes per minute
├── master_bars: [MasterBar] time and key signature, repeats, endings, markers,
│                            tempo changes, directions (segno, coda, D.S. …)
└── tracks: [Track]          name, instrument (General MIDI), mix, chord diagrams, lyrics
    └── staves: [Staff]      tuning, capo, frets, notation and tab shown (two staves: a grand staff)
        └── bars: [Bar]      the clef; one bar for each master bar
            └── voices: [Voice]  one to four
                └── beats: [Beat]  a duration; notes, a rest or nothing; beat effects; a chord diagram
                    └── notes: [Note]  string and fret (or a pitch); tie, dynamic, note effects

The conventions:

• Strings are numbered from 1, the highest. Staff::tuning lists the open strings’ pitches from string 1 down; standard guitar tuning is [64, 59, 55, 50, 45, 40].
• Pitches are MIDI note numbers, 60 = middle C.
• Time is exact. Positions and lengths are Ratios of a whole note (num / den, den > 0); durations are a note value, dots and tuplets, outermost first.
• The shape must hold: every track has at least one staff; every staff has exactly one bar per master bar; every bar has at least one voice. A voice’s beats should fill its bar’s time signature; oxiguitar-cli info checks that (“Bar check”).
• Show what the file says. Don’t infer keys, add notation the file didn’t ask for or fix the music; OxiGuitar has tools for that, which the user chooses.

Several types have no default (a master bar’s time signature, a beat’s duration, a note’s place, a bar’s clef), so you fill them in. The template’s score:

/// The score: one guitar track in standard tuning, every bar in 4/4.
fn score(info: ScoreInfo, tempo: f64, bars: Vec<Vec<Beat>>) -> Score {
    let master_bar = MasterBar {
        time_signature: TimeSignature {
            numerator: 4,
            denominator: 4,
        },
        key_signature: KeySignature {
            fifths: 0,
            mode: KeyMode::Major,
        },
        repeat_start: false,
        repeat_end: None,
        alternate_endings: 0,
        marker: None,
        double_bar: false,
        triplet_feel: TripletFeel::None,
        tempo_changes: Vec::new(),
        directions: Vec::new(),
    };
    let staff = Staff {
        tuning: STANDARD.to_vec(),
        fret_count: FRETS,
        show_standard: true,
        show_tablature: true,
        // One bar of the staff for each master bar, each with at least one
        // voice: the core refuses a score shaped otherwise.
        bars: bars
            .iter()
            .map(|beats| Bar {
                clef: Clef::Treble8vb,
                voices: vec![Voice {
                    beats: beats.clone(),
                }],
            })
            .collect(),
        ..Staff::default()
    };
    Score {
        info,
        initial_tempo: tempo,
        tempo_label: String::new(),
        master_bars: vec![master_bar; bars.len()],
        tracks: vec![Track {
            name: "Guitar".into(),
            short_name: "Gtr.".into(),
            // General MIDI 25: steel-string acoustic guitar (from 0).
            instrument: Instrument {
                midi_program: 25,
                percussion: false,
            },
            mix: Mix {
                volume: 0.8,
                pan: 0.0,
                mute: false,
                solo: false,
            },
            staves: vec![staff],
            ..Track::default()
        }],
    }
}

oxiguitar-cli info FILE and oxiguitar-cli render FILE --png page show what OxiGuitar makes of your score, faster than the window (Testing).

Writing

pub trait FormatWriter: Send + Sync {
    fn write(&self, score: &score::Score) -> Result<Written, Error>;
}

pub struct Written {
    pub data: Vec<u8>,
    pub losses: Vec<LogEntry>,
}

A writer returns the whole file, and in losses everything the format can’t hold and left out, like the reader’s log: the user is told before anything is lost. It must answer any score the core can hold, the empty one included (the contract suite writes one). The template’s:

impl FormatWriter for ExampleTab {
    fn write(&self, score: &Score) -> Result<Written, Error> {
        let mut out = format!("{MAGIC}\n");
        let mut losses = Vec::new();
        for (key, value) in [("title", &score.info.title), ("artist", &score.info.artist)] {
            if !value.is_empty() {
                out.push_str(&format!("{key}: {}\n", value.replace('\n', " ")));
            }
        }
        out.push_str(&format!(
            "tempo: {}\n",
            score.initial_tempo.clamp(20.0, 400.0)
        ));
        // Whatever the format can't hold is left out and listed as a loss,
        // so the user is told before anything is lost.
        if score.tracks.len() > 1 {
            let n = score.tracks.len() as u32 - 1;
            add_n(&mut losses, "only the first track is written", "", n);
        }
        let Some(staff) = score.tracks.first().and_then(|t| t.staves.first()) else {
            return Ok(Written {
                data: out.into_bytes(),
                losses,
            });
        };
        if staff.tuning != STANDARD {
            add(
                &mut losses,
                Level::Warning,
                "the track is written in standard tuning",
                String::new(),
            );
        }
        for (i, bar) in staff.bars.iter().enumerate() {
            let place = format!("bar {}", i + 1);
            let four_four = TimeSignature {
                numerator: 4,
                denominator: 4,
            };
            if score
                .master_bars
                .get(i)
                .is_some_and(|m| m.time_signature != four_four)
            {
                add_n(&mut losses, "every bar is written in 4/4", &place, 1);
            }
            if bar.voices.len() > 1 {
                add_n(&mut losses, "only the first voice is written", &place, 1);
            }
            let beats = bar.voices.first().map_or(&[][..], |v| &v.beats[..]);
            let mut tokens: Vec<String> = beats
                .iter()
                .map(|b| token(b, &place, &mut losses))
                .collect();
            if tokens.len() > PER_BAR {
                add_n(
                    &mut losses,
                    "notes after the eighth of a bar are left out",
                    &place,
                    1,
                );
                tokens.truncate(PER_BAR);
            }
            if tokens.is_empty() {
                tokens.push("-".into());
            }
            out.push_str(&tokens.join(" "));
            out.push('\n');
        }
        Ok(Written {
            data: out.into_bytes(),
            losses,
        })
    }
}

Writers are declared and called through the plugin host today, and the contract suite checks them; the window’s Save and Export menus use them from the editor’s release (OxiGuitar 0.5).

One format per plugin

Our own plugins each read one format (Guitar Pro 3, 4, 5, 6 and 7–8 are five plugins), so a format can be added, improved or retired on its own, and a user loads only the code for files they open. You may put several formats in one plugin; consider whether your users would rather have them apart.