SheetSage2: A Primer on ABC Transcription in Generative Audio

As you already know, a song recording contains voices and instruments mixed together, but what you may not know is that SheetSage2 can turn that sound into an editable musical outline. The output gives musicians something they can inspect, and gives another music model structured information to learn from.

Its main output is a lead sheet: a melody with chord symbols. Think of the tune you'd hum, with the chords that accompany it. SheetSage2 also predicts information such as beats and song sections. You get a useful outline of the song, rather than a complete score for every instrument. This process is called transcription. The model is writing down music from an existing recording.

One way it writes that music is ABC notation. Despite the name, ABC here is a text format for musical scores. Calling it an ABC training method would be misleading.

For example, the letter M followed by a colon and four slash four specifies four-four meter. The letter K followed by a colon and F sharp minor specifies the key. Other text represents the notes and their lengths. Those characters describe music in a form you can read and edit.

How does the recording become that text? First, an audio encoder turns the sound into features the model can work with. A decoder uses those features to predict musical events, including melody notes and chords. Then a notation builder organizes the predictions into ABC. The learning happens in the model; ABC is how the resulting score is represented.

That score has a role in building YuE2, the project's music-generation system. The project uses extracted scores as symbolic training targets. In everyday terms, they provide musical structure for the system to learn to predict.

There's also a separate use after training. You can review an ABC score from SheetSage2 and give it to YuE2 alongside lyrics and a requested style. The score then helps guide the music it generates. Training targets help build the model; this supplied score helps steer a particular output.

The reported transcription results are promising. On a pop-music benchmark called RWC-Pop, the vocal-melody F1 score rises from about sixty-three for SheetSage1 to about eighty-three for SheetSage2. F1 balances finding the correct notes against making incorrect predictions.

But this test measures pitch classes. A low C and a high C count as the same pitch class, so that result doesn't establish correct octaves. And the sources we reviewed don't isolate how much ABC itself improves YuE2's generated music.

The useful connection is that SheetSage2 turns recordings into editable scores, which can support learning and guide generation. That's a concrete role for symbolic music, with evidence for better transcription and an open question about ABC's separate contribution.