Semantic Web and Semantic Audio technologies

132nd AES Convention, 26th-29th of April, Budapest, Hungary

The emerging Semantic Web provides a powerful framework for the expression and reuse of structured data. Recent efforts have brought this framework to bear on the field of Semantic Audio, as well as information management in audio applications. This tutorial will provide an introduction to Semantic Web concepts and how they can be used in the context of music-related studies. We will outline the use of the Resource Description Framework (RDF) and related ontology and query languages. Using practical examples, we will demonstrate the use of the Music and Studio Ontologies, and show how they facilitate interoperability between audio applications and linked data sets on the Web. We will explore how signal processing tools and results can be described as structured data and utilised in audio production.

This tutorial focuses on the intersection of the fields of Semantic Audio and the Semantic Web.

Fig 1. The areas around the intersection of Semantic Audio and the Semantic Web are described in more detail.

Intended audience

This tutorial is targeted at researchers or students in Semantic Audio Analysis and Music Information Retrieval who may benefit from using the Web of Linked data as well as semantic audio tools that utilise Semantic Web technologies.

Metadata practitioners, archivists, and audio engineers interested in Semantic Audio applications and metadata management in the recording studio may also find it useful, and finally developers of Web-based music applications and mash-ups.

Outline

  • Introduction to Semantic Audio and Semantic Web Technologies
  • Motivations for using Semantic Web technologies in Semantic Audio
  • Semantic Web Applications
  • Short Hands on Session (1) Demonstrating how to query using SPARQL.
  • The Music Ontology
  • The Studio Ontology
  • Semantic Audio Tools
  • Short Hands on Session (2) demonstrating the use of Sonic Annotator and SAWA
  • Semantic Audio in Music Production

Tutorial Slides

The tutorial slides are available in pdf.

Hands on sessions

Short hands on sessions will guide those who are new to Semantic Audio or the Semantic Web to access Linked Data resources (SPARQL end-points) and use high level tools for extracting meaningful information from audio content.

In particular we will use DBPedia and the SPARQL-Wrapper library in Python, and Sonic Annotator.

To install SPARQL-Wrapper you may use:

$ sudo easy_install SPARQLWrapper

and test it by executing the python code:


# import the library
from SPARQLWrapper import SPARQLWrapper, JSON

# (1) wrap the dbpedia SPARQL end-point
endpoint = SPARQLWrapper("http://dbpedia.org/sparql")

# (2) set the query string
endpoint.setQuery("""
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dbpr: <http://dbpedia.org/resource/>
SELECT ?label
WHERE { dbpr:Dave_Brubeck rdfs:label ?label }
""")

# (3) select the return format (e.g. XML, JSON etc...)
endpoint.setReturnFormat(JSON)

# (4) execute the query and convert into Python objects
results = endpoint.query().convert()

# (5) interpret the results: 
for res in results["results"]["bindings"] :
	print res['label']['value']

Note: The JSON returned by the SPARQL endpoint is converted to nested Python dictionaries, therefore additional parsing is not required.

To install and use Sonic Annotator, download the Sonic Annotator binary for your operating system, as well as a few Vamp Plugins (see the resources section below on where to find these), and then execute the commands:

# basic help
$ sonic-annotator -h

# list available transforms:
$ sonic-annotator -l

# run a simple feature extractor
$ sonic-annotator -d vamp:transform:key -w rdf-stdout path/to/audio/file.mp3

In the last command you need to substitute vamp:transform:key with a key you obtained when using the list (-l) switch, and path/to/audio/file.mp3 with an existing .wav or .mp3 file on your machine.

Resources

1) Content-based audio analysis: