Chapter 5 A Small Application j 46

Albums by Artist

We might also want to include the query from the previous section and that’s fairly easy to do now. Our album.sql le looks like the following:

1-- name: list-albums-by-artist

2 -- List the album titles and duration of a given artist

3select album.title as album,

5from album

6join artist using(artistid)

Later in this section, we look at the calling Python code.

Top-N Artists by Genre

Let’s implement some more queries, such as the Top-N artists per genre, where we sort the artists by their number of appearances in our playlists. This ordering seems fair, and we have a classic Top-N to solve in SQL.

The following extract is our application’s genre-topn.sql le. The best way to implement a Top-N query in SQL is using a lateral join, and the query here is using that technique. We will get back to this kind of join later in the book and learn more details about it. For now, we can simplify the theory down to lateral join allowing one to write explicit loops in SQL:

1-- name: genre-top-n

2 -- Get the N top tracks by genre

3select genre.name as genre,

4case when length(ss.name) > 15

5then substring(ss.name from 1 for 15) || '…'

6else ss.name

7end as track,

8 artist.name as artist

9 from genre

10left join lateral

11/*

12* the lateral left join implements a nested loop over

13* the genres and allows to fetch our Top-N tracks per

Chapter 5 A Small Application j 47

14* genre, applying the order by desc limit n clause.

15*

16* here we choose to weight the tracks by how many

17* times they appear in a playlist, so we join against

18* the playlisttrack table and count appearances.

19*/

20(

21select track.name, track.albumid, count(playlistid)

24where track.genreid = genre.genreid

25group by track.trackid

26order by count desc

27limit :n

28)

29/*

30* the join happens in the subquery's where clause, so

31* we don't need to add another one at the outer join

32* level, hence the "on true" spelling.

33*/

34ss(name, albumid, count) on true

35join album using(albumid)

36join artist using(artistid)

37order by genre.name, ss.count desc;

Here, we loop through the musical genres we know about, and for each of them, we fetch the n tracks with the highest number of appearances in our registered playlists (thanks to the SQL clauses order by count desc limit :n). This correlated subquery runs for each genre and is parameterized with the current

genreid thanks to the clause where track.genreid = genre.genreid. This

where clause implements the correlation in between the outer loop and the inner one.

Once the inner loop is done in the lateral subquery named ss then we join again with the album and artist tables in order to get the artist name, through the album.

The query may look complex at this stage. The main goal of this book is to help you to nd it easier to read and gure out the equivalent code we would have had to write in Python. The main reason why writing moderately complex SQL for this listing is e ciency.

To implement the same thing in application code you have to:

1.Fetch the list of genres (that’s one select name from genre query)

2.ThenforeachgenrefetchtheTop-Nlistoftracks, whichisthe ss subquery

Chapter 5 A Small Application j 48

before ran as many times as genres from the application

3.Then for each track selected in this way (that’s n times how many genres you have), you can fetch the artist’s name.

That’s a lot of data to go back and forth in between your application and your database server. It’s a lot of useless processing too. So we avoid all this extra work by having the database compute exactly the result set we are interested in, and then we have a very simple Python code that only cares about the user interface, here parsing command line options and printing out the result of our queries.

Another common argument against the seemingly complex SQL query is that you know another way to obtain the same result, in SQL, that doesn’t involve a lateral subquery. Sure, it’s possible to solve this Top-N problem in other ways in SQL, but they are all less e cient than the lateral method. We will cover how to read an explain plan in a later chapter, and that’s how to gure out the most e cient way to write a query.

For now, let’s suppose this is the best way to write the query. So of course that’s the one we are going to include in the application’s code, and we need an easy way to then maintain the query.

So here’s the whole of our application code:

#! /usr/bin/env python3

# -*- coding: utf-8 -*-

import anosql import psycopg2 import argparse import sys

PGCONNSTRING = "user=cdstore dbname=appdev application_name=cdstore"

class chinook(object):

"""Our database model and queries""" def __init__(self):

self.pgconn = psycopg2.connect(PGCONNSTRING) self.queries = None

for sql in ['sql/genre-tracks.sql', 'sql/genre-topn.sql', 'sql/artist.sql', 'sql/album-by-artist.sql', 'sql/album-tracks.sql']:

queries = anosql.load_queries('postgres', sql) if self.queries:

for qname in queries.available_queries: self.queries.add_query(qname, getattr(queries, qname))

else:

self.queries = queries

Chapter 5 A Small Application j 49

def genre_list(self):

return self.queries.tracks_by_genre(self.pgconn)

def genre_top_n(self, n):

return self.queries.genre_top_n(self.pgconn, n=n)

def artist_by_albums(self, n):

return self.queries.top_artists_by_album(self.pgconn, n=n)

def album_details(self, albumid):

return self.queries.list_tracks_by_albumid(self.pgconn, id=albumid)

def album_by_artist(self, artist):

return self.queries.list_albums_by_artist(self.pgconn, name=artist)

class printer(object):

"print out query result data"

def __init__(self, columns, specs, prelude=True):

"""COLUMNS is a tuple of column titles, Specs an tuple of python format strings

"""

self.columns = columns self.specs = specs

self.fstr = " | ".join(str(i) for i in specs)

if prelude: print(self.title()) print(self.sep())

def title(self):

return self.fstr % self.columns

def sep(self): s = ""

for c in self.title():

s += "+" if c == "|" else "-" return s

def fmt(self, data):

return self.fstr % data

class cdstore(object):

"""Our cdstore command line application. """

def __init__(self, argv): self.db = chinook()

parser = argparse.ArgumentParser(

description='cdstore utility for a chinook database', usage='cdstore <command> [<args>]')

subparsers = parser.add_subparsers(help='sub-command help')

genres = subparsers.add_parser('genres', help='list genres') genres.add_argument('--topn', type=int)

Chapter 5 A Small Application j 51

Of course, we can change our --topn parameter and have the top three tracks per genre instead:

Now if we want to change our SQL query, for example implementing another way to weight tracks and select the top ones per genre, then it’s easy to play with the query in psql and replace it once you’re done.

As we are going to cover in the next section of this book, writing a SQL query happens interactively using a REPL tool.