iOS Web Apps and Media Session API

I’ve been rewriting my podcast/audiobook PWA (progressive web app). More about that soon. For now I’m excited about one feature.

PWA support on iOS has been lacklustre compared to Android. At least that was true when I switched back to iOS a couple of years ago. At times critical features like background audio playback and IndexedDB have been outright broken on iOS. I had pretty much given up developing my PWA because of such issues.

Well it turns out I missed Safari adding support for the Media Session API after I first built my app. This allows me to do something like this:

navigator.mediaSession.metadata = new MediaMetadata({
  title: 'Web Streams Explained',
  artist: 'Syntax - Tasty Web Development Treats',
  artwork: [{
    src: "https://example.com/artwork.jpg"
  }],
});

It works somewhat in macOS:

Although not ever image size and format works. Podcast artwork is too large I’m guessing?

I’ve yet to get any image working on the iPhone:

I’ve seen reports suggesting it did work and is now broken but fixed in the current beta. Classic Safari.

29th March 2023 update: iOS Safari 16.4 has fixed this bug.

To solve the “artwork is too big” issue I’m using the browser to generate a smaller image on the fly. The canvas element has a drawImage to resize it and toBlob to generate a temporary URL. For example:

let blobURL;
const image = new Image();
image.src = artwork[0].src;
image.addEventListener('load', async () => {
  const canvas = document.createElement('canvas');
  canvas.width = 128;
  canvas.height = 128;
  const context = canvas.getContext('2d');
  context.drawImage(image, 0, 0, canvas.width, canvas.height);
  canvas.toBlob((blob) => {
    if (!blob) return;
    if (blobURL) URL.revokeObjectURL(blobURL);
    blobURL = URL.createObjectURL(blob);
    navigator.mediaSession.metadata = new MediaMetadata({
      title: 'Web Streams Explained',
      artist: 'Syntax - Tasty Web Development Treats',
      artwork: [
        {
          src: blobURL,
          type: blob.type,
          sizes: `${canvas.width}x${canvas.height}`
        }
      ]
    });
  });
});

This works on macOS so I hope the future iOS Safari fix accepts it.

The Media Session API can also do this:

// "audio" is an HTML <audio> node
const forward = () => (audio.currentTime += 30);
const backward = () => (audio.currentTime -= 30);
navigator.mediaSession.setActionHandler('seekbackward', backward);
navigator.mediaSession.setActionHandler('previoustrack', backward);
navigator.mediaSession.setActionHandler('seekforward', forward);
navigator.mediaSession.setActionHandler('nexttrack', forward);

These actions allow me to customise the seek duration when skipping forward and backward. iOS shows the “10” icon regardless of the actual duration. I bind it to the track actions too giving my headphones preferential functionality. When I listen to a podcast I never want to skip the entire thing.

Once all this works it’ll be a huge quality of life improvement. When I built “v1” almost 2 years ago nothing like this was possible. iOS didn’t even show the seek buttons.

It’s good to see Safari development improving. Standards support is even leading in some cases. Updates and bug fixes are released at a quicker cadence. It’s probably the only Apple software that is getting better. The quality elsewhere, especially on macOS, is declining.

Update for 22nd September 2023

At some point my artwork stopped working in iOS. Now Safari 17 is out and it still does not work properly.

I’ve found that if you provide a small image, 96×96 for example, it will appear on the small player (like the screenshot above). If you tap the image it transitions to the fullscreen player. However, the image becomes pixellated; it does not swap to a larger image.

If you only provide a large image, like 512×512, the small player is a grey box (no image). Only after you tap to enlarge the player does the image appear. It then remains in place.

So basically the only way to guarantee an image is to use a small one.

Just Safari things!