Friday, January 5, 2018

Configuring Google Home to play music with iTunes

Welcome


This post explains how I configured a Google Home (GH) to play music through iTunes on Windows.  This makes the GH handle requests like "Hey Google, play Pentatonix Christmas" by playing the closest appropriate material in iTunes over our airplay speaker system (it doesn't play the music on the GH itself).  Here's a video of the system working:



Motivation


My family is heavily ingrained in Airplay, Apples proprietary system used to play to and sync many different speakers / audio systems.  We've got several Airplay speakers, and several Apple Airport Express's that are plugged into some old speaker systems that my Dad built.  On top of that, I've got every album our family has ever bought loaded onto iTunes on the home computer (running Windows 7), and that computer perpetually plays music throughout the house via Airplay.  Anyone in the family can jump on the iOS remote App to play music / adjust volume as desired.  The system works great (as long no one is using the microwave), and has for years.

Over Christmas, we got a new Google Home (thanks Fritz!).  If you ask the GH to play music, it will try to play internet radio through itself (it's a tiny speaker too).  After doing this once, it was immediately obvious to my folks and I that the Google Home should play through our iTunes & Airplay setup, and not through itself.  I found no way to do this correctly, so I hacked together a working solution.

Application Pipeline


The most time consuming process was determining exactly how to get from point A (GH verbal cue) to point B (play respective material through iTunes).  My original goal was have the GH talk directly to the iTunes computer over LAN.  I couldn't find a way to do that, so my goal shifted to minimizing the number of services the data would need to move through in order to get the job done.  This is the pipeline I arrived at:


IFTTT

IFTTT is a popular glue application.  It allows you to glue different apps / services you already have together.  Google Assistant supports IFTTT, so it's the first step in our process.  We'll setup several IFTTT applets to steal the GH commands play, play X, loudersofter, next, etc, and send them where we want them.

Push Bullet

IFTTT is great, but the only 3rd party support they offer is through WebHooks.  I didn't want our home computer to be facing the internet, so the IFTTT actions are all PushBullet notifications.  PushBullet offers an API where you can connect and fetch the latest notifications (or if you keep the connection open, they can be pushed directly to you).

My Own Application

This application connects to a PushBullet server, waits for a new notification, parses it, uses a bit of NLP to figure out what to play with iTunes, and plays the material in iTunes.  The application is written in C#, configured to boot on machine startup in the background, and communicates with iTunes using the very dated iTunes COM SDK.  The application is able to fetch all song / album / artist data from iTunes directly using the COM SDK, however there is no clean way that I'm aware of with the SDK to "play songs by Third Eye Blind."  The workaround is a designated playlist, called Automated, that we fill with songs that match our query and then play.

Follow my Footsteps


Start by making accounts for your Google Home, for IFTTT and for PushBullet.  Open the Google Home application and make sure your device is setup and on the network.  Now, open the IFTTT website, and we'll setup several different IFTTT Applets.  You'll need to sign in to Google Assistant and PushBullet within the IFTTT App.

IFTTT

Here are my IFTTT Applets.  Note the inclusion of "please" in front of reserved phrases like "play" and "pause".  Google typically doesn't let you override the standard home commands, but if you include the keyword "please", the API will let you override them, and in practice, you don't need to say please:


All of these use the Say a simple phrase (Google Assistant) trigger and the Push a note (Push Bullet) action.  The triggers can be whatever you want them to be, but if you're trying to use the software I've developed the actions have to identical to what I have.  The last Applet takes in custom phrases with the Say a phrase with a text ingredient trigger.  It uses the same Push a note action as before, but this time it includes the ingredient as a message for the notification.



You can test these without anything else setup, and if you've done everything correctly, the GH should respond with the correct reply.

Push Bullet

Log on to the Push Bullet website, go to settings, and click Create Access Token.  PushBullet will show a long key that corresponds to your account.  Copy it (or save it to a text file).  You'll need to enter it into my custom C# Application.

My Custom Application

Here is the installer for the most recent Windows Application.

Here is source code for the Windows Application. 

The iTunes COM SDK should be installed when you installed iTunes.  If you're having difficulties, make sure that iTunes is installed in the default directory.  I don't know how long apple will continue to support the iTunes COM SDK, but it at least works with iTunes 12.7.2.60 on Windows 7, 8, & 10.

When you run my custom application for the first time, you'll have to copy the PushBullet access token from into the GUI.  It should remember it in the future.  I've gone ahead and set the application to run on startup on our machine.  If it has a valid PushBullet secret, it will minimize itself shortly after launch to the taskbar (where you can only exit by right clicking and selecting exit).

That's it!


Good luck!


EDIT 1:

Push bullet accounts periodically expire (around every 120 days).  If the system stops working, try logging into PushBullet on your browser and trying again.

33 comments:

  1. Sam, this is sweet & works perfectly. Once question: when I tell it to play, instead of "unpausing" it restarts a playlist (starts over instead of continuing where it left off). I typically use it to randomly play albums so instead of picking up where it left off, it selects a new random album each time. Any suggestions on how to make it simply "unpause"?

    ReplyDelete
    Replies
    1. Dave, glad it works for you. The issue you're describing was in my software. I've fixed it, and updated the installer for you. Reinstall, and let me know if that works.

      Delete
    2. Awesome thx! Still having some issues...now it still stops (doesn't pause) though will not play. Maybe it's because you're using iTunesAppClass()?.Resume() now while "Stop" still uses iTunesAppClass()?.Stop() instead of iTunesAppClass()?.Pause()?

      Delete
    3. Makes sense to me! I've changed Stop() to Pause()

      Delete
    4. You are the man. OK now Pause() is actually pausing which is great...however nothing's playing still. Maybe it should be Play() instead of Resume() after all? I'd try this myself though don't know how to create the .msi after I compile the code. :( Feel free to explain that so I can self-serve!

      Delete
    5. Back to Play() it is! You can build the .msi by right clicking the iTunesGoogleHomeSetup project inside Visual Studio and clicking build, the MSI will be in the iTunesGoogleHomeSetup folder.

      Delete
    6. Working swimmingly now...thx again!

      Delete
  2. Hi Sam, this is pretty much the solution for everything I've researched about voice control with iTunes, but in my case I can only play, pause or skip tracks... If I say for example "play Highway to Hell" nothing happens, would you be so kind to help me with this?

    Thank you so much in advance

    ReplyDelete
    Replies
    1. I saw in your video the commands appearing on the matching side and nothing appears in mine as well

      Delete
    2. I'll give it my best shot! Does the google home echo back "I'm playing XXX with iTunes"?

      Delete
    3. Thanks, really!

      Yes, it does, but then nothing happens on iTunes
      If I just ask to play, it starts playing the first song on the playlist

      Delete
    4. Hmmm. can you post a screen shot of the play ingredient recipe?

      Delete
    5. I believe this is what you are looking for, please let me know
      I just asked to play some song/artist
      https://s14.postimg.cc/ol56zofjl/image.png

      Delete
    6. Thanks for that image. It's a null pointer error (some of your songs have null names / artists / albums. I'm not sure why they are null and not the empty string). I've updated the code, try this new version. Hopefully it works, and if it doesn't, hopefully we get a different error! Post an image if it is a new error.

      Delete
    7. I did get a different error
      Now, this is what appears on the app then it just gets stuck and I have to manually end it via task manager, still plays and stops but when I ask for a song or artist it just gets stuck.. Let me know if you need more info from my side
      https://image.ibb.co/ezzqpH/itunes.png

      Delete
    8. There is a chance your computer is still loading statistics about your library. Let' it run for some time and see if it ever comes through. Also, I've got a new build up that should be slightly faster.

      Delete
    9. Welp... 36.292 hahaha
      Okay, I'll get the new build and try to run it for an hour or two and I'll report back :)

      Delete
    10. Hi, little update, it works!
      But it takes forever to put the command into action...
      If I ask to pause/play it does it as soon as google stops speaking, but when asking for songs/artists it takes forever
      See below:
      https://image.ibb.co/inXfeS/ghme.png
      Is it because I have a ton of songs in my library? I would assume it's that...

      Delete
    11. Yea, it is because you have a ton of songs in your library. The way the iTunes Comm SDK tracks songs changes based on what playlist / sorting you are in, so when you ask the app to play a single song I have to loop through all songs in the library and put the ones that match the query into the "Automated" playlist. If I had a better understanding of the Comm SDK I might be able to find a workaround, but for now, you're stuck.

      As a workaround: you should get much faster results by asking the Google Home to "play playlist XYZ", because if the playlist already exists we don't have to create the automated playlist. So if you make an "AC/DC" playlist and say play playlist ACDC it should respond much faster.

      Delete
  3. This works perfectly! Thank you! My only issue is that whenever I restart my computer, I have to load a new Pushbullet access number. I have the shortcut to start on startup via shell:startup but it always needs a new access number. Am I missing something?

    ReplyDelete
  4. Hmmmm... Does the application print out paths on startup? Something like:

    Path: C:\ProgramData\Something Silly\iTunes Google Home
    Key: C:\ProgramData\Something Silly\iTunes Google Home

    ???

    ReplyDelete
    Replies
    1. Just following up. Nothing is printing out on startup

      Delete
    2. I just uploaded a new build for you. You should uninstall + redownload + reinstall. Maybe a 50 / 50 shot this fixed your not remembering the app token issue. If it didn't fix your issue, it has better logging so we can see what's happening.

      As per why it's not playing playlists, what does it say on the right when you say "Play playlist test"??

      Delete
    3. I will check for you when I get back home on Monday. Thanks for the update and I will try it out then!

      Delete
    4. The Pushbullet code is now stored and I no longer have to enter it on restart! Thanks!
      Playlist isn’t playing though. Still trying different things, but it isn’t a feature I use really, so I am happy with it as is 🙂

      Delete
    5. You're welcome! What does it say on the right hand side of the screen (yellow text) when you say "Play playlist test"

      Delete
  5. I don’t have anything “printing out”. The only iTunes Google Home related item immediately showing from it starting at shell is the pushbullet secret - start pushbullet screen. The console: and the matching: fields are just black and empty.

    ReplyDelete
  6. After entering the new Pushbullet access token, it minimizes properly. I expanded it, and the console field is going through its pushbullet heartbeat timestamps. The matching: field remains blank and black.

    ReplyDelete
  7. After playing with it, everything works great otherwise, except it doesn’t play a playlist. That doesn’t matter much to me personally as I mainly use the other features but when I say “play playlist test” GH responds but can’t connect to the playlist. After awhile it ends up playing something else instead.

    ReplyDelete
  8. Playing with again for the first time in a while...in your app I can enter a Pushbullet Secret, though I can't click "Start Pushbullet" (it's grayed out). Any help?

    ReplyDelete
  9. ...plus I'm getting no Console messages. Thx!

    ReplyDelete