iPhone OS 3.0 music: totally incompatible
With the new OS version, Apple totally changed up the database format. Now it’s based on a whole bunch of SQLite files and there are also a few files in a format similar to the old proprietary one. There are more than likely still hashes in the critical files, and there’s also a suspicious pair of files that appear to be entirely encrypted. The SQLite format is open and certainly better than the old one, but someone still needs to interface a media player to it.
The upshot of this is that a whole new support library will need to be written or large changes in libgpod need to happen to support the new SQLite format. The DBVersion hack also isn’t going to work – either someone needs to reverse engineer the complete hashing algorithm (and maybe that encrypted file stuff), or the MusicLibrary binary on the phone needs to be patched. So, if you’re currently syncing music with free software, you’ll want to hold off on upgrading to 3.0.
Patching the check out of MusicLibrary looks trivial enough, so although it’s not as easy as the DBVersion hack, it isn’t a real showstopper. The hash algorithm looks the same as the old one, or at least quite similar (and just as badly obfuscated). Those encrypted files do scare me a bit though.
June 21st, 2009 at 5:05 pm
Marcan dont let the encryption scare you too badly, the lowest level the routines will be found is in the kernel
3.0 kernel does not have access to the AES engine without mods so apple cant use it on anything in userland 
June 21st, 2009 at 5:16 pm
I’m not so concerned about the encryption per se (I know that can’t be using any hardware security anyway). It’s just that, if the code that handles these files is obfuscated like the signature stuff, it’s going to be hard to reverse engineer and more intrusive to patch around.
June 21st, 2009 at 7:18 pm
yeah the obfuscation they do is unbelievable sometimes, they like to hide things. but i find the hardest part when it comes to reversing any userland areas is the constant Branching to and from thumb, they dont seem to like sitting with 1 instruction set to long
June 22nd, 2009 at 10:42 pm
I installed 3.0 over the weekend and couldn’t get Amarok to play nicely with it. I had reckoned that if I left it a week after the FW was released, any MobileMediaPlayer changes would have been quickly documented by other folks. How wrong I was!
I want to issue a call to arms to the OSS community to quash the bug that is MobileMediaPlayer once and for all. We need an OPEN media player on the iPhone that acts as a drop-in replacement for MobileMediaPlayer. Something like dTunes, but REPLACES the standard player with something unencumbered by DRM / obfuscated hashes / etc. My development skills lie in other areas, but surely there is a demand for such a beast?
Let’s stop chasing the iTunes / MobileMediaPlayer dragon and just build our own solution.
June 23rd, 2009 at 3:47 pm
@ankh: replacing MobileMediaPlayer means replacing lot of other frameworks, like the syncing daemon and the own MobileMediaPlayer framework [that 3rd party apps can use, in 3.0 software] meaning more incompatibility. It’s easy to interpret how the whole stuff works [maybe using a USB packet logger] and write a opensource library that most media players can use
June 23rd, 2009 at 4:06 pm
The syncing stuff is easy. I wrote a proper usbmux clone for linux (see http://git.marcansoft.com/ ) and I there are other projects which implement the syncing protocol and such. Syncing using open source is also easy, just set up an rsync server or an sftp server or something like that.
The main reason to maintain compatibility with Apple’s music library implementation is to keep App Store framework compatibility.
June 23rd, 2009 at 4:41 pm
Sure… Maintaning framework compatibility, this was my point. BTW, nice code. Are u planning to study and do the necessary changes for writing the new library format to iPhone/iPod?
June 29th, 2009 at 3:15 am
Well here is some interesting info:
I have a jailbroken Ipod Touch 1G 3.0
So I moved the new iTunes_Control Folder Structure over to some other place, than allowed gtkpod to create an old structure (2.x) there..
I then added some music. I tried to access that music, and it of course failed.. with no music listed on my ipod touch
I switched my system Over to a Windows Vista boot, then ran iTunes 8.2..
it saw my iPod, and also saw the music that I added (not the photos though), all listed there..
after an initial connection (not syncing any music just connecting up)
New I can see the music I added in my iPod.. (album covers too!)
Hope that sheds some light for you guys…
I’m going to use this method of getting music onto my ipod until something better comes along..
June 29th, 2009 at 3:18 am
I know that iTunes will happily see old-format databases (after all, it has to deal with every single iPod to date still, and older iPhone OS versions). However, I’m sure if you look at the folder structure after having iTunes connect to it you’ll find the new-format databases. In fact, it’s quite likely that this exact method that you saw is how the music gets seamlessly migrated to the new format after updating the iPhone OS using iTunes.
July 13th, 2009 at 2:40 pm
Just to be clear, does this affect just the syncing of the iPhone with iTunes and Linux? In other words, can I just hook up my iPhone using the USB, have it mount, and then drag music, photos, etc to the iPhone? From there, if that is possible, how and where. My biggest concern is turning my iPhone into a brick once I connect it to the computer. Any new info would be great.
Thanks for what you do
July 16th, 2009 at 7:38 am
This could very well be the reason for the encrypted file:
http://www.informationweek.com/blog/main/archives/2009/07/apple_kills_pal.html
September 4th, 2009 at 6:05 am
This may not be the best place to ask, but has anyone managed to write any software to sync 3.0 iPhones yet, or made any progress towards it?
September 27th, 2009 at 4:33 am
Ubuntu wiki looks like they have a new updated way to get it to work….BUT….it is by virtual machine and of course valid M$ XP OS
September 2nd, 2010 at 8:42 pm
I like this site presented and it has offered myself some sort of commitment to succeed for some reason, so thanks.
February 23rd, 2011 at 4:05 am
With havin so much content do you ever run into any problems of plagorism or copyright violation? My website has a lot of unique content I’ve either written myself or outsourced but it looks like a lot of it is popping it up all over the web without my authorization. Do you know any ways to help protect against content from being stolen? I’d really appreciate it.