LASTSCRAPE IS DEPRECATED

Users wishing to import their listening habits from Last.fm to Libre.fm should use the LastToLibre scripts. Last.fm introduced a number of web-service methods making it possible to retrieve data without needing to scrape pages, LastToLibre makes use of these and as such is much more reliable than the old Lastscrape mechanism.

--

Using lastscrape

Overview

This page provides instructions for creating a dump of your [http://last.fm last.fm] listening history and import it into [http://turtle.libre.fm libre.fm]. These instructions are geared towards Debian and Ubuntu GNU/Linux users. Feel free to add instructions for running these scripts on your operating system of choice. There are two scripts involved with this process. The first, lastscrape.py_', is a python program for pulling listened tracks from last.fm. The second, '_import.py, is a python program for pushing listened tracks.

Important Notice: if your last.fm profile is set to "private" the lastscrape.py script won't work!!

Scraper GUI

An easy-to-use program for migrating your listening history may be at [http://encukou.github.com/lastscrape-gui/].
It provides an interface for all the steps below (scraping, removing duplicates, and uploading; obtaining the prerequisites may also be easier, especially for Windows users).

Obtaining lastscrape.py_' and '_import.py

These two scripts are located in the [http://savannah.nongnu.org/svn/?group=librefm libre.fm subversion repository].

import.py also depends on gobble.py, which you should put in the same directory as it:

Installing Dependencies

Install the dependencies for lastscrape.py


 $ sudo apt-get install python-beautifulsoup

Beautiful Soup 3.1.0 is known to give errors.[http://www.crummy.com/software/BeautifulSoup/3.1-problems.html] Problems have also been reported with version "3.0.7-1" from Ubuntu repositories. Use version 3.0.7a instead. You may use 3.0.7a by placing a copy of the file locally:


 $ cd /path/to/lastscrape.py  

 $ wget http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-3.0.7a.py -O BeautifulSoup.py 

 $ ./lastscrape.py

Running lastscrape.py


 $ ./lastscrape.py INSERT_LASTFM_USERNAME_HERE lastfm_dump.txt

You might need to run


 $ chmod +x lastscrape.py

first, to get it executable.

Running import.py

This is important, before importing, you should check your dump file for duplicate entries otherwise the import might crash half way which is not a pleasant experience. If your last.fm account was created before 2007, you will probably have duplicates entries due to last.fm bugs. To remove the duplicates lines in the dump file, you can open the file with vi and do:


 :%s/^\(.*\)\n\1$/\1/

Or you can just type from console (under Linux/Unix with awk installed):


 $ awk '!x[$0]++' lastfm_dump.txt > lastfm_dump_no_duplicates.txt

To check whether the duplicates have been removed, do:


 $ cat lastfm_dump_no_duplicates.txt | wc -l; cat lastfm_dump.txt | sort -u | wc -l

And check that the numbers are the same. There are many other ways to remove duplicates, just check online [http://www.google.com/search?q=remove+duplicates+lines+from+file].

Remove the first line ("Artist Track Timestamp") of the dump file with an editor.


 $ ./import.py INSERT_LIBREFM_USERNAME_HERE lastfm_dump.txt

The script will ask you for your password and then submit tracks in batches of 50.

If your username has spaces, replace them with +. Eg: if your username is "John Doe", then do


$ ./import.py John+Doe lastfm_dump_no_duplicates.txt

If you get gobble errors related with BADTIME just synchronise your system date using ntpdate


 # ntpdate pool.ntp.org

If you are trying to submit to a different server, use the -s option. For example:


 $ ./import.py -s http://turtle.libre.fm/ John+Doe lastfm_dump_no_duplicates.txt

Other distros

You're invited to add distro-specific instructions here.

Arch Linux

As lastscrape doesn't work with Beautiful Soup 3.1 you'll have to downgrade momentarily to 3.0.7a.

Search for beautiful-soup PKGBUILD on /var/abs/community/devel.

Then change pkgver_' to 3.0.7a and '''source''' to http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-$pkgver.tar.gz. Remove the '_md5sum line.

Also, you can get the beautiful-soup 3.0.7a from [http://repos.archlinux.org/viewvc.cgi/community/devel/beautiful-soup/PKGBUILD?viewh1.markup&revision=1.9&root=community&pathrevCURRENT community CVS]. Just remember to change the source url.

Run makepkg -i and you're done.

Fedora

In Fedora 10, install Beautiful Soup using yum:


 # yum install python-BeautifulSoup

Follow the instructions as shown in Debian/Ubuntu section.

Gentoo Linux


# emerge -av beautifulsoup

Follow the instructions as listed in the general sections above.

openSUSE

In openSUSE (or SuSE), install Beautiful Soup using zypper:


# sudo zypper install python-BeautifulSoup

Follow the instructions as shown in Debian/Ubuntu section.

Mac OS X

Just follow the instructions as listed in the general sections above, downloading !BeautifulSoup with the wget command instead of apt-get.

Windows

Get the installer for Python 2.5 (tested) or 2.6 from [http://www.python.org/download/releases/ python.org] and install.

Download [http://www.crummy.com/software/BeautifulSoup/ BeautifulSoup 3.0.7] (version 3.1.0 does not work), unzip and put the file !BeautifulSoup.py_' into subfolder '_..\Lib\site-packages of your Python install.

Put the two Scripts into your Python folder, run a command prompt and use the scripts with the same commands as above.

So, if your last.fm user-name is user you would type the following into the command prompt in your python 2.5 directory:


python lastscrape.py user myscrapedscrobbles.lfm

Your scrobbled files will start printing out to the screen, once it's done check for duplicates as described and type in:


python import.py INSERT_LIBREFM_USERNAME_HERE myscrapedscrobbles_checkedForDuplicates.lfm