QNAP 412 Turbo Nas – no more disk space

I have a QNAP 412 Turbo Nas and ran out of disk space. No problem you say?
Well.. apparently QNAP’s are very fragily when it comes to 100% disk space usage.

Symptoms could be:

  • • console: ls freezes, rm’s freeze
  • • winscp/ftp/web admin: cannot delete files (times out)
  • • reboot: NAS becomes unavailable, doesn’t show up in your network anymore, QFinder can’t find your nas
  • • even my Windows 7 box became unstable because I mapped a few network drives to my shares

15-10-2013 10-09-11

I’ve tried upgrading my firmware (remove your disks first! use 1 old hdd) but this doesn’t help. As soon as you insert your old drives, it returns to slowmotion mode.. (aka. timeouts).

I then contacted QNAP via their support forum and here’s their solution:

  1. Turn off your QNAP NAS
  2. Pull out all HDD’s
  3. Restart QNAP without the HDD’s
  4. Use QFinder to detect the IP
  5. Put back the HDD’s
  6. Don’t follow the installation wizard (web)! Instead, connect via SSH (use putty?), login credentials will probably be admin/admin
  7. Execute these commands (mount the raid):

 

mdadm -A /dev/md9 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
mdadm -A /dev/md0 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3
cd /mnt
(Under /mnt)
mkdir HDA_ROOT
cd /share
(Under /share)
mkdir MD0_DATA
 
mount /dev/md9 /mnt/HDA_ROOT
mount /dev/md0 /share/MD0_DATA -t ext4
(or mount /dev/md0 /share/MD_DATA -t ext3)
  1. Your shares should now be accesible again in /share/MD0_DATA so you can delete some data. QNAP recommends atleast 3GB free space.
  2. Reboot your NAS with this command: ‘reboot’
  3. Your NAS should now be accesible again.

 

So all in all this fixes the problem and I still have all my data. But I do think QNAP should have added an extra fail-safe to prevent this from happening. We’re not all technical people and most people won’t even know what SSH is..

Automatically create .m3u playlists with bash (for QNAP iTunes Server)

One of the cool features of my new QNAP 412 Turbo Nas is iTunes Server. This basicly allows you to create playlists, and share them among your home network. I haven’t figured out yet if it’s also possible to sync these to your iOS devices, but we can atleast (easily) share our music library to all our devices.

Qnap comes with so called ‘smart playlist’, but I don’t like those. I prefer my playlists per CD or folder. In order for this to work we’ll have to create an m3u playlist for EACH folder you want to share, this can be a daunting task.. Here’s a way to automate this. This script uses recursion to find all audio files in the parent folder + child folders. Since iTunes uses the m3u’s filename as playlist title, this script will use the folder name for this.

SSH to your server (QNAP default user = admin), and cd yourself to your music folder. Now create a bash script and give it execute permission:

cd /Qmultimedia/Music/
touch makePlaylists.sh
chmod +x makePlaylists.sh
vim makePlaylists.sh

Now enter the following code. I found this neat script here, and stripped it down to the necessary commands, and implemented the removal of old m3u’s (in case you delete/update Music folders).

#!/bin/bash

IFS=$'\n'

/opt/bin/find . -type f -name "*.m3u" -exec rm -f {} \; # remove all existing m3u's

M3Ulist="`pwd`/M3UfileList.txt"
rm -f $M3Ulist

indexCurrDir ()
{
FileList="" # initialize empty variable FileList
for FileTypes in "ogg" "mp3" "flac" "wav" # loop over file types.
do
FindFiles=$(/opt/bin/find $(pwd) -type f -iname "*.$FileTypes" | sort)
FileList=$FileList$FindFiles
done

if [ "${#FileList}" != "0" ] # do not write m3u file if file list is empty
then
CurrDir=$(pwd)
echo "$CurrDir"
m3uName=$(basename $CurrDir)

echo "Writing m3u playlist."
echo "$FileList" > "${m3uName}.m3u"
echo "$CurrDir/${m3uName}.m3u" >> "$M3Ulist"
fi
}

AllDirs=$(/opt/bin/find $(pwd) -type d | sort)

for Directory in $AllDirs
do
cd "$Directory"
indexCurrDir
done

exit 0

Now you can easily create m3u’s for your complete music folder by running:

/Qmultimedia/Music/makePlaylists.sh

You can also put this in a cronjob for automating things. Here’s more info on cronjobs on a QNAP.

Subliminal on QNAP

Installing Subliminal

So installing Subliminal on QNAP (in my case, a QNAP 412 Turbo Nas) isn’t out of the box with qpkg’s like for example SickBeard, CouchPotato or SABnzbd. For all who’s wondering, these tools are the ultimate combination for managing and updating your media library.

Subliminal is the successor of Periscope, also written in Python. It allows your to download subtitles for movies/tv shows fairly easy. Here’s a quick install guide.

First, connect to your NAS via SSH with the ‘admin’ user.
Next, make sure you update your IPKG’s + add some new ones:

ipkg update                             
ipkg upgrade                            
ipkg install git                         
ipkg install textutils	                
ipkg install python27	                
ipkg install py27-setuptools	    
ipkg install mlocate

So now you’ve got all the utlities you need so let’s download the latest version of Subliminal:

git clone -b master https://github.com/Diaoul/subliminal.git /tmp/subliminal

Now let’s install Subliminal.

cd /tmp/subliminal
/opt/bin/python2.7 setup.py install

You now should have a working copy of Subliminal on your NAS. Now how do you use this wonderful piece of software?

You can manually retrieve subs using this command:

# subliminal -l LANG DIR
subliminal -l en /Qmultimedia/Movies/

But here’s a better way. We’re lazy right?

Automating Subliminal with Sickbeard and Couchpotato

1) using cronjobs
Cronjobs is a command/script which runs regularly. Normally you’d use crontab for this, but QNAP automatically cleans this each reboot..
So according to their docs, we’ll have to do some other way. We’ll pick method 1.

echo "1 22,0 * * * /share/MD0_DATA/.qpkg/Optware/local/bin/subliminal -l en /Qmultimedia/Movies/" >> /etc/config/crontab
echo "#1 22,0 * * * /share/MD0_DATA/.qpkg/Optware/local/bin/subliminal -l en /Qmultimedia/Series/" >> /etc/config/crontab
crontab /etc/config/crontab

You now have a cronjob which checks for new subtitles for new movies at 22 and 24 hour each day. Don’t overdo do this, since you’re basicly scraping all the subtitle sites. Once or twice a day should do.

2) using extrascripts
Currently only sickbeard has this well documented.
Using the ‘locate’ command I found out where my SickBeard install is located.

cd /share/MD0_DATA/.qpkg/SickBeard/
vim config.ini

And find the extrascript and replace it with:

extra_scripts = "findSubs.sh"

Now create this script:

vim findSubs.sh

Insert script

# http://code.google.com/p/sickbeard/wiki/AdvancedSettings
subliminal -l en $1

Now quit vim and chmod this bash script.

chmod +x findSubs.sh

Congrats! Sickbeard will now automaticlly call this script as soon as an episode is downloaded meaning you will have subtitles almost directly after downloading them.

EDIT:
I noticed these cronjobs aren’t always running.. after quite some debugging I found a workaround. The reason the cronjobs aren’t working is because Subliminal is having problems scanning directories with lots of media. Here’s how I fixed it:

1 22,0 * * * /share/MD0_DATA/Multimedia/_crons/subs_series.sh
10 22,0 * * * /share/MD0_DATA/Multimedia/_crons/subs_movies.sh

subs_series.sh

#!/bin/bash
 
find /Qmultimedia/Series/ -type d | while read dir;
do
        /share/MD0_DATA/.qpkg/Optware/local/bin/subliminal -l en "$dir" --cache-dir /Qmultimedia/_crons/cache/
        echo "$dir"
done
 
touch /Qmultimedia/_crons/logs/series_`date +\%Y-\%m-\%d-\%H-\%M-\%S`-cron.log
~

subs_movies.sh

#!/bin/bash
 
find /Qmultimedia/Movies/ -type d | while read dir;
do
        /share/MD0_DATA/.qpkg/Optware/local/bin/subliminal -l en "$dir" --cache-dir /Qmultimedia/_crons/cache/
        echo "$dir"
done
 
touch /Qmultimedia/_crons/logs/movies_`date +\%Y-\%m-\%d-\%H-\%M-\%S`-cron.log