Set Mac OS X Dock Size in Pixels

Published on

This is a one line command you can run in your terminal. Replace 64 with the your desired size

defaults write com.apple.dock tilesize -int 64; killall Dock
  • OS X
Loading...

Implementing a Database File Storage Backend for Django

Published on

Disclaimer: In many cases, storing files in the database is a BAD idea. Your database will easily become bloated and the performance can degrade rapidly. See this StackExchange post for more information.

I think one of the most interesting and fun exercise any Django developer can do is to implement a database storage system. It doesn't take long to implement one, and you get a much better understanding on how storage works in Django.

I was originally inspired by this question on Stack Overflow and ended up implementing the entire backend in one afternoon. The entire implementation of the model and storage class was surprisingly short.

The only implementation required for writing a custom storage backed is the Storage class itself. Since files are being saved to the database here, Django models can be used to help simplify accessing and saving files.

# models.py

@python_2_unicode_compatible
class DBFile(models.Model):

    content = models.BinaryField(editable=False)
    name = models.CharField(max_length=255, unique=True)
    size = models.IntegerField(default=0)
    created_on = models.DateTimeField(auto_now_add=True)
    updated_on = models.DateTimeField(auto_now=True)

    def save(self, *args, **kwargs):
        self.size = len(self.content)
        super(DBFile, self).save(*args, **kwargs)
# storage.py

class DBStorage(Storage):

    def __init__(self, base_url=None):
        if base_url is None:
            base_url = settings.MEDIA_URL
        if not base_url.endswith('/'):
            base_url += '/'
        self.base_url = base_url

    def _open(self, name, mode='rb'):
        f = DBFile.objects.get(name=name)
        return ContentFile(f.content)

    def _save(self, name, content):
        name = self.get_available_name(name, max_length=255)
        DBFile.objects.create(content=content.read(), name=name)
        return name

    def get_valid_name(self, name):
        return name[:255]

    def delete(self, name):
        DBFile.objects.filter(name=name).delete()

    def exists(self, name):
        return DBFile.objects.filter(name=name).exists()

    def size(self, name):
        return DBFile.objects.only('size').get(name=name).size

    def url(self, name):
        return urljoin(self.base_url, filepath_to_uri(name))

    def created_time(self, name):
        return DBFile.objects.only('created_on').get(name=name).created_on

    def modified_time(self, name):
        return DBFile.objects.only('updated_on').get(name=name).updated_on

For a complete implementation, see django-db-storage. It is a custom storage I built based on the code above and comes with an admin interface that makes accessing and managing files easy.

  • django
  • python
Loading...

Installing Older Versions of Python with Brew

Published on

I created a brew tap for installing older versions of python on my own machine. The source code is located at https://github.com/derekkwok/homebrew-python.

Prerequisites

The formula openssl needs to be installed and linked, otherwise the python installed will be without ssl support.

$ brew install openssl
$ brew link --force openssl

Install

Start by adding the brew tap derekkwok/python

$ brew tap derekkwok/python

Adding this brew tap will make all the formulas in the Github repository derekkwok/python available to be brew installed. The following formulas are available in this tap:

  • python26 - version 2.6.9
  • python31 - version 3.1.5
  • python32 - version 3.2.6
  • python33 - version 3.3.6
  • python34 - version 3.4.4

Example 1: installing python 3.4

$ brew install python34
$ python3.4 --version
Python 3.4.4

Example 2: installing python 2.6

$ brew install python26
$ python2.6 --version
Python 2.6.9

These formulas will not overwrite any existing symlinks to python and python3. As an example, if the python3 formula is installed from the official brew repository, the python3 symlink will still point to the original brew installation of python.

The older versions of python will work with virtualenv

$ mkvirtualenv venv --python=/usr/local/bin/python3.4
(venv) $ python --version
Python 3.4.4
(venv) $ pip --version
pip 7.1.2 from ... (python 3.4)

Uninstall

To uninstall, use the commands brew uninstall and brew untap E.g.

$ brew uninstall python26 python34
$ brew untap derekkwok/python
  • python
  • homebrew
Loading...

Encoding Anime from MKV to MP4

Published on

By far the most common distribution format for fansubbed videos is Matroska. This is a container format, and unfortunately it won't play on a device with only MP4 support - such as chromecast, Apple TV, or PS4.

This is a guide on converting MKV to MP4, and will also include burning subtitles directly into the video file.

Required Software

This guide uses ffmpeg, and it can do almost anything related to video formats. It is also used in many popular video conversion software.

Make sure that ffmpeg is compiled with libass and libfdk support.

Encoding Basics

Let's start by extracting the subtitles out of a video. If the video already has subtitles burned in, nothing will be extracted.

ffmpeg -i input.mkv subtitles.ass

A file named subtitles.ass will be created. ass is the standard extension used for Advanced SubStation Alpha subtitles.

Most fansubbed anime use fonts (e.g. karaoke) that aren't installed on computers. These fonts can be extracted with the command

ffmpeg -i input.mkv -dump_attachment:t

These attachments - most of which are usually font files - need to be placed in a directory where the operating system recognizes them. For example, on Ubuntu, the files can be placed in ~/.fonts directory.

We can then re-encode the video to mp4 with subtitles burned in:

ffmpeg -i input.mkv -vf ass=subtitles.ass output.mp4

Additional Options

Additional commands can be passed into improve the file size, speed of encoding, video quality, audio quality and CPU requirements for playing back the video. Here's a few that I use:

-crf 23

The crf value affects video quality and should be a number between 0 and 51. The lower the value, the higher the quality of the video and larger file size. For 10-bit videos, the crf value can range from 0 to 63. A crf value of 18 is generally considered almost lossless.

-tune animation

This is a built-in ffmpeg tuning for encoding animation video files.

-preset medium

The preset options affects encoding speed and output file size. Using a slower preset will provide smaller file size at the cost of slower encoding. Options include: ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo.

-profile:v high

profile:v can be used control support for playing back videos. For example, baseline can be played on almost any devices, while high will limit playback to Apple TV 3, iPhone 4s and later. A more detailed compatibility chart can be found at https://trac.ffmpeg.org/wiki/Encode/H.264#Compatibility.

Encoding Audio

The preferred audio codec to use is AAC, designed to be the successor of MP3. If the source file's audio codec is already AAC, just copy the stream:

-c:a copy

If you need to encode, specify AAC as the codec, and add a bitrate for the stream:

-c:a libfdk_aac -b:a 160k

Putting it all together

Commands can get very long, don't be surprised if your command ends up looking like this:

ffmpeg -i input.mkv \
       -vf ass=subtitles.ass \
       -crf 23 \
       -tune animation \
       -preset medium \
       -profile:v high \
       -c:a lib_fdkaac \
       -b:a 160k \
       output.mp4

I have a script which I use for video encoding. See https://gist.github.com/derekkwok/4077509

Additional Information

See http://ffmpeg.org/trac/ffmpeg/wiki/x264EncodingGuide for more information on encoding.

  • ffmpeg
  • python
Loading...

Implementing a Compressed TextField in Django

Published on

I recently answered an interesting question on Stack Overflow - how can text be stored compressed in the database?

Django itself doesn't offer anyway to store data compressed, but this is actually quite easily implemented using zlib (also used by gzip for compression)

class CompressedTextField(models.TextField):

    def __init__(self, compress_level=6, *args, **kwargs):
        self.compress_level = compress_level
        super(CompressedTextField, self).__init__(*args, **kwargs)

    def to_python(self, value):
        value = super(CompressedTextField, self).to_python(value)
        return zlib.compress(value.encode(), self.compress_level)

    def get_prep_value(self, value):
        value = super(CompressedTextField, self).get_prep_value(value)
        return zlib.decompress(value).decode()

Here is an example using CompresssedTextField

class MyModel(models.Model):

    # zlib offers compression levels 0-9
    #    0 is no compression
    #    9 is maximum compression
    big_text = CompressedTextField(compress_level=9)

As an example, I generated 1000 words of lorem ipsum that used 7,411 bytes. After compression it was reduced down to 2,360 bytes - a savings of 5,051 bytes or 68% reduction in size!

As a side note, most databases already have some built-in form of compression for stored data, which means this solution may not yield actual storage benefits. Another down side of using such a field is django filters such as contains and icontains no longer work.

  • django
  • python
Loading...