0 comments

I’ve been using GratisDNS for many, many years and have pretty much been happy with the technical side of the service. There have been a few glitches here and there but seeing the big picture, I rarely had any problems. Except for a few times when their DNS servers have been extremely slow to pick up changes which has been a huge annoyance.

I have looked in their support forums a few times, just to check if there was any news from the people behind GratisDNS about why zones weren’t being created, changes weren’t being delegated etc. More often than not, there wasn’t any news and we users were just forced to wait and see what happened. I have been stuck in situations with customers for whom I host mail or websites, where I have had to explain to them that an external dependency was the reason for their domain not working correctly. On a side note, when browsing the support forums (which are in Danish), more often than not forum admins (or “DNS Gurus” as they are called there) would answer users in an extremely rude and obnoxious way. It seems they might have forgotten that they are actually providing a service and their users who really are the main reason for their existence.

Yesterday I needed to delegate a new domain to GratisDNS, so I created the zone using the admin interface (this was at about 10am) and set about waiting for them to reload their name servers at some time between 1pm and 2pm (like it says in their wiki). After 2pm I could still not query the zone on their primary server, so redelegating the domain wasn’t possible. I tweeted about it and a friend replied on Facebook that I should take a look at QuickDNS.

I created a user, created my first zone and tried to query their DNS server. It answered. I requested a delegation from DK-Hostmaster and it went through straight away. The time it took from creating a user to pressing “OK” on the delegation form and receiving a positive response was less than 4 minutes.

Another really cool thing about QuickDNS is the way you can use templates to build up your zones. So I can have templates for DNS setup for all my different systems (like mail servers, IPv6 addressing, web, admin, stats etc.) and just piece them together for each of my zones. This makes it really easy to handle quite complex DNS setups where many domains don’t necessarily have the exact same setup.

This morning I delegated the rest of my domains to QuickDNS. As soon as the change has propagated through the rest of the internet (I’ll give it 24 hours to be on the safe side) I’ll be deleting my user from GratisDNS. They can keep their slow systems and rude admins to themselves - the zone I created on their servers yesterday still isn’t answering.

Oh – and Merry Christmas Smile

0 comments

WARNING: This post is long and pretty technical and probably quite boring if streaming video isn’t your thing. It’s mostly meant as a reminder to myself as to how I got this working :-)

I’ve always been kind of keen on streaming video, webcams and so on and have for some time tried to work out the best way of delivering a live/almost live video feed to my iPhone (for free that is).

Last night I finally got a breakthrough while searching the web for possibilities. I had for some time had a pure HTML5 stream running using Theora/OGG encoded with VLC, but this was only viewable with Google Chrome.

HTTP Live Streaming is basically a trick used by Apple and a few others to deliver video content over anything from 3G to WI-FI connections. It works by splitting a video into small chunks and then downloading them one by one to the client which then puts them back together. To know which bits to download, the protocol makes use of an index file which is really just a list of URLs. All this is pretty easy with static content – just transcode your video into the correct format (H264 in an MPEG2 container) and chop it up. Then provide an index file (.m3u8) and point your device at it. There is software out there that can do this for you.

But for live video there’s a challenge. First of all, there is no set length of the video – it just goes on and on. So what most of the HTTP Live Streaming software out there does is that it stores the video feed in chunks, just like with a static video, and then dynamically updates the index file to point to the last say 5 chunks. The client continuously downloads the index file, so it always knows the name of the chunk to download and play next. The main drawback of this method is that the content the client sees is not really live, but instead has a latency of <chunk size * number of chunks> – usually about 60 seconds.

However, if you can live with this (and I think most people can if it’s just for fun) here’s how to set it up using the latest VLC Nightly Build which contains the httplive module. To get this to work, you must be running a web server on the same machine as your webcam.

  • First of all, download and install the latest VLC Nightly
  • Next, create a directory somewhere in your web root, where you want the video to reside. I chose c:\inetpub\wwwroot\live
  • Next up is to figure out the right options with which to launch VLC. It turns out that quite a few are needed; my commandline looks like this:

    "C:\Program Files (x86)\VideoLAN\VLC\vlc.exe" dshow:// --sout=#transcode{vcodec=h264,vb=512,scale=1,acodec=none,sfilter=marq{marquee="[%%d-%%m-%%Y %%H:%%M:%%S]",position=8,size=18},venc=x264{aud,profile=baseline,level=30,keyint=15,
    bframes=0,ref=1,nocabac}}:duplicate{dst=std{access=livehttp{seglen=10,delsegs=true,numsegs=5,
    index=c:/inetpub/wwwroot/live/mystream.m3u8,index-url=http://homeserver.local/live/mystream-########.ts},mux=ts{use-key-frames},dst=c:/inetpub/wwwroot/live/mystream-########.ts},dst=std{access=http,mux=ts,dst=:8090/video.mp4}}

    Take note of the paths and URLs in there – they have to be correct. Here’s a description of what’s going on:
    • The first part is just the path to the VLC executable.
    • dshow:// tells VLC to take the first available DirectShow device (my webcam).
    • --sout=sets up the default output chain. What follows is a chain of setup options that describe the output.
      • #transcode{ we want to transcode the input (from the webcam) to one or more different formats
      • vcodec=h264use the h.264 codec for output
      • vb=512 use a video bitrate of 512kb/s
      • scale=1 don’t scale the video – output the same size as the input
      • acodec=none I don’t want audio in this video so I explicitly unset the Audio Codec.
      • sfilter=marq{marquee="[%%d-%%m-%%Y %%H:%%M:%%S]",position=8,size=18}filter that adds the current time and date to the video stream.
      • venc=x264{aud,profile=baseline,level=30,keyint=15,
        bframes=0,ref=1,nocabac}
        Sets up the video encoding profile for the h.264 stream. This is to ensure that the iPhone can display the stream.
      • } ends the transcode section.
    • :duplicate{ Tell VLC that we want to chain multiple outputs. In this case I want 2 – one for HTTP Live Streaming and also a regular HTTP MP4/h.264 stream for viewing with VLC.
    • dst=std{access=livehttp{seglen=10,delsegs=true,numsegs=5,
      index=c:/inetpub/wwwroot/live/mystream.m3u8,index-url=http://homeserver.local/live/mystream-########.ts},mux=ts{use-key-frames},dst=c:/inetpub/wwwroot/live/mystream-########.ts

      This is quite a mouthful. I’ll split this one up by itself:
      • access=livehttp Set up the livehttp stream
      • seglen=10The length of each segment, in seconds
      • delsegs=true Delete segments that have rolled out of scope
      • numsegs=5 The number of segments to have queued up
      • index=c:/inetpub/wwwroot/live/mystream.m3u8 This is the path to where the index file will be located. It has to be in the web-accessible directory we set up earlier.
      • index-url=http://homeserver.local/live/mystream-########.ts This is the URL to the file fragments that will be embedded in the index file. The hash-characters have to be there – they are placeholders for the always-incrementing index file numbers.
      • mux=ts{use-key-frames} Set up the multiplexer and tell it to embed the key frames we set up in the video encoding profile earlier
      • dst=c:/inetpub/wwwroot/live/mystream-########.ts Finally, this is the file name template for the stream segments.
    • The last dst= part in the command line is just to set up the alternative H.264/MPEG4 stream that can be viewed using VLC from a PC.
  • Now that we have the command line set up, there is one more thing to be aware of. When running under Windows, the livehttp module won’t be able to automatically overwrite the index file every time a new segment is written to disk. To overcome this, it instead writes the new index file to a temporary file in the same directory. What I then did was to create a small PowerShell script that could copy the temp file over the index file at an interval shorter than the segment length.
    This is done in 4 lines of code. I placed the file in the output directory and named it rename.ps1:
    while ($true) {
        Copy-Item mystream.m3u8.tmp mystream.m3u8
        Start-Sleep -s 2
    }
  • Next up we have to configure the web server to spit out the correct MIME type when serving the .m3u8 and the .ts files, so that the client device knows what’s coming. I added the following two MIME mappings in my IIS:
    .m3u8  application/x-mpegURL
    .ts    video/MP2T
  • Now you’re just about ready to start serving video. All you have to do is set up an HTML5 page on a web server that you can access from your iPhone: 
    <!DOCTYPE html>
    <html>
    <head>
        <title>Live Cam</title>
    </head>
    <body>
        <div id="player">
            <video autoplay="true" controls="controls" width="640" height="480">
                <source src="http://homeserver.local/live/mystream.m3u8" />
                Your browser does not support HTML5 streaming!
            </video>
        </div>
    </body>
    </html>
  • What I did locally on my server was to set up a small .cmd file that could start the stream and file copier all at once. 
    @echo off
    start "" "c:\program files (x86\VideoLan\VLC\vlc.exe” [use the command line from above]
    
    echo Running rename script... do NOT close this window!
    powershell c:\inetpub\wwwroot\live\rename.ps1

That should be all there is to it! If you get any interesting streams up and running, paste a link to them in the comment section below :-)

0 comments

DISCLAIMER: This is really pretty basic stuff. – I was just really tired the other night and couldn’t really grasp all the 1’s and 0’s.

Anyway, just to set things straight, and in case someone else might be stuck with a similar problem, here’s a quick explanation.

 

To follow the example from the other day:

We have two numbers taken from our Facility and Priority enumerations and have selected 19 and 3. We know that the maximum value for the first number is 23 and for the second it’s 7.

In the example, we will be storing the result in single byte. In case you’ve forgotten how binary works, each bit, starting right to left, is twice the value of the bit before. I’ll try to illustrate how this looks:

1286432168421

So the maximum value you can store in 8 bits is 255. Now with the basics down, on to how the method for calculating the PRI field in a syslog message:

Calculating the PRI value

We know that the maximum facility value is 23 and the maximum priority is 7. Using the method for calculating the PRI value, that gives us a maximum possible value of 19x8+7=159 and thus we can easily stay within the bounds of a single byte.

Moving onwards:

The Facility number 19 in binary would be

00010011

We multiply 19x8. In binary terms, multiplying by 8 is the same as left-shifting your bits 3 places:

10011000

Okay, now we add the priority value (3) to the above value. That gives us

10011011

Converted to decimal, this gives 155 (surprise! this was also what we got when we did our little sum: 19x8+3).

Getting back to where we started

So now, to convert back to the original values, here’s what goes on.

To find the Facility (19), we simply right-shift our value 3 places (we left-shifted before, remember, by multiplying by 8?)

so, right-shift this 3 places:

10011011

and we end up with:

00010011

Which is the same as the original Facility number we started out with.

Now to find the priority number, we take our 8 bits and find the values in the three least-significant bits (three bits because the maximum value of the priority field is 7). This is done using an AND operation:

10011011

&

00000111

=

00000011

Which is 3 :-)

So the magic isn’t really all that magic at all. It’s just a question of knowing where your bits are (no pun intended!)

The above method can easily be extended to house pretty much any values (or more of them) – it’s just a question of shifting back and forth and keeping track of what is where.

0 comments

Okay, so while working on the syslog target for NLog, I of course studied the syslog RFC to properly be able to send messages in the correct format.

A syslog message always starts of with a PRI field which is a composite of a Facility number and a Severity number. Facility numbers range from 0 to 23 and Severity numbers from 0 to 7:

enum SyslogFacility
{
    Kernel = 0,
    User = 1,
    Mail = 2,
    Daemons = 3,
    Authorization = 4,
    Syslog = 5,
    Printer = 6,
    News = 7,
    Uucp = 8,
    Clock = 9,
    Authorization2 = 10,
    Ftp = 11,
    Ntp = 12,
    Audit = 13,
    Alert = 14,
    Clock2 = 15,
    Local0 = 16,
    Local1 = 17,
    Local2 = 18,
    Local3 = 19,
    Local4 = 20,
    Local5 = 21,
    Local6 = 22,
    Local7 = 23
}
 
enum SyslogSeverity
{
    Emergency = 0,
    Alert = 1,
    Critical = 2,
    Error = 3,
    Warning = 4,
    Notice = 5,
    Informational = 6,
    Debug = 7
}

Okay, so to generate the PRI field, we first multiply the Facility by 8 and then add the Severity.
Example:
A message with Facility Local3and Severity Error gives the following:

19x8+3=155.

So the PRI field in this syslog message would contain the value 155, letting the syslog server identify how to categorize the message.

Now, when the syslog server receives the message, a relatively simple operation allows the server to identify the Facility and Severity of the message.

To find the Facility, the PRI value is run through a bitwise right-shift three times, which leaves us the correct value (19)

int intFacility = intPri >> 3;

To find the Severity, the PRI value is run through an AND 0x7. This again leaves us with the correct Severity (3)

int intSeverity = intPri & 0x7;

My question is not so much how this works (I’ve done the maths on paper and yes it works) but WHY. I just don’t get the mechanics of it.

Why exactly right shift 3 times and why exactly AND 0x7 – how are these two numbers established?

Could I use a similar method to build my own little system containing two parameters with an arbitrary number of values in each? If so, is there a simple way of telling how to first combine the two into a single value for transmitting over the wire, and then how do I figure out how to get back to the two original values, knowing only the combined value and the number of possible values in the two lists?

If someone can explain this to me in relatively simple terms, I’d be really happy (I might even buy you a beer!)

0 comments

It’s been quite a while since I last blogged. A lot has been going on and I haven’t really had time to work on any pet projects. But last week, a little thing came up that I thought I should solve.

I’ve started using NLog for my logging purposes (it seems that log4net has slowed to a complete standstill with no releases since some time in 2008). So far I’m really happy with NLog and it seems to nicely support all my logging needs. However I stumbled on a situation where I needed to send log messages to a Syslog server at work and discovered that there was no Syslog target out of the box in NLog (as there is in log4net).

So what to do? Simple, write my own ;-)

To begin with I thought I’d need to fork out my own copy of the NLog source on GitHub and wire-up my Syslog target right in the innards of the system. So that was what I started out doing. However it seemed like an awful lot of wiring-up was needed to be able to support “just another way of sending messages”.

I stumbled across a chap called Ryan Farley’s blog – he had made his own custom target for NLog to notify Growl clients, so I quickly had a look at his source code up on GitHub.

I decided that I could use the same method of plugging in to NLog as he had done, for my simple needs. And what better to do that hack it together and upload it to GitHub also (did I say I love Git? – and I’ve only just scratched the surface so far).

Using NLog.Targets.Syslog

First of all, grab a copy of the binary and dump the NLog.Targets.Syslog.dll file in your application’s bin folder (the same place where the NLog.dll is).

Update: Just get it through NuGet :-)

Then edit your NLog.conf file to contain something like this:

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

    <extensions>
        <add assembly="NLog.Targets.Syslog" />
    </extensions>
    
    <targets>
        <target name="syslog" type="Syslog" syslogserver="" port="" facility=""/>
    </targets>

    <rules>
        <logger name="*" minLevel="Trace" appendTo="syslog"/>
    </rules>
</nlog>

Be sure to enter your own syslog server name, port and a syslog facility name in the relevant places. Fire up your application and voila! – messages should start appearing on your remote syslog.

Feel free to grab a copy of the source code for NLog.Targets.Syslog from my GitHub repository.