Simon Wadsworth

Master of Science | Computer Scientist

Hurricane Electric DNS Hosting Service
Published Fri 23 November 2012 by Simon Wadsworth in Internet

I recently wanted to move my DNS from my web hosts cPanel managed system because I really dislike cPanel (they are soon moving to Plesk, but this provided the perfect option to seperate my services). I looked around for DNS hosting services and looked at Zonomi a New Zealand based paid service with DNS servers located around the world. However I was worried about the cost as one of my domains alone would take up the allocated records.

I asked around and was told about Hurricane Electrics DNS service. It’s free and feature packed. It can handle a load of record types (including SSHFP) and can even do reverse zones if you need them. One of it’s neat features is built in dynamic DNS. I previously had a CNAME in my DNS pointing to a dyndns address from no-ip.com. Now it’s just a A record. Awesome!

They are a service I would highly recommend.

If you domain registrar allows you to specify the IP addresses of the DNS servers then you can use this to configure vanity NS records, by changing the NS records at your registrar to something like ns1.your-domain.com and the IP address of ns1.he.net etc up to ns5. ns2-ns5 provide a Dual Stacked IPv6 service, which is a neat benefit.

Finally if you are worried about query times for users outside the states, it’s not that bigger deal as most visitors will be using recursive DNS servers provided by their ISPs or employers and will cache your records, so subsequent users will receive the responses fast until they expire and need to be re-requested.

Thanks to Brad Cowie for pointing me to them

Networks Internet DNS



Reducing Web Server load using Amazon S3
Published Mon 11 July 2011 by Simon Wadsworth in Internet

Anyone who runs a website, will know that eventually a website will (hopefully) become so large, and popular that one server is simply not enough to host all the content or load that is thrown at it. A common method to reduce this is just to add more servers in and load balance them. But what if you can’t afford more servers. Well there is a very cheap alternative. This is Amazon’s S3 hosting. It is a cloud storage technology provided by Amazon Web Services, which provide extra features, likely access control, enabling public access and setting custom headers. The ultimate goal would be to use a fully fledged Content Distribution Network, but for starters Amazon S3 easily does the trick. All you have to pay for is the storage space you use, and data you actually transfer.

So how does this help, well by placing your content (images, video, even say CSS) on Amazon S3 and using an Amazon S3 address to link to the content, then the end user will pull the content from Amazon S3, reducing the number of connections needed on your server, and the amount of data your server needs to send, enabling it to answer other requests faster. Not only that you can provide Cache tags on the files meaning that the client will cache the file, to stop you incurring extra costs of the end user requesting the file all the time. Not only this, it makes it faster for the user.

I use Amazon S3 on my blog, and by assigning the S3 bucket name as a CNAME on my domain, I can use a nice URL to access my content, making it look highly personalized. Not only that if you are using Wordpress their are a number of addons that allow for Amazon S3 integration, my favourite being WP Total Cache, which will upload the files that it thinks should be served statically, and automatically rewrite the URLs to them. Not only that, if you change to Amazon CloudFront it will easily allow you to change to that.

So if you are having issues with your website being overloaded with traffic especially when it is images etc, try moving it to Amazon S3. And once it is in their, if you decide you need to added power of the Amazon CDN, it’s extremely simple to setup and use your pre-existing content in S3 as the source.

Networks Amazon S3



Limiting bandwidth on Apache
Published Wed 22 June 2011 by Simon Wadsworth in Internet

Now I don’t actually need to enforce data limits on my Apache virtual hosts, because I only host myself, so what I use is irrelevant, but I thought it would be interesting to find out how to do it. I know there are modules out there, but the one I found which I like is mod_cband. It works really well, and enables you to enforce data limits, but also speed limits and number of connections if you are experiencing heavy load say and wish to throttle it a bit. These instructions are based on Ubuntu Server 10.04LTS.

So to start I downloaded the mod_cband source code, available here. After extracting it you then need to compile it. To compile it you need to have APXS2 installed, which I installed by using the following command

sudo apt-get install apache2-prefork-dev

I assume if you are using the threaded version you would need to install the apache2-threaded-dev package. Now after you have installed this package, you can execute ./configure to begin the first phase. This will check dependencies etc and tell you if you are missing anything. You shouldn’t do, but if you are, resolve them before you continue.

Next you need to actually compile the source code. I had a problem here with the Makefile. You need to alter the Makefile slightly. You need to change the line that reads

APXS_OPTS=-Wc,-Wall -Wc,-DDST_CLASS=3

and add in -lm so that it now reads

APXS_OPTS=-lm -Wc,-Wall -Wc,-DDST_CLASS=3

If you don’t you will get an error when you try to start Apache.

Now execute make and when that is completed execute sudo make install. You will get some warnings here regarding comparison of different types, but they shouldn’t effect the running of the module. This will compile the library and install it into the correct location. Now to check that it is enabled type sudo a2enmod cband (assuming of course you are using a2enmod, otherwise you will have to manually edit the configuration files to check that it is there.

Now restart Apache and it should load everything correctly. Now it is time to configure Apache virtual hosts to limit bandwidth. There are many configuration options here, so I will only explain how to enforce Data limits. For other options refer to the mod_cband documentation.

To enable mod_cband your virtual hosts MUST have a ServerName directive, and all cband directives must come after this. If you don’t Apache will throw all sorts of warnings when you try and start it. So the simplest and quickest way to enable a data limit on a virtual host is by placing the following directive in the configuration.

CBandLimit 10M

That will place a limit of 10Megabytes on the user, which is pretty small, and I am sure no one would actually give something that small, but it is very simple to setup. There are many other options you can configure, like a page to send when the bandwidth is used up, or what HTTP code to send, and many more. You can also configure a page that allows you to see the status of the virtual hosts and their various restrictions.

Networks Apache Servers



Updating METADATA on Amazon S3 objects
Published Wed 15 June 2011 by Simon Wadsworth in Internet

So I host the static content from my blog on the Amazon S3 Simple Storage Service. This allows me to remove some of the load of my server for static content. However this means that over time I need to pay money for the S3 hosting, and if I have a lot of requests this could end up costly. So how do I get around this. Well by setting the Content-Control META tag onto the objects in S3, I can ensure that the static content is cached by the remote user for however long I want. In this case I have set it for 7 days. However updating all the files in S3 would take a long time to do manually, so I use this Python code to update the objects in my S3 bucket.

I had to modify it to support encoding as I use gzip encoding on some of the static content to reduce the amount of data needing to be transferred.

from boto.s3.connection import S3Connection

connection = S3Connection('API_KEY', 'API_SECRET')

buckets = connection.get_all_buckets()

for bucket in buckets:
    for key in bucket.list():
        print('%s' % key)
        encoding = None
        if key.name.endswith('.jpg'):
            contentType = 'image/jpeg'
        elif key.name.endswith('.gif'):
            contentType = 'image/gif'
        elif key.name.endswith('.png'):
            contentType = 'image/png'
        elif key.name.endswith('.css.gzip'):
            encoding = 'gzip'
            contentType = 'text/css'
        elif key.name.endswith('.js.gzip'):
            contentType = 'application/x-javascript'
            encoding = 'gzip'
        elif key.name.endswith('.css'):
            contentType = 'text/css'
        elif key.name.endswith('.js'):
            contentType = 'application/x-javascript'
        else:
            continue
        if encoding is not None:
            key.metadata.update({
                'Content-Type': contentType,
                'Cache-Control': 'max-age=604800',
                'Content-Encoding': encoding
            })
        else:
            key.metadata.update({
                'Content-Type': contentType,
                'Cache-Control': 'max-age=604800'
            })
            key.copy(
                key.bucket.name,
                key.name,
                key.metadata,
            )
            key.set_acl('public-read')

Amazon S3 Programming Cache



Enabling IPv6 on a home network
Published Mon 13 June 2011 by Simon Wadsworth in Internet

IPv6 is the next generation internet protocol. Currently few ISPs provide it to the customers, and therefore uptake is slow. However if you wish to have access to the IPv6 world now then there are options. If you only have a single machine than a tunnel is fine, but however if you wish to add it to an entire network then you need something more. If you have a spare old machine lying around, or a machine running Linux that is always on, then you can configure that as a router and use it to provide IPv6 to your LAN.

I have IPv6 connectivity to all the machines that are connected to my network. To achieve this, I use an Ubuntu Linux box as a router, which has a tunnel configured. This allows all the computers to connect onto the IPv6 internet transparently. This is a guide on how I did it.

I use sixxs.net as my IPv6 tunnel provider. They provide the use of the aiccu client which allows the configuration and setup of the tunnel automatically. It creates a interface sixxs which is one end of the tunnel. First things first, you need to register an account at sixxs.net. After your account is approved you are able to create an IPv6 tunnel. This will only allow you to connect one machine, but it is essential before you will be able to enable access to other machines. This will take a while to get approved, but once approved you can install the aiccu client. On Ubuntu you can install it using:

sudo apt-get install aiccu

During setup it will ask you to enter information regarding your tunnel, most likely your sixxs.net login information. Once entered it should authenticate and complete the installation. If it hasn’t started automatically, you need to start it.

sudo service aiccu startOr on older version of Ubuntu try sudo /etc/init.d/aiccu start

Then it will configure the tunnel and you should be able to connect to IPv6 sites. You can try this by typing traceroute6 ipv6.google.com. The next thing to do is to provide IPv6 addresses to your network. To do this, you must apply for a Subnet from sixxs. You will receive a /48 subnet, for which you assign /64s to your network. To distribute your prefix announcement onto your network you need something like radvd installed. Again on Ubuntu it is as simple as typing

sudo apt-get install radvd

Now once radvd is installed, you need to edit the configuration file. This is usually stored in /etc/radvd.conf. So open it up and you want to enter the following:

interface eth0
{
  AdvSendAdvert on;
  AdvManagedFlag on;
  prefix 2001:4232:532::/64
  {
    AdvOnLink on;
    AdvAutonomous on;
    AdvRouterAddr on;
  };
};

The prefix is from the subnet that sixxs has assigned you. In this case I was assigned 2001:4232:532::/48, so I chose to use the /64 of this for simple setup.

Now of course your interface that is connected to your IPv4 LAN, so what will now be the interface on your router not connected to IPv6 web, should have a static IP assigned to it. This makes it easier to remember, and use. So I just assigned 2001:4232:523::1 to eth0. I won’t cover how to do this, as it is relatively simple if you have done any networking in Linux before.

You now need to tell the linux kernel that you want it to forward traffic for IPv6. To enable IPv6 forwarding you need to edit /etc/sysctl.conf and add the following lines:

net.ipv6.conf.all.forwarding=1
net.ipv6.conf.default.forwarding=1

Now save this file and reboot. When the machine comes back up, check that aiccu and radvd have started ( I find I always have to start aiccu manually). If this is the case then your other machines should have Global IPv6 addresses assigned to them using the prefix you gave radvd. However I found this was not enough to allow my other machines to connect to the internet. After specifying the default route on the router as the IP at the sixxs end of the tunnel, all traffic from eth0 was then routed out over my tunnel, and all the other machines appeared to have native IPv6 connectivity, and were globally addressable. You therefore need to ensure that your machines have firewalls installed, and if you like setup IPv6 iptables on the router. This is what I have done to filter traffic that is not wanted in the network. Also as your IPv6 address will be based on your MAC Address, you can be easily tracked based on it. Windows by default enabled privacy extensions, but Linux does not. To enable this on your Linux clients edit /etc/sysctl.conf and add these lines:

net.ipv6.conf.wlan0.use_tempaddr=2
net.ipv6.conf.all.use_tempaddr=2
net.ipv6.conf.default.use_tempaddr=2

If you have eth0 then replace wlan0 with eth0 or add an extra line for each different interface. all and default should cover all of them, but I like to specify them individually as well just to be safe. I will write another article regarding IPv6 tables at a later date.

Networks IPv6 Linux sixxs



World IPv6 Day
Published Sun 05 June 2011 by Simon Wadsworth in Internet

World IPv6 day is on June 8, 2011. World IPv6 Day is a day where several large organisations, such as Google, Facebook, Yahoo, Akamai etc will offer there content of IPv6 for a day. I’m ready for it, and so on June 8, I will be browsing these sites in IPv6 for the day!

Here is what I get when I traceroute from my machine to ipv6.google.com

Tracing route to ipv6.l.google.com [2404:6800:8004::68]
over a maximum of 30 hops:

1 1 ms <1 ms 2001:4428:450::1
2 28 ms 26 ms 27 ms gw-113.wlg-01.nz.sixxs.net [2001:4428:200:70::1]
3 29 ms 26 ms 27 ms ge0-1-6.v6wlg0.acsdata.co.nz [2001:4428:0:6::1]
4 39 ms 50 ms 38 ms ge0-0-2321.v6akl1.acsdata.co.nz [2001:4428:0:911::4]
5 38 ms 39 ms 38 ms ten-0-0-0-134.bdr01.akl02.akl.VOCUS.net.au [2402:7800:110:511::d]
6 43 ms 38 ms 38 ms ten-0-2-0-400.bdr01.akl01.akl.VOCUS.net.au [2402:7800:110:1::1a]
7 62 ms 65 ms 62 ms 2402:7800:0:1::ca
8 62 ms 63 ms 94 ms 2402:7800:0:2::92
9 64 ms 63 ms 64 ms 2001:4860::1:0:9f7
10 67 ms 73 ms 70 ms 2001:4860:0:1::d7
11 63 ms 142 ms 65 ms 2404:6800:8004::68

Trace complete.

Networks IPv6



Yahoo unlocks IMAP access
Published Mon 01 November 2010 by Simon Wadsworth in Internet

Up until recently it has been near impossible to access Yahoo IMAP through any client that wasn’t the Zimbra Client, or else a device like a Blackberry or Apple iPhone etc. Zimbra sends a special command to the Yahoo IMAP servers which authenticate it as an allowed client. I had been using a modified version of Thunderbird that sent this command also, as I prefer the interface and search in Thunderbird.

However it now seems that Yahoo have allowed access to their IMAP servers without this command, and as a result all clients should be able to connect now. I have set up my unmodified Thunderbird on my Ubuntu laptop to connect and it worked fine. I also tried Outlook to see if it worked, and it worked without a hitch. One thing however that is not clear is whether or not this is temporary or whether Yahoo are now offering it free to all users. It is possibly part of their plans to become more competitive in the Webmail market, after having suffered a 10% loss in the number of users in the last year to rival services. This has made it the second largest behind Microsoft’s Windows Live Mail. Google has had a 21% increase in the number of users, and they of course offer IMAP access.

Incoming Server Settings
IMAP Server: imap.mail.yahoo.com
IMAP Port: 993
IMAP security: SSL/TLS

Outgoing Server Settings
SMTP Server: smtp.mail.yahoo.com
SMTP Port: 465
SMTP security: SSL/TLS

The SMTP requires authentication, the same username and password you use to authenticate to the IMAP server.

I find that if your email address is user@yahoo.com then user will work as your username, but I have not tried the full email as the username.

IMAP Networks Yahoo


CONTACT

Email
contact at simonwadsworth dot geek dot nz

Address
Hamilton, Waikato
New Zealand

Phone
+64 27 403 4531
02825516452@2talk.co.nz

SOCIAL LINKS

Generated with Pelican

Created by BLACKTIE.CO