For example, you can use your database to protect you against high-impact attacks such as SQL-injection.

I created a presentation about this topic a while ago You can download this presentation here:

http://mini.louwrentius.com/files/designingsecureapplications.pdf

A short summary of the points made.

• Truly understand your application and their requirements.
• Do not create a monolithic application, create separate applications. For example, at least separate front office and back office.
• Run those applications under different operating system users or ideally on different servers, residing in different network segments.
• It suddenly makes sense to put your database server in a separate secure network segment as opposed to running it on the same box as the application server.
• Do not use a single database account with root-like privileges.
• Create separate database accounts for separate application components. Only assign those privileges required for that application. White-list privileges within the database. This is key.
• Understand that for end-user authentication, 'select username,password from user' kinda privs is not required!
• Use stored procedures and functions wisely. By only providing access to functions, views and stored procedures, while preventing access to tables, you can significantly reduce the impact of SQL-injection or other application level security breaches.
• In any case, understand that an attacker can never obtain more database privileges than the database account used. Even if the entire application server is compromised. This is especially important for your internet-facing applications.
• Use your database as an extra layer of defense.

Your systems are still not protected against privately known vulnerabilities and if hackers have zero-day exploits for such vulnerabilities, you are clearly having a false sense of security.

There couldn't be a better example than a high-risk vulnerability MS12-020 regarding the Microsoft Remote Desktop Protocol interface, as present on TCP-port 3389. Any unpatched Microsoft Windows-based server or desktop system can be compromised through this vulnerability. If the system is vulnerable and TCP-port 3389 is accessible, it is over. Your data is compromised.

Now, how many people knew about this vulnerability and for how long?

As we speak, someone may be reading these very words on your computer, just remotely, because of an undisclosed, unknown vulnerability. That sounds like paranoia, but it isn't.

Zero-day exploit market

There is a whole zero-day exploit market. Exploits are sold at enormous prices, as high as $100.000+ dollars. Only those who have the means (money) and a need for them will pay such prices. Buyers often tend to be government agencies and such.

There is no doubt in my mind that the computer I'm currently working on is affected by high-risk vulnerabilities I don't know of. It is very likely that for some of them, exploits exist. But look at the risk: who is going to spend a $100.000 exploit on me? But is the intelectual property of your company worth that much? Might sound way more realistic already, doesn't it?

You may hope that zero-day exploits are sold to trustworthy governments, but the marked is free. Anyone with sufficient means can buy them. Some sellers may scrutinize to whom they sell, but others?

This whole zero-day exploit market is a problem. Exploit-sellers have nothing to gain and only to loose from public disclosure of the vulnerability. As long it is undiscovered, it can be used by buyers. All parties involved in this market benefit from keeping systems insecure. From keeping systems unpatched.

So instead of informing the vendor of a security vulnerability so the public can be protected, knowledge of the vulnerability is sold to the highest bidder who then does who knows what with it.

For most organisations and people, the upside is that nobody will spend a $100.000 on you if you're not worth it. The reason is that every time an exploit is used, it can be discovered, rendering the exploit useless once a security patch is released.

Protecting against zero-day exploits

The question is then what to do against this kind of threat. What can you do to protect yourself against the risk of zero-day exploits if you perceive the risk as realistic towards your organisation.

The answer is a security strategy of defense in depth. It is not a solution that ends all problems, but it decreases the risk that your organisation gets compromised. It is about trying to diminishing risk to acceptable levels.

Assume that you will get compromised. Then, think about what can be done to reduce the impact of the hack. Will only one server get hacked, or the entire internal company network?

Defense in depth is the principle that you do not rely on one single security measure to protect systems and services from a compromise. There are many ways to implement such a strategy and I will name a few.

1. Only expose those services towards the internet that are required for production.
2. Make sure you have proper network segmentation in place, systems should not provide a stepping stone for an attacker to enter your internal company network.
3. Never expose management interfaces such as RDP towards the internet directly, use an additional security layer (white list IP address or use VPN).
4. Establish an emergency patch-policy to make sure that all systems are patched outside regular maintenance windows if high-risk vulnerabilities are reported.
5. Monitor the heck out of your environment. Carefully try to log and alert to those events that may indicate a security breach.
6. Audit your systems, regularly check for misconfigurations and resolve them.
7. Select hardware and software vendors based on their security track record.
8. Use different vendors and brands for different defensive layers.
9. Consider internet off-limits for end-user systems processing sensitive information

Software is vulnerable so prepare for the worst.

Design

I have a Mac mini running Linux that acts as my internet router. The closet that houses the cable modem is not a friendly environment for such a device and there is not a good location for it. The closet is also outside of my house, behind a door not too well protected. So this is why I keep my router inside my house.

From this closet, one UTP cable terminates in the living room, the other in the basement. This configuration has a very big problem. How do I run two different networks over one wire?

I have to connect my iMac to my 'internal' home network. However, the Mac mini must be connected to both the internet network segment (connected to the cable modem) and the home network. All through a single UTP cable.

Therefore I use VLANs. I transport both the internet network and the local home network though one cable. VLAN 10 is for internet, VLAN 20 for my local home network. For this all to work you need managed switches that support 802.1q.

How traffic flows

So let's say that the server is accessing the internet to obtain the latest Linux security updates. How does this network traffic flow through the infrastructure (click to enlarge)?

All internet traffic must flow through the router. Thus, even if the traffic from the basement travels through the switch next to the cable modem, it must first travel to the router in the living room. There the router decides if the traffic is permitted to go out to the internet and thus enter the internet VLAN.

Pros and cons

Pros:

• Just a single cable to the living room
• no extra USB-based ethernet adapters required for the Mac mini
• Mac mini resides in save and computer friendly environment

Cons:

• Managed switches supporting VLANs are relatively expensive

Every person who has to maintain some kind of Iptables-based firewall should really look into LIFS. It will make managing your firewall much more convenient.

For more advanced purposes. LFS allows you to create object groups. These are groups of individual hosts, networks or services (tcp/udp).

Look at this example of object groups in action. Read and understand.

HTTP_SERVICES="
    80/tcp
   443/tcp
"

WEB_SERVER_1=192.168.0.10
WEB_SERVER_2=192.168.0.11

WEB_SERVERS="
    $WEB_SERVER_1
    $WEB_SERVER_2
"

allow_in any "$WEB_SRVERS" any "$HTTP_SERVICES"

As you can see, a single firewall rule in fact creates 4 rules, one for each host and port. This functionality can be found in commercial based firewalls but it is not build-in into Iptables. LIFS fixes this.

LIFS is a continuation of LFS, which has been discontinued.

Update 18 February 2012

There is one problem. When the robot is not connected to the charger, the batteries are depleted very fast. Even if the batteries are not entirely depleted and the robot can still display the menu, the clock loses it's time. Every time the robot gets a too low charge, you have to set the date and the time, which is a bit of a hassle. This does not happen often though. The robot seems to be consistently operating properly.

Update 1 March 2012

It seems that the batteries have worsened so bad that the device cannot clean my living room without 3x recharging. I have to return the product for repair. I had the device scheduled to clean every other day, about 4x per week.

Update 20 March 2012

I received a brand new device that is now charging. I hope this one will last longer.

Update 23 March 2012

It seems that the brand new robot is also flawed, it just goes nuts. Seems to be up-to-date regarding software, so have to return this one also. (read below!)

Update 31 March 2012

I did not return this device and did some additional cleaning cycles. All cylces where performed withouth problems. The device choked om some cloth and some cables I forgot to cleanup, but it does seem to operate properly. So I will keep it.

Uodate 4 May 2012

Still works like a charm. I'm currently very hapy with it. If the batteries hold up, this device is really worth the money.

Original article:

So I bought a robotic vacuum cleaner. The first question is 'why would you spend some serious money on such a device? On a toy?'. I have some rationalisations for buying this device, but honestly, one reason is that sometimes I just like to buy a new toy. Something to play with. Excuse me for being human. In this blog post I want to explain to you why I bought a Neato XV-15 and not another product.

Now I did say that I have some rationalisations, so let's start. One rationalisation is that I hate vacuum cleaning. Since I have two cats, vacuum cleaning once a week may not be enough. And I'm not going to clean more frequently. So you can accept it or if you can spare a little dough, buy a robotic vacuum cleaner that cleans your house when you're not at home.

So let's introduce the Neato XV-15.

The Neato XV-15 Vacuum cleaning robot

The XV-15 robot is made by Neato Robotics, a young startup that seems to be started purely for this device. The company started with the XV-11 for the US market, and the XV-15 is identical except that it is meant for the European market. A new XV-12 has also been announced, which seems to be identical to the other two machines, except for the color (white).

The robot automatically vacuums your house while you're away or minding your own business. I't can't do anything else, but not having to vacuum all the time is kinda cool, right?

I bought the XV-15 in The Netherlands for 500 euros. The XV-11 can be had for around $400 excluding taxes or maybe even for less at Amazon. Not very cheap, but competitively priced compared to other robots on the market.

How the robot works

The XV-15 has a rubber brush at the front that rotates quite fast and that brush scoops up the dirt. Just behind the rubber brush, a vacuum mouth is present. Anything sucked up through that mouth enters the dustbin. The actual vacuum motor is at the back of the dustbin, protected by the dust filter of the dustbin. The XV-15 is a true vacuum and Neato claims that vacuuming power is way stronger than any other robot on the market. Based on the noise, that may be true.

On top of the XV-15 you can find an LCD screen for configuring the robot and the turret housing its special secret weapon: laser sight. This is the cool part.

The XV-15 has a laser system mounted on top that allows the robot to locate objects and walls. It is capable of creating a map of its surroundings. Anything the laser can 'see' will be avoided. The robot will not bump into any objects it can see. This is in stark contrast to products like the iRobot Roomba, which just bumps into everything. The XV-15 does have a front bumper though, because anything below the laser turret cannot be seen. Thus the robot does bump into things occasionally but it does a hard job trying not to.

The laser system is not just for preventing collisions with furniture. Being able to generate a map of your house allows the robot to clean your house in an efficient manner. Robots like the Roomba just randomly zigzag through your house. If you do that long enough, chances are high that most of your house gets cleaned, which it will.

The XV-15 only covers each spot once, and thus is able to clean your house much faster. It first cleans the perimeter of a room, hugging the walls. It then cleans the room in straight lines, like a swimmer in a pool. It remembers where it has cleaned or not and will come back later to a spot if something (like humans or pets) was occupying an area that can now be cleaned.

My living-room, kitchen and entrance are cleaned in 40 minutes. An area of 40 square meters or about 420 square feet.

When you see the XV-15 doing it's job, you may tend to stare at it longer than you may want to. It's just fascinating to see the device effortlessly navigating around your house. And it doesn't need stuff like battery operated 'light houses' like the Roomba's need. It is truly autonomous except for emptying the dust bin.

The XV-15 seems to divide the rooms it detects in parts and will start cleaning those parts one after another. As said earlier, the robot will continue cleaning where it had left off if the batteries are low and needs recharging.

The robot has no problem detecting stairs. Neato has also provided a roll of magnetic strip that can be used as a boundary marker. The robot will not cross this strip and will clean around it.

However, how smart the XV-15 may be, you need to make your house robot-proof. The first time you start cleaning with the Neato, it is advised to monitor it's progress and 'fix' difficult spots in your house. I have no experience with other robots, but I think that this is true for all of them.

The robot is just low enough that it can clean underneath my central heating radiators, which is very nice. It also has no trouble cleaning under my bed, an area which seems to collect dust very fast.

The robot has never had any problems finding the base. It gently wiggles it's behind towards the base until it has a connection. It then informs you with a sound that it has finished cleaning.

Docking station

The XV-15 comes with a docking station that allows the device to automatically recharge for the next run. The XV-15 will return to the docking station if the batteries are low. When recharged, the XV-15 will continue cleaning where it left off. If you have a single story apartment, the XV-15 will thus clean the entire apartment all by itself, even if it can't clean your home in one take on a single battery charge. After recharging, the unit will just return to the spot where it aborted cleaning to recharge and continue cleaning.

The docking station allows you to put excess power cord into the station itself, to keep cable clutter to a minimum. You can also reroute the cable to exit the station from either the left side or right side.

Scheduling

The robot can start cleaning with a press of the big orange button. The robot will start cleaning and return to the docking station when finished. Ideally, you want to have the robot clean the house when you're not around. Fortunately you can set a schedule for all seven days of the week.

The robot has a clear LCD screen with a very easy menu for setting the clock and entering a schedule. A few simple buttons allows you to enter a schedule, which probably has to be done once. I have it set to clean every other day except for the weekend.

Scheduling is extremely simple: for all seven days of the week, you can configure a start time or choose not to clean that day. That's all.

Noise level

When you start the XV-15 for the first time, you will be surprised by the of noise this little device generates. The vacuum motor is loud, but the rubber brush adds an additional roaring and rattling sound to it that is just almost unbearable.

The rubber brush keeps hitting the floor causing the loud rattling sound. I had to add some felt strips on the bottom to raise the robot a little bit from the ground. This eliminated the rattling, but the robot is very loud. Keep this in mind.

I think the noise level is the biggest downside of this robot.

Cleaning performance

The picture shows what the XV-15 can collect during a sweep. I dit not perform any scientific tests to verify the cleaning performance of the robot, but any visible dirt is always devoured by the robot. I'm personally very pleased with the results.

I found a source written in Italian that seems to suggest that the XV-15 does a significantly worse job of cleaning stuff (67%) than the Roomba 780 (97%) robot, but it is an artificial test that does not use the stuff it is supposed to clean: (fine) dust and hair. However, it thus may be possible that the dumb Roomba's clean better. I don't know.

I only can tell you that even if you clean daily and you have some pets, you will find quite some stuff inside the dustbin after each run.

Maintenance

The iRobot Roomba range of products seem to require quite some maintenance. The biggest issue with the Roombas is the fact that you need to clean out hair from the bearings and brushes after each run. This is not necessary with the XV-15.

I don't know how much time cleaning of a Roomba takes, but I have an issue with that: why bother with a robot if you have to clean the robot instead of the house itself? Yes cleaning the robot takes less time, but it's probably no fun either.

The only thing that you need to do when the XV-15 is finished: empty the dustbin and clean the filter. That will take no longer than 30 seconds I guess. No need to clean up the brush or bearings. It is of course advised to inspect the brush and bearings now and then.

Checking the condition of the rubber brush and bearings is very easy. The brush guard can be removed without tools in seconds. Removing the rubber brush is just as easy and cleaning the axles shouldn't take long if ever required. I've never had to clean the brush itself. It seems that hair gets sucked up and doesn't stick to the brush.

Inside the box

The XV-15 comes with an additional rubber brush and four additional filters. According to Neato, you need to replace the filter every three to six months, depending on the frequency of your cleaning schedule. At 16 euros ($20) for 4 filters, that's not a big deal I guess.

I couldn't find any details on how long the rubber brush will last.

Updating the software

If you take a closer look at the back of the robot, you will notice that at the left side of the big exhaust vent, two small ports are present: for power (only useful if you do not use the docking station) and a USB port. The USB port can be used to update the software of the robot to the latest version.

Please note that Neato does not suply a USB cable so you need to get a mini USB cable when you want to update the software (firmware) of the robot. Bad news for Apple and Linux users: the firmware update software only runs on Windows. You can update the robot from Windows running inside VMware (Workstation or Fusion).

Take a look at Neato's update page to see if new updates are available.

Hacking the XV-15

When the robot is connected to the computer through USB, you can communicate with the device through Hyperterminal or Minicom. If you like hacking your robot, continue reading here.

Conclusion

I'm quite happy with the robot. The biggest question is how long this device will last. At first, a robot like this seems a bit as a toy and it may be, but it is a pretty darn useful one.

The lack of maintenance compared to the other robots is a big plus to me. If you have to spend time on cleaning the robot itself, where is the benefit?

To me, the only downside is the noise.

It can't vacuum the stairs. It can't vacuum in every corner. But the device can clean the majority of your house more often than you would probably have done yourself.

Additional sources

Robot Reviews

Very cool youtube film showing the robot through 'near infrared' view.

Wouldn't it be nice if that time can be reduced? Even to 5 seconds?

Although not enabled by default, you can enable so called 'bitmaps'. As I understand it, a bitmap is basically a map of your RAID array and it charts which areas need to be resynced if a drive fails.

This is great, because I have the issues that of every 30 reboots, sometimes a disk won't get recognized and the array is degraded. Adding the disk back into the array will mean that the system will be recovering for 5+ hours.

I enabled Bitmaps and after adding a missing disk back into the array, the array was recovered instantly.

Isn't that cool?

So there are two types of bitmapsL

1. internal: part of the array itself
2. external: a file residing on an external drive outside the array

The internal bitmap is integrated in the array itself. Keeping the bitmap up to date will probably affect performance of the array. However I didn't notice any performance degradation.

The external bitmap is a file that must reside on a EXT2 or EXT3 based file system that is not on top of the RAID array. So this means that you need an extra drive for this or need to use your boot drive for this. I can imagine that this solution will have less impact on the performance of the array but it is a bit more hassle to maintain.

I enabled an internal bitmap on my RAID arrays like this:

mdadm --grow /dev/md5 --bitmap=internal

This is all there is to it. You can configure an external bitmap like this:

mdadm --grow /dev/md5 --bitmap=/some/directory/somefilename

There probably will be some performance penalty involved, but it does not seem to affect sequential throughput, which is the only thing that is important for my particular case.

For most people, I would recommend configuring an internal bitmap, unless you really know why you would have to use an external bitmap.

Why using a VPN with your iPhone?

1. Security: all data is encrypted and cannot be read by malicious people trying to eavesdrop on your data.
2. Performance: my subjective experience is that a VPN can speed up web browsing, it seems to reduce latency.

Introduction

I am assuming that you use:

• an iPhone as the VPN client
• a Debian-based Linux distro, such as Debian or Ubuntu

We will use the following software:

• openswan
• xl2tpd
• pppd

To setup the VPN, we need to configure the following steps:

1. install the software
2. configure IPSec
3. configure L2TP
4. configure PPP
5. open up the appropriate firewall Ports
6. setup firewall rules to forward traffic between the iPhone and Internet
7. configuring the iPhone

This set of instructions is 90% based on instructions on peen.net made by Niels Peen (Groeten!). I borrowed some other stuff from this blog.

I use openswan for IPSec support because strongswan does not support NAT by default. I just want to use software as part of the operating system and don't like to have to maintain manually compiled versions. This is why.

Initial assumptions

• You are using a Linux host as the VPN server
• The server is accessible from the internet or the appropriate UDP ports are forwarded to the box.
• You have full control over the box and it's firewall configuration.
• Your iPhone has an unfiltered internet connection. If UDP is blocked, this type of VPN is not for you.

Install the software

First, we start with installing all required software:

apt-get install openswan xl2tpd pppd

Configure IPSec

Now we start with configuring the software. First we start with IPSec:

/etc/ipsec.conf

config setup
    nat_traversal=yes
    protostack=netkey

conn L2TP-PSK
    authby=secret
    pfs=no
    rekey=no
    type=tunnel
    esp=aes128-sha1
    ike=aes128-sha-modp1024
    ikelifetime=8h
    keylife=1h
    left=<internet ip of router/server>
    leftnexthop=%defaultroute
    leftprotoport=17/1701
    right=%any
    rightprotoport=17/%any
    rightsubnetwithin=0.0.0.0/0
    auto=add
    dpddelay=30
    dpdtimeout=120
    dpdaction=clear

Some notes about this configuration:

• We use a secret or password for authentication. Sources on the internet seem to suggest that the iPhone cannot handle certificates.
• we must configure the dead peer detection rules at the bottom or else you cannot reconnect to the VPN when returning from sleep.

We thus also need to configure an encryption secret (password) for the IPSec tunnel.

/etc/ipsec.secrets

%any %any | PSK "thisismysupersecretpassword"

It is smart to choose a strong (long) password.

Configure L2TP

Inside the directory /etc/xl2tpd you have to edit xl2tpd.conf like this:

[global]
debug network = yes
debug tunnel = yes

[lns default]

ip range = 10.0.1.201-10.0.1.240
local ip = 10.0.1.200
require chap = yes
refuse pap = yes
require authentication = yes
name = <put some name here>
ppp debug = yes
pppoptfile = /etc/ppp/options.xl2tpd
length bit = yes

The "ip range" is within your internal network. It is a range outside of your DHCP-scope. The "ip range" must not include the "local ip". This IP address is dedicated to your Linux host.

Important: once the VPN setup is working properly Turn off all debugging options (set them to 'no'). Otherwise, your logs will fill up very quickly because every time a packet is transmitted, this is logged.

Configure PPP

Now we must configure PPP. Edit /etc/ppp/options.xl2pd and make it look like this:

ipcp-accept-local
ipcp-accept-remote
ms-dns <address of your local or remote dns server>
noccp
auth
crtscts
idle 1800
mtu 1410
mru 1410
nodefaultroute 
debug
lock
proxyarp
connect-delay 5000

Note that you must enter a valid DNS server that must be reachable by the VPN client (iPhone) through the tunnel.

We are almost there. Now we must also configure a password for the PPP connection. Edit /etc/ppp/chap-secrets and make it look like this:

* * thisissomesecretpassword *

This password is not related to the IPSec password. I think it is wise to configure different passwords for IPSec and PPP.

Configuring the firewall

An IPSec + L2TP + PPP VPN requires the following ports to be opened:

• 500/udp
• 4500/udp
• 1701/udp

You must open these ports in your firewall yourself.

Configuring traffic forwarding rules

If you use a Linux box with IPtables, you may already have a functioning configuration. However, this line is required for traffic forwarding to work:

iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE

You must replace the correct IP addresses according to your configuration. You may also have to enable traffic forwarding like this:

echo 1 > /proc/sys/net/ipv4/ip_forwarding

A detailed firewall configuration guide is outside the scope of this tutorial.

If you use IPtables for your local firewall, you may be interested in my "Linux Firewall script" (shameless plug alert).

Configuring the iPhone

To configure a VPN profile, goto settings -> general -> network -> vpn (at the bottom). Choose 'Add VPN Configuration..."

1. Enter a description
2. Enter the IP address or DNS name of your Linux box.
3. The 'account' field can be anything you like.
4. Leave RSA SecurID off.
5. The Password is the PPP password configured in /etc/ppp/chap-secrets
6. The IPSec secret (/etc/ipsec.secrets) goes into the 'Secret' field.
7. Keep 'Send All Traffic' enabled.

If the connection succeeds, a VPN symbol will show up in the iPhone status bar. All traffic from then on will flow through the VPN.

It may not immediately work. Look in /var/log/auth.log and /var/log/daemon.log for debug messages.

Once it is working properly, disable all debug settings in xl2tpd.conf and restart the daemon.

Final remarks

You may have to tweak the 'dead peer detection' within the IPSec configuration. When the iPhone comes out of sleep, the VPN connection cannot be reinitiated right away, which is inconvenient.

Also, I'm not sure what the impact is on battery life.

Many organisations do not seem too concerned about a very old vulnerability regarding Ethernet-based networks called ARP-poisoning. Basically ARP-poisoning means that an attacker-controlled system steals the identity of another legitimate server, thus drawing all network traffic away from the legitimate server to the attacker-controlled system. Then, the attacker can do with that traffic as he or she sees fit. The attacker will be performing a man-in-the-middle attack. Please note that such an attack is trivial using tools as Cain and Abel or Dsniff.

It is often the case that many different server systems are placed in a single network segment or VLAN. That implies that any of these systems poses a threat to each other. It takes just one hacked system to compromise network traffic between all the other systems. This is especially a threat to all unencrypted network traffic, but encrypted sessions may also be attacked if clients don't check the sever's identity.

Unless you've actually implemented proper network segmentation using separate (V)LANS and filter traffic between these network segments through firewalling, your environment may be at risk. In that case, please understand that it takes just one single web application containing just one vulnerability to compromise the entire environment.

Not everybody has implemented proper network segmentation and firewalling, preventing these kind of attacks. And it takes quite some labour to change all that. So what can you do, assuming that you want to do something right now?

In general, in a shared network environment, as described, the only way to make sure that data in transit is kept confidential and unmodified is to make proper use of encryption, identification and authentication.

The solution

To secure web traffic, there is already a fairly easy solution: using HTTPS or HTTP over SSL. The most difficult part is getting a valid SSL certificate and configuring the HTTP server to use it.

But if you want to transfer files between servers or between clients and servers? How about that?

Is there actually an easy way to securely transfer files between two hosts? From what I can see, the answer is "no". Security comes with some additional effort and it isn't easy.

The first problem is to understand what 'secure' actually means. To me, it means that data is not stolen or modified by an attacker during transit.

There are three requirements to make sure that confidentiality and integrity is guaranteed:

1. data in transit is encrypted;
2. the client authenticates the server;
3. the server authenticates the client.

Encryption prevents a man-in-the-middle attacker from eavesdropping or altering data. And if the client verifies the identity of the server, the attacker cannot impose the real, genuine server. This part may be overlooked. "I'm using SSL, thus encryption, so I am safe, right?". That is a firm negative. The client is identified with a password, passphrase or client-side SSL certificate. But how does the client identify the server? If the client doesn't verify the identity of the server, you might as well turn encryption off.

For the Windows platform, there is no native solution. Most of the time, files are transfered using SMB and thus your files can be grabbed from the wire and you may be transferring your files to some impostor instead of the genuine host.

The other often-used solution is FTP. And everyone knows that the biggest problem with FTP is the lack data encryption. All communication, including credentials for authentication, are transmitted in plain-text.

Without any additional third-party software, it is impossible to securely transfer files between Windows hosts, except for one solution that I have never seen used: IPsec. IPsec is used to encrypt any network traffic between two host, thus also SMB traffic.

The Unix world has only one solution without using third-party tools and that is transferring files using SSH as a secure transport. But SSH is also used for secure shell access to hosts and it may be difficult to prevent shell access and still allow file transfers.

So now there is a new tendency to use FTP over SSL. You have the same inconvenience as with HTTPS: you need to install a valid SSL certificate on each FTPS server. And although this does improve security, encryption is still useless if the client side system does not properly validate the server's identity.

Furthermore FTP uses a control channel for commands and a separate data channel to transfer the actual data. You want both channels to be encrypted, but that may not be the default. Check your FTP server's configuration to make sure this is the case.

Implementations

To implement FTP over SSL on Windows, you might want to take a look at the Filezilla server. An FTP server that also supports FTP over SSL. It had some security vulnerabilities in the past but not too many. To me, it is a better solution than to expose TCP-port 445 to other systems. The SMB service doesn't have a good security track-record.

For Unix environments, take a look at VSFTPD. The Very Secure FTP daemon, is written by Cris Evans, who works on the security team for Google. The irony is that although VSFTPD itself doesn't seem to be affected by any security vulnerability itself, the hosting provider hosting the software was compromised by an the attacker. This attacker put a back-door in a specific VSFTPD release.

Anyway, I still recommend VSFTPD as it is very well-document and the configuration is simple.

If one of these solutions is not an option for your particular situation, you might think about using your existing insecure file transfer method on top of a VPN connection that handels authentication and encryption, such as OpenVPN. But setting up OpenVPN within such an environment may also be cumbersome.

Recent events regarding compromised certificate authorities show that the trust model SSL-authentication often leans upon may be broken. You must be sure which certificate authorities to trust. If you have your own certificate authority, make sure you take every precaution to keep it secured.

Question: should the client rely on build in CA certificates, the sames as present in your browser? Or are you going to configure the client to accept only the single certificate of the server?

If you have any other suggestions for a simple solution to securely transfer files between hosts, feel free to leave a comment.

Therefore, I've created a script that monitors the used space of a volume and deletes the oldest file if a certain threshold is reached.

The script will keep on deleting the oldest file present on disk until used capacity is below the threshold.

So you can tell the script to monitor volume /storage and delete old files if the used capacity is bigger than 95 percent.

The script works like this:

./deleteoldfiles.sh <mount point> <percentage>

The mount point represents a volume or physical disk. The percentage represents the maxium used capacity threshold.

The script reads the output of the 'df -h' command to determine 'disk' usage.

Example:

bash-3.2$ ./deleteoldfiles.sh /Volumes/usb 92

DELETE OLD FILES 1.00

Usage of 90% is within limit of 92 percent.

How let's see what happens when the threshold is exceeded.

bash-3.2$ sudo ./deleteoldfiles.sh /Volumes/usb 92

DELETE OLD FILES 1.00

Usage of 97% exceeded limit of 92 percent.
Deleting oldest file /Volumes/usb/a/file02.bin
Usage of 91% is within limit of 92 percent.

Here you notice that an old file is deleted and that the script checks again if there is now enough free space. If not, another file would have been deleted.

If you have a need for it, have fun. It was a fun little scripting exercise.

The script works under Linux and Mac OS X.

I boot my iMac from an external FW800 SSD. I found out that it is impossible to encrypt this disk using the new FileVault as part of Lion.

Furthermore, I also found out that if you have a disk with a Bootcamp partition FileVault will also refuse to start the encryption process. I'm not trying to encrypt the Bootcamp volume, just the bootable Mac OS X Lion installation.

It may be advised to stay away from Lion if you need a setup similar to this one and also need disk encryption.