Mount a disk permanently with ‘fstab’.

Having a hard drive in your Linux system is no fun unless you mount it! So why would you want to have to manually mount the drives after every reboot? In today’s article I will go over the steps of writing your disk information to the Linux ‘fstab’ file to allow you to automatically mount a drive’s partition every time you reboot your system.

Linux uses a file named ‘fstab’ to keep track of what partitions should be mounted when the system starts-up and the location to mount said partitions. The ‘fstab’ file is located in ‘/etc/’ and can be edited directly. To edit the file you can use any text editor; in this article I will be using the ‘Vim’ editor which can be obtained in Ubuntu by using the following command:

sudo apt-get install vim 

In order to put your drives information into the ‘fstab’ file you will need the following information:

1) The file system that the partition has been formatted with. (e.g. ext4)

2) The UUID of the partition. This can be obtained by issuing the following command:

blkid -p /dev/<drive>

3) The mount point (aka, the place where you would like to mount the disk.) This can be any folder that you choose as long as it has the correct permissions and is empty. It is best that you make a new folder in ‘/mnt/’ using the mkdir command.

Once you have compiled a list of the above info, you can open the file and take a look at the contents. Use the following command to do so:

vim /etc/fstab

Your file should look similar to the following:

fstab

Note: Your file may be opened as “read-only”. Depending on the distribution you are using, you may have to enter the ‘sudo’ command before opening the file in Vim.

As you can see there are three entries in my ‘fstab’ file; proc, / (root) and swap (which has no mount point.). The entries are written in the following format:

<file system> <mount point> <type> <options> <dump> <pass>

The file system is either the drive designation in the format /dev/<drive> (e.g. /dev/sdb1), the “Label” of the drive or the UUID of the drive.

The mount point is the place where you would like the partition to be mounted and therefor accessible to the system and the user.

The type is the file system that the partition is formatted with ( e.g. ext4)

The options, dump and pass tell the system what options the partition should be mounted with, whether or not to copy the drive to a backup (this is hardly used in modern times) and what order to mount the disks. The pass command can have three values with the following designations:

0 = no check (basically the entry is ignored)

1 = root file system (this drive should be mounted before other drives)

2 = other file system (this drive is not a system drive and should be mounted after the system drives)

Using the example drive we installed in my previous posts, we should create and entry for our new drive. To do so from within Vim, press ‘i’ to enter ‘insert’ mode so you can edit the file. Once you are in ‘insert’ mode enter the following information at the bottom of the ‘fstab’ file:

Note: You will be using the UUID of the drive. You should obtain the UUID with the command shown previously in this post.

 UUID=<UUID> /mnt/mountpoint ext4 defaults 0 0

Where <UUID> is the UUID of the partition you would like to mount.

After you have entered the above information exit out of ‘insert’ mode by pressing the ‘Esc’ key and then save and exit the file with by typing ‘:wq’.

In order to test the entry you should reboot your system and then use the ‘df’ command to view the mounted drives. You disk should show up as mounted to the location you specified as the mount point. If the drive is not shown in the output of the ‘df ‘ command then you should double check to make sure that you entered the correct information in ‘fstab’.

This article completes the series of posts that shows you, step-by-step, how to get a drive up and running from scratch on your Linux system. If you have any questions or comments, or you find any mistakes, please let me know!

Format a hard drive in Linux.

In the past couple of articles I have shown how to detect a new drive under Linux as well as how to partition the disk and prepare it for use. I have also gone over how to test a disk to make sure that it is reliable. Now I am going to cover how to format a drive using the Linux utility ‘mkfs’. In this post I will go over formatting a drive with the ‘ext4’ file-system which is a commonly used and (somewhat) modern file-system.

Note: Most of the commands shown below require you to have ‘root’ privileges. If you are running Ubuntu you will want to preface the commands with ‘sudo’ to get elevated privileges. I have kept the commands simple by not including the ‘sudo’ preface. 

In order to format the disk drive we must know a few things about the disk. First, you need to have the hard drive designation (e.g. sda). Secondly, you should have the number of the partition that you would like to format (e.g. sda1). I have shown how to find this information in my previous posts. In the following examples I will be using the first partition on ‘sdb’, so we will be formatting ‘sdb1’.

Once you have that information on hand you can execute the following command to format the disk drive:

mkfs.ext4 /dev/sdb1

Keep in mind that by using the ‘ext4’ file system your drive will most likely not be able to be read or written to by a Windows based system. If you are formatting a thumb drive using this method you will want to format the disk using FAT32 file-system, in which case you would use the mkfs.vfat command. If you are going to be preparing a hard drive for use on a Windows machine, you would want to format the disk using the ‘ntfs’ file-system by using the mkfs.ntfs command.

You will have to wait for a while depending on the size of the partition you are formatting. When the format is complete you will be presented with a command prompt as shown below.

mkfs.ext4

Now that you have a formatted disk you are going to want to be able to write and read data to and from the disk. In order to be able to do this you will have to ‘mount’ the disk to a directory on your system. This is a relatively simple processes which includes creating a mount point (a folder) and then mounting the file-system you just created to that directory.

To create a directory enter the following:

mkdir <location>

For example you could create a mount point in the ‘/mnt’ directory named ‘mountpoint’ with the following:

mkdir /mnt/mountpoint

Lastly, you will want to mount the disk to the mount-point you just created.

mount /dev/sdb1 /mnt/mountpoint

To ensure that the disk was correctly mounted, run the ‘df’ command by simply typing:

df

In addition to viewing the disk you mounted above, you can also use the ‘df’ command to view the location of all currently mounted file-systems as shown below.

df_command

Now you can browse to the disk using the ‘cd’, or “change directory” command as follows:

cd /mnt/mountpoint

Keep in mind that once you reboot your computer the drive will be un-mounted and you will have to manually re-mount the drive using the ‘mount’ command. In tomorrows article I will show you how to permanently mount the disk by using the ‘fstab’ file.

Changing the file format of a text file using the Vim editor.

After a long days installation of a new VoIP phone system at the office I am going to make today’s post a brief, but hopefully helpful one. I often use Vim to edit text files, but I also on occasion need to open said text files on a windows machine where nothing but WordPad is available. I have attempted to use Notepad for this task but often times it strips hard returns and tabs from the file making it impossible to read. After editing the file with WordPad you would inevitably save it and then open it again with Vim to make sure that no extra formatting was added. Usually, everything seems to be fine, but looking closely you will discover two things:

1) The file has become tagged as ‘executable’ in the Linux system.

Executable_File

2) The file now shows ‘dos’ as the file format type in Vim.

dos_format

The solution to the first problem (well, it’s really is more of an annoyance) can be achieved by issuing the following command:

chmod -x

The fact that the file format has now been changed to ‘dos’ can lead to problems down the road (e.g. when writing scripts). You are definitely going to want to ensure that the file format gets changed back to ‘unix’ format before you attempt to run your code on a Unix-like OS, but how?

Here is how you would change the file format from within the Vim editor back to ‘unix’:

:set fileformat=unix

If you  have ever wondered exactly why editing a file with anything other than Notepad in Windows was frowned upon to such a degree by the programming community, now you have the answer (and the solution).

Creating partitions using ‘fdisk’.

In today’s post we will go over how to partition a disk in Linux using the ‘fdisk’ utility and how to format a disk using the ‘mkfs’ utility. If you are still wondering how to get a disk to be recognized by the OS you can refer to yesterdays article here. After the disk has been installed and you have verified that the disk is being seen by the operating system the next step is to partition the disk.

Partitions (or in some operating systems referred to as “slices”) allow you to divide a disk into separate logical sections that you will then format with a file-system. In most Linux distributions you will have a ‘boot’ partition that contains the needed information to boot into the OS; a ‘system’ partition where your main operating system file structure is stored; and a ‘swap’ partition which is an area of the disk the OS can use as memory for operations. On most modern day Linux distros (and Windows for that matter), the partitions will be created and formatted for you automatically. In this article we will be explaining how to create a single partition that spans the entire disk. Of course you can create more than one partition on a physical hard drive but creating one large partition is useful when adding a new disk that will be used as a storage drive.

To partition a disk we will be using the ‘fdisk’ utility. This utility has been around for many years and is getting a little outdated in that it can only handle partitions up to 2TB in size. So if you are working with a disk that is larger than 2TB and you want to create a single, all encompassing partition you would be better off using a utility such as ‘parted’ (more on that in another post). The main reason I am using the ‘fdisk’ utility in this example is that it is easy to use and works very well.

To begin formatting the disk open a command prompt and type the following command:
(Note: ‘sudo’ is required if you are using a distro such as Ubuntu. All instructions are written assuming that you are using Ubuntu as this is the most popular desktop distribution at the time of this writing.)

sudo fdisk /dev/<hard drive>

Where <hard drive> is the name of the drive you are working with (e.g. /dev/sda). You should now be prompted with a prompt that looks like this:

Command (m for help):

From here you can press the ‘m’ key, then enter for a list of commands. Before we partition the disk, lets make sure that there are no existing partitions on the drive by pressing ‘p’ then enter. This will present you with a list of partitions that are on the disk. The list should be empty, meaning that there are no partitions on the disk. If there are partitions on the disk you can remove them by pressing the ‘d’ key then enter. You will be prompted on which partition you would like to remove.

Disclaimer:

Keep in mind that all commands that you place using the ‘fdisk’ utility are not “committed” to the disk (that is they are not written to the drive) until you issue the ‘w’ command. Even with that in mind, you should proceed with caution as removing or editing partitions on a disk will render existing data unreadable!

To create a partition on a disk you use the ‘n’ command followed by enter. You will be presented with the option of creating a “primary” or “extended” partition. In this scenario we will be creating a “primary” partition that spans the entire disk. To create the primary partition type “p” then enter. Next, you will presented with options on how large you want the partition to be. In this case, since we are creating a single partition that spans the entire disk we can simply press enter to accept the default values.

Next we will want to define the type of partition that we are going to be creating. You can define the partition type by issuing the ‘t’ command followed by enter. You will then be prompted to select the partition you would like to define – in this case we will just press enter as we only have one partition on the disk. If you where working with multiple partitions you would select the desired partition number.

After selecting the partition to define a type to, you will be prompted with a rather scary looking message asking for the “Hex code”. At this prompt you are going to want to press the “L” key to display a list of codes and their associated partition types. In this example we are going to define a “Linux” type partition which uses hex code ’83’.

After defining the partition parameters as explained above we are ready to write the partition to the disk. This is the point of no return so to speak and if you venture past this point of the article you will be making changes to your drive that could potentially cause you to loose information. Of course if this is a brand new drive that you are using for storage (as we are assuming in this article) then you will have no existing data on the disk.

To write the partition to the disk, issue the ‘w’ command and press enter. This will begin the processes of writing the partition tables to the disk. Depending on the size of your disk and other factors this can take a minute to complete. Once you are presented with your regular command prompt the processes has been completed and you now have a disk with a partition that is ready for formatting.

In tomorrows post, I will go over the processes of formatting the partition using the ‘mkfs’ utility included in most Linux distributions.

As always, if you have any questions leave me a comment!

Everything is a text file, or… How to tell a drive is connected in Linux.

Working with Linux can be a daunting experience at first, but once you get the basics down you will find that things are usually pretty simple as long as you know the right commands.

One thing to keep in mind is something that someone, somewhere once told me; “Everything in Linux is a text file”.

In other operating systems (I’m looking at you Windows!) everything is hidden from the user. OS’s like Windows store everything in binary blobs that are only directly readable by a computer. The user must use a utility (like Microsoft Management Console) to ask the computer to translate the binary into a user readable format. With Windows, even the log files are stored as binary blobs (Event Viewer anyone?).

But with Linux, everything is a text file. Not only logs but also system information is essentially a text file. For starters, let’s look at Linux directory named ‘/proc’. This is a pseudo-file-system mount-point where the system stores information used by the ‘kernel’. For the laymen, this is information about the inner workings of your Linux operating system. Everything from memory usage to process information can be pulled from the many files located in this directory.

In this article I will focus on how to retrieve disk information for use in connecting a new disk-drive to Linux. We will assume that the drive is fresh out of the box and has never been used before.

After plugging in the drive and powering on the machine the first thing we will want to do is check to make sure that the drive is being seen by the OS. On a Windows machine you would go into the ‘Computer Management’ MMC and view the Disk Management information. On Linux you can essentially think of the ‘/proc’ directory as your ‘Computer Management console. To view the most basic disk information on your Linux box you will want to view a file named ‘diskstats’ located in the ‘/proc’ directory. We can view this file with the following command:

cat /proc/diskstats

Below you can view the output of the command on my Linux box.

cat proc diskstats

In Linux hard drives are named in the format: connection type (sd=SCSI); drive letter (a,b,c…); parition (1,2,3…). So as you can see from the output I have a SCSI attached drive (Its a Serial ATA drive) that has been assigned the letter ‘a’ and it has a couple of partitions in it (sda1,sda2, sda5). This is my system drive and the partition structure of 1,2,5 is normal for a system disk (boot, system, and swap). If I enter in the new disk and check the file again we will hopefully see a ‘sdb’ show up in the list. At this point the disk will not have any partitions on it so we should just see the simple ‘sdb’ without any partition numbers.

Now that we have verified that the disk has been detected by the system kernel we can go ahead and start to get that disk ready for use. Preparing a disk consists of two task: creating one or more partitions and formatting the partitions. In tomorrows post I will talk about how to use the ‘fdisk’ utility to partition your drive and how to go about formatting the partition once it has been created.

Here we have a perfect example of how easy Linux can be if you just keep in mind that, “Everything is a text file”. What the system sees is essentially what the user sees and even though most of the files contained in the ‘/proc’ directory are read-only, you can obtain quite a bit of information from them. If you are new to Linux, one of the first things I would recommend is that the user uses the ‘cat’ command to view the files located in the ‘/proc’ directory. You cant hurt anything by just looking and you will get a better idea of what the system sees by exploring. That is part of the fun of Linux; you, as the user, has complete freedom to explore the system!

Using SMART to test a hard drive.

SMART stands for Self-Monitoring, Analysis and Reporting Technology and is available on most mechanical disks that are sold today. Using SMART to test the ‘health’ of a disk consists of two steps:

1 – Run a SMART test to get an initial report.

2 – Run a second SMART report.

In this article I will briefly  go over how to use the smartctl Linux utility that is included in the smartmontools  package.

If you are using Ubuntu you can get the smartmontools package by entering the following command:

sudo apt-get install smartmontools

Before testing the drive, make sure that SMART is enabled on the disk with the following command:

sudo smartctl -i /dev/<drive> | grep "SMART support"

After you have verified that SMART is indeed enabled on the drive, run a test to get an initial reading:

sudo smartctl --test=short /dev/<drive>

Note: <drive> is used to indicate the desired drive to run the test on.

The ‘short’ test will take approximately 2 mins to run. You should wait the recommended time before proceeding to run the report again. Running the ‘short’ test will give you an initial test to let you know if there are any glaring problems reported by the drives SMART software. If you want to run a more thorough test on the drive, you may want to consider running the test using the ‘long’ value. Even better, if you have a fresh disk or you don’t care about the data on the disk and you need to test it rigorously you should use a utility such as shred to write randomly to the disk before running the second report.

To run the second report run the following command:

sudo smartctl -a /dev/<drive>

If the report indicates that the drive PASSED the test then you can be pretty sure that the drive is healthy.

This has been a brief overview of testing the drive’s capability to pass a SMART test. If you have any questions please feel free to comment on this post and I will respond with clarifications.

How to place a chest in a bedroom.

I have been playing Dwarf Fortress for the past couple of weeks (much to my fiance’s chagrin) and I consider myself to be doing fairly well in getting the basics down; keeping dwarfs happy, drunk and alive.

My first couple of forts lasted about 20 mins before I started over and the first fort I really felt good about ended up getting overrun with goblins and everyone died. This is something that I feel that I am going to have to come to terms with if I am going to pursue this game any further (after all, losing is fun), but the most difficult thing about this game has to be the user interface.

The game was written by Tarn Adams aka ‘Toady One’ (with his brothers input) and as I’m playing I can’t help but feel like I am deep inside the man’s head. Everything, from the inconsistent menu selection buttons (seemingly random, sometimes you scroll with the up-down arrows, sometimes with the +/-) to the naming of in-game items, reveal a design that seems to have been put together in a stream-of-consciousness fashion. It’s actually kind of interesting to be able to get such an intimate idea of the character and personality of the creator of the game – something which you rarely get from most game developers – although it can also be annoying at times.

One thing in particular that had me scratching my head, was getting a damn “chest” into the bedrooms. According to the wiki (http://dwarffortresswiki.org/index.php/Bedroom), the dwarfs will have a happy life if you place the following in their bedrooms: a bed, a cabinet (for clothes) and a chest (for God knows what).

So I direct a carpenter to make a chest. Chest done. Now lets place the chest in the room…

Where the heck on the ‘build’ menu is there a chest???

Nowhere on the menu

Looking closely, I see that there is a non-descript “container” on the list. Let’s try that. However, when I press enter, I get a “bag” placed in the room. So how the heck do I get a “chest” in the room? Going through the menu again, I now see that there is a sub-selection area where you can choose what type of “container” you would like to place.

Chest menu

Come to find out, some menus have sub-menus and some items are placed directly from the “build” menu… Did the creator felt that some items are more important than others? Is a cabinet not a type of container? I suppose there is some logic to this and I don’t want this to end up being a rant about “how hard this game is”, but I just wanted to put that out there. Out of everything that the Toady One could do to make this game better, I would have to say that standardizing the menus would be at the top of my list.

By the way, here is my bedroom area with the chests ready to be placed!!! (The chests are the Æ symbols.)

BedRooms

What version of a Linux Distro are you using???

There comes a time in every Linux user’s life when they need a refresher as to what version of a Linux distro they are currently running. Fortunately there is a quick and easy utility for pulling the distribution-specific information” called ‘lsb_release‘.

You can look up the manual page for more detailed information (man lsb_release) and a run through of the various switches, but if you simply need the distro name and version number you can use the following command:

lsb_release -d

..and you should receive something similar to this:

Description: Ubuntu 12.04.2 LTS

Another useful use of this utility is to grab the “codename” of the distro you are running. This can be especially handy when running Ubuntu as they use a random word based on a particular letter of the alphabet.
In order to pull this info using the lsb_release utility enter:

lsb_release -c

…and receive the codename of your distro:

Codename: precise

Stay tuned for more command line utilities that help with everyday tasks! I will be categorizing them all under the category ‘Linux Commands’ for easy reference. If you have any requests or suggestions, be sure to comment and I will create articles based on those suggestions.

BackupPC Installation Walk-through – Introduction

BackupPC is a great piece of software that does what is sets out to do and does it well. Its goal is to provide a complete, enterprise level backup solution that is built on open software and it is truly a great piece of software that demonstrates how, by combining common Linux utilities, you can create a wonderful end product. Unfortunately it is often overlooked in favor of paid software suites that are bloated and often times overkill for the (what should be simple) job of backing up files.

One thing to keep in mind is what exactly a backup solution should offer. It is not enough to simply make a copy of the files you want to protect to some media. A true backup solution is one that maintains incremental changes in such a way that a certain file from a desired time frame is readily available for restore.

Most businesses are seriously in the dark ages when it comes to backups. Many are still using tape backups, even when hard drive capacity is going up and prices are coming down at alarming rates. Multiple terabytes of storage on a single medium was unheard of only years ago and the fact that you can now get 3TB+ disks for relatively cheap should be an indication that hard drive backups are the way to go. This is where BackupPC excels, although it is fully capable of performing backups to old fashioned tape drives as well.

In the following series of blog posts I am going to demonstrate, step by step, exactly how I setup a complete backup system for a client using sub $700 dollars worth of hardware and absolutely no money on OS and BackupPC (although donations to products and tools that help you get things done is always encouraged 😉 Upon completion my goal is to provide you with a step-by-step guide to setting up your own backup infrastructure that can be duplicated time and time again whether it be for a business client or for personal use (the netbook I am writing this on runs BackupPC to backup files to a MicroSD card).

Welcome to my blog!

Hello!

This is the very first post of my blog here on Atari911.com where I will be posting material related to some of the more technical aspects of technology.

I know that sounds kinda scary (and redundant?) but I promise that I will do my best to make it fun! All I really want is a place where people can go to get insight into some of the technology that I use on a regular basis. Things like; Linux, BackupPC, Windows products (including server) and maybe even a little Dwarf Fortress thrown in on the side (that’s right, I’m going to blog about my experience).

So this will evolve into a place where a wide range of information is available on a wide range of technologies, ideas… etc.

Love Jason XOXO