FS
Documentation

Backup3G/User Guide/Media

From Documentation

(Difference between revisions)
Jump to: navigation, search
Revision as of 01:06, 19 April 2006
Moff (Talk | contribs)
(Media Support Tables)
← Previous diff
Revision as of 07:57, 20 April 2006
Moff (Talk | contribs)

Next diff →
Line 8: Line 8:
*media usage reports and volume labeling. *media usage reports and volume labeling.
-'''Note -''' References to tapes and tape drives in backup3G and in this manual generally apply to any type of removable media.+ 
 +----
 +;Note: References to tapes and tape drives in backup3G and in this manual generally apply to any type of removable media.
 +----
<br> <br>
== Media Database == == Media Database ==
-The media database is kept on your backup3G master host, and stores information+The media database is kept on your backup3G master host, and stores information about all your removable media – both used and unused – and the data they contain. The main components of the database are:
-about all your removable media – both used and unused – and the data they contain.+;''media details'': information about all the media which can be used for backup and recovery—e.g. media type, usage, location
-The main components of the database are:+;''media contents'': information about data stored on the media, such as online indexes and lists of base directories that have been backed up
-;''media details'': information about all the media which can be used for backup and+;''media support tables'': information that describes how media are used in your organization—e.g. media types, locations, label types.
-recovery—e.g. media type, usage, location+ 
-;''media contents'': information about data stored on the media, such as online indexes+ Figure 9 — Media contents listing
-and lists of base directories that have been backed up+
-;''media support tables'': information that describes how media are used in your+
-organization—e.g. media types, locations, label types.+
-Figure 9 — Media contents listing+The database is set up when you first configure the backup3G master host (see Add Media to the Media Library on page 148). These details are automatically updated as each backup is run and when you use the options in the Media Management module.
-The database is set up when you first configure the backup3G master host (see Add 
-Media to the Media Library on page 148). These details are automatically updated as 
-each backup is run and when you use the options in the Media Management module. 
-Information on Media 55 
<br> <br>
== Media Management Concepts == == Media Management Concepts ==
-Removable media are used in backup3G to store data for backups and other purposes.+Removable media are used in backup3G to store data for backups and other purposes. Typically, high-capacity tapes are used (e.g. SDLT, LTO, DLT), but low capacity legacy tapes (e.g. Exabyte, DAT), diskettes, and optical media can equally well be used.
-Typically, high-capacity tapes are used (e.g. Exabyte, DAT, DLT), but lowcapacity+
-legacy tapes, diskettes, and optical media can equally well be used.+
'''Electronic labels''' '''Electronic labels'''
-Media are identified by a unique number. Tapes can be electronically labeled with+Media are identified by a unique number. Tapes can be electronically labeled with their media number to prevent live data being accidentally overwritten.
-their media number to prevent live data being accidentally overwritten.+
'''Media tracking''' '''Media tracking'''
-You can use the media management module to track details of your media over their+You can use the media management module to track details of your media over their whole life-cycle. For example, you can enter details of a new batch of tapes, display media contents, search through online indexes for backed-up files, and record when
-whole life-cycle. For example, you can enter details of a new batch of tapes, display+
-media contents, search through online indexes for backed-up files, and record when+
media are moved between locations. media are moved between locations.
-Media can be stored in several locations, for example different tape libraries and offsite+Media can be stored in several locations, for example different tape libraries and offsite storage areas. Jukeboxes and silos are also types of media location.
-storage areas. Jukeboxes and silos are also types of media location.+
'''Media usage''' '''Media usage'''
-Backup tapes have a retention period which is defined in the backup job. Once the+Backup tapes have a retention period which is defined in the backup job. Once the expiry date has passed, the data is not deleted, but the tape becomes available again. Unused and expired tapes are called scratch tapes, and the set of media available to be used is called the scratch pool.
-expiry date has passed, the data is not deleted, but the tape becomes available again.+
-Unused and expired tapes are called scratch tapes, and the set of media available to+
-be used is called the scratch pool.+
-Backup3G tries to balance media usage, by selecting the least-recently used scratch+Backup3G tries to balance media usage, by selecting the least-recently used scratch volume.
-volume.+
<br> <br>
=== How backup3G Tapes are Structured === === How backup3G Tapes are Structured ===
-Figure 10 shows the layout of a backup tape created by backup3G. The first file is a+Figure 10 shows the layout of a backup tape created by backup3G. The first file is a short (1 KB) file containing the electronic label. Subsequent files contain the backup data, followed by an end-of-data marker.
-short (1 KB) file containing the electronic label. Subsequent files contain the backup+
-data, followed by an end-of-data marker.+
File # Backup of Method/Format File # Backup of Method/Format

Revision as of 07:57, 20 April 2006

An essential element of the backup and recovery mechanism is effective media management. This section explains some media concepts and provides examples of good practice. Managing Removable Media on page 115 describes how to manage and use media through backup3G.

Backup3G’s facilities include:



Note
References to tapes and tape drives in backup3G and in this manual generally apply to any type of removable media.


Contents

Media Database

The media database is kept on your backup3G master host, and stores information about all your removable media – both used and unused – and the data they contain. The main components of the database are:

media details
information about all the media which can be used for backup and recovery—e.g. media type, usage, location
media contents
information about data stored on the media, such as online indexes and lists of base directories that have been backed up
media support tables
information that describes how media are used in your organization—e.g. media types, locations, label types.
Figure 9 — Media contents listing

The database is set up when you first configure the backup3G master host (see Add Media to the Media Library on page 148). These details are automatically updated as each backup is run and when you use the options in the Media Management module.



Media Management Concepts

Removable media are used in backup3G to store data for backups and other purposes. Typically, high-capacity tapes are used (e.g. SDLT, LTO, DLT), but low capacity legacy tapes (e.g. Exabyte, DAT), diskettes, and optical media can equally well be used.


Electronic labels

Media are identified by a unique number. Tapes can be electronically labeled with their media number to prevent live data being accidentally overwritten.


Media tracking

You can use the media management module to track details of your media over their whole life-cycle. For example, you can enter details of a new batch of tapes, display media contents, search through online indexes for backed-up files, and record when media are moved between locations.

Media can be stored in several locations, for example different tape libraries and offsite storage areas. Jukeboxes and silos are also types of media location.


Media usage

Backup tapes have a retention period which is defined in the backup job. Once the expiry date has passed, the data is not deleted, but the tape becomes available again. Unused and expired tapes are called scratch tapes, and the set of media available to be used is called the scratch pool.

Backup3G tries to balance media usage, by selecting the least-recently used scratch volume.


How backup3G Tapes are Structured

Figure 10 shows the layout of a backup tape created by backup3G. The first file is a short (1 KB) file containing the electronic label. Subsequent files contain the backup data, followed by an end-of-data marker.

File # Backup of Method/Format 2 / (root) Full cpio 3 /opt Full cpio 4 /var Full cpio 5 $APPL_HOME/db Full cpio

Figure 10 — Sample layout of a backup3G tape


Media Set

A multi-volume backup is written to a set of media. A media set is identified by a ‘Set ID’, which is the media number of the first volume in the set. Each volume also has a ‘Sequence Number’, which shows its position in the set.

Example: a backup job writes to three tapes, whose media numbers are 2012, 2038 and 2001. The media set is identified as follows:

Table 5 — Media set ID and sequence number Media No. Set ID Sequence No. 2012 2012 1 2038 2012 2 2001 2012 3


Scratch Media

Scratch media are tapes and disks that are available to be written by backup jobs. A tape may be included in the scratch pool for two reasons.

A volume is determined to be scratch according to information held about it in the media database, not by any mark or record on the tape or diskette itself. Until they are overwritten, scratch or expired tapes still contain the last files written to them.

A media set can also be manually scratched via the Maintain > Scratch option in the Media module. When you do this, each volume in the set is marked as ‘scratch’ and the link between them (the Set ID) is removed. Note that backup3G’s record of the contents of that volume is also lost. If you might need to search for or recover files from old backups, it’s best to leave them as expired and not to mark them as scratch.

A volume with a null expiry date never expires; this equates to a retention period of ‘forever’. To reuse a volume like this you must manually mark it as ‘scratch’ as described above.

Caution - Don’t try to force a volume to expire by editing its expiry date. This is not a safe method of scratching media, because the volume may be part of a media set containing active data, and if there is an online index it will not be removed.


Media Balancing and Backup Cycling

Some sites use a few tapes to do all their regular backups. For example, each week they may cycle through a set of tapes physically labeled ‘Monday’, ‘Tuesday’ and so on. The disadvantage of this scheme is that media containing vital backup data are used the most heavily and are subject to a higher risk of wear and damage. Exabyte tapes, for instance, carry a manufacturers recommended maximum of 40 uses.

Instead, backup3G balances tape usage by prompting the operator to load from the scratch pool the tape with the earliest ‘date last written’.

The scratch pool method has other advantages over tape cycling.

However, should you wish to, you can still implement cycling of backup media within backup3G as follows.

  1. Define a special media type, with the same details as the media normally used for this backup job. Name it (for example) exabyte-dly
  2. Add enough media to complete exactly one cycle—e.g. for a twelve-month cycle add twelve tapes of type exabyte-yrly.
    Label each tape electronically with the media number, and physically with a sticky label showing the media number and month—for example:

501 January 502 February 503 March …

  1. In the backup job, select this media type. Choose a retention period so that each volume expires the day before it is due to be reused.

Each time the backup job runs, only one tape of the correct type will ever be available—the one from the same point in the previous cycle that has just expired.


Media Life Cycle

A typical backup tape might go through the following stages.


Recording media on the system


Using media for backup or archive

the media table. The name of the base directory for each backup step is stored in the media contents table.


Tracking tape location


Recovery or expiry


Media Support Tables

The media database comprises the media table plus several support tables, which contain information about the way media are used at your site. These tables are updated whenever backup and recovery jobs run. You can customize the media support tables through backup3G configuration > Maintain tables.


Site/Media Location

A media location is a place where media can be stored. There are several different types of location:

onsite
the main media library or tape vault
offsite
another branch or office, or a long-term archive facility
load_device
a silo or jukebox, from where tapes are loaded automatically into a drive
load_area
a place where media are stored temporarily before being loaded manually into a drive—for example a trolley.

You should add to the location table any places where media may be kept, including temporary locations and auto-loading devices such as jukeboxes. Any location that will not be used for media storage should be defined as location type ‘other’.

A tape stacker or trolley is in effect a temporary staging area from which tapes can be loaded. However a load device can also be a semi-permanent storage location, for example a jukebox containing up to several hundred tapes, where most tapes are just left in the unit.

Tapes allocated to an auto-loading location are assigned slot numbers. If backup3G detects that a tape has been loaded in the wrong slot, it corrects the slot number in the media table.

Backup3G has facilities to track media movements between locations, for example to register that archive tapes have been sent offsite, and to list which tapes are due to be returned onsite. Under the Media module, you can View media by location and Move selected media sets on- or off-site.

An offsite location is a place away from the main onsite location where media can be stored indefinitely, for example in a vault on another floor or in another building.

Offsite media must be returned onsite before they can be used in a backup or recovery job or allocated to a load device.


Usage type

Describes what a volume is currently used for. For example, tapes containing backup data have type ‘backup’, and tapes containing archive data have type ‘archive’. Unused media are set to type ‘scratch’.


Maintenance Procedures

You can keep a history of corrective or preventive maintenance done to media, such as tape cleaning and low-level formatting. You can add your own site’s procedures to the ‘Media Maint Procedure’ table.


Media type

Each size and/or density of storage medium is stored as a unique media type. You can specify which media type a backup should request. For example, you could use the same drive to back up a large filesystem to a 5-gigabyte Exabyte tape, and smaller backups to 2-gigabyte tapes. You can add new formats to the ‘Media Type’ table.

The media type table also includes the part size and capacity, which are used when writing multi-part backups.


Media label type

Media should be electronically labeled with the media number. This helps to avoid volumes containing live data from being accidentally overwritten. backup3G supports several label formats:

COSstdtape writes a short file containing the media number and ‘date last written’ to the start of a tape, and an end-of-data marker after the last file on the tape. This is recommended for tapes that will be used for appending backups. COSstdBOT writes the media number and ‘date last written’ to the start of a tape, but doesn’t write an end-of-data marker. end_label writes the label to the end of a diskette. This prevents the label being overwritten when a new backup is done. sV_filesys System V filesystem labels


Media Table Locking

Processes that update media details (such as backup and recovery jobs, labelling a volume, and adding a media batch) lock the media table briefly while the new details are written to the database.

If the media table is temporarily locked when a second process tries to update it, the lock will usually have been released by the time the second process tries again. Interactive users are prompted to press a button to try again; unattended processes such as backup jobs retry automatically.

If a process locks out a backup for an excessive length of time, backup3G will kill that process rather than hold up a backup. For example, if a user has left open the ‘Change media details’ prompt form, backup3G will cancel the user’s process to release the lock and allow backups to continue.