C H A P T E R 4

Releasing

Releasing is the process by which the releaser makes disk cache space available by identifying archived files and releasing their disk cache copy. This makes room for other files to be created or staged from archive media. The releaser can release only archived files. A released file has no data on the disk cache.

The Sun StorEdge SAM-FS software automatically invokes the releaser process when a site-specified disk threshold is reached. Alternatively, you can use the `release`(1) command to release a file's disk space immediately or to set releasing parameters for a file.

You can also specify that files be released immediately after archiving, that files never be released, or that files be partially released.

This chapter describes the releasing process and releaser operations. It contains the following sections:

- Releaser Process Overview

- Releaser Concepts

- About Partial Releasing and Partial Staging

- About the releaser.cmd File

- Planning Releaser Operations

- Running the Releaser Manually

##

Releaser Process Overview

When file system utilization exceeds its configured high water mark, the file system management software invokes the releaser, which does the following:

1. Reads the `releaser.cmd` file and collects the directives that control the release process.

2. Scans the file system and collects information about each file.

3. Begins releasing files in priority order.

A file system can contain thousands of files. Keeping track of the release priority for every file can be wasteful because releasing only several large files might return the file system to its low water mark. However, the releaser must examine the priority of each file or risk missing the best candidates for release. The releaser does this by identifying the first 10,000 candidates. It then discards subsequent candidates if they do not have a priority greater than the lowest-priority candidate among the first 10,000.

After the releaser has determined the priority of the first 10,000 candidates, it selects the files with the highest priority for release. After releasing each file, the releaser checks whether the file system cache utilization is below the low water mark. If so, it stops releasing files. If not, it continues releasing the files in priority order.

If the releaser has released all 10,000 candidates and the file system is still above the low water mark, it starts over and identifies 10,000 new candidates.

The releaser exits if it cannot find any viable candidates. This can occur, for example, if files do not yet have archive copies. If this happens, the Sun StorEdge SAM-FS software starts the releaser again after one minute has elapsed.

The high and low water marks are set with the `high=`percent and `low=`percent file system mount options. For more information about these mount options, see the `mount_samfs`(1M) man page.

##

Releaser Concepts

This section describes concepts that are basic to the releaser process:

- Age - The amount of elapsed time from a given event until now. A file's inode keeps track of the following times for use by the releaser:

- Residence-change time

- Data-modified time

- Data-accessed time

You can view these times by using the `sls`(1) command with the `-D` option. Each time has a corresponding age. For example, if it is 10:15 a.m., a file with a modify time of 10:10 a.m. has a data-modified age of five minutes. For more information about the `sls`(1) command, see the `sls`(1) man page.

- Candidate - A candidate is a file that is eligible to be released. A file is not a candidate under the following circumstances:

- The file is already offline.

- The file has not been archived.

- The `archiver.cmd` command file specifies the `-norelease` attribute for the file, and the required copies have not yet been made.

- The file is marked as damaged.

- The file is not a regular file. It is a directory, block, character-special file, or pipe.

- The archiver is staging the file to make an additional copy. The file becomes eligible for release after the archiver stages it.

- The age of the file is negative. This can occur for NFS clients with inaccurate clock settings.

- The file is marked to never be released. You can use the `release`(1)`-n` command to specify this.

- The file was staged at a time in the past that is less than the minimum residence time setting. For more information, see The min_residence_age Directive: Specifying a Minimum Residence Time.

- The file was flagged for partial release, through the `release`(1) command's `-p` option, and it is already partially released.

- The file is too small.

- Priority - A priority is a numeric value that indicates the rank of a candidate file based on user-supplied weights that are applied to numeric attributes of that candidate. The overall priority is the sum of two types of priority: age priority and size priority.

Candidate files with numerically larger priorities are released before candidates with numerically smaller priorities.

- Weight - The weight is a numeric value that biases the priority calculation to include file attributes in which you are interested and to exclude file attributes in which you are not interested. For example, the size attribute of a file is excluded from the priority calculation if the size weight is set to 0. Weights are floating-point values from 0.0 to 1.0.

- Partial release - With partial release, a beginning portion of the file remains in disk cache while the rest of the file is released. Partial release is useful with utilities such as `filemgr`(1) that read the beginning of a file.

##

About Partial Releasing and Partial Staging

Releasing and staging are complementary processes. Files can be completely released from online disk cache after they are archived, or a site can specify that the beginning of a file (the stub) remain in disk cache while the remainder of the file is released. Partially releasing a file provides immediate access to data in the file stub without requiring that the file be staged first.

As system administrator, you can specify both the default partial release size and the maximum size of the stub to remain online when a file system is mounted. You can set these through the `mount`(1M) command or in the File System Manager software. See the File System Manager online help for more information.

The `mount`(1M) command options are as follows:

- `-o partial=`n option - Sets the default size (n) of a file stub to remain online. The `-o partial=`n setting must be less than or equal to the `-o maxpartial=`n setting.

- `-o maxpartial=`n option - Sets the maximum size (n) of a file stub to remain online.

You can specify the default stub size for a file by specifying the `-p` option on the `release`(1) command or the `p` option on the `sam_release`(3) library routine. To specify different-sized file stubs for different types of files or different applications, a user can specify the `-s` option on the `release`(1) command or the `s` option on the `sam_release`(3) library routine. The `-s` and `s` values must be less than the `-o maxpartial` value used with the `mount`(1M) command when the file system was mounted.

Another mount option, `-o partial_stage=`n, allows a system administrator to establish how much of a partial release stub must be read before the rest of the file is staged. Reading past the `-o partial_stage=`n size specification initiates the stage of the file.

By default, the `-o partial_stage=`n option is set to the size of the partial release stub. Changing this value affects file staging as follows:

- If the `-o partial_stage=`n option is set to the size of the partial release stub, the default behavior prevents the file from being staged until the application reaches the end of the partial release stub. Waiting until the end of the stub is reached causes a delay in accessing the rest of the file.

- If the `-o partial_stage=`n option is set to a value smaller than the partial release stub, the file is staged after the application crosses the threshold set by the `-o partial_stage=`n option. This reduces the chance of a delay in accessing the rest of the file data.

For example, assume that the following options are in effect:

- `-o partial_stage=16` (16 kilobytes)

- `-o partial=2097152` (2 gigabytes)

- `-o maxpartial=2097152` (2 gigabytes)

The `filemgr`(1) program reads the first 8 kilobytes of a file. The file is not staged. A video-on-demand program reads the same file, and the file is staged after it reads past the first 16 kilobytes of the file. The application continues reading the 2 gigabytes of disk data while the archive tape is mounted and positioned. When the video-on-demand program reads past 2 gigabytes of file data, the application reads immediately behind the staging activity. The application does not wait because the tape mounting and positioning is done while the application reads the partial file data.

Several command-line options affect whether a file can be marked for partial release. Some options are enabled by the system administrator, and others can be enabled by individual users. The following sections describe the release characteristics that can be set by the various types of users.

### System Administrator Option Summary

As system administrator, you can change the maximum value and default value for partial release when the file system is mounted. The mount(1M) options in TABLE 4-1 affect partial release. For more information about the mount(1) command, see the mount_samfs(1M) man page.

TABLE 4-1 Mount Options for Partial Release
mount(1M) Option | Effect
`-o maxpartial=`n | Determines the maximum amount of space, in kilobytes, that can remain in online disk cache if a file is marked for partial release. The maximum value is 2,097,152 kilobytes, which is 2 gigabytes. The minimum value is 0, which prevents any file from being partially released. If `-o maxpartial=0` is specified, the partial release feature is disabled, released files are released completely, and no portion of a file remains in disk cache. Users cannot override the value specified on this option after the file system is mounted. By default, the n argument is set to 16. This setting enables users to mark files for partial release with the maximum amount remaining on disk being 16 kilobytes.
`-o partial=`n | Sets a default amount of space, in kilobytes, to remain in disk cache if a user marks a file for partial release by using the `release`(1) command's `-p` option. The n argument must be at least 8, but it can be as great as the value specified for the `-o maxpartial=`n option. Because some applications do not need access to the entire file to complete their work, this option can be used to ensure that applications have the needed beginnings of files available to them. At the same time, using this option prevents files from being staged unnecessarily. By default n is 16.
`-o partial_stage=`n | Specifies that when a partially released file is accessed, n bytes of the file must be read before the entire file is staged from the archive media. This value is typically set to be lower than the amount of the `-o partial` setting. For n, specify an integer from 0 to the `-o maxpartial` specification. By default, this is 16, or whatever value was specified for the `-o partial` option.
`-o stage_n_window=`n | Specifies the amount of data to be staged at any one time to n. For n, specify an integer from 64 to 2,048,000. The default is 256 kilobytes. This option applies only to files that have the `stage -n` attribute set.

### User Option Summary

As system administrator, you can set maximum and default values for the size of a file stub that can remain in disk cache after the file is released. You can also determine whether the partial release feature is enabled for a particular file system.

By using the `release`(1) command and the `sam_release`(3) library routines, however, a user can set other release attributes and can specify the files to be marked for partial release. The command and library options that determine partial release attributes are shown in TABLE 4-2. For more information about the `release`(1) command, see the `release`(1) man page. For more information about the `sam_release`(3) library routine, see the `sam_release`(3) man page.

TABLE 4-2 User Release Options
Options | Effect
`release`(1) command and `-p` option or `sam_release`(3) library routine and `p` option | The `-p` and `p` options mark the named file for partial release. If these options are used, the amount of the file remaining in online disk cache after the file is released depends on the value of the `-o partial=`n option that was set when the file system in which the file resides was mounted. These options cannot be used to specify the number of bytes to remain online.
`release`(1) command and`-s partial_size` option or `sam_release`(3) library routine and `s` option | The `-s` and `s` options mark the named file for partial release, and they specify the amount of the file to remain in online disk cache. The arguments to the `-s` or `s` options specify the amount, in kilobytes, to remain online. A user cannot specify that the amount of a file remaining online be greater than the amount specified for the `-o maxpartial=`n value when the file system was mounted. If the user's value is greater than the value for the file system, the value for the file system is used, and the user's specification is ignored.

##

``About the `releaser.cmd` File

The `/etc/opt/SUNWsamfs/releaser.cmd` file consists of directive lines that specify site-specific releasing actions. The `releaser.cmd` file can contain directives for setting the release priority, specifying a log file, and other actions.

The following subsections describe the `releaser.cmd` directives:

- Specifying Age-Related and Size-Related Release Priority Directives

- The fs Directive: Specifying Directives for Individual File Systems

- The no_release and display_all_candidates Directives: Specifying Debugging

- The min_residence_age Directive: Specifying a Minimum Residence Time

- The logfile Directive: Specifying a Log File

- The rearch_no_release Directive: Inhibiting Releasing for Rearchived Files

- The list_size Directive: Adjusting the Size of the Releaser Candidate List

- Specifying Release Attributes for All Files in an Archive Set

For more information about these directives, see the `releaser.cmd`(4) man page.

Some global releasing directives can be configured using the File System Manager software. See the File System Manager online help for more information.

### Specifying Age-Related and Size-Related Release Priority Directives

Files are released from a file system according to a priority order determined by directives defined in the `releaser.cmd` file. Both file age and file size are considered. By default, sites release the largest, oldest files first, leaving the smallest, newest files on disk. The following sections show how the releaser considers a file's age and size when determining the release priority of files in a file system.

For additional information about releaser directives, see the `releaser.cmd`(4) man page.

#### File Age

The releaser considers the following possible ages when determining the age-related component of a file's release priority:

- The age since it was last accessed

- The age since it was last modified

- The age since it changed residency in disk cache

By default, the age of a file is the more recent of the file's three ages.

In some cases, a simple age derived from the most recently accessed time, modified time, and residence-changed time is preferred. In other cases, you might want the access age of a file to take precedence over the modification age. You can use directives to specify that a weighted age priority be used when calculating the release priority for a file.

CODE EXAMPLE 4-1 shows the formats of the age priority directive.

CODE EXAMPLE 4-1 Age Priority Directive Formats
    weight_age = float
    weight_age_access = float
    weight_age_modification = float
    weight_age_residence = float

- The `weight_age` directive specifies that a file's default age (the smaller of the file's access, modification, or residence age) be given a weighting factor. For float, specify a floating-point number from 0.0 through 1.0. By default, float = `1.0`.

This directive cannot be specified in conjunction with the `weight_age_access`, `weight_age_modification`, or `weight_age_residence` directives.

- The `weight_age_access`, `weight_age_modification`, and `weight_age_residence` directives specify that a file's age be determined based on one, two, or three of these possible ages. For float, specify a floating-point number from 0.0 through 1.0. By default, float = `1.0`.

These directives cannot be specified in conjunction with the `weight_age` directive.

If the `weight_age_access`, `weight_age_modification`, and `weight_age_residence` directives are all used, the age-related priority for a file is calculated as follows:

1. File age data is gathered for each file's possible age.

2. File age data is multiplied by the weighting factors specified in the `releaser.cmd` file.

3. The products of the multiplication are added together, as shown in the following equation.

file access age * `weight_age_access` + file modification age * `weight_age_modification` + file residency age * `weight_age_residence` _________________________________________________ = age_related_priority

CODE EXAMPLE 4-2 shows lines in a `releaser.cmd` file specifying that only the file's residence age be considered, and that the modification age and the access age be ignored, when the release priority of a file is calculated.

CODE EXAMPLE 4-2 `releaser.cmd` File Fragment
    weight_age_residence = 1.0
    weight_age_modify = 0.0
    weight_age_access = 0.0

After a file's age-related priority is calculated, it is multiplied by the file's size-related priority. The size-related priority is calculated as described in the following section.

#### File Size

The releaser considers a file's size when determining the size-related component of a file's release priority. The size of the file (in 4-kilobyte blocks) is multiplied by the weight specified for the `weight_size` directive to determine the size-related component of a file's release priority.

The format of the `weight_size` directive is as follows:

    weight_size = float

For float, specify a floating-point number from 0.0 through 1.0. By default, float = 1.0.

CODE EXAMPLE 4-3 shows a `releaser.cmd` file specifying that the file size is to be ignored for all files in the `samfs1` and `samfs2` file system when the release priority of a file is calculated.

CODE EXAMPLE 4-3 `releaser.cmd` File
    # releaser.cmd file
    logfile = /var/adm/default.releaser.log
    weight_size = 0.0
    #
    fs = samfs1
    weight_age = 1.0
    logfile = /var/adm/samfs1.releaser.log
    #
    fs = samfs2
    weight_age_modify = 0.3
    weight_age_access = 0.03
    weight_age_residence = 1.0
    logfile = /var/adm/samfs2.releaser.log

### The `fs` Directive: Specifying Directives for Individual File Systems

You can use the `fs =` family-set-name directive in the `releaser.cmd` file to indicate that the directives that follow the `fs =` directive apply only to the named file system.

This directive has the following format:

    fs = family-set-name

For family-set-name, specify the name of a family set in the `mcf` file.

Directives preceding the first `fs =` directive are global and apply to all file systems. Directives following the `fs =` directive override global directives. The directives described in this chapter can be used either as global directive or as directives specific to one file system.

The `releaser.cmd`(4) man page includes examples of the `fs =` directive.

### The `no_release` and `display_all_candidates` Directives: Specifying Debugging

The `no_release` and `display_all_candidates` directives can be useful for tuning or debugging the releaser. These directives are as follows:

- The `no_release` directive prevents files from being removed from online disk cache. You can use this directive to check the directives in the `releaser.cmd` without actually releasing files.

This directive has the following format:

    no_release

- The `display_all_candidates` directive writes the names of all release candidates to the log file.

This directive has the following format:

    display_all_candidates

These directives are helpful during debugging because the releaser writes the names of release candidates to the log file, but it does not physically release them from the file system.

### The `min_residence_age` Directive: Specifying a Minimum Residence Time

The `min_residence_age` directive enables you to specify the minimum amount of time that a file must reside in a file system before it becomes a candidate for release.

This directive has the following format:

    min_residence_age = time

For time, specify a time in seconds. The default time is 600, which is 10 minutes. There is no practical minimum or maximum time setting.

### The `logfile` Directive: Specifying a Log File

If a `logfile` directive is specified in the `releaser.cmd` file, the releaser either appends its activity log to the indicated file, or creates the file if it does not exist.

This directive has the following format:

    logfile = filename

For filename, specify the name of a log file.

CODE EXAMPLE 4-4 shows a sample log file (some lines have been wrapped to fit on the page).

CODE EXAMPLE 4-4 Releaser Log File Example
    Releaser begins at Wed Apr 28 17:29:06 2006
    inode pathname          /sam1/.inodes
    low-water mark          24%
    weight_size             1
    weight_age              1
    fs equipment ordinal    1
    family-set name         samfs1
    started by sam-amld?    yes
    release files?          yes
    display_all_candidates? no
    ---before scan---
    blocks_now_free:    3481504
    lwm_blocks:         3729362
    ---scanning---
    10501 (R: Wed Apr 21 18:47:50 CDT 2006) 10001 min, 500 blks /sam1/testdir0/filevp
    10500 (R: Wed Apr 21 18:48:10 CDT 2006) 10000 min, 500 blks /sam1/testdir0/filewq
    ...
    ---after scan---
    blocks_now_free:      3730736
    lwm_blocks:           3729362
    archnodrop: 0
    already_offline: 0
    bad_inode_number: 0
    damaged: 0
    extension_inode: 0
    negative_age: 0
    nodrop: 1
    not_regular: 9
    number_in_list: 675
    released_files: 202
    too_new_residence_time: 0
    too_small: 2
    total_candidates: 675
    total_inodes: 1376
    wrong_inode_number: 0
    zero_arch_status: 689
    zero_inode_number: 0
    zero_mode: 0
    CPU time: 2 seconds.
    Elapsed time: 10 seconds.
    Releaser ends at Wed Apr 28 17:29:16 2006

The `releaser`(1M) man page describes the information contained in the log file. Because the size of the log increases with each releaser run, you should plant to rotate the log file, or omit the `logfile` keyword.

### The `rearch_no_release` Directive: Inhibiting Releasing for Rearchived Files

By default, files marked for rearchiving are released. If the `rearch_no_release` directive is specified in the `releaser.cmd`(4) file, the releaser does not release the files marked for rearchiving. This directive has the following format:

    rearch_no_release

### The `list_size` Directive: Adjusting the Size of the Releaser Candidate List

You can use the `list_size` directive to specify the number of releaser candidates. If you notice that the releaser makes multiple file system scans before it releases the number of files needed to get to the low water mark, you might want to consider raising this value to a level greater than the default of 10,000. This might occur in a file system that contains many small files. You can get information about releaser activities from the releaser log file.

This directive has the following format:

    list_size = number

For number, specify an integer from 10 through 2,147,483,648.

### Specifying Release Attributes for All Files in an Archive Set

Most directives in the `archiver.cmd` file affect archiving, but the archive set assignment directive enables you to specify release attributes that apply to all files in an archive set.

TABLE 4-3 shows archive set assignment values that pertain to releasing.

TABLE 4-3 Archive Set Assignment Values for Releasing
Directive | Effect
--- | ---
`-release a` | Specifies that the files in the archive set should be released after the first archive copy is made. Do not use this option if you are making more than one archive copy of each file. In such a situation, copy 1 would be staged in order to make copy 2.
-release d | Reset to default.
`-release n` | Specifies that the files in the archive set should never be released.
`-release p` | Specifies that the files in the archive set should be partially released after archiving.

For more information about these and the other `archiver.cmd` directives, see Archiving.

##

Planning Releaser Operations

It is necessary to decide the characteristics of files in cache for your site. It is wasteful to load a tape if you are staging only a few kilobytes, so you may want to configure your system to retain small files in cache. CODE EXAMPLE 4-5 shows the directives to use in the `releaser.cmd` file to release the largest files first.

CODE EXAMPLE 4-5 Directives to Release the Largest Files First
    weight_size = 1.0
    weight_age = 0.0

Alternatively, you might want to retain recently modified files in cache, since a recently modified file might be modified again soon. This avoids the overhead involved when the file is staged to enable modification. In this case, use the second set of age weights. CODE EXAMPLE 4-6 shows the directives to use in the `releaser.cmd` file to weight files from the oldest modified to the most recently modified.

CODE EXAMPLE 4-6 Directives to Release Oldest Modified Files First
    weight_size = 0.0
    weight_age_access = 0.0
    weight_age_modify = 1.0
    weight_age_residence = 0.0

However, most situations are not this straightforward.

Assume that you want to release the largest files first. There are hundreds of small files that are the same size, and there are several large files. Eventually, the releaser releases all the large files. If `weight_age` `=` `0.0` is specified, the releaser then releases the small files in essentially random order because they are all the same size and have the same release priority.

In this scenario, you could set `weight_age` `=` `0.01` in order to release the oldest of the equally-sized small files first.

Alternatively, you might set `weight_size` `=` `1.0` and `weight_age` `=` `0.01`.

These directives violate the largest-first policy by counting smaller, less recently accessed files as better candidates than larger, more recently accessed files. However, you can reduce this effect by making `weight_age` smaller than `weight_size`.

For example, with the previous settings, a 4-kilobyte file that was staged 100 minutes ago and an 8-kilobyte file that was just staged both have the same release priority However, if you set `weight_age` `=` 0.001, a 4-kilobyte file must have been staged 1,000 minutes ago to have the same priority as the 8-kilobyte file that was just staged.

For assistance in adjusting priority weights, you can use the `no_release` and `display_all_candidates` directives and run the releaser manually to obtain a list of candidates in priority order.

##

Running the Releaser Manually

From time to time, you might want to run the releaser manually. For this, you need to know the mount point of the file system and the low water mark that you want the releaser to reach.

For example, to release files in the `/sam1` file system until it reaches 47 percent full, log in as root and type the following:

    # /opt/SUNWsamfs/sbin/sam-releaser /sam1 47 1.0

Command-line options override options specified in the `releaser.cmd` file. As the releaser runs, it writes information to your screen and to the releaser log file, if one is specified in the `releaser.cmd` file. For more information, see the `sam-releaser`(1M) man page.

Copyright © 2006, Sun Microsystems, Inc. All Rights Reserved.