Oracle8 Visual Information Retrieval Cartridge User's Guide Release 1.0.1 A55255-02 |
Library | Product | Contents | Index

# 4
Visual Information Retrieval Reference

The Visual Information Retrieval Cartridge library consists of:

- object data types (ODTs) -- see Section 4.1.
- methods -- see Section 4.2.
- operators -- see Section 4.3.

The examples in this chapter assume that the stockphotos table, referred to in Chapter 3, has been created and filled with some photographic images. The table was created using the following SQL statement:

    CREATE TABLE stockphotos (photo_id NUMBER, photographer (VARCHAR2(64),
                 annotation (VARCHAR2(255), photo ORDSYS.ORDVIRB);

When you are storing or copying images, you must first create an empty BLOB in the table. Use the empty_blob( ) function, which has the following format:

    ORDSYS.ORDVIRB(empty_blob(),NULL,NULL,NULL,NULL,NULL,NULL,NULL)

## 4.1 Object Data Types (ODTs)

The Visual Information Retrieval Cartridge object data types (ODTs) are as follows:

- ORDVirB: supports images stored in an Oracle8 BLOB
- ORDVirF: supports images stored in an Oracle8 external file (BFILE)

This section presents reference information on the object data types.

For more information on BLOBs and BFILEs, see Oracle8 Application Developers Guide, Chapter 6: Large Objects (LOBs).

## ORDVirB Object Data Type

The ORDVirB ODT supports storage and retrieval of image data in a BLOB within an Oracle database. This object data type is defined as follows:

    CREATE TYPE ORDVIRB
    (
    -- TYPE ATTRIBUTES
    content             BLOB,
    height              INTEGER,
    width               INTEGER,
    contentLength       INTEGER,
    fileFormat          VARCHAR2(64),
    contentFormat       VARCHAR2(64),
    compressionFormat   VARCHAR2(64),
    signature           RAW(2000),
    -- METHOD DECLARATION
      MEMBER PROCEDURE copyContent(dest IN OUT ORDVIRB),
      MEMBER PROCEDURE setProperties(SELF IN OUT ORDVIRB),
      MEMBER PROCEDURE process    (SELF    IN OUT ORDVIRB,
                                   command  IN     VARCHAR2)
      MEMBER PROCEDURE processCopy(command  IN     VARCHAR2,
                                   dest     IN OUT BLOB)
    );

Where:

- content: Stored image
- height: Height of the image in pixels
- width: Width of image in pixels
- contentLength: Size of the on-disk image file in bytes
- fileFormat: File type of image (such as, TIFF, JFIF)
- contentFormat: Type of image (such as, monochrome, 8-bit gray scale)
- compressionFormat: Compression type of image
- signature: Feature vector (signature) for content-based retrieval

In PL/SQL data is moved with the DBMS LOB package. From the client, data is moved via OCI LOB calls. The ORDVirB ODT does not supply piece-wise routines for moving data.

## ORDVirF Object Data Type

The ORDVirF ODT supports storage and retrieval of image data in external files (BFILEs, which are not stored in the database). BFILEs are assumed to be read-only, and this is reflected in the member procedures. This object data type is defined as follows:

    CREATE TYPE ORDVIRF
    (
      -- TYPE ATTRIBUTES
    content             BFILE,
    height              INTEGER,
    width               INTEGER,
    contentLength       INTEGER,
    fileFormat          VARCHAR2(64),
    contentFormat       VARCHAR2(64),
    compressionFormat   VARCHAR2(64),
    signature           RAW(2000),
    -- METHOD DECLARATION
    MEMBER PROCEDURE copyContent(dest IN OUT ORDVIRB),

    MEMBER PROCEDURE setProperties(SELF IN OUT ORDVIRF),

    MEMBER PROCEDURE processCopy(command IN VARCHAR2,
                                 dest    IN OUT BLOB)
    );

Where:

- content: Stored image
- height: Height of the image in pixels
- width: Width of image in pixels
- contentLength: Size of the on-disk image file in bytes
- fileFormat: File type of image (such as, TIFF, JFIF)
- contentFormat: Type of image (such as, monochrome, 8-bit gray scale)
- compressionFormat: Compression type of image
- signature: Feature vector (signature) for content-based retrieval

## 4.2 Methods

This section presents reference information on the methods used for image manipulation.

The Visual Information Retrieval Cartridge methods are as follows:

- CopyContent( ): copies only the image portion of a BLOB or BFILE to a BLOB.
- Process( ): processes an image in place (for example, modifies it or converts it to another format); available for BLOBs only.
- ProcessCopy( ): copies an image and processes the copy (for example, modifies it or converts it to another format); available for BLOBs and BFILEs.
- SetProperties( ): obtains and stores the attributes of an image.

For more information on object types and methods, see the Oracle8 Concepts manual.

## CopyContent() Method

### Format

CopyContent (dest IN OUT BLOB);

### Description

Copy an image without changing it.

### Parameter

##### dest

The destination of the new image.

### Usage

Copies the image data into the supplied BLOB.

### Example

Create a copy of the image in image1 into myblob:

    image1.CopyContent(myblob);

## Process( ) Method

### Format

Process (command IN VARCHAR2 );

### Description

Performs one or more image processing techniques on a BLOB, writing the image back on itself.

### Parameter

##### command

A list of image processing changes to make for the image.

### Usage

You can change one or more of the image attributes shown in Table 4-1. See Appendix A, "File and Compression Formats" for information on the supported format combinations.

##### Table 4-1 Image Processing Parameters

Parameter Name | Usage | Values
--- | --- | ---
fileFormat | file format of the image | GIFF, TIFF, PCXF, PICT, RASF, CALS, BMPF, TGAF, JFIF
contentFormat | imagetype/pixel/data format | MONOCHROME, RAW, 4BITGRAYSCALE, 4BITGREYSCALE, 8BITGRAYSCALE, 8BITGREYSCALE, 1BITLUT, 2BITLUT, 4BITLUT, 8BITLUT, 16BITRGB, 24BITRGB, 32BITRGB, 24BITPLANAR
scale | scale factor (for example, 0.5 or 2.0); preserves aspect ratio | <FLOAT> positive
xscale | X-axis scale factor (default is 1) | <FLOAT> positive
yscale | Y-axis scale factor (default is 1) | <FLOAT> positive
compressionFormat | compression type/format | JPEG, SUNRLE, BMPRLE, TARGARLE, LZW, LZWHDIFF, FAX3, FAX4, HUFFMAN3, Packbits, GIFLZW
compressionQuality | compression quality | MAXCOMPRATIO, MAXINTEGRIT, LOWCOMP, MEDCOMP, HIGHCOMP
cut | window to cut (origin.x origin.y width height) | (Integer Integer Integer Integer) maximum value is 65535

Note: When specifying parameter values that include floating-point numbers, you must use double quotation marks (" ") around the value. If you do not, this may result in incorrect values being passed and you will get incorrect results.

### Examples

Change the file format of image1 to GIF:

    image1.process('fileFormat=GIFF');

Change image1 to use lower quality JPEG compression and double the size of the image, preserving the aspect ratio:

    image1.process('compressionFormat=JPEG, scale="2.0"');

Note that changing the length on only one axis (for example, xscale=2.0) does not affect the length on the other axis, and would result in image distortion.

## ProcessCopy( ) Method

### Format

ProcessCopy (command IN VARCHAR2,

dest IN OUT BLOB);

### Description

Copies an image BLOB to another BLOB.

### Parameters

##### command

A list of image processing changes to make for the image in the new copy.

##### dest

The destination of the new image.

### Usage

See Table 4-1, "Image Processing Parameters".

### Example

Copy an image, changing the file format, compression format, and data format in the destination image:

    create or replace procedure copyit is
     imgB1     ORDSYS.ORDVIRB;
     imgB4     ORDSYS.ORDVIRF;
     mycommand   VARCHAR2(400);
    begin
      select col2 into imgB1 from ordimgtab  where col1 = 1;
      select col2 into imgB4 from ordimgtab  where col1 = 4 for update;
      mycommand:= 'fileFormat=tiff compressionFormat = packbits
      contentFormat = 8bitlut';
      imgB1.processcopy(mycommand,imgB4.content);
      imgB4.setproperties;
      update ordimgtab set col2 = imgB4 where col1 = 4;
    end;

## SetProperties( ) Method

### Format

SetProperties();

### Description

Writes the characteristics of an image (BLOB or BFILE) into the appropriate attribute fields.

### Parameters

None

### Usage

After you have copied or stored an image, call this method to set the characteristics of the new image content.

This procedure sets the following information about an image:

- height in pixels
- width in pixels
- data size of the on-disk image in bytes
- file type (TIFF, JFIF, and so forth)
- image type (monochrome, 8-bit gray scale, and so forth)
- compression type (JPEG, LZW, and so forth)

Note that SetProperties( ) does not create the signature required for visual information retrieval. See the Analyze( ) operator in Section 4.3 for details.

### Example

Select the type, and then set the attributes using the SetProperties procedure.

    imgB1 ORDSYS.ORDVIRB;
    .
    .
    .
    select col2 into imgB1 from ordimgtab where col1 = 1 for update;
    imgB1.setProperties;
    dbms_output.put_line('image width = '|| imgB1.width );
    dbms_output.put_line('image height = '|| imgB1.height );
    dbms_output.put_line('image size = '|| imgB1.contentLength );
    dbms_output.put_line('image file type = '|| imgB1.fileFormat );
    dbms_output.put_line('image type = '|| imgB1.contentFormat );
    dbms_output.put_line('image compression = '|| imgB1.compressionFormat );
    -- Note: signature not meaningful as displayed output.

Example output:

    image width = 360
    image height = 490
    image size = 59650
    image file type = JFIF
    image type = 24BITRGB
    image compression = JPEG

## 4.3 Operators

The Visual Information Retrieval Cartridge operators are located in the
ORDSYS.VIR package. The operators, which are specific to content-based retrieval, are as follows:

- Analyze( ): Produces the signature of an image.
- Convert( ): Converts the signature to either big or little endian byte order based on the architecture of the host machine. This operator can also convert a Viisage facial signature to a signature usable by the Score( ) and Similar( ) operators.
- Score( ): Compares two signatures, considers the weights for the visual attributes, and computes an overall score that is the weighted sum of the distances. For example, in Section 2.3.3, overall scores (weighted sums) of 61.5 and 24 were computed for comparisons of the same two imags using two different sets of weights.
- Similar( ): Compares two signatures and determines whether or not the images match, based on the weights and threshold; returns 1 if the computed distance measure (weighted average) is less than or equal to the threshold value, and otherwise returns 0.

For ease of use, you can create a local synonym for the ORDSYS.VIR package. Connect to your schema and issue the following command:

    SVRMGR> CREATE SYNONYM VIR FOR ORDSYS.VIR;

After creating the synonym, you can use it in calls to the operators: VIR.Analyze( ), VIR.Score( ), and so forth. Note that you must have the default CREATE SYNONYM privilege.

This section presents reference information on the operators.

## Analyze( ) Operator

### Format

Analyze(image IN BLOB, signature OUT RAW);

or

Analyze(image IN BFILE, signature OUT RAW);

### Description

Analyzes an image BLOB or BFILE, derives information relating to the visual attributes (including a score for each), and creates the image signature.

### Parameters

##### image

The BLOB or BFILE to be analyzed.

##### signature

The vector to contain the signature.

### Usage

The Analyze( ) operator creates the image attribute signature, which is necessary for any content-based retrieval. Whenever you are working with a new or changed image, you should also use the SetProperties( ) method to set the other image characteristics.

Signatures for facial images can be created using an optional third-party software package from Viisage Technology, Inc. After creating a facial signature, Visual Information Retrieval cartridge can convert the signature to a standard format and then compare the signatures using the Score( ) and Similar( ) operators.

### Example

Create the signatures for all images in the stockphotos table.

    DECLARE
       temp_image   ORDSYS.ORDVirB;
       temp_id      INTEGER;
       cursor c1 is select id, image from stockphotos;
    BEGIN
       OPEN c1;
       LOOP
          fetch c1 into temp_id, temp_image;
          EXIT WHEN c1%NOTFOUND;

     -- Generate signature and set the properties for the image.
          ORDSYS.VIR.Analyze(temp_image.content, temp_image.signature);
          temp_image.setProperties;
          UPDATE stockphotos SET photo = temp_image WHERE photo_id = temp_id;
       END LOOP;
       CLOSE c1;
    END;

## Convert( ) Operator

### Format

Convert(signature IN OUT RAW, operation IN VARCHAR2);

### Description

Converts the image signature to a format usable by the host machine.

### Parameters

##### signature

The signature of the image, as created by the Analyze( ) operator. Data type is raw(2000).

##### operation

The operation specifies what processing is done to the image signature. The following operations are available:

Operation Keyword | Description
--- | ---
BYTEORDER | Converts the signature to the natural byte order of the host machine.
VIISAGE | Converts the signature from the format used for Viisage face-recognition to a signature usable by the Score( ) and Similar( ) operators.

### Usage

The signature is converted to the format of the host platform regardless of its initial state.

This procedure is useful if the database is stored on a remote system, but you want to do your processing locally. If your host machine is from Sun Microsystems, Inc., the Convert( ) operator sets the signature to the big endian byte order. On an Intel Corporation machine, the operator converts the signature to the little endian byte order. Note that the images themselves are platform-independent; only the signatures need to be converted.

### Example

The following example converts the signature of the image with `photo_id=1` to the platform of the host system:

    DECLARE
       myimage ORDSYS.ORDVirB;
       myid      INTEGER;
    BEGIN
       SELECT photo INTO myimage FROM stockphotos WHERE photo_id=1;
       ORDSYS.VIR.Convert(myimage.signature,'BYTEORDER');
       UPDATE stockphotos SET photo=myimage;
    END;

## Score( ) Operator

### Format

Score(signature1 IN RAW,

signature2 IN RAW,

weightstring IN VARCHAR2);

where weightstring is: 'globalcolor="val", localcolor="val", texture="val", structure="val"'

or: 'facial=1'

### Description

Compares the signatures of two images and returns a number representing the weighted sum of the distances for the visual attributes.

### Parameters

##### signature1

The signature of the comparison image (the image with which other images are being compared to test for matches). Data type is raw(2000).

##### signature2

The signature of the image being compared with the comparison image. Data type is raw(2000).

##### weightstring

A list of weights to apply to each visual attribute. Data type is VARCHAR2. The following attributes can be specified:

Attribute | Description
--- | ---
globalcolor | The weight value (0.0 to 1.0) assigned to the global color visual attribute. Data type is number. Default is 0.0.
localcolor | The weight value (0.0 to 1.0) assigned to the local color visual attribute. Data type is number. Default is 0.0.
texture | The weight value (0.0 to 1.0) assigned to the texture visual attribute. Data type is number. Default is 0.0.
structure | The weight value (0.0 to 1.0) assigned to the structure visual attribute. Data type is number. Default is 0.0.
facial | The two signatures are Viisage facial signatures. When comparing facial signatures, no other attributes can be included in the weightstring. Data type is number, and must be set to 1 if used.

Note: When specifying parameter values that include floating-point numbers, you should use double quotation marks (" ") around the value. If you do not, this may result in incorrect values being passed and you will get incorrect results.

### Returns

This function returns a FLOAT value between 0.0 and 100.0.

### Usage

Before the Score( ) operator can be used, the image signatures must be created with the Analyze( ) operator.

The Score( ) operator can be useful when an application wants to make finer distinctions about matching than the simple "Yes" or "No" returned by Similar( ). For example, using the number for weighted sum returned by Score( ), the application might assign each image being compared to one of several categories, such as Definite Matches, Probable Matches, Possible Matches, and Non-Matches. The Score( ) operator can also be useful if the application needs to perform special processing based on the degree of similarity between images.

The weights supplied for the four visual attributes are normalized prior to processing such that they add up to 100 percent. To avoid confusion and meaningless results, you should develop a habit of always using the same scale, whether 0 to 100 or 0.0 to 1.0.

You must specify at least one of the four image attributes or the facial attribute, in the weightstring. You cannot combine the facial attribute with any of the other attributes.

### Example

The following example finds the weighted sum of the distances between image 1 and the other images in the stockphotos table, using the following weights for the visual attributes:

- global color: 0.2
- local color: 0.2
- texture: 0.1
- structure: 0.5

This example assumes that the signatures have already been created using the Analyze( ) operator and they are stored in the database.

    DECLARE
       weighted_sum     NUMBER;
    BEGIN
       SELECT Q.photo_id,
              ORDSYS.VIR.Score(S.photo.signature,
                                     Q.photo.signature,
                                     'globalcolor="0.2"

                                      localcolor="0.2"

                                      texture="0.1"

                                       structure="0.5"') weighted_sum
       FROM stockphotos Q, stockphotos S
         WHERE S.photo_id=1 and Q.photo_id !=S.photo_id;
    END;

The following shows possible output from this example. The last image has the highest score, and therefore is the best match of the test image (`photo_id=1`). Changing the weights used in the scoring would lead to different results.

    PHOTO_ID   WEIGHTED_SUM
    ---------- -------------
             2    3.79988
             3    76.0807
             4    47.8139
             5    80.451
             6    91.2473
    5 rows selected.

## Similar( ) Operator

### Format

Similar(signature1 IN RAW

signature2 IN RAW,

weightstring IN VARCHAR2

threshold IN FLOAT);

where weightstring is: 'globalcolor="val", localcolor="val", texture="val", structure="val"'

or: 'facial=1'

### Description

Determines whether or not two images match. Specifically, compares the signatures of two images, computes a weighted sum of the distance between the two images using weight values for the visual attributes, compares the weighted sum with the threshold value, and returns the integer value 1 if the weighted sum is less than or equal to the threshold value, and otherwise returns 0.

### Parameters

##### signature1

The signature of the comparison image (the image with which other images are being compared to test for matches).

##### signature2

The signature of the image being compared with the comparison image.

##### weightstring

A list of weights to apply to each visual attribute. Data type is VARCHAR2. The following attributes can be specified:

Attribute | Description
--- | ---
globalcolor | The weight value (0.0 to 1.0) assigned to the global color visual attribute. Data type is number. Default is 0.0.
localcolor | The weight value (0.0 to 1.0) assigned to the local color visual attribute. Data type is number. Default is 0.0.
texture | The weight value (0.0 to 1.0) assigned to the texture visual attribute. Data type is number. Default is 0.0.
structure | The weight value (0.0 to 1.0) assigned to the structure visual attribute. Data type is number. Default is 0.0.
facial | The two signatures are Viisage facial signatures. When comparing facial signatures, no other attributes can be included in the weightstring. Data type is number, and must be set to 1 if used.

Note: When specifying parameter values that include floating-point numbers, you should use double quotation marks (" ") around the value. If you do not, this may result in incorrect values being passed and you will get incorrect results.

##### threshold

The threshold value with which the weighted sum of the distances is to be compared. If the weighted sum is less than or equal to the threshold value, the images are considered to match. This range of this parameter is from 0.0 to 100.0.

### Returns

This operator returns an integer value of 0 (not similar) or 1 (match).

### Usage

Before the Similar( ) operator can be used, the image signatures must be created with the Analyze( ) operator.

The Similar( ) operator is useful when the application needs a simple "Yes" or "No" for whether or not two images match. The Score( ) operator is useful when an application wants to make finer distinctions about matching or to perform special processing based on the degree of similarity between images.

The weights supplied for the four visual attributes are normalized prior to processing such that they add up to 100 percent. To avoid confusion and meaningless results, you should develop a habit of always using the same scale, whether 0 to 100 or 0.0 to 1.0.

You must specify at least one of the four image attributes, or the facial attribute in the weightstring. You cannot combine the facial attribute with any of the other attributes.

### Example

This example checks the first image against all of the other images in the table and determines if there are any matches, using a threshold value of 75 and the following weights for the visual attributes:

- global color: 20
- local color: 20
- texture: 10
- structure: 50

    DECLARE

    BEGIN
       SELECT Q.photo_id FROM stockphotos Q, stockphotos S
          WHERE S.photo_id=1 and Q.photo_id != S.photo_id
          AND  ORDSYS.VIR.Similar(S.photo.signature,
                                        Q.photo.signature,
                                       'globalcolor="20"

                                        localcolor="20"

                                        texture="10"

                                            structure="50"', 75)=1;
    END;

The following shows a possible output from this example. See the Example section of the Score( ) operator for a different result using the same images and weights.

    PHOTO_ID
    ----------
             3
             5
             6
    3 rows selected.

Prev | Next
| Copyright © 1997 Oracle Corporation. All Rights Reserved. |
Library | Product | Contents | Index