Skip Headers

Oracle® JRockit Virtual Edition User's Guide Release 2.0.0 Part Number E22518-01 |
Home | Contents | Contact Us

Previous | Next
| View PDF

# 7.2 Run-Time Problems

This section provides information to help you diagnose and solve problems that might occur when you run virtual machines (created by using Oracle JRockit Virtual Edition) on Oracle VM.

This section contains the following topics:

- How to Diagnose and Solve Run-Time Problems

- Fixing Problems in the File System of a Virtual Machine

- Solutions to Specific Run-Time Problems

- Frequently Asked Questions: Run-Time

- Viewing the Virtual Machine Log File

- Using Crash Files

- Using Diagnostic Commands

- Using the Diagnostic Service

## 7.2.1 How to Diagnose and Solve Run-Time Problems

This section describes a roadmap that you can follow to diagnose and solve Image Tool problems efficiently.

1. Verify whether you created the virtual machine as described in Chapter 3, "Creating Virtual Machines on Oracle VM."

2. Go to the Oracle VM Manager and check the status of the virtual machine.

   If the status is `Error`, view the details of the virtual machine and click the link to the log file. The Oracle VM Manager User's Guide ("Troubleshooting" section) provides information to help you troubleshoot virtual machine problems by using log messages.

3. Fix problems in the file system of the virtual machine by following the procedure described in Section 7.2.2, "Fixing Problems in the File System of a Virtual Machine."

4. Check whether Section 7.2.3, "Solutions to Specific Run-Time Problems" contains the solution to the problem.

5. Check whether Section 7.2.4, "Frequently Asked Questions: Run-Time" contains the answer to your question.

6. Contact Oracle for help, as described in Section 7.3, "Contacting Oracle for Help."

## 7.2.2 Fixing Problems in the File System of a Virtual Machine

The virtual machine might crash due to problems in its file system. The problems could, for example, be caused by interruptions while executing Image Tool options (such as `-f` and `-r`) that access the file system, and result in the following error message:

    EBUSY - Device or resource busy! (Filesystem already mounted!)

When the virtual machine restarts, Oracle JRockit Virtual Edition checks the file system automatically.

If the virtual machine cannot be restarted, you can find and fix file system problems by running the following Image Tool command.

Note:

Repairing a file system of a virtual machine image could lead to loss of data. So, before attempting to repair the file system, back up the contents in virtual machine disk by copying the virtual machine image.

    java -jar jrockitve-imagetool.jar --repair vm_cfg [prompt_type]

In this command, `vm_cfg` is the path and name of the virtual machine configuration file (`vm.cfg`), and `prompt_type` indicates whether problems found should be fixed automatically (`auto`, the default value), not fixed at all (`check`), or fixed only after the user specifically confirms that each problem should be fixed (`prompt`).

For more information, see --repair in Appendix A, "Image Tool Command-Line Option Reference."

## 7.2.3 Solutions to Specific Run-Time Problems

ERROR: Configured IP [...] in use by MAC

When the IP address specified for the server is already in use, the following error message is displayed:

    00:00:01 [net WRN] Configured IP ip_address in use by MAC: mac_address
    00:00:02 [net WRN] Network stack initialization FAILED: 98

Use the `-r` option to verify the IP address. Make sure that the IP address is correct and none of the running virtual machines use the same address. If the problem persists, contact your system administrator to obtain a new IP address.

For more information about using the `-r` option to reconfigure a virtual machine, see Appendix A, "Image Tool Command-Line Option Reference."

My Virtual Machine Is Freezing

If you know the IP address of the virtual machine, ping it.

- A failed ping indicates that the virtual machine is no longer running; it has crashed or hung (see "My Virtual Machine Has Crashed").

  Note that the ping can also fail if network security features, such as a firewall, are enabled in your system or network.

- A successful ping indicates that the network stack, scheduler, and other operating system functions are working.

  Try to determine where the virtual machine is freezing by reviewing the thread dump. For information about generating a thread dump, see "How do I take thread dumps?".

My Virtual Machine Has Crashed

When a virtual machine crashes, the Oracle JRockit JVM creates a text crash file in the current working directory of the virtual machine; it also writes the crash file to `stderr` and to the virtual console log.

Review the crash file to determine the cause of the crash. For more information, see Section 7.2.5, "Using Crash Files."

Error: 2, 'Invalid kernel', 'xc_dom_parse_elf_kernel: corrupted ELF image\n'

When a virtual machine starts, Oracle VM copies the Oracle JRockit Virtual Edition ELF (executable and linking format) kernel image to the `/var/run/xend/boot` directory, and then reads the image. If the `/var` disk partition is full (for example, on account of frequent of core dumps in the `/var/xen/dump` directory), Oracle VM cannot copy the ELF kernel image to the `/var/run/xend/boot` directory; so it displays the error message.

Make sure that the `/var` disk partition has sufficient space.

## 7.2.4 Frequently Asked Questions: Run-Time

Where can I find crash files?

See "Location of the Oracle JRockit JVM and Oracle JRockit Virtual Edition Crash Files".

How do I take thread dumps?

A thread dump is a snapshot of the state of all threads that are part of a virtual machine image.

To take thread dumps in Oracle JRockit Virtual Edition, do one of the following:

- Run the `print_threads` diagnostic command from Oracle JRockit Mission Control.

- Press `Ctrl+\` (or the `9` key) in the virtual machine console. This is equivalent to sending a SIGQUIT signal to the process in a Linux environment

If you cannot generate thread dumps, force the virtual machine to crash by using the `xm dump-core` command. Provide the core dump file to Oracle Support for further diagnosis and troubleshooting.

For more information about troubleshooting by using thread dumps, see the Oracle® JRockit Diagnostics and Troubleshooting Guide at `http://download.oracle.com/docs/cd/E15289_01/doc.40/e15059/toc.htm`.

How do I send a Diagnostic Command?

Use Oracle JRockit Mission Control.

For more information, see Section 7.2.6, "Using Diagnostic Commands."

Where should I store the ctrlhandler.act file?

Save the `ctrlhandler.act` file in the current working directory of the file system of the virtual machine. For information about finding out the current working directory, see "How do I find out the current working directory of a virtual machine image?".

How do I shut down the virtual machine from the virtual console?

Press the numeric key '`5`' and then press `Ctrl+c` in the virtual machine console. This is equivalent to sending a SIGINT signal.

How do I send a SIGABRT signal?

Oracle JRockit Virtual Edition does not provide an option to send a SIGABRT signal.

Use the `xm dump-core` command to generate a core dump.

How do I find out the current working directory of a virtual machine image?

You can find out the current working directory by running the following command:

Caution:

You must shut down the virtual machine before running this command.

    java -jar jrockitve-imagetool.jar -r vm_cfg get working-dir

In this command, `vm_cfg` is the path and name of the virtual machine configuration file (`vm.cfg`).

## 7.2.5 Using Crash Files

When the JVM crashes, Oracle JRockit Virtual Edition creates a text crash file (`.dump`) and a binary crash file. Note that if the Oracle JRockit Virtual Edition kernel (rather than the JVM) notices the crash first, then the crash files are not generated.

For more information about crash files and crash file sizing, see the Oracle® JRockit Diagnostics and Troubleshooting Guide at `http://download.oracle.com/docs/cd/E15289_01/doc.40/e15059/toc.htm`.

Location of the Oracle JRockit JVM and Oracle JRockit Virtual Edition Crash Files

- The name of the Oracle JRockit text crash file is `jrockit.4711.dump`. It is written to the current working directory in the file system of the virtual machine.

  For information about finding out the current working directory, see "How do I find out the current working directory of a virtual machine image?"

  Note:

  Oracle JRockit Virtual Edition does not support using the environment variable `JROCKIT_DUMP_PATH` to specify a different location for the text crash file.

- The Oracle JRockit Virtual Edition binary crash file is written to the `/var/xen/dump` directory on the Oracle VM Server.

Enabling Creation of Crash Files on Oracle VM

The virtual machine configuration file (`vm.cfg`) contains a parameter `on_crash`, which defines the action to be performed in the event of a system crash, as shown in the following example.

    name="wls-ve"
    bootloader="/usr/bin/pygrub"
    memory=256
    disk=['tap:aio:/OVS/seed_pool/wls-ve/system.img,sda1,w']
    vif=['']
    on_crash="coredump-destroy"

For a virtual machine image created by using Oracle JRockit Virtual Edition, the possible values for the `on_crash` parameter are `restart`, `destroy`, `coredump-restart` and `coredump-destroy`.

Note:

For crash files to be written, sufficient disk space must exist in the `/var` partition on the Oracle VM Server.

Disabling Creation of Crash Files

- To disable creation of the text crash file, use the `-XXnoJrDump` option.

- To disable creation of the binary crash file, change the value of the `on_crash` parameter in the `vm.cfg` file.

Caution:

Changing the `on_crash` parameter in the `vm.cfg` file renders the product unsupported.

## 7.2.6 Using Diagnostic Commands

You can send diagnostic commands to a running virtual machine in one of the following ways:

- By using the Ctrl-Break handler from Oracle JRockit Mission Control.

- By using the JRockit Management Console in Oracle JRockit Mission Control.

Note:

The default Ctrl-Break handler (triggered by pressing `Ctrl+\`) is the `print_threads` command. However, if the `ctrlhandler.act` file is available in the current working directory, the diagnostic commands listed in the file are executed.

For more information about using the Ctrl-Break handler and JRockit Management Console, see the Oracle® JRockit JDK Tools Guide at `http://download.oracle.com/docs/cd/E15289_01/doc.40/e15061/toc.htm`.

## 7.2.7 Using the Diagnostic Service

You can configure a diagnostic service called `diag` with the Image Tool and send certain network diagnostic commands to a running virtual machine.

Enabling the diagnostic service

You can enable the diagnostic service `diag` by executing the `-r (--reconfigure)` command as follows:

    java -jar jrockitve-imagetool.jar -r vm_cfg enable service diag

where `vm_cfg` represents the path and name of the configuration file (`vm.cfg`) of the virtual machine to be configured.

For more information about the `-r` option, see "-r (--reconfigure)".

Configuring the diagnostic service

To send the network diagnostic commands to a running virtual machine, you must first define the commands in a script file (`*.script` file). The script file must contain any or all of the following supported commands:

- `traceroute hostname|ip`

- `ping hostname|ip`

- `lookup hostname|ip`

After you enable the `diag` service, configure the virtual machine to use the script file. To configure services in a virtual machine, use the `--reconfigure-service` command as follows:

    java -jar jrockitve-imagetool.jar --reconfigure-service vm_cfg diag set script-file my.script

For more information about the `--reconfigure-service` option, see "--reconfigure-service".

Note:

If you disable the `diag` service for a virtual machine, you must enable and reconfigure the service again for the virtual machine.

To view the configured script for a running virtual machine, execute the following command:

    java -jar jrockitve-imagetool.jar --reconfigure-service vm_cfg diag get script-file [outfile]

To see the latest result of the diagnostic service:

    java -jar jrockitve-imagetool.jar --reconfigure-service vm_cfg diag get result [outfile]

The service also provides you with an option to shut down the virtual machine if any of the diagnostics commands fails. For example, if the `traceroute` command fails to trace a host, you can shut down the virtual machine. The default behavior of the service is not to shut down (`false`) if the command fails. If you want to modify this option, reconfigure the service as follows:

    java -jar jrockitve-imagetool.jar --reconfigure-service vm_cfg diag set fail-on-error true|false

## 7.2.8 Viewing the Virtual Machine Log File

The log messages generated by Oracle JRockit Virtual Edition are useful for diagnosing and troubleshooting run-time problems.

The messages written to the standard output streams (`stdout` and `stderr`) are also written to a log file, which, by default, is `/jrockitve/log/jrockitve.log` within the file system of the virtual machine image. You can change the log file name and location by using the `-r vm_cfg set console-log-path` operation of the Image Tool.

Note:

Oracle VM Server logs are in the `/var/log/xen` directory of the Oracle VM server. The log file is named `xend.log`. It contains only Oracle VM server-related messages.

To view the contents of the virtual machine log file (if it exists within the file system of the virtual machine), shut down the virtual machine and then run the following command:

    java -jar jrockitve-imagetool.jar --get-log vm_cfg [output-file]

In this command, `vm_cfg` is the path and name of the virtual machine configuration file (`vm.cfg`), and `output-file` is the path and name of the file to which you want the contents of the log file to be written. If you do not specify the output file, the output of the command is printed to `stdout`.

Alternatively, you can specify the location of the log file on a mounted NFS drive and read the log file at run time without using SSH. For more information about NFS mounts, see Section B.3.1.2, "mounts (mandatory)."