Importing Files into OME

Both the web-ui and ome can be used to import files. However, the web-ui only informs you whether the imports succeeded or not. For debugging or developing purposes, I suggest you use ome to take advantage of its more verbose error reporting.

There are, broadly, two classes of files users might want to import to OME (1) Images in proprietary formats or (2) OME-XML files. OME-XML files can not only define Images, Analysis Modules and Chains, but also most other OME objects such as datasets, projects, and experimenters.

The import engine (the perl source code can be found under src/perl2/OME/ImportEngine) is responsible for taking image files in external formats and importing them into OME. It uses a delegation pattern, in which each external format is recognized and imported by its own subclass of OME::ImportEngine::AbstractFormat. The Import Engine passes a group of files (import candidates) to each of the format sub-classes. Each format sub-class removes files from the group that it recognizes. Any left over files are, by definition, of an unsupported format. Thereby it is easy to extend the Import Engine to other proprietary image format types.

Importing Images

OME allows users to import multi-dimensional microscope images into the OME data hierarchy (images --> datasets --> projects). These images are available for further examination and processing. We currently support Applied Precision SoftWorx (Deltavision), Metamorph STK and HTD, Molecular Dynamics GEL, TIFF, and OME-XML image formats. As a result of our partnership with the developers of VisBio, their viewer can be used to import all of the many common image formats it understands directly into OME.

A lot of information about the biology work-flow is known at acquisition time, e.g. a detailed description of the microscope, what filters were used for each channel, etc. This information is stored with the image's pixels as meta-data, and is accessible through OME's various interfaces.

All images in OME are associated with an experimenter and a dataset. The image's experimenter attribute is set implicitly based on which user imported the image. After import the image's owner can be changed using ome data chown. You can specify the dataset explicitly and the images will be imported into the dataset with that name, or if the specified dataset doesn't yet exist a new dataset with that name will be created for you. If a dataset isn't specified, ome import will create one for you with an auto-generated name based on the import time.

The following command imports all files in the working directory that begin with CT and associates them with the dataset Test Images.

ome import -d "Test Images" CT*

Importing files
         1/8: [IN PROGRESS] Starting import
         5/8: [IN PROGRESS] Uploaded CT-MONO2-16-ort
         7/8: [IN PROGRESS] Importing
         8/8: [IN PROGRESS] Executing import chain
         8/8: [FINISHED] Imported 4 images from 4 files. 5 scanned. 0 unknown format,
                         1 duplicates, 0 errors.
Successfully Imported:
1197: CT-MONO2-8-abdo
1198: CT-MONO2-16-ort
1199: CT-MONO2-16-brain
1200: CT-MONO2-16-ankle-MATLAB

You can use the -r flag to re-import images into OME. This is normally only useful for debugging or developing image format readers.

VisBio supports several more file formats and can upload data directly from VisVio to an OME database. This is another viable method for importing files to OME.

Importing Analysis Modules and Chains

Before an analysis chain can be executed, it must be imported into OME. Due to dependency checking, an analysis chain can only be imported after all the modules it utilizes have been successfully imported.

You don't need to specify a dataset if importing OME Analysis Modules or Chains.

ome import MatlabModules.ome 

Since XML files have LSIDs which are globally unique, and its a serious data corruption issue to have duplicate globally unique LSIDs, you cannot re-import OME-XML files.

Suppose you wish to change modules or chains once they have been successfully imported to OME. You cannot simply import the new OME-XML files to overwrite the old ones. One possible work-around is to change the module and chain's names (and their LSIDs).

I found that its easier to simply capture a snapshot of OME's contents before your modules and chains have been imported. When you will want to re-import analysis modules, you must restore your database from archive to a point before the modules were imported. To do this you will probably want to use ome data.