Preprocessing: Tile Stitching

In the rapidly evolving field of digital pathology, handling and processing high-resolution histopathology images is a critical task. This is especially true in the context of whole-slide imaging (WSI), a technique that has revolutionized the analysis of tissue samples by digitizing entire microscope slides at a gigapixel scale. However, the large size of these images presents a significant challenge in terms of data management and analysis. To address this, the images are often segmented into smaller, manageable, and overlapping segments known as tiles. The real challenge, and the focus of this tutorial, is in the accurate reconstruction of these tiles to reform the complete image—a process known as tile stitching.

PathML offers the TileStitcher class, which is the Python adaptation of an existing Groovy script used in QuPath which is available here. The TileStitcher class reimplements the functionality of its Groovy counterpart, allowing for the extraction of tile coordinates from the baseline TIFF tags followed by seamlessly stitching them and writing the stitched image as a pyramidal OME-TIFF file.

This tutorial will walk you through the process of using TileStitcher class to collect, parse, and stitch large sets of tiled TIFF images then saving the reconstructed image.

Prerequisites

Before using the TileStitcher class, we need to install the necessary software and configure the environment.

Software Installation

The TileStitcher class requires QuPath and OpenJDK. Here is how to install them:

1. Download and install QuPath from its GitHub release page. Here we are using version 0.3.1.

wget https://github.com/qupath/qupath/releases/download/v0.4.3/QuPath-0.4.3-Linux.tar.xz

Unzip

tar -xvf QuPath-0.4.3-Linux.tar.xz

Make executable

chmod u+x /path/to/QuPath/bin/QuPath

2. Download and Install OpenJDK 17

```bash wget https://download.oracle.com/java/17/latest/jdk-17_linux-x64_bin.deb sudo apt install ./jdk-17_linux-x64_bin.deb

   Set the Java path according to your installation method. If you have set up your environment using PathML, set the Java path to /opt/conda/envs/pathml. Otherwise, adjust it to the appropriate path on your system.

Environment Configuration

To use TileStitcher, we need to set the correct paths to the QuPath library and OpenJDK. For this, we need to add the paths to the environment variables JAVA_HOME, CLASSPATH, and LD_LIBRARY_PATH.

The JAVA_HOME environment variable should be set to the path where the JDK is installed. The CLASSPATH environment variable should include paths to all the QuPath library jar files. In the initialization of TileStitcher, these environment variables are used to start the Java Virtual Machine (JVM) and import the necessary QuPath classes.

Best Practices and Considerations for Using the TileStitcher Module

1. JVM Session Management

The TileStitcher module utilizes jpype to manage the JVM sessions, a departure from the python-javabridge used in other parts of the package. This difference can cause conflicts when trying to run modules concurrently within the same Python environment. Hence, it is advisable to avoid running TileStitcher operations in the same notebook where python-javabridge dependent modules are running.

2. Restarting Kernel to Re-initialize JVM

The jpype does not allow the JVM to be restarted within the same Python session once it has been terminated. As a result, to run the TileStitcher module again or to switch back to modules that use python-javabridge, a kernel restart might be necessary.

3. Segregating Workflows

To mitigate potential conflicts, consider segregating workflows based on the JVM management package they depend on. It is recommended to use separate notebooks or scripts for operations involving TileStitcher and for those involving modules that are dependent on python-javabridge.

Using TileStitcher

Ensure QuPath and JDK installations are complete before proceeding.

Initialization

The class is initialized with several parameters:

• qupath_jarpath: List of paths to QuPath JAR files.

• java_path: Custom path to Java installation. If set, JAVA_HOME will be overridden.

• memory: Allocated memory for the JVM (default: “40g”).

• bfconvert_dir: Directory for Bio-Formats conversion tools.

During initialization, TileStitcher sets up the Java Virtual Machine (JVM) and imports necessary QuPath classes. It also includes error handling for Java path configurations and JVM startup issues.

JVM Startup

The _start_jvm method initiates the JVM with specified memory and classpath settings. It imports necessary QuPath classes upon successful startup, ensuring compatibility with Java 17.

import glob
import os
from pathml.preprocessing.tilestitcher import TileStitcher
from pathml.utils import setup_qupath


# Set the path to the JDK
os.environ["JAVA_HOME"] = "/usr/lib/jvm/jdk-17/"

# Use setup_qupath to get the QuPath installation path
qupath_home = setup_qupath("../../tools1/tools1/")

if qupath_home is not None:
    os.environ["QUPATH_HOME"] = qupath_home

    # Construct the path to QuPath jars based on qupath_home
    qupath_jars_dir = os.path.join(qupath_home, "lib", "app")
    qupath_jars = glob.glob(os.path.join(qupath_jars_dir, "*.jar"))
    qupath_jars.append(os.path.join(qupath_jars_dir, "libopenslide-jni.so"))

    # Create an instance of TileStitcher
    stitcher = TileStitcher(qupath_jars)
else:
    print("QuPath installation not found. Please check the installation path.")

./tools/bftools/bfconvert ./tools/bftools/bf.sh
bfconvert version: Version: 7.0.1
Build date: 16 October 2023
VCS revision: 20e58cef1802770cc56ecaf1ef6f323680e4cf65
Setting Environment Paths
Java Home is already set
JVM was already started

import jpype

jpype.isJVMStarted()

False

Image Stitching with TileStitcher

Once TileStitcher is initialized, it’s capable of stitching together tiled images.

• Method: run_image_stitching

• Inputs:

  • A list of TIFF files or a directory containing TIFF files.

  • Output file path.

• Optional Parameters:

  • downsamples: Specify the number of downsample levels (e.g., [1,4,8]). Defaults to levels read from the tiles.

  • separate_series: If set to True, it downloads bftools and extracts the base level image from the stitched image.

input_files = glob.glob("path/to/tiles/*.tif")
output_file = "path/to/output.ome.tif"
stitcher.run_image_stitching(input_files, output_file)

Demo

import jpype

jpype.isJVMStarted(), jpype.getJVMVersion()

(False, (0, 0, 0))

import glob
import os

# Set the path to the JDK
os.environ["JAVA_HOME"] = "/opt/conda/envs/pathml"
os.environ["PATH"] += os.pathsep + os.path.join("/opt/conda/envs/pathml", "bin")

from pathml.preprocessing.tilestitcher import TileStitcher
from pathml.utils import setup_qupath


# Use setup_qupath to get the QuPath installation path
qupath_home = setup_qupath("./tools/")

if qupath_home is not None:
    os.environ["QUPATH_HOME"] = qupath_home

    # Construct the path to QuPath jars based on qupath_home
    qupath_jars_dir = os.path.join(qupath_home, "lib", "app")
    qupath_jars = glob.glob(os.path.join(qupath_jars_dir, "*.jar"))
    qupath_jars.append(os.path.join(qupath_jars_dir, "libopenslide-jni.so"))

    # Create an instance of TileStitcher
    stitcher = TileStitcher(qupath_jars)
else:
    print("QuPath installation not found. Please check the installation path.")

Using JVM version: (17, 0, 10) from /opt/conda/envs/pathml/lib/jvm/lib/server/libjvm.so
Importing required QuPath classes

jpype.isJVMStarted(), jpype.getJVMVersion()

(True, (17, 0, 10))

# Specify the folder path where the list of .tif files are present, here we are using a folder path that has single tif file for demo purposes.
infile_path = "../tests/testdata/tilestitching_testdata/"
outfile_path = "./output/tile_stitching_demo.ome.tif"

import time

start = time.time()
# Run the image stitching process
stitcher.run_image_stitching(
    infile_path, outfile_path, downsamples=[1], separate_series=True
)
end = time.time()

19:07:21.270 [main] [INFO ] q.l.i.s.b.BioFormatsServerOptions - Setting max Bio-Formats readers to 8
19:07:21.900 [main] [ERROR] q.l.i.s.o.OpenslideServerBuilder - Could not load OpenSlide native libraries
java.lang.UnsatisfiedLinkError: no openslide-jni in java.library.path: /opt/conda/envs/pathml/lib/python3.9/site-packages/cv2/../../lib64:/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
        at java.base/java.lang.System.loadLibrary(System.java:1993)
        at org.openslide.OpenSlideJNI.<clinit>(OpenSlideJNI.java:55)
        at org.openslide.OpenSlide.<clinit>(OpenSlide.java:53)
        at qupath.lib.images.servers.openslide.OpenslideServerBuilder.<clinit>(OpenslideServerBuilder.java:90)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789)
        at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729)
        at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
        at qupath.lib.images.servers.ImageServerProvider.getServerBuilders(ImageServerProvider.java:191)
        at qupath.lib.images.servers.ImageServerProvider.getPreferredUriImageSupport(ImageServerProvider.java:223)
19:07:21.901 [main] [INFO ] q.l.i.s.o.OpenslideServerBuilder - If you want to use OpenSlide, you'll need to get the native libraries (either building from source or with a packager manager)
and add them to your system PATH, including openslide-jni.
19:07:24.717 [main] [WARN ] q.l.i.writers.ome.OMEPyramidWriter - Deleting existing file /home/jupyter/sreekar/projects/tilestitching/pathml/examples/./output/tile_stitching_demo.ome.tif
19:07:24.733 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Writing Sparse image (1 regions) to /home/jupyter/sreekar/projects/tilestitching/pathml/examples/./output/tile_stitching_demo.ome.tif (series 1/1)
19:07:24.734 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Setting series 0 compression to zlib
19:07:24.734 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Writing resolution 1 of 1 (downsample=1.0, 12 tiles)
19:07:24.736 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Writing plane 1/1
19:07:35.528 [main] [INFO ] q.l.i.writers.ome.OMEPyramidWriter - Plane written in 10792 ms
Image stitching completed. Output file: ./output/tile_stitching_demo.ome.tif
bfconvert version: Version: 7.1.0
Build date: 11 December 2023
VCS revision: 05c7b2413cfad19a73b619c61ddf77ca2d038ce7
./output/tile_stitching_demo.ome.tif
OMETiffReader initializing ./output/tile_stitching_demo.ome.tif
[OME-TIFF] -> ./output/tile_stitching_demo_separated.ome.tif [OME-TIFF]
Reading IFDs
Populating metadata
Reading IFDs
Populating metadata
        Converted 1/7 planes (14%)
        Converted 7/7 planes (100%)
Overwriting existing Creator attribute: OME Bio-Formats 6.12.0
[done]
2.023s elapsed (162.28572+47.857143ms per plane, 507ms overhead)
bfconvert completed. Output file: ./output/tile_stitching_demo_separated.ome.tif
Original stitched image deleted: ./output/tile_stitching_demo.ome.tif