Rejection sampling. Integrative postprocessing. XML/Star support for ctf / tilt specification in match_template. GPL license. Streamlined analyzer and composable filters.

Author: maurerv

doc/conf.py

@@ -61,11 +61,9 @@
 }
 
 html_theme_options = {
-    # "logo": {
-    #     "text": "pytme",
-    #     "image_light": "_static/index_api.svg",
-    #     "image_dark": "_static/index_api.svg",
-    # },
+    "logo": {
+        "text": "pytme",
+    },
     "header_links_before_dropdown": 4,
     "icon_links": [
         {

doc/quickstart/index.rst

@@ -33,7 +33,6 @@ User Guide
    postprocessing/motivation
    postprocessing/example
    postprocessing/summary
-   integrations
 
 
 .. toctree::

doc/quickstart/installation.rst

@@ -5,7 +5,7 @@ Installation
 
 .. _installation-section:
 
-This section provides instructions on how to install |project|. We recommend creating an installation enviroment for a clean and isolated setup. Available options for different use cases are outlined in the tabs below.
+We recommend creating a virtual environment for a clean and isolated setup.
 
 .. tab-set::
 

doc/quickstart/license.rst

@@ -10,11 +10,11 @@ The following subsections provide templates for the execution of |project| on di
 
 .. note::
 
-    The ``estimate_ram_usage.py`` script computes an initial memory estimate on a given template matching case for clusters that require memory reservations.
+    The ``estimate_memory_usage.py`` script computes an initial memory estimate on a given template matching case for clusters that require memory reservations.
 
     .. code-block:: bash
 
-        estimate_ram_usage.py --help
+        estimate_memory_usage.py --help
 
 SLURM (EMBL)
 ------------

doc/quickstart/matching/cluster.rst

@@ -111,7 +111,10 @@ The final orientations can be passed to ``postprocess.py`` via the ``--orientati
 
     from tme import Orientations
     orientations = Orientations.from_file("orientations.tsv")
-    orientations.to_file("orientations.star", name = "/path/to/your/tomogram")
+    orientations.to_file(
+        "orientations.star",
+        source_path="/path/to/your/tomogram"
+    )
 
 
 References

doc/quickstart/postprocessing/example.rst

@@ -4,7 +4,7 @@
 Motivation
 ==========
 
-Postprocessing identifies regions of high similarity from the template matching output, which we refer to as peaks. Each peak corresponds to an occurence of the template in the target, which is fully characterized by a translation and rotation vector. Identifying peaks is challenging by itself, hence |project| implements a variety of analysis strategies for optimal performance.
+Postprocessing identifies regions of high similarity from the template matching output, which we refer to as peaks. Each peak corresponds to an occurence of the template in the target, which is fully characterized by a translation and rotation matrix. Identifying peaks is challenging by itself, hence |project| implements a variety of analysis strategies for optimal performance.
 
 
 Background
@@ -18,14 +18,14 @@ Lets recall an example from the preprocessing section. The highest peak is locat
     import numpy as np
     import matplotlib.pyplot as plt
     import matplotlib.patches as patches
-    from skimage.feature import match_template
 
     from tme import Density
+    from tme.cli import match_template
 
     target = Density.from_file("../../_static/examples/preprocessing_target.png").data
     template = Density.from_file("../../_static/examples/preprocessing_template.png").data
 
-    result = match_template(target, template, pad_input=True)
+    result = match_template(target, template)[0]
     ij = np.unravel_index(np.argmax(result), result.shape)
     x, y = ij[::-1]
 

doc/quickstart/postprocessing/motivation.rst

@@ -4,41 +4,31 @@
 Summary
 =======
 
-The ``postprocess.py`` command-line tool can be used to analyze the results generated by ``match_template.py``.
+The ``postprocess.py`` tool analyzes results generated by ``match_template.py`` to identify and characterize top-scoring peaks from template matching.
 
 .. code-block:: bash
 
     postprocess.py --help
 
-Top-scoring peaks can be identified using
+.. tip::
 
-.. code-block:: bash
-
-    postprocess.py \
-        --input_file output.pickle \
-        --output_prefix output \
-        --output_format orientations \
-        --min_distance 30 \
-        --min_boundary_distance 20 \
-        --num_peaks 1000
-
-Identifying no more than 1,000 top-scoring peaks that are separated by at least 20 voxel from each other and 30 voxel from the boundaries. Different peak calling strategies can be chosen from using the ``--peak_caller`` argument.
+    From version 0.3.0 onwards, postprocessing supports advanced multi-input and background correction. Multiple input files can be specified to distinguish between different macromolecular species, with corresponding class identifiers made available in orientations and RELION output formats. Additionally, multiple background corrections can be applied simultaneously via ``--background_file``, enabling users to account for various noise sources beyond single backgrounds (e.g., from ``--scramble_phases``) and incorporate complex cellular environments such as membrane backgrounds.
 
 Depending on the subequent use case, different ``output_format`` options are available and outlined below.
 
 .. tab-set::
 
     .. tab-item:: Orientations
 
-        A tab-separated file *output.tsv* will be created in the process containing eight columns. The z, y and x column correspond to the translation, the euler_z, euler_y and euler_x column to the rotation used to obtain the column score. The detail column contains peak caller specific information.
+        A tab-separated file *output.tsv* will be created in the process containing eight columns. The x, y and z column correspond to the translation, the euler_x, euler_y and euler_z column to the rotation used to obtain the column score. The detail column contains peak caller specific information.
 
         .. code-block:: bash
 
             postprocess.py \
                 --input_file output.pickle \
                 --output_prefix output \
                 --output_format orientations \
-                --min_distance 30 \
+                --mask_edges \
                 --min_boundary_distance 20 \
                 --num_peaks 1000
 
@@ -52,7 +42,8 @@ Depending on the subequent use case, different ``output_format`` options are ava
                 --input_file output.pickle \
                 --output_prefix output \
                 --output_format relion4 \
-                --min_distance 20 \
+                --mask_edges \
+                --min_boundary_distance 20 \
                 --num_peaks 1000
 
 
@@ -66,6 +57,8 @@ Depending on the subequent use case, different ``output_format`` options are ava
                 --input_file output.pickle \
                 --output_prefix output \
                 --output_format alignment \
+                --mask_edges \
+                --min_boundary_distance 20 \
                 --num_peaks 10
 
     .. tab-item:: Extraction
@@ -78,9 +71,9 @@ Depending on the subequent use case, different ``output_format`` options are ava
                 --input_file output.pickle \
                 --output_prefix output \
                 --output_format extraction \
-                --min_distance 20 \
-                --num_peaks 500 \
-                --peak_caller PeakCallerMaximumFilter
+                --mask_edges \
+                --min_boundary_distance 20 \
+                --num_peaks 100
 
     .. tab-item:: Average
 
@@ -92,13 +85,28 @@ Depending on the subequent use case, different ``output_format`` options are ava
                 --input_file output.pickle \
                 --output_prefix average \
                 --output_format average \
-                --min_distance 20 \
-                --num_peaks 500 \
-                --peak_caller PeakCallerMaximumFilter
+                --mask_edges \
+                --min_boundary_distance 20 \
+                --num_peaks 100
+
+
+    .. tab-item:: Pickle
+
+        The code below will apply background correction, and create a new pickle file containing the corrected scores. The output is intended for visual assessment of the normalization procedure, and can be reused for postprocessing.
+
+        .. code-block:: bash
+
+            postprocess.py \
+                --input_file output.pickle \
+                --output_prefix output_new \
+                --output_format pickle \
+                --background_file background1.pickle background2.pickle
 
 .. note::
 
-    Orientations are following the conventions outlined in [1]_. We use a right-handed coordinate system with orthogonal X, Y and Z axes. Euler angles are expressed using intrinsic ZYZ convention, with the first rotation around the Z-axis, the second around the new Y-axis and the third around the new Z-axis (see :py:meth:`euler_to_rotationmatrix <tme.rotations.euler_to_rotationmatrix>`). The default orientation the z-unit vector (0, 0, 1).
+    Orientations are following the conventions outlined in [1]_. We use a right-handed coordinate system with orthogonal X, Y and Z axes. Euler angles are expressed using intrinsic ZYZ convention, with the first rotation around the Z-axis, the second around the new Y-axis and the third around the new Z-axis (see :py:meth:`euler_to_rotationmatrix <tme.rotations.euler_to_rotationmatrix>`). The default orientation is the z-unit vector (0, 0, 1).
+
+
 
 
 References