maspipeline

Preprocessing pipeline for genomic prediction using microhaplotype markers

License: GPL v3 Rust

GitHub: https://github.com/bowo1698/maspipeline

General information

We adapted and optimised the GVCHAP pipeline from Prakapenka et al. (2020) to handle large genomic data and utilise microhaplotype markers in genomic prediction. The pipeline consists of four key stages: (1) phasing, (2) converting phased data to haplotypes, (3) defining microhaplotype blocks, and finally, (4) encoding them into microhaplotype genotypes. The pipeline was optimised with Rust, so it has more efficient memory and enhanced scalability due to parallelisation. For phasing, we only used Beagle as the pipeline has not been developed for other phasing methods yet (e.g., FindHap, FImpute, etc.). In addition, we encourage the use of parallelisation techniques when phasing with Beagle.

While haplotypes refer to long-range combinations of alleles along a chromosome, microhaplotypes can be defined as several SNPs covering a short DNA segment, typically 125 – 150 bp. So in this pipeline, conducted by haplotype-hybrid, we provide fixed window (SNP counts) and LD-based methods to define fully haplotype blocks and microhaplotype segments, respectively. When the --method snp-count-simple argument is used, it simply defines haplotype blocks depending on how many SNPs are set in each block. In contrast, when the --method ld-haploblock along with --haplotype-type micro arguments are used, it will discover microhaplotypes based on LD and window constraints. However, both methods will result in similar data outputs.

Pipeline at a glance

The pipeline runs in four stages, executed in order. Each stage has its own dedicated page with the detailed command line, inputs, outputs, and parameter reference:

# Stage What it does Tool / dep Page
1 Data preparation Clean and filter raw VCF — chromosome naming, biallelic filtering, sample-id checks plink, convert-to-vcf (CSV path) Data preparation
2 Phasing Resolve diploid genotypes into haploid maternal/paternal strands Beagle Phasing
3 Genotype-to-haplotype Split each phased genotype into two haploid columns and emit per-chromosome haploid files convert-from-vcf Genotype to haplotype
4 Microhaplotype discovery Discover microhaplotype blocks (LD-based / fixed-window) and emit the final genotype matrix haplotype-hybrid Microhaplotype discovery

The microhaplotype genotype matrix produced by Stage 4 is consumed by masreml::build_G_mh() and masbayes::construct_wah_matrix(). See Input data → Microhaplotypes for the on-disk schema expected by the R packages.

Installation

We provide three tools for the microhaplotype discovery and genotyping preprocessing pipeline:

  1. convert-to-vcf: Converts genotype data from CSV format to standard VCF (Variant Call Format) file
    • Input: CSV file with SNP genotypes (0/1/2 coding)
    • Output: VCF file compatible with phasing software (e.g., Beagle)
  2. convert-from-vcf: Extracts phased haplotype data from VCF files
    • Input: Phased VCF file (after running phasing software)
    • Output: Separated haplotype files per chromosome for downstream analysis
    • Generates SNP map file for position information
  3. haplotype-hybrid: Discovers microhaplotype segments using LD-based haploblock identification and performs genotyping
    • Input: Phased haplotype files and SNP map
    • Methods: LD-based haploblock discovery (Jonas et al. 2017) or fixed-window segmentation
    • Output: Microhaplotype block definitions and numerical genotype matrix
    • Implements Criterion-B scoring for optimal haplotype selection

We provide two installation options:

  1. Build from source (Recommended): Compile using Rust’s Cargo package manager
  2. Pre-compiled binaries (Quick start): Download platform-specific executables from the Releases page

We strongly encourage building from source using Cargo, as different operating systems and hardware architectures may require specific optimisations.

1. Build from source

Prerequisites for building from source

  • Rust toolchain (version 1.70 or later recommended): Install from rustup.rs

Build instructions

# 1. Clone the repository
git clone https://github.com/bowo1698/maspipeline.git
cd MicrohapsSel

# 2. Navigate to preprocessing directory
cd preprocessing

# 3. Build all tools in release mode (optimized)
cargo build --release

# 4. Compiled binaries will be available in:
# ./target/release/convert-to-vcf
# ./target/release/convert-from-vcf
# ./target/release/haplotype-hybrid

# 5. (Optional) Add to system PATH for global access
# For Linux/macOS:
export PATH="$PWD/target/release:$PATH"
# Add to ~/.bashrc or ~/.zshrc to make permanent

# For Windows (PowerShell):
$env:Path += ";$PWD\target\release"
# Add to system environment variables to make permanent

# 6. Verify installation
./convert-to-vcf --help
./convert-from-vcf --help
./haplotype-hybrid --help

2. Pre-compiled binaries

Download platform-specific binaries from the links below:

Linux:

# Download and extract
wget https://github.com/bowo1698/maspipeline/releases/download/v1/microhaplotype-tools-x64-linux.tar.gz
tar -xzf microhaplotype-tools-x64-linux.tar.gz
cd microhaplotype-tools-x64-linux

# Make executable
chmod +x convert-to-vcf convert-from-vcf haplotype-hybrid

# Test installation
./convert-to-vcf --help
./convert-from-vcf --help
./haplotype-hybrid --help

# (Optional) Move to system PATH for global access
sudo mv convert-to-vcf convert-from-vcf haplotype-hybrid /usr/local/bin/

macOS (Intel x86_64):

# Download and extract
curl -L -O https://github.com/bowo1698/maspipeline/releases/download/v1/microhaplotype-tools-x64-macos.tar.gz
tar -xzf microhaplotype-tools-x64-macos.tar.gz
cd microhaplotype-tools-x64-macos

# Remove macOS quarantine attribute and make executable
chmod +x convert-to-vcf convert-from-vcf haplotype-hybrid
xattr -d com.apple.quarantine convert-to-vcf convert-from-vcf haplotype-hybrid

# Test installation
./convert-to-vcf --help
./convert-from-vcf --help
./haplotype-hybrid --help

# (Optional) Move to system PATH for global access
sudo mv convert-to-vcf convert-from-vcf haplotype-hybrid /usr/local/bin/

macOS (Apple silicon ARM64):

# Download and extract
curl -L -O https://github.com/bowo1698/maspipeline/releases/download/v1/microhaplotype-tools-arm64-macos.tar.gz
tar -xzf microhaplotype-tools-arm64-macos.tar.gz
cd microhaplotype-tools-arm64-macos

# Remove macOS quarantine attribute and make executable
chmod +x convert-to-vcf convert-from-vcf haplotype-hybrid
xattr -d com.apple.quarantine convert-to-vcf convert-from-vcf haplotype-hybrid

# Test installation
./convert-to-vcf --help
./convert-from-vcf --help
./haplotype-hybrid --help

# (Optional) Move to system PATH for global access
sudo mv convert-to-vcf convert-from-vcf haplotype-hybrid /usr/local/bin/

Windows:

# Download (using PowerShell)
Invoke-WebRequest -Uri "https://github.com/bowo1698/maspipeline/releases/download/v1/microhaplotype-tools-x64-windows.zip" -OutFile "microhaplotype-tools-x64-windows.zip"

# Extract
Expand-Archive -Path microhaplotype-tools-x64-windows.zip -DestinationPath microhaplotype-tools-x64-windows
cd microhaplotype-tools-x64-windows

# Test installation (no chmod needed on Windows)
.\convert-to-vcf.exe --help
.\convert-from-vcf.exe --help
.\haplotype-hybrid.exe --help

# (Optional) Add to system PATH:
# 1. Copy full path of current directory: Get-Location
# 2. Search "Environment Variables" in Windows Start menu
# 3. Edit "Path" variable → Add new entry with the copied path

Note: On macOS, if you still get security warnings after using xattr, go to System Preferences → Security & Privacy → General, and click “Allow Anyway” for each blocked binary.

Want to help us?

Contributions are welcome and very beneficial! You can email me to improve the Rust implementation, add a new model, documentation, benchmarks, or bug reporting. I will appreciate, really.

License

GPL-3 License — see LICENSE file.

Copyright (c) 2025 Agus Wibowo

Contact

References

  • Da, Y. Multi-allelic haplotype model based on genetic partition for genomic prediction and variance component estimation using SNP markers. BMC Genet. 16, 144 (2015)

  • Jonas, D. et al. Alternative haplotype construction methods for genomic evaluation. Journal of Dairy Science 99, 6 (2016)

  • Prakapenka, D. et al. GVCHAP: A computing pipeline for genomic prediction and variance component estimation using haplotypes and SNP markers. Front. Genet. 11, 282 (2020)

Development Team

Lead Developer: Agus Wibowo James Cook University

Supervisors:

  • Prof. Kyall Zenger
  • Dr. Cecile Massault