⚠️ Warning: Code should be refactored in a next release

This repository contains the implementation of mosaicking and occlusion recovery algorithms for fetoscopy, developed as part of a thesis project. The project focuses on panoramic image reconstruction and SLAM techniques for medical imaging applications.

• reconstruction_algorithm_RESNET.py - Main reconstruction algorithm using ResNet features
• reconstruction_algorithm_RESNET_VLAD.py - Reconstruction with ResNet and VLAD descriptors
• reconstruction_algorithm_VGG.py - Reconstruction algorithm using VGG features
• SLAM_LoFTR_new_dataset.py - SLAM implementation with LoFTR on new dataset
• SLAM_LoFTR_simulation.py - SLAM simulation using LoFTR
• SLAM_ORB.py - SLAM implementation with ORB features
• SLAM_SIFT.py - SLAM implementation with SIFT features
• project_robotics.py - Mosaicking robotic project script
• resnetfinal.pth - Pre-trained ResNet50 weights from FetReg Challenge 2021

Scripts for generating statistical boxplots from experimental data.

Implementation of statistical tests for validating experimental results.

• blitting_tries/ - Attempts to implement blitting optimization for 3D scatterplot k-means clustering
• code_imported/ - Scripts imported from external projects
• template_matching_tries/ - Template matching algorithm experiments
• tries-LoFTR_images/ - LoFTR demonstration images

• exposure_fusion/ - Mertens exposure fusion implementation (MATLAB)
• LoFTR/ - LoFTR repository
• SuperPointPretrainedNetwork/ - SuperPoint implementation
• VGG/ - VGG network implementations (PyTorch and Keras)
• VLAD_master/ - VLAD descriptor repository

• dataset/ - Frame sequences organized in subfolders
• dataset_videos/ - Reconstructed video sequences

Primary dataset used for thesis experiments, with frames organized in subfolders.

• dictionary_file_npy/ - VGG feature descriptors (NumPy format)
• experiment_files/ - Experimental results (Excel format)
• output_panorama_images/ - Generated panoramic images by experiment
• output_panorama_video/ - Panoramic reconstruction videos
• similarity_matrices/ - Complete similarity matrices
• similarity_matrices_rt_gb_9/ - Real-time similarity matrices with Gaussian blur (kernel size 9)
• visual_dictionary/ - VLAD method dictionaries (Pickle format)

• Boxplot_images/ - Statistical visualization outputs
• boxplots_npy/ - Data files for boxplot generation
• output_panorama_images/ - Final panoramic reconstructions
• output_panorama_video/ - Final panoramic videos
• sanity_check_panorama/ - Relocalization assessment images

• code_Bano/ - Implementation of Bano et al. (2020) methods
• examples_generation_code/ - Scripts for generating presentation materials
• mask_cut/ - MICCAI 2021 dataset masks

Scripts for generating and processing similarity matrices.

Previous versions of reconstruction algorithms.

Earlier SLAM implementations.

Previous VGG-based feature extraction code.

• Python 3.x
• PyTorch
• OpenCV
• NumPy
• Additional dependencies as specified in individual script files

Each main algorithm file can be run independently. Refer to individual scripts for specific usage instructions and parameter configurations.

This work focuses on:

• Mosaicking: Creating panoramic views from fetoscopic video sequences
• Occlusion Recovery: Handling visual obstructions in medical imaging
• SLAM: Simultaneous Localization and Mapping for surgical navigation
• Feature Matching: Using various descriptors (LoFTR, SIFT, ORB, VGG, ResNet)

• MICCAI 2020 Dataset
• MICCAI 2021 Dataset
• FetReg Challenge 2021
• Bano et al. (2020)

This repository represents research code that requires refactoring for production use. The codebase includes experimental implementations and legacy code that should be cleaned up in future releases. This work was comtribute by @ChiaraLena