ROV-Based Multi-Sensor Dataset: Synchronized Camera and Sonar images taken in the Tropical Waters of the Red Sea, Eilat

Description:This dataset consists of approximately 46,928 synchronized image pairs collected by the  Blue-ROV2. The images were captured using a machine-vision camera (IDS UI-3260CP-C-HQ)  and a BluePrint Oculus M1200d Forward-Looking Sonar (FLS). Both sensors were installed with the FLS tilted 15 degrees downward to achieve optimal coverage of the terrain and optimal FOV overlap.The data was collected to train and evaluate a comprehensive perception and obstacle avoidance framework. Context:This dataset is the second installment in our collection of synchronized multi-sensor underwater  datasets, aimed at enabling advanced research in multi-modal sensor fusion, obstacle  detection, and navigation for autonomous underwater vehicles (AUVs). The data was collected  using the Blue-ROV2 Remotely Operated Vehicle (ROV) in the tropical waters of the Red Sea,  off the coast of Eilat, Israel. This data captures diverse underwater environments and is part of a  research project focused on developing fusion models for improved obstacle detection and  navigation in AUVs. Content:The data encompasses several sites within the tropical waters of the Red Sea, Eilat, including  corals, rocks, shipwrecks, man-made structures, piers, and caves. The ROV platform was  operated by divers, ensuring accurate positioning and coverage. Data was acquired at depths  ranging from 3 to 12 meters at different times from dawn to dusk. Dataset Composition:SiteRecording SessionImage PairsDescriptionTropical Site 120221211_09250620221211_13325210,9157,978Pier, rocks, coralsTropical Site 220221212_09582120221212_1413089,9008,475Man-made structure, rocks, coralsTropical Site 320221213_1025429,390Rocks, coralsTotal 46,928   The dataset is organized into separate sessions, each representing a specific dive or  experiment. Within each session, data is further categorized into modalities: camera (FLC  images), sonar (FLS images), and depth. Each modality directory contains the corresponding  data files in PNG format for images and CSV format for depth data. Each modality directory includes:A camera.csv file for the camera modality that maps each image file to its respective  timestamp.A sonar.csv file for the sonar modality that maps each image file to its respective timestamp.The depth data in depth.csv formatted with timestamp and value.Additionally, a samples.json file documents the relationship between uni-modal and  multi-modal samples, enabling easy association of data from different modalities. Technical Details:Camera: IDS UI-3260CP-C-HQImage dimensions: 1936x1216 pixels (downscaled to 968 × 608 for this dataset)Sensor type: Sony IMX249 1/1.2" CMOSLens: Tamron M112FM06Captured bit depth: 8-bitFrame rate: 5 HzSonar: BluePrint Oculus M1200dOperating frequency: 1.2 MHz (low frequency mode)Maximum range: 40 m (set to 15 m for this dataset)Horizontal aperture: 130°Vertical aperture: 20°Number of beams: 512Angular resolution: 0.6°Beam separation: 0.25°Image resolution: 544x300 pixelsCoordinate system: PolarFrame rate: 5 HzDepth: Blue-Robotics Ping2 Sonar Altimeter and EchosounderFrequency: 115 kHzSource Level: 198 dB re 1µPa @ 1mBeamwidth: 25 degreesTypical Minimum Range: 0.3 m (1 ft)Typical Usable Range: 100 m (328 ft)Range Resolution: 0.5% of rangeDepth Rating: 300 m (984 ft)Data format: CSVColumns:timestamp: Unix timestamp (seconds)value: Depth value (meters)Sample rate: 5 Hz Example File Tree Layout:${session}/${dataset}/camera/camera.csv00000001.png00000002.png…sonar/sonar.csv00000001.png00000002.png…depth/depth.csvsamples.json Example File Content: camera.csvtimestamp,filename1644234340.181234,00000001.png1644234343.375667,00000002.png sonar.csvtimestamp,filename1644234340.181234,00000001.png1644234343.375667,00000002.png depth.csvtimestamp,value1644234340.181234,5.41644234343.375667,6.1 samples.json{    "samples": [        {            "camera": [                0            ],            "depth": [                0            ],            "sonar": [                0            ]        },        {            "camera": [                1            ],            "depth": [                1            ],            "sonar": [                1            ]        }]By providing synchronized and aligned camera, sonar imagery, and depth data, this dataset  enables researchers to explore novel algorithms and techniques for multi-modal sensor fusion in  the context of autonomous underwater vehicles operating in the tropical waters of the Red Sea.AcknowledgementsThe data in this repository is part of the DeeperSense project that received funding from the European Commission, Program H2020-ICT-2020-2 ICT-47-2020, Project Number: 101016958.