ACOUSLIC-AI challenge data¶
An explanation of how to view, understand, and navigate the images and annotation masks is available on the Datasets: data visualization page.
Data splits¶
This challenge is structured into three distinct data splits with the following characteristics and use cases:
- Public Training and Development Set (300 cases): This set is accessible to all participants for the purpose of developing AI models during the Preliminary Development Phase. The data, released under a CC BY-NC-SA license, encompass cases from three Public Health Units (PHUs) in Sierra Leone. Imaging data, annotation masks and reference abdominal circumference measurements per sweep have been released via this link.
- Hidden Validation and Tuning Set (10 cases): Designed to support a live, public leaderboard, this set aids in model selection and tuning during the Preliminary Development Phase. It consists of previously unseen data from two PHUs in Tanzania.
- Hidden Test Set (290 cases): Utilized for benchmarking AI models at the conclusion of the Final Test Phase, this set contains unseen testing data from two PHUs in Tanzania (190 cases) and from Radboudumc, the Netherlands (100 cases).
Imaging data¶
Data acquisition¶
Imaging data included in the ACOUSLIC-AI challenge was acquired by novice users (1 hour training) with a low-cost portable probe (MicrUs Pro-C60S, Telemed, Lithuania) connected to a smartphone. The users were blinded to the ultrasound images during acquisition and performed free-hand sweeps according to instructions provided. The instructions were presented in the smartphone screen and guided them through the obstetric sweep protocol (OSP, figure 1) proposed by Stigter et al., 2011. The OSP is a blind-sweep acquisition protocol consisting of six sweeps over the gravid abdomen: three transverse sweeps (1-3) in caudocranial direction, and three sagittal sweeps (4-6) from participants' left to right.
Figure 1: The Obstetric Sweep Protocol (DeStigter et al, 2011). In orange, transverse sweeps (1 - 3); in violet, sagittal sweeps (4 - 6).
Data characteristics¶
Training, validation and test cases consist of a pair of 2D B-mode ultrasound sweep data and abdominal circumference annotations. Both the sweeps and annotations correspond to a series of 840 frames of shape 744x562 pixels and a fixed spacing of 0.28 mm/pixel. The annotations correspond to pixel masks of the abdomen on individual frames, and pertain to either of two categories: optimal and suboptimal planes for abdominal circumference measurement. Per frame, the annotation pixels adopt one of three values: pixel value 0 indicates no annotation (background), pixel value 1 indicates a mask drawn on an optimal plane, and pixel value 2 indicates a mask drawn on a suboptimal plane. Cases are also accompanied by the corresponding abdominal circumference reference value (in mm) for each sweep where annotations are available.
Annotations: fetal abdomen masks¶
Annotations were provided through manual drawing of ellipses on the appropriate frames. Each ellipse drawn is identified as optimal (an ideal frame to measure abdominal circumference) or sub-optimal (a frame that could be used to measure abdominal circumference, although it is not quite perfect for the task).
Annotation Details¶
Ellipse annotations in all three sets were obtained from manual annotations performed by human readers on every initial and final frame where the corresponding structure (transverse plane of the abdomen) and type (optimal/suboptimal) were observed. Annotations on intermediate frames were automatically generated using linear interpolation. In cases where the size or position of the structure changed greatly, additional manual annotations were provided to ensure accuracy in the interpolation process. All ellipse annotations were filled in to provide participants with pixel mask annotations rather than ellipse contours. Per frame, the annotation pixels adopt one of three values: pixel value 0 indicates no annotation (background), pixel value 1 indicates a mask drawn on an optimal plane, and pixel value 2 indicates a mask drawn on a suboptimal plane. Pixel annotations extending beyond the field of view of the ultrasound beam were set to zero.
Training set¶
For all 300 cases in this set, annotations were performed by two readers with 20 hours of training in acquiring and analyzing blind-sweep ultrasound data. Their experience extends over two years, with one reader dedicating a total of 120 hours and the other 300 hours to analyzing such data. Each reader annotated cases independently, with an approximate distribution of 50% cases each.
Hidden validation & tuning and test set¶
For cases in the Hidden Validation and Tuning Set (10 cases) and in the Hidden Test Set (282 cases), the annotation process was performed by two readers with higher expertise. A radiologist in training with substantial experience in prenatal ultrasound annotated all cases. Subsequently, these annotations were reviewed and corrected by a sonographer with 37 years of experience.
Reference abdominal circumference measurements¶
For all cases in all three data splits, fetal abdominal circumference measurements were derived from the best available — optimal, otherwise suboptimal — annotation masks on each of the six sweeps, wherever present. These reference measurements serve as a standard for evaluating the accuracy of the algorithms' estimations.