Submission of inference containers to the preliminary development phase¶

Grand challenge algorithm template¶

Submissions to the Preliminary Development Phase and Final Test Phase of the ACOUSLIC-AI challenge should be in the form of inference containers submitted as Grand Challenge (GC) algorithms. GC algorithms are Docker containers that encapsulate the trained AI models—including both the architecture and model weights—and all necessary components. These components enable the container to load cases (stacked fetal ultrasound images), generate predictions (fetal frame number and fetal abdomen segmentation masks), and store the outputs (*.mha segmentation files and *.json frame number files) for subsequent evaluation.  Please note that you should use your own computing resources, or your choice of public platform, for model training.

Example code for such submissions can be found on ACOUSLIC-AI-baseline-algorithm GitHub repository, which currently supports an example implementation of a supervised nnUNet-based solution. There, you will find detailed step-by-step instructions to integrate and package your own algorithm running into a Docker container. Before implementing your own algorithm using this template, we recommend that you first upload a GC algorithm based on this unaltered template. This initial step is important because it may take multiple submissions to resolve any issues with it. In the following sections, we will provide a walkthrough of how to do this.

Uploading algorithm container to Grand Challenge¶

Please refer to these instructions to learn how to set up docker on your local system and package an algorithm container. Once you have successfully followed those steps, you can proceed to upload your algorithm as a Grand Challenge algorithm as follows:

Click on "+ Add a new algorithm"  or navigate to Submit > Preliminary Development Phase and complete the form on the page you are referred to:¶

Fill the Inputs and Outputs fields with the appropriate interfaces, as shown in the figure:

This step is very important, as these are the expected interfaces for any ACOUSLIC-AI challenge algorithm submission.  To learn more about GC input/output interfaces, visit: 

https://grand-challenge.org/components/interfaces/algorithms/

Once complete, click the "Save" button. At this point, your GC algorithm has been created and you are on its homepage. Now, click the "Containers" tab on the left panel, and then the "Upload a Container" button. Select your container image (.tar.gz file) for upload. After the upload is complete, click "Save". It typically takes 20-60 minutes till your container has been activated (depending on the size of your container). After its status is "Active", test out your container with a sample, unprocessed training case from the Training and Development Dataset.¶

Submission to Preliminary Development Phase - Validation and Tuning Leaderboard¶

Once you have your trained AI model uploaded as a fully-functional GC algorithm, you're now ready to make submissions to the Preliminary Development Phase of the challenge! Navigate to the "Preliminary Development Phase Submissions" page, select your algorithm, and click "Save" to submit. If there are no errors and evaluation has completed successfully, your score will be up on the leaderboard (typically in less than 24 hours).¶

⚠️ Please double-check all rules to make sure that your submission is compliant. Invalid submissions will be removed and teams repeatedly violating any/multiple rules will be disqualified. Also, please try-out your uploaded GC algorithm with a sample case before making a full submission to the leaderboard.¶

Submission to Final Test Phase - Testing Leaderboard¶

Once you have developed and tested your model(s), you may submit the best performing one to the Final Test Phase. You may do so as explained in the previous section.¶

For this Final Test Phase, please note the following:¶

• Only one submission per team is allowed. Teams who submit more than once will be disqualified. The Preliminary Development Phase will remain open during this period to allow for testing your model before your final submission.
• Along with your submission, you must provide:
  • A valid and publicly accessible link to the GitHub repository containing your model. This repository should be under the Apache 2.0 license.
  • A PDF file with the following details:
    • Full description of all team members, including:
      • Name and surname
      • Affiliation(s)
      • Email address
      • Grand-challenge username (if applicable)
    • A clear explanation of your method. This explanation should be detailed enough to allow others to reproduce your proposed solution. The document does not need to follow the traditional structure of a research article: there is no need to include introduction, results, and discussion sections. Instead, it should be formatted like a methods section of a paper, as explained in the following subsection.

Methods Explanation (to be included in PDF file)¶

Your submission should include a comprehensive explanation of your method. This explanation should cover the following aspects, where applicable:

Data Processing:¶

• How the data was cleaned and pre-processed.
• How the data was split for training.

Network Architecture:¶

• Description of the neural network architecture(s) used.
• Hyperparameters chosen.

Training Details:

• Information on the training process, including the number of epochs and any techniques used to improve training (e.g., early stopping, data augmentation).

Post-processing:¶

• Any post-processing steps applied to the predicted masks.