Dataset Processing Instructions

This is an overview guide on how to upload a dataset to the server and process it

Paper Dataset For those of you interested in processing the dataset used in the paper please download the following paper zip. This zip contains two datasets that already meet the requirement of (1).

1) Prepare your Dataset for Processing

To upload a dataset please ensure that the structure follows
the following folder structure. This is the most crucial part as we are very strict about folder structure.
There must be no extraneous files, folders, or images outside of the given structure.
Also please be sure to only have currently supported image files, such as TIFF, JPG, JPEG, and PNG.

Dataset Folder/Dataset Zip Structure Example


            - Biopsy 1 

            -- Glom 1 

            --- Image 1.tiff 

            --- Image 2.png 

            --- ... 

            -- Glom 2 

            --- Image 1.tiff 

            --- Image 2.jpg 

            --- ... 

            -- ... 

            - Biopsy 2 

            -- Glom 1 

            --- ... 

            -- Glom 2 

            --- ... 

            --- ... 

            - Biopsy 3 

            ...

Note: the names of the folders/images can be anything, in fact they should be more descriptive, but the structure must be the same

2) Create your Dataset Online

Visit the following page to create a new dataset to process. You will want to create a new dataset to generate reports specific to that dataset.
For example a normal patients dataset or a fabry patients dataset.
Each dataset, as described above, will have a set of biopsies and glomerulus images within those biopsies

Create a Dataset

Once opened you can fill out the name of the dataset, for example Normals taken 9-7-2022, then optionally fill out a description. Finally click create.

2) Upload the Dataset

You have two options for uploading a dataset. You may either go to the datasets page and click Upload Zip to upload a small zip file through the browser. The other option is to use the Desktop Application to upload larger datasets with more feedback.

Option A) Upload Online

First go to your datasets

All Datasets

Once there click the Upload Zip button for your respective dataset. Then follow the instructions on that screen

Option B) Desktop Application Installation Option

You may skip this step if you already have the desktop application installed, but if you run into any errors uploading please try installing the latest version from here before reporting any errors to us. Thanks!

FPW Uploader for Windows

Here are the steps for the desktop application

Once you have the installer downloaded please follow the instructions to have it installed to your system.
The app name should be called FPW Uploader. Please find it and open it, which depends on where you installed it in the first step.
Once you have the app opened there will be a login screen. Use the same credentials as you did for this site to login to the application.
On the selector screen there are two required things. First you should select the dataset you wish to upload to, and then click the Choose Folder option to select the folder with your dataset.
Before clicking Upload, please verify that the dataset is a folder and not a zip file, and that it follows the structure as in step #1
Once you're sure of this, press the Upload button and watch for errors. The first step will be a folder verification that checks the image contents and folder structure.
The last page will be the text logging and progress of the upload. Depending on your internet speed, server usage, and other factors such as dataset size please be patient as it might take a while.

3) Process the dataset

Now that the dataset is successfully on the server, all of the processing can now be fully managed online.
Processing the dataset in done sequentially, in most cases, which is covered on the workflow page of the specific dataset.
To view the workflow page please visit the datasets page and click Process on the specified dataset you wish to process.
The overall workflow to process a dataset is as follows:

1) Pre-screening: Screen the images by toggling the Prediction and Vision switches to not include the segmentation masks or processing of that particular image.
2) Prediction: Once started this will generate all of the segmentation masks for the images, except those that the Prediction toggle is turned off, and store them on the server.
3) Post-screening: The segmentation masks will be overlayed on the original image with a preview for you to verify the results of the model and exclude any bad images from being process by the vision script.
4) Vision: This task will process the segmentation masks using various techniques to try to map slits to membranes and measure the foot process lengths and save the results to a database.
5) Results: Allows for various exports of the dataset, which is under active development, and allows you to start tasks that generate reports. Some tasks might take longer than others.

4) Download the results

On the same Results page of the dataset overview, which is navigated to by the Process button on a dataset, you can view all of the previous exported files.
On the Exports side of the Results, you can select the export you wish to download and then click Download.