A new web application streamlines deep learning workflows through a project-based, multi-user environment.
Esri has released a new web app for users who want to integrate deep learning into their imaging workflows. Deep Learning Studio, available with the release of ArcGIS Enterprise 11, provides a collaborative environment where multiple users can work together on an image-based project that includes deep learning. With the app, multiple users can work on a single project and perform deep learning taskssuch as collecting training samples, training deep learning models, and performing large-scale inference.
The application combines several things at once: a front-end experience for deep learning tasks that are part of backend raster analysis, a collaborative environment that distributes otherwise tedious deep learning tasks across multiple users, and a complete end-to-end deep learning workflow, i.e. offered through a user-friendly project wizard (Figure 3). Additionally, users can customize their own workflow if needed.
Although the app does not require any local software installation, it does require both ArcGIS Enterprise and ArcGIS Image Server, as these provide the data and analysis tools accessible through the app. ArcGIS Enterprise is Esri’s comprehensive mapping and analysis platform, available on-premises, in the cloud, or a mix of both. ArcGIS Image Server is part of ArcGIS Enterprise and provides a distributed computing and storage system that powers the analytical processing and dissemination of large collections of imagery, elevation data, rasters, and other remote sensing data.
How the app works
The app is available through the app launcher on the ArcGIS Enterprise portal website. After starting the application, users can create a new project, name it, and select the type of task you want this deep learning model to perform. It can be object detection, pixel classification or object classification. Model creation involves two steps: data preparation and model creation. After that, you can use it on other imagery data, which is called running inference. Esri has released a video which describes in detail how the application works.
Data preparation and model creation workflows
The data preparation process is a four-step workflow. First, you select an imagery layer from a configured raster data store. Second, you define the tags you want to collect, i.e. the process of augmenting data with informative tags. Third, you invite other members of your organization to label the data and define geographic work areas for each of these members so that a set of images is divided into multiple areas that each member gets a part of. All of these steps use a simple project wizard where you can simply click on a set of buttons showing the different options for a new project.
After completing the data preparation workflow, you can begin collecting training samples, which uses scanning tools to collect representative training samples of objects of interest. Labels are created manually in the imagery data. Since this process is rather tedious and repetitive, it makes sense to use this application to divide a data set into multiple areas and distribute the work among multiple users. After tagging, the app provides a dashboard to the project manager where he or she can review project samples and approve or reject them. After the labeling process is completed, the training data is exported, after which the deep learning model can be trained.
Train and run a large-scale deep learning model
First, you select the training samples (called image chips) and specify the type of model you want to train. The miniature train parameters are defined in the app documentation. Then start the training process, which takes a few minutes. The resulting deep learning model can be published to an ArcGIS portal as a deep learning package, which includes the trained model along with a model definition file. This model is now ready to be used on other large-scale imagery data (using the inference tool running in the app).