Astrocyte
Astrocyte is an application based on artificial intelligence dedicated to training neural networks on 2D images for various applications. Through a highly flexible graphical user interface users can bring in their own image samples and train neural networks to perform classification, object detection, segmentation and noise reduction. Astrocyte allows visualizing and interpreting models for performance/accuracy as well as exporting these models to files for later use at runtime into Teledyne DALSA’s Sapera and Sherlock platforms.
Key Features
- Graphical User Interface for rapid application development.
- Training and deployment on user PC for full privacy (no cloud connection required).
- Multiple deep learning architectures for a wide range of applications.
- Continual Learning (aka Lifelong Learning) in classification for further learning at runtime.
- Automatic generation of annotations via Semi-Supervised Object Detection (SSOD).
- Access to hyperparameters for highly flexible training, including selection of neural network type.
- Graphical visualization of training progress and model performance.
- Availability of training heatmaps for model assessment and runtime heatmaps for object location.
- Export of model file to interface with Sapera Processing and Sherlock for runtime inference.
- Pre-trained models for reduced training effort (lower number of samples required).
- Automatic generation and conditioning of training image files through live video acquisition from Teledyne and 3rd party cameras.
Deep Learning Architectures
Astrocyte supports the following deep learning architectures.
Classification
Classification involves predicting which class an item belongs to. Some classifiers are binary resulting in a yes/no decision. Others are multi-class and can categorize an item into one of several categories. Classification is used to solve problems like detect identification, character recognition, presence detection, food sorting, etc. Astrocyte supports the following classification neural networks: Resnet-18, Resnet-50, Resnet-101. Astrocyte also supports continual classification allowing further training at inference time.
Anomaly Detection
Anomaly Detection is the identification of rare occurrences, items or events of concern due to their differing characteristics from majority of the processed data. Anomaly Detection is a binary classifier dedicated to identifying good and bad samples. Unlike regular classification Anomaly Detection can train on unbalanced datasets (i.e. large number of good samples and small number of bad samples). Anomaly Detection is used on any application involving identification of defects on a surface or scene. Astrocyte supports the following anomaly detection neural networks: Alexnet and VGG16.
Object Detection
Object Detection involves localizing one or more objects of interest in an image. It combines the two tasks of localizing and classifying objects into one single execution. The output of Object Detection includes bounding box and a class label for each of the objects of interest. Object Detection is used to solve problems like presence detection, object tracking, defect localization and sorting, etc. Astrocyte supports the following object detection neural networks: SSD300, SSD512 and SSDLite.
Segmentation
Image segmentation involves dividing input image into segments to simplify image analysis. Segments represent objects or parts of objects and are composed of groups of pixels. Image segmentation sorts pixels into larger components
eliminating the need to consider individual pixels as units of observation. Image segmentation is a critical process in computer vision and is used for defect sorting/qualification, food sorting, shape analysis, etc. Astrocyte supports the following segmentation neural networks: DeepLabV3-Resnet-50, DeepLabV3-Resnet-101, Unet.
Noise Reduction
Image denoising aims to reconstruct a high-quality image from its degraded observation. It represents an important building block in real applications such as digital photography, medical image analysis, remote sensing, surveillance and digital entertainment. Astrocyte supports the following noise reduction neural networks: Residual Channel Attention Network (RCAN).
Astrocyte Graphical User Interface
Creating Dataset
Generating image samples
- Connect to a Teledyne or 3rd party camera to acquire live video
- Save images while acquiring live video stream (manually from click or automatic)
- Images are prepared for training while acquiring (padding/squishing/cropping)
Importing image samples
- Import from different storage locations (local, remote, cloud)
- Secure data import with credentials and encryption.
- File selection based on folder layout, prefix/suffix and regular expressions.
- Image file formats: PNG, JPG, BMP, GIF and TIFF.
- Automatic (random) or manual distribution of images into training and validation datasets.
- Adjustable image size for optimizing memory usage.
Importing/creating annotations (ground truth)
- From common databases such as Pascal VOC and MS COCO.
- From user-defined text files and parsing scheme.
- Bounding box visual editing for object detection.
- Semi-Supervised Object Detection (SSOD) allowing automated generation of bounding boxes from a dataset containing a percentage of unlabeled images.
Visualizing/editing dataset
- Image display and zoom.
- Annotation display as overlay graphics on image.
- Annotation selection, deletion and editing.
- Manual editing of annotations on individual samples.
Training Model
- Training on system GPU. See minimum requirements below.
- Selection of device (when multiple devices available)
- Choice of deep learning models for optimal accuracy.
- Access to hyperparameters such as learning rate, number of epochs, batch size, etc., for customization of training execution.
- Hyperparameters pre-set with default values commonly used.
- Image augmentation available for artificially increasing the number of training samples via transformations such as rotation, warping, lighting, zoom, etc.
- Training session cancelling and resuming.
- Progress bar with training duration estimation.
- Graph display of progress including accuracy and training loss at each iteration (epoch).
Model Validation
- Statistics on model training.
- Metrics on model performance: accuracy, recall, mean average precision (mAP), intersection over union (IoU).
- Model testing interface to perform validation of the model on either training, validation, entire, or user-defined dataset with possibility of reshuffling samples.
- Display of confusion matrix (graph showing intersection between prediction and ground truth). Interactive selection of individual images.
- Display of heatmaps for visualization of hot regions in classification.
- Inference on sample images for testing inside Astrocyte.
Model Export-Import
- Proprietary model format compatible with Sapera Processing and Sherlock†.
- Model contains all information required for performing inference: model architecture, trained weights, metadata such as image size and format.
- Multiple model management. Models stored in Astrocyte internal storage.
- Model can be imported into user application via Sapera Processing or Sherlock†.
Integration with Sapera Processing and Sherlock
- Both Sapera Processing and Sherlock include an inference tool for each of the supported model architecture.
- Model files are imported into the inference tool and ready for execution on live video stream.
- The inference tool can be coupled with other image processing tools such as blob analysis, pattern matching, barcode reading, etc.
- To be used in conjunction with Sapera LT for acquiring images from Teledyne DALSA cameras and frame-grabbers.
- Examples available with source code.
Licensing
- Astrocyte requires either Sapera AI SDK or Sherlock AI SDK to operate.
- If no license is present Astrocyte will run in evaluation mode for 60 days.
- AI inference is enabled in Sapera Processing with the Sapera Group 4 Runtime license.
- AI inference is enabled in Sherlock with Sherlock AI Runtime license.
System Requirements
| Operating System | Windows 10 64-bit |
| GPU | Minimum recommended: NVIDIA GeForce GTX 1070 with 8GB RAM or equivalent. |