Generic, lightweight python interface to unify loading and evaluating neural networks trained in machine learning libraries such as Keras, PyTorch, Torch7 and Caffe.
emu is a lightweight interface to deep learning libraries to simplify the creation of analysis toolchains. The focus is on processing images with pretrained image recognition models, manipulating parameters (e.g. to lesion a set of units) as well as reading parameters and output values of arbitrary layers. Hence, no training is supported with emus interface. Currently supported machine learning libraries (backends) are:
emus design pattern is the adapter pattern, meaning it converts the interface of the backend libraries to a unified interface NNAdapter with the following core functionality:
forward(4d_numpy_tensor)- Process a batch of images.
preprocess(list_of_images)- Preprocess a list of images for use with the neural network.
get_layers()- Get an ordered dictionary assigning a layer type to each layer name. The order is equal to the sequence of layers within that model.
get_layeroutput(layer_name)- After forward has been called, get the output of layer with name layer_name.
get_layerparams(layer_name)- Get the weights and bias parameter of layer with name layer_name.
set_weights(layer_name, weight_tensor)- Assign new weights to the given layer
set_bias(layer_name, bias_tensor)- Assign a new bias to the given layer
To use emu with a specific backend, you have to instantiate the respective backend class, as the NNAdapter does itself not contain any functionality besides a generic preprocessing function.
Overview over implemented classes inheriting NNAdapter:
- Keras:
KerasAdapter - pytorch/Torch7:
TorchAdapter - Caffe:
CaffeAdapter
The instantiation of these classes slightly differ as they depend on the quirks of the backends. Please see the documentation of these classes or have a look at the example notebooks at the bottom of this readme.
Forward two images through a Keras model and read the output of the first convolutional layer.
import emu
from emu.keras import KerasAdapter
# initialize model from Keras stock model zoo
mean = emu.keras.imagenet_mean
nn = KerasAdapter('ResNet50', 'imagenet', mean, std=None, inputsize=(224,224,3), keep_outputs=['conv1'], use_gpu=True)
# define two example images
# Note that it is sufficient to pass file paths.
# The preprocess() function automatically loads the images with RGB color channels.
images = ['data/images/img1.jpg',
'data/images/img5.png']
# preprocess and forward images
batch = nn.preprocess(images)
predictions = nn.forward(batch)
# read output of conv1
conv1_output = nn.get_layeroutput('conv1')
# get weights of conv1
weight, bias = nn.get_layerparams('conv1')- General
- numpy (>= 1.11.1)
- scikit-image (>= 0.12.3)
- Using Keras as backend
- Keras
- and one of:
- Using pytorch/torch7 as backend
python setup.py install- Find pretrained models:
- Keras:
- Model Zoo
After installation, use pretrained models via passing an available architecture name to the KerasAdapter,
e.g.:
KerasAdapter(model_cfg='ResNet50', model_weights='imagenet'). See Available models
- Model Zoo
After installation, use pretrained models via passing an available architecture name to the KerasAdapter,
e.g.:
- PyTorch:
- Model Zoo
After installation, use pretrained models via passing an available architecture name to the TorchAdapter,
e.g.:
TorchAdapter(model_fp='resnet18'). See Available models
- Model Zoo
After installation, use pretrained models via passing an available architecture name to the TorchAdapter,
e.g.:
- Using emu to estimate mean and standard deviation of pretrained caffe or torch models.
- Lesioning/Altering parameters of models
This package is named after the bird, which as the functionality in this package cannot run backwards.