Link: https://dzone.com/articles/malware-detection-with-convolutional-neural-networ
In this post, we will learn about artificial network architectures and how to use one of them (Convolutional Neural Networks) to help malware analysts and information security professionals to detect and classify malicious code.
Malware is a nightmare for every modern organization. Attackers and cybercriminals are always coming up with new malicious software to attack their targets. Security vendors are doing their best to defend against malware attacks but, unfortunately, with millions of malware discovered monthly, they cannot achieve that. Thus, novel approaches such as deep learning are needed.
Before diving into the technical details and the steps for the practical implementation of the DL method, it is essential to learn and discover the other different architectures of artificial neural networks. The major artificial neural networks are discussed now.
This excerpt is taken from the book Mastering Machine Learning for Penetration Testing by Packt Publishing. This book teaches you extensive skills to become a master at penetration testing using machine learning with Python.
Convolutional Neural Networks (CNNs)
Convolutional Neural Networks (CNNs) are a deep learning approach to tackle the image classification problem, or what we call computer vision problems, because classic computer programs face many challenges and difficulties to identify objects for many reasons, including lighting, viewpoint, deformation, and segmentation.
This technique is inspired by how the eye works, especially the visual cortex function algorithm in animals. CNN are arranged in three-dimensional structures with width, height, and depth as characteristics. In the case of images, the height is the image height, the width is the image width, and the depth is RGB channels.
To build a CNN, we need three main types of layers:
Convolutional layer: A convolutional operation refers to extracting features from the input image and multiplying the values in the filter with the original pixel values
Pooling layer: The pooling operation reduces the dimensionality of each feature map
Fully-connected layer: The fully-connected layer is a classic multi-layer perceptrons with a softmax activation function in the output layer
To implement a CNN with Python, you can use the following Python script:
import numpy
from keras.datasets
import mnist
from keras.models
import Sequential
from keras.layers
import Dense
from keras.layers
import Dropout
from keras.layers
import Flatten
from keras.layers.convolutional
import Conv2D
from keras.layers.convolutional
import MaxPooling2D
from keras.utils
import np_utils
from keras
import backend
backend.set_image_dim_ordering(‘th’)
model = Sequential()
model.add(Conv2D(32, (5, 5), input_shape = (1, 28, 28), ))
model.add(MaxPooling2D(pool_size = (2, 2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(128, ))
model.add(Dense(num_classes, ))
model.compile(, , metrics = [‘accuracy’])
Recurrent Neural Networks (RNNs)
Recurrent Neural Networks (RNNs) are artificial neural networks where we can make use of sequential information, such as sentences. In other words, RNNs perform the same task for every element of a sequence, with the output depending on the previous computations. RNNs are widely used in language modeling and text generation (machine translation, speech recognition, and many other applications). RNNs do not remember things for a long time.
Long Short Term Memory networks
Long Short Term Memory (LSTM) solves the short memory issue in recurrent neural networks by building a memory block. This block sometimes is called a memory cell.
Hopfield networks
Hopfield networks were developed by John Hopfield in 1982. The main goal of Hopfield networks is auto-association and optimization. We have two categories of Hopfield network: discrete and continuous.
Boltzmann Machine Networks
Boltzmann machine networks use recurrent structures and they use only locally available information. They were developed by Geoffrey Hinton and Terry Sejnowski in 1985. Also, the goal of a Boltzmann machine is optimizing the solutions.
Malware Detection With CNNs
For this new model, we are going to discover how to build a malware classifier with CNNs. But I bet you are wondering how we can do that while CNNs are taking images as inputs. The answer is really simple, the trick here is converting malware into an image. Is this possible? Yes, it is. Malware visualization is one of many research topics during the past few years. One of the proposed solutions has come from a research study called Malware Images: Visualization and Automatic Classification by Lakshmanan Nataraj from the Vision Research Lab, University of California, Santa Barbara.
The following diagram details how to convert malware into an image:
The following is an image of the Alueron.gen!J malware:
This technique also gives us the ability to visualize malware sections in a detailed way:
By solving the issue of how to feed malware machine learning classifiers that use CNNs by images, information security professionals can use the power of CNNs to train models. One of the malware datasets most often used to feed CNNs is the Malimg dataset. This malware dataset contains 9,339 malware samples from 25 different malware families. You can download it from Kaggle.
These are the malware families:
Allaple.L
Allaple.A
Yuner.A
Lolyda.AA 1
Lolyda.AA 2
Lolyda.AA 3
C2Lop.P
C2Lop.gen!G
Instant access
Swizzor.gen!I
Swizzor.gen!E
VB.AT
Fakerean
Alueron.gen!J
Malex.gen!J
Lolyda.AT
Adialer.C
Wintrim.BX
Dialplatform.B
Dontovo.A
Obfuscator.AD
Agent.FYI
Autorun.K
Rbot!gen
Skintrim.N
After converting malware into grayscale images, you can get the following malware representation so you can use them later to feed the machine learning model:
The conversion of each malware to a grayscale image can be done using the following Python script:
For feature selection, you can extract or use any image characteristics, such as the texture pattern, frequencies in image, intensity, or color features, using different techniques such as Euclidean distance, or mean and standard deviation, to generate later feature vectors. In our case, we can use algorithms such as a color layout descriptor, homogeneous texture descriptor, or global image descriptors (GIST). Let’s suppose that we selected the GIST; pyleargist is a great Python library to compute it. To install it, use PIP as usual:
# pip install pyleargist=.0.1
As a use case, to compute a GIST, you can use the following Python script:
Here, 320 refers to the first 320 values while we are using grayscale images. Don’t forget to save them as NumPy arrays to use them later to train the model.
After getting the feature vectors, we can train many different models, including SVM, k-means, and artificial neural networks. One of the useful algorithms is that of the CNN.
Once the feature selection and engineering is done, we can build a CNN. For our model, for example, we will build a convolutional network with two convolutional layers, with 32 * 32 inputs. To build the model using Python libraries, we can implement it with the previously installed TensorFlow and utils libraries.
So, the overall CNN architecture will be as in the following diagram:
This CNN architecture is not the only proposal to build the model, but at the moment we are going to use it for the implementation.
To build the model and CNN in general, I highly recommend Keras. The required imports are the following:
As we discussed before, the grayscale image has pixel values that range from 0 to 255, and we need to feed the net with 32 * 32 * 1 dimension images as a result:
We will train our network with these parameters:
To build the architecture, with regards to its format, use the following:
To compile the model, use the following:
Malware_Model.compile(.losses.categorical_crossentropy, .optimizers.Adam(),metrics=['accuracy'])
Fit and train the model:
Malware_Model.fit(train_X, train_label, ,,,validation_data=(valid_X, valid_label))
As you noticed, we are respecting the flow of training a neural network that was discussed in previous chapters. To evaluate the model, use the following code:
Promises and Challenges in Applying Deep Learning to Malware Detection
Many different deep network architectures were proposed by machine learning practitioners and malware analysts to detect both known and unknown malware; some of the proposed architectures include restricted Boltzmann machines and hybrid methods.
New approaches to detect malware and malicious software show many promising results. However, there are many challenges that malware analysts face when it comes to detecting malware using deep learning networks, especially when analyzing PE files because to analyze a PE file, we take each byte as an input unit, so we deal with classifying sequences with millions of steps, in addition to the need of keeping complicated spatial correlation across functions due to function calls and jump commands.
转载请注明：江海志の博客 » Malware Detection With Convolutional Neural Networks in Python