In this chapter, structure activity correlation analysis is performed using DNN.
First, let’s build a simple prediction model using DNN. Here we use the same data as in Chapter 9. First, create a classification model and label the Positive label as [0, 1] and the Negative label as a [1, 0] two-dimensional OneHot vector. If you create a model using Keras Model object, you can get the expected value of each of the above two dimensions. You can use Numpy’s Argmax function to know which class it is likely to belong to.
|
Note
|
OneHot vector is a vector in which one value is 1 and the other is 0. When considering a classification problem of 10 classes, such as [1, 0, 0, 0, 0, 0, 0, 0, 0, 0], a vector such that somewhere is 1 and the remaining 9 are 0 I can express a class. In the above example, there are two classes of Positive / Negative, so the OneHot vector is two-dimensional. |
Import the required libraries.
from rdkit import Chem, DataStructs
from rdkit.Chem import AllChem, Draw
from rdkit.Chem.Draw import IPythonConsole
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, f1_score
from tensorflow.python.keras.layers import Iput
from tensorflow.python.keras.layers import Dense
from tensorflow.python.keras.layers import Dropout
from tensorflow.python.keras.layers import Activation
from tensorflow.python.keras.Model import ModelNext, read the data. In Chapter 9 we put "POS" / "NEG" in the list of labels, so it was a one-dimensional representation, but this time it is two-dimensional.
mols = []
labels = []
with open("ch09_compounds.txt") as f:
header = f.readline()
smiles_index = -1
for i, title in enumerate(header.split("\t")):
if title == "CANONICAL_SMILES":
smiles_index = i
elif title == "STANDARD_VALUE":
value_index = i
for l in f:
ls = l.split("\t")
mol = Chem.MolFromSmiles(ls[smiles_index])
mols.append(mol)
val = float(ls[value_index])
if val < 1000:
labels.append([0,1]) # Positive
else:
labels.append([1,0]) # Negative
labels = np.array(labels)Next, create classification models and regression models sequentially.
The first is a regression model, and the input uses the same ECFP as in Chapter 9. In order to construct DNN, it is necessary to specify the dimension of input data explicitly, so we define the variable nBits.
|
Tip
|
Specifying an appropriate integer in random_state for train_test_split is useful for verification because the same data is obtained each time. |
nBits = 2048
fps = []
for mol in mols:
fp = AllChem.GetMorganFingerprintAsBitVect(mol, 2, nBits=nBits)
arr = np.zeros((1,))
DataStructs.ConvertToNumpyArray(fp, arr)
fps.append(arr)
fps = np.array(fps)
x_train1, x_test1, y_train1, y_test1 = train_test_split(fps, labels, random_state=794)Create a neural network whose inputs are 2048 dimensions, the total connection layer of 300 neurons is three layers, and the final output layer is two. We used ReLU for the activation function and Softmax for two-dimensional multiclass classification for the output layer.
The Dropout layer plays a role to prevent overlearning by randomly deleting neurons.
In Keras, a model is built by calling the compile function after defining the model. The optimizer and loss need to be changed according to the purpose. This time, I used 'categorical_crossentropy' as loss and 'adam' as optimizer, but there are many other than links:https://keras.io/ja/optimizers/[adam optimzer], so it is actually a trial and error case to determine which is appropriate.
|
Tip
|
ReLU is often used because it can overcome the problem of gradient disappearance of Sigmoid function. |
# Define DNN classifier model
epochs = 10
inputlayer1 = Input(shape=(nBits, ))
x1 = Dense(300, activation='relu')(inputlayer1)
x1 = Dropout(0.2)(x1)
x1 = Dense(300, activation='relu')(x1)
x1 = Dropout(0.2)(x1)
x1 = Dense(300, activation='relu')(x1)
output1 = Dense(2, activation='softmax')(x1)
model1 = Model(inputs=[inputlayer1], outputs=[output1])
model1.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])NOTE: Keras provides a Sequential model, which can be used to describe the network more simply than the example above (Functional API). The reason we defined the model in the Functional API is that it is easy to handle multiple inputs and more complex models if you get used to it. If you are interested in writing Sequential please check out the official site and Qiita
|
Note
|
DNN optimizes the model while iterating the Backpropagation procedure, which compares the actual value with the predicted value predicted based on the initial randomly generated weight, and updates the weight so as to minimize the difference (LOSS). You It is Epochs that specifies the number of repetitions. You may seem to get smarter as you increase Epochs, but there is a risk of computational cost and over-learning, so it is not good if it is long. Observe Loss, Accuracy, etc. and find the appropriate number of Epochs. |
Using training data, we will adjust the weight to reduce the error between the correct value and the predicted value for each Epoch. If it is learned using a sufficient amount of training data and it is repeated too much, the generalization performance of the model will be reduced since the same training data will be learned over and over again.
To evaluate overfiting, you can evaluate the accuracy of the training set / validation set for each Epoch and plot to improve the accuracy of the training set while checking whether the accuracy of the validation set does not change or becomes worse. Keras has a function of Early stopping, and even if learning continues for a certain number of times, if the performance of the model does not change, learning can be stopped there.
See the introduction and references of Early stopping for more information.
After building the model, you can do fit / predict in the same way as Scikit-learn.
hist1 = model1.fit(x_train1, y_train1, epochs=epochs)Finally, let’s visualize the result.
%matplotlib inline
import matplotlib.pyplot as plt
plt.plot(range(epochs), hist1.history['acc'], label='acc')
plt.legend()
plt.plot(range(epochs), hist1.history['loss'], label='loss')
plt.legend()In this example, the model has good accuracy around 6Epoch.
Next, verify with test data.
y_pred1 = model1.predict(x_test1)
y_pred_cls1 = np.argmax(y_pred1, axis=1)
y_test_cls1 =np.argmax(y_test1, axis=1)
confusion_matrix(y_test_cls1, y_pred_cls1)A little subtle ,,,,
The regression model is basically the same as the classification problem above. This time it is a regression, so the last output layer is the value itself, ie one dimensional The activation function is 0-1 in Sigmoid etc., so it is Linear. The training data uses the code of Chapter 9.
from math import log10
from sklearn.metrics import r2_score
pIC50s = []
with open("ch09_compounds.txt") as f:
header = f.readline()
for i, title in enumerate(header.split("\t")):
if title == "STANDARD_VALUE":
value_index = i
for l in f:
ls = l.split("\t")
val = float(ls[value_index])
pIC50 = 9 - log10(val)
pIC50s.append(pIC50)
pIC50s = np.array(pIC50s)
x_train2, x_test2, y_train2, y_test2 = train_test_split(fps, pIC50s, random_state=794)Next, define the model. Note that the Loss part is MSE, unlike the classification model above.
epochs = 50
inputlayer2 = Input(shape=(nBits, ))
x2 = Dense(300, activation='relu')(inputlayer2)
x2 = Dropout(0.2)(x2)
x2 = Dense(300, activation='relu')(x2)
x2 = Dropout(0.2)(x2)
x2 = Dense(300, activation='relu')(x2)
output2 = Dense(1, activation='linear')(x2)
model2 = Model(inputs=[inputlayer2], outputs=[output2])
model2.compile(optimizer='adam', loss='mean_squared_error')If you can do this, the rest is the same.
hist = model2.fit(x_train2, y_train2, epochs=epochs)
y_pred2 = model2.predict(x_test2)
r2_score(y_test2, y_pred2)
plt.scatter(y_test2, y_pred2)
plt.xlabel('exp')
plt.ylabel('pred')
plt.plot(np.arange(np.min(y_test2)-0.5, np.max(y_test2)+0.5), np.arange(np.min(y_test2)-0.5, np.max(y_test2)+0.5))What do you think. The prediction model looks a bit like UnderEstimate. The DNN needs to tune a number of parameters, such as the number of layers to overlap, the percentage of dropouts, the number of neurons in the hidden layer, and the type of activation function. This example was hard-coded, but it is also interesting to compare the performance of the models by changing various parameters.
So far, we have created models of RandomForest and DNN using molecular fingerprints as input. One of the reasons why DNN has received a great deal of attention is that models can recognize feature quantities even if people do not extract feature quantities.
For example, in image classification, a human defined the feature quantity called SIFT, and a model was created using this as an input, but the current DNN basically uses the pixel information of the image itself.
In terms of chemoinformatics, SIFT is equivalent to a molecular fingerprint. So isn’t it possible to improve DNN’s performance by changing this (input) to a more primitive expression? It is extremely natural to think that. In 2015, Alan Aspuru-Guzik et al’s group at Harvard University proposed the Neural Fingerprint/NFP as a challenge.
The differences between ECFP and NFP used so far are shown by citing figures in their papers.
ECFP (Circular Fingerprints) converts information from each atom of input molecules to atoms in the vicinity of N (N is arbitrary) into Hash values (Mod in this example) to arbitrary values, and converts them into vectors of fixed length was. Roughly speaking, it is an image such as using the one where the presence or absence of the partial structure is corrected to the bit information of 0/1. On the other hand, NFP introduced this time is similar in concept to ECFP, but the part of Hash function is Sigmoid, and the part to be discretized with Mod is Softmax. Therefore, it is expected that input datasets will generate molecular fingerprints more flexibly than ECFP.
A number of implementations have been published to GitHub since this paper was published, but each implementation does not work with Keras 1.x or Keras / Tensorflow, even if the Backend is Theano or Keras / Tensorflow There are a lot of environment-dependent things that are surprisingly difficult to handle. Unfortunately there is no one that works in the environment we built this time, so I created one that works with Keras 2.x / Python 3.6 based on this code .
SIFT was proposed in 1999. According to the original paper, the difficulty in dealing with pixels themselves in object (image) recognition seems to be in dealing with objects that differ in position, rotation, size (scale), light intensity, etc. It seems that various methods have been studied to convert these fluctuating values into universal features. There is no way to use the pixels themselves, but the machine learning that I started with python, which I purchased when studying machine learning , has an example of learning and classifying human face image data. Here, with the pixel data as input, the feature of the face is extracted and classified by principal component analysis. I have not been able to find a document that was clearly valid on this question, but I think it was valid depending on the task. Please comment if you have any details.
git clone https://github.com/iwatobipen/keras-neural-graph-fingerprint.gitIf you look at the code in the example.py file, you will find the atmosphere somehow. In the previous examples, molecule representations were generated using RDKit for this example, but this time the fingerprint itself is learned by DNN.
So, representing molecules as a graph is the input. As Atom_matrix, (max_atoms, num_atom_features) is used as Edge_matrix, (max_atoms, max_degree) as bond_tensor, and three matrices (max_atoms, max_degree, num_bond_features) are used. Since each molecule has a different number of atoms, max_atoms defines the maximum number of atoms. By doing this, it becomes input of the same matrix size for each numerator and batch learning becomes possible.
If you want to execute Example, please enter the following command.
python example.pyReference links: - NGF-paper - DeepChem-paper - keiserlab - HIPS NFP - Theano base - for keras1.x - ericmjl/graph_fp - DeepChem - SIFT original Paper - Early stopping