Skip to content

Commit e4cb37a

Browse files
cosmic-cortexcosmic-cortex
committed
add: Keras integration jupyter notebook added
1 parent f617b2a commit e4cb37a

File tree

3 files changed

+242
-103
lines changed

3 files changed

+242
-103
lines changed

‎docs/source/content/examples/Keras-integration.rst‎

Lines changed: 0 additions & 102 deletions
This file was deleted.

‎docs/source/content/examples/Keras_integration.ipynb‎

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"Keras models in modAL workflows\n",
8+
"=============================\n",
9+
"\n",
10+
"Thanks for the scikit-learn API of Keras, you can seamlessly integrate Keras models into your modAL workflow. In this tutorial, we shall quickly introduce how to use the scikit-learn API of Keras and we are going to see how to do active learning with it. More details on the Keras scikit-learn API [can be found here](https://keras.io/scikit-learn-api/).\n",
11+
"\n",
12+
"The executable script for this example can be [found here](https://github.com/cosmic-cortex/modAL/blob/master/examples/keras_integration.py)!"
13+
]
14+
},
15+
{
16+
"cell_type": "markdown",
17+
"metadata": {},
18+
"source": [
19+
"Keras' scikit-learn API\n",
20+
"-----------------------\n",
21+
"\n",
22+
"By default, a Keras model's interface differs from what is used for scikit-learn estimators. However, with the use of its scikit-learn wrapper, it is possible to adapt your model."
23+
]
24+
},
25+
{
26+
"cell_type": "code",
27+
"execution_count": 1,
28+
"metadata": {},
29+
"outputs": [
30+
{
31+
"name": "stderr",
32+
"output_type": "stream",
33+
"text": [
34+
"/home/namazu/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
35+
" from ._conv import register_converters as _register_converters\n",
36+
"Using TensorFlow backend.\n"
37+
]
38+
}
39+
],
40+
"source": [
41+
"import keras\n",
42+
"from keras.models import Sequential\n",
43+
"from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D\n",
44+
"from keras.wrappers.scikit_learn import KerasClassifier\n",
45+
"\n",
46+
"# build function for the Keras' scikit-learn API\n",
47+
"def create_keras_model():\n",
48+
" \"\"\"\n",
49+
" This function compiles and returns a Keras model.\n",
50+
" Should be passed to KerasClassifier in the Keras scikit-learn API.\n",
51+
" \"\"\"\n",
52+
"\n",
53+
" model = Sequential()\n",
54+
" model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28, 28, 1)))\n",
55+
" model.add(Conv2D(64, (3, 3), activation='relu'))\n",
56+
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
57+
" model.add(Dropout(0.25))\n",
58+
" model.add(Flatten())\n",
59+
" model.add(Dense(128, activation='relu'))\n",
60+
" model.add(Dropout(0.5))\n",
61+
" model.add(Dense(10, activation='softmax'))\n",
62+
"\n",
63+
" model.compile(loss='categorical_crossentropy', optimizer='adadelta', metrics=['accuracy'])\n",
64+
"\n",
65+
" return model"
66+
]
67+
},
68+
{
69+
"cell_type": "markdown",
70+
"metadata": {},
71+
"source": [
72+
"For our purposes, the ``classifier`` which we will initialize now acts just like any scikit-learn estimator."
73+
]
74+
},
75+
{
76+
"cell_type": "code",
77+
"execution_count": 2,
78+
"metadata": {},
79+
"outputs": [],
80+
"source": [
81+
"# create the classifier\n",
82+
"classifier = KerasClassifier(create_keras_model)"
83+
]
84+
},
85+
{
86+
"cell_type": "markdown",
87+
"metadata": {},
88+
"source": [
89+
"Active learning with Keras\n",
90+
"---------------------------------------\n",
91+
"\n",
92+
"In this example, we are going to use the famous MNIST dataset, which is available as a built-in for Keras."
93+
]
94+
},
95+
{
96+
"cell_type": "code",
97+
"execution_count": 3,
98+
"metadata": {},
99+
"outputs": [],
100+
"source": [
101+
"import numpy as np\n",
102+
"from keras.datasets import mnist\n",
103+
"\n",
104+
"# read training data\n",
105+
"(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
106+
"X_train = X_train.reshape(60000, 28, 28, 1).astype('float32') / 255\n",
107+
"X_test = X_test.reshape(10000, 28, 28, 1).astype('float32') / 255\n",
108+
"y_train = keras.utils.to_categorical(y_train, 10)\n",
109+
"y_test = keras.utils.to_categorical(y_test, 10)\n",
110+
"\n",
111+
"# assemble initial data\n",
112+
"n_initial = 1000\n",
113+
"initial_idx = np.random.choice(range(len(X_train)), size=n_initial, replace=False)\n",
114+
"X_initial = X_train[initial_idx]\n",
115+
"y_initial = y_train[initial_idx]\n",
116+
"\n",
117+
"# generate the pool\n",
118+
"# remove the initial data from the training dataset\n",
119+
"X_pool = np.delete(X_train, initial_idx, axis=0)[:5000]\n",
120+
"y_pool = np.delete(y_train, initial_idx, axis=0)[:5000]"
121+
]
122+
},
123+
{
124+
"cell_type": "markdown",
125+
"metadata": {},
126+
"source": [
127+
"Active learning with data and classifier ready is as easy as always. Because training is *very* expensive in large neural networks, this time we are going to query the best 200 instances each time we measure the uncertainty of the pool."
128+
]
129+
},
130+
{
131+
"cell_type": "code",
132+
"execution_count": 4,
133+
"metadata": {},
134+
"outputs": [
135+
{
136+
"name": "stdout",
137+
"output_type": "stream",
138+
"text": [
139+
"Epoch 1/1\n",
140+
"1000/1000 [==============================] - 4s 4ms/step - loss: 1.5794 - acc: 0.4790\n"
141+
]
142+
}
143+
],
144+
"source": [
145+
"from modAL.models import ActiveLearner\n",
146+
"\n",
147+
"# initialize ActiveLearner\n",
148+
"learner = ActiveLearner(\n",
149+
" estimator=classifier,\n",
150+
" X_training=X_initial, y_training=y_initial,\n",
151+
" verbose=1\n",
152+
")"
153+
]
154+
},
155+
{
156+
"cell_type": "markdown",
157+
"metadata": {},
158+
"source": [
159+
"To make sure that you train only on newly queried labels, pass ``only_new=True`` to the ``.teach()`` method of the learner."
160+
]
161+
},
162+
{
163+
"cell_type": "code",
164+
"execution_count": 5,
165+
"metadata": {},
166+
"outputs": [
167+
{
168+
"name": "stdout",
169+
"output_type": "stream",
170+
"text": [
171+
"Query no. 1\n",
172+
"Epoch 1/1\n",
173+
"100/100 [==============================] - 1s 10ms/step - loss: 2.0987 - acc: 0.3300\n",
174+
"Query no. 2\n",
175+
"Epoch 1/1\n",
176+
"100/100 [==============================] - 1s 7ms/step - loss: 2.1222 - acc: 0.3300\n",
177+
"Query no. 3\n",
178+
"Epoch 1/1\n",
179+
"100/100 [==============================] - 1s 8ms/step - loss: 2.0558 - acc: 0.4900\n",
180+
"Query no. 4\n",
181+
"Epoch 1/1\n",
182+
"100/100 [==============================] - 1s 9ms/step - loss: 1.6943 - acc: 0.4700\n",
183+
"Query no. 5\n",
184+
"Epoch 1/1\n",
185+
"100/100 [==============================] - 1s 12ms/step - loss: 1.5865 - acc: 0.6200\n",
186+
"Query no. 6\n",
187+
"Epoch 1/1\n",
188+
"100/100 [==============================] - 1s 14ms/step - loss: 1.8714 - acc: 0.3500\n",
189+
"Query no. 7\n",
190+
"Epoch 1/1\n",
191+
"100/100 [==============================] - 1s 14ms/step - loss: 1.3940 - acc: 0.6700\n",
192+
"Query no. 8\n",
193+
"Epoch 1/1\n",
194+
"100/100 [==============================] - 1s 14ms/step - loss: 2.1033 - acc: 0.3200\n",
195+
"Query no. 9\n",
196+
"Epoch 1/1\n",
197+
"100/100 [==============================] - 1s 11ms/step - loss: 1.5666 - acc: 0.6700\n",
198+
"Query no. 10\n",
199+
"Epoch 1/1\n",
200+
"100/100 [==============================] - 1s 12ms/step - loss: 2.0238 - acc: 0.2700\n"
201+
]
202+
}
203+
],
204+
"source": [
205+
"# the active learning loop\n",
206+
"n_queries = 10\n",
207+
"for idx in range(n_queries):\n",
208+
" print('Query no. %d' % (idx + 1))\n",
209+
" query_idx, query_instance = learner.query(X_pool, n_instances=100, verbose=0)\n",
210+
" learner.teach(\n",
211+
" X=X_pool[query_idx], y=y_pool[query_idx], only_new=True,\n",
212+
" verbose=1\n",
213+
" )\n",
214+
" # remove queried instance from pool\n",
215+
" X_pool = np.delete(X_pool, query_idx, axis=0)\n",
216+
" y_pool = np.delete(y_pool, query_idx, axis=0)"
217+
]
218+
}
219+
],
220+
"metadata": {
221+
"kernelspec": {
222+
"display_name": "Python 3",
223+
"language": "python",
224+
"name": "python3"
225+
},
226+
"language_info": {
227+
"codemirror_mode": {
228+
"name": "ipython",
229+
"version": 3
230+
},
231+
"file_extension": ".py",
232+
"mimetype": "text/x-python",
233+
"name": "python",
234+
"nbconvert_exporter": "python",
235+
"pygments_lexer": "ipython3",
236+
"version": "3.6.5"
237+
}
238+
},
239+
"nbformat": 4,
240+
"nbformat_minor": 2
241+
}

‎docs/source/index.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ modAL is an active learning framework for Python3, designed with *modularity, fl
4242
content/examples/bayesian_optimization
4343
content/examples/query_by_committee
4444
content/examples/bootstrapping_and_bagging
45-
content/examples/Keras-integration
45+
content/examples/Keras_integration
4646

4747
.. toctree::
4848
:glob:

0 commit comments

Comments
 (0)