From 603547bfac039b9ac44b75874416681b66a2c68f Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 22:49:18 +0530 Subject: [PATCH 1/8] Highlighted syntax for sample NN code --- README.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 7be7d91..5377b6b 100644 --- a/README.md +++ b/README.md @@ -8,11 +8,13 @@ Pretty Tensor provides a set of objects that behave likes Tensors, but also support a chainable object syntax to quickly define neural networks and other layered architectures in TensorFlow. - result = (pretty_tensor.wrap(input_data, m) - .flatten() - .fully_connected(200, activation_fn=tf.nn.relu) - .fully_connected(10, activation_fn=None) - .softmax(labels, name=softmax_name)) +```py +result = (pretty_tensor.wrap(input_data, m) + .flatten() + .fully_connected(200, activation_fn=tf.nn.relu) + .fully_connected(10, activation_fn=None) + .softmax(labels, name=softmax_name)) +``` Please look here for full documentation of the PrettyTensor object for all available operations: From a9387e9597ba221ab256352d04a0d84b3aa7cb12 Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 22:55:22 +0530 Subject: [PATCH 2/8] Highlighted syntax for import code --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5377b6b..e3adfba 100644 --- a/README.md +++ b/README.md @@ -38,8 +38,10 @@ The easiest installation is just to use pip: ## Quick start ### Imports - import prettytensor as pt - import tensorflow as tf +```py +import prettytensor as pt +import tensorflow as tf +``` ### Setup your input my_inputs = # numpy array of shape (BATCHES, BATCH_SIZE, DATA_SIZE) From f8a7613a374c87e706eba6200fb02901f805098a Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 22:56:14 +0530 Subject: [PATCH 3/8] Highlighted syntax for setting up the inputs --- README.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index e3adfba..0322aff 100644 --- a/README.md +++ b/README.md @@ -44,11 +44,13 @@ import tensorflow as tf ``` ### Setup your input - my_inputs = # numpy array of shape (BATCHES, BATCH_SIZE, DATA_SIZE) - my_labels = # numpy array of shape (BATCHES, BATCH_SIZE, CLASSES) - input_tensor = tf.placeholder(np.float32, shape=(BATCH_SIZE, DATA_SIZE)) - label_tensor = tf.placeholder(np.float32, shape=(BATCH_SIZE, CLASSES)) - pretty_input = pt.wrap(input_tensor) +```py +my_inputs = # numpy array of shape (BATCHES, BATCH_SIZE, DATA_SIZE) +my_labels = # numpy array of shape (BATCHES, BATCH_SIZE, CLASSES) +input_tensor = tf.placeholder(np.float32, shape=(BATCH_SIZE, DATA_SIZE)) +label_tensor = tf.placeholder(np.float32, shape=(BATCH_SIZE, CLASSES)) +pretty_input = pt.wrap(input_tensor) +``` ### Define your model softmax, loss = (pretty_input. From 9a81952cc61d2148487b0f09aafb6adb9266eb62 Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 22:57:03 +0530 Subject: [PATCH 4/8] Highlighted syntax for defining the model --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 0322aff..92ed5ed 100644 --- a/README.md +++ b/README.md @@ -53,9 +53,11 @@ pretty_input = pt.wrap(input_tensor) ``` ### Define your model - softmax, loss = (pretty_input. - fully_connected(100). - softmax_classifier(CLASSES, labels=label_tensor)) +```py +softmax, loss = (pretty_input. + fully_connected(100). + softmax_classifier(CLASSES, labels=label_tensor)) +``` ### Train and evaluate accuracy = softmax.evaluate_classifier(label_tensor) From 066de2ad822850c3790eb98fff608b144f52eed7 Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 22:58:10 +0530 Subject: [PATCH 5/8] Highlighted syntax for "Train and evaluate" --- README.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 92ed5ed..ebf3d6e 100644 --- a/README.md +++ b/README.md @@ -60,19 +60,21 @@ softmax, loss = (pretty_input. ``` ### Train and evaluate - accuracy = softmax.evaluate_classifier(label_tensor) +```py +accuracy = softmax.evaluate_classifier(label_tensor) - optimizer = tf.train.GradientDescentOptimizer(0.1) # learning rate - train_op = pt.apply_optimizer(optimizer, losses=[loss]) +optimizer = tf.train.GradientDescentOptimizer(0.1) # learning rate +train_op = pt.apply_optimizer(optimizer, losses=[loss]) - init_op = tf.initialize_all_variables() +init_op = tf.initialize_all_variables() - with tf.Session() as sess: - sess.run(init_op) - for inp, label in zip(my_inputs, my_labels): - unused_loss_value, accuracy_value = sess.run([loss, accuracy], - {input_tensor: inp, label_tensor: label}) - print 'Accuracy: %g' % accuracy_value +with tf.Session() as sess: + sess.run(init_op) + for inp, label in zip(my_inputs, my_labels): + unused_loss_value, accuracy_value = sess.run([loss, accuracy], + {input_tensor: inp, label_tensor: label}) + print 'Accuracy: %g' % accuracy_value +``` ## Features From 8ff482c5eb6331522bdc0e9dc79cd279b4433f59 Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 22:59:22 +0530 Subject: [PATCH 6/8] Highlighted syntax for Terse section --- README.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index ebf3d6e..cf2f022 100644 --- a/README.md +++ b/README.md @@ -105,11 +105,12 @@ You've already seen how a Pretty Tensor is chainable and you may have noticed that it takes care of handling the input shape. One other feature worth noting are defaults. Using defaults you can specify reused values in a single place without having to repeat yourself. - - with pt.defaults_scope(activation_fn=tf.nn.relu): - hidden_output2 = (pretty_images.flatten() - .fully_connected(100) - .fully_connected(100)) +```py +with pt.defaults_scope(activation_fn=tf.nn.relu): + hidden_output2 = (pretty_images.flatten() + .fully_connected(100) + .fully_connected(100)) +``` Check out the documentation to see [all supported defaults](docs/pretty_tensor_top_level.md#defaults_scope). From 46aa0dd187a923545be976954f80af659260c8bb Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 23:01:35 +0530 Subject: [PATCH 7/8] Highlighted syntax for "Code matches model" section --- README.md | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index cf2f022..8dd7fc0 100644 --- a/README.md +++ b/README.md @@ -105,6 +105,7 @@ You've already seen how a Pretty Tensor is chainable and you may have noticed that it takes care of handling the input shape. One other feature worth noting are defaults. Using defaults you can specify reused values in a single place without having to repeat yourself. + ```py with pt.defaults_scope(activation_fn=tf.nn.relu): hidden_output2 = (pretty_images.flatten() @@ -121,32 +122,37 @@ Sequential mode lets you break model construction across lines and provides the subdivide syntactic sugar that makes it easy to define and understand complex structures like an [inception module](http://arxiv.org/abs/1409.4842): - - with pretty_tensor.defaults_scope(activation_fn=tf.nn.relu): - seq = pretty_input.sequential() - with seq.subdivide(4) as towers: - towers[0].conv2d(1, 64) - towers[1].conv2d(1, 112).conv2d(3, 224) - towers[2].conv2d(1, 32).conv2d(5, 64) - towers[3].max_pool(2, 3).conv2d(1, 32) +```py +with pretty_tensor.defaults_scope(activation_fn=tf.nn.relu): + seq = pretty_input.sequential() + with seq.subdivide(4) as towers: + towers[0].conv2d(1, 64) + towers[1].conv2d(1, 112).conv2d(3, 224) + towers[2].conv2d(1, 32).conv2d(5, 64) + towers[3].max_pool(2, 3).conv2d(1, 32) +``` ![Inception module showing branch and rejoin](inception_module.png) Templates provide guaranteed parameter reuse and make unrolling recurrent networks easy: - output = [], s = tf.zeros([BATCH, 256 * 2]) +```py +output = [], s = tf.zeros([BATCH, 256 * 2]) - A = (pretty_tensor.template('x') - .lstm_cell(num_units=256, state=UnboundVariable('state')) +A = (pretty_tensor.template('x') + .lstm_cell(num_units=256, state=UnboundVariable('state')) - for x in pretty_input_array: - h, s = A.construct(x=x, state=s) - output.append(h) +for x in pretty_input_array: + h, s = A.construct(x=x, state=s) + output.append(h) +``` There are also some convenient shorthands for LSTMs and GRUs: - pretty_input_array.sequence_lstm(num_units=256) +```py +pretty_input_array.sequence_lstm(num_units=256) +``` ![Unrolled RNN](unrolled_lstm.png) From a949571baa8ed4c3112017cb223b27fd2bf89046 Mon Sep 17 00:00:00 2001 From: Rishit Dagli <39672672+Rishit-dagli@users.noreply.github.com> Date: Sun, 17 May 2020 23:03:40 +0530 Subject: [PATCH 8/8] Highlighted syntax in "Extensible" section --- README.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8dd7fc0..b6537ee 100644 --- a/README.md +++ b/README.md @@ -161,7 +161,9 @@ pretty_input_array.sequence_lstm(num_units=256) You can call any existing operation by using `apply` and it will simply subsitute the current tensor for the first argument. - pretty_input.apply(tf.mul, 5) +```py +pretty_input.apply(tf.mul, 5) +``` You can also create a new operation There are two supported registration mechanisms to add your own functions. `@Register()` allows you to create a @@ -170,9 +172,11 @@ a new value. Name scoping and variable scoping are handled by the framework. The following method adds the leaky_relu method to every Pretty Tensor: - @pt.Register - def leaky_relu(input_pt): - return tf.select(tf.greater(input_pt, 0.0), input_pt, 0.01 * input_pt) +```py +@pt.Register +def leaky_relu(input_pt): + return tf.select(tf.greater(input_pt, 0.0), input_pt, 0.01 * input_pt) +``` `@RegisterCompoundOp()` is like adding a macro, it is designed to group together