|
| 1 | +# MobileNet |
| 2 | + |
| 3 | +MobileNets are small, low-latency, low-power models parameterized to meet the resource constraints of a variety of use cases. They can be built upon for classification, detection, embeddings and segmentation similar to how other popular large scale models, such as Inception, are used. |
| 4 | + |
| 5 | +MobileNets trade off between latency, size and accuracy while comparing favorably with popular models from the literature. |
| 6 | + |
| 7 | +This TensorFlow.js model does not require you to know about machine learning. |
| 8 | +It can take as input any browser-based image elements (`<img>`, `<video>`, `<canvas>` |
| 9 | +elements, for example) and returns an array of most likely predictions and |
| 10 | +their confidences. |
| 11 | + |
| 12 | +For more information about MobileNet, check out this readme in |
| 13 | +[tensorflow/models](https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md). |
| 14 | + |
| 15 | +## Usage |
| 16 | + |
| 17 | +There are two main ways to get this model in your JavaScript project: via script tags or by installing it from NPM and using a build tool like Parcel, WebPack, or Rollup. |
| 18 | + |
| 19 | +### via Script Tag |
| 20 | + |
| 21 | +```html |
| 22 | +<!-- Load TensorFlow.js. This is required to use MobileNet. --> |
| 23 | +<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.10.3"> </script> |
| 24 | +<!-- Load the MobileNet model. --> |
| 25 | +<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet@0.1.0"> </script> |
| 26 | + |
| 27 | +<!-- Replace this with your image. Make sure CORS settings allow reading the image! --> |
| 28 | +<img id="img" src="cat.jpg"></img> |
| 29 | + |
| 30 | +<!-- Place your code in the script tag below. You can also use an external .js file --> |
| 31 | +<script> |
| 32 | + // Notice there is no 'import' statement. 'mobilenet' and 'tf' is |
| 33 | + // available on the index-page because of the script tag above. |
| 34 | +
|
| 35 | + const img = document.getElementById('img'); |
| 36 | +
|
| 37 | + // Load the model. |
| 38 | + mobilenet.load().then(model => { |
| 39 | + // Classify the image. |
| 40 | + model.classify(img).then(predictions => { |
| 41 | + console.log('Predictions: '); |
| 42 | + console.log(predictions); |
| 43 | + }); |
| 44 | + }); |
| 45 | +</script> |
| 46 | +``` |
| 47 | + |
| 48 | +### via NPM |
| 49 | + |
| 50 | +```js |
| 51 | +// Note: you do not need to import @tensorflow/tfjs here. |
| 52 | + |
| 53 | +import * as mobilenet from '@tensorflow-models/mobilenet'; |
| 54 | + |
| 55 | +const img = document.getElementById('img'); |
| 56 | + |
| 57 | +// Load the model. |
| 58 | +const model = await mobilenet.load(); |
| 59 | + |
| 60 | +// Classify the image. |
| 61 | +const predictions = await model.classify(img); |
| 62 | + |
| 63 | +console.log('Predictions: '); |
| 64 | +console.log(predictions); |
| 65 | +``` |
| 66 | + |
| 67 | +## API |
| 68 | + |
| 69 | +#### Loading the model |
| 70 | +`mobilenet` is the module name, which is automatically included when you use |
| 71 | +the <script src> method. When using ES6 imports, mobilenet is the module. |
| 72 | + |
| 73 | +```ts |
| 74 | +mobilenet.load( |
| 75 | + version?: 1, |
| 76 | + alpha?: 0.25 | .50 | .75 | 1.0 |
| 77 | +) |
| 78 | +``` |
| 79 | + |
| 80 | +Args: |
| 81 | +- **version:** The MobileNet version number. Currently only accepts and defaults to version 1. In the future we will support MobileNet V2. |
| 82 | +- **alpha:** Controls the width of the network, trading accuracy for performance. A smaller alpha decreases accuracy and increases performance. Defaults to 1.0. |
| 83 | + |
| 84 | +Returns a `model` object. |
| 85 | + |
| 86 | +#### Making a classification |
| 87 | + |
| 88 | +You can make a classification with mobilenet without needing to create a Tensor |
| 89 | +with `MobileNet.classify`, which takes an input image element and returns an |
| 90 | +array with top classes and their probabilities. |
| 91 | + |
| 92 | +If you want to use this for transfer learning, see the `infer` method. |
| 93 | + |
| 94 | +This method exists on the model that is loaded from `mobilenet.load`. |
| 95 | + |
| 96 | +```ts |
| 97 | +model.classify( |
| 98 | + img: tf.Tensor3D | ImageData | HTMLImageElement | |
| 99 | + HTMLCanvasElement | HTMLVideoElement, |
| 100 | + topk?: number |
| 101 | +) |
| 102 | +``` |
| 103 | + |
| 104 | +Args: |
| 105 | +- **img:** A Tensor or an image element to make a classification on. |
| 106 | +- **topk:** How many of the top probabilities to return. Defaults to 3. |
| 107 | + |
| 108 | +Returns an array of classes and probabilities that looks like: |
| 109 | + |
| 110 | +```js |
| 111 | +[{ |
| 112 | + className: "Egyptian cat", |
| 113 | + probability: 0.8380282521247864 |
| 114 | +}, { |
| 115 | + className: "tabby, tabby cat", |
| 116 | + probability: 0.04644153267145157 |
| 117 | +}, { |
| 118 | + className: "Siamese cat, Siamese", |
| 119 | + probability: 0.024488523602485657 |
| 120 | +}] |
| 121 | +``` |
| 122 | + |
| 123 | +#### Getting activations |
| 124 | + |
| 125 | +You can also use this model to get intermediate activations or logits as |
| 126 | +TensorFlow.js tensors. |
| 127 | + |
| 128 | +This method exists on the model that is loaded from `mobilenet.load`. |
| 129 | + |
| 130 | +```ts |
| 131 | +model.infer( |
| 132 | + img: tf.Tensor3D | ImageData | HTMLImageElement | |
| 133 | + HTMLCanvasElement | HTMLVideoElement, |
| 134 | + endpoint?: string |
| 135 | +) |
| 136 | +``` |
| 137 | + |
| 138 | +- **img:** A Tensor or an image element to make a classification on. |
| 139 | +- **endpoint:** The optional endpoint to predict through. You can list all the endpoints with `model.endpoint`. These correspond to layers of the MobileNet model. If undefined, will return 1000D unnormalized logits. |
0 commit comments