{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"execution": {},
"id": "view-in-github"
},
"source": [
"
"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"# Bonus Tutorial: Deploying Neural Networks on the Web\n",
"\n",
"**By Neuromatch Academy**\n",
"\n",
"__Content creators:__ Sam Ray, Vladimir Haltakov, Konrad Kording\n",
"\n",
"__Production editors:__ Spiros Chavlis"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"---\n",
"# Tutorial Objectives\n",
"\n",
"In this tutorial, you will learn the basics of how to deploy your deep learning models as web applications using some modern frameworks and libraries. In this tutorial, you will learn to:\n",
"\n",
" - Serve web pages with Flask\n",
" - Apply the MVVM design pattern to write maintainable code\n",
" - Create an interactive UI for your service\n",
" - Deploy your deep learning models as a REST API\n",
" - Deploying your application on Heroku"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"# @markdown\n",
"from IPython.display import IFrame\n",
"from ipywidgets import widgets\n",
"out = widgets.Output()\n",
"with out:\n",
" print(f\"If you want to download the slides: https://osf.io/download/p6wty/\")\n",
" display(IFrame(src=f\"https://mfr.ca-1.osf.io/render?url=https://osf.io/p6wty/?direct%26mode=render%26action=download%26mode=render\", width=730, height=410))\n",
"display(out)"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"---\n",
"# Setup\n",
"\n",
"Run the following cells to install and include important dependencies."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install dependencies\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"# @title Install dependencies\n",
"!pip install --upgrade jupyter-client --quiet\n",
"!pip install Flask-RESTful flasgger pyngrok --quiet"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install and import feedback gadget\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"# @title Install and import feedback gadget\n",
"\n",
"!pip3 install vibecheck datatops --quiet\n",
"\n",
"from vibecheck import DatatopsContentReviewContainer\n",
"def content_review(notebook_section: str):\n",
" return DatatopsContentReviewContainer(\n",
" \"\", # No text prompt\n",
" notebook_section,\n",
" {\n",
" \"url\": \"https://pmyvdlilci.execute-api.us-east-1.amazonaws.com/klab\",\n",
" \"name\": \"neuromatch_dl\",\n",
" \"user_key\": \"f379rz8y\",\n",
" },\n",
" ).render()\n",
"\n",
"\n",
"feedback_prefix = \"Bonus_DeplooyModels\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Modified functions from the `flask-ngrok` package which doesn't work with the latest version of `ngrok`\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"# @title Modified functions from the `flask-ngrok` package which doesn't work with the latest version of `ngrok`\n",
"import time\n",
"import json\n",
"import atexit\n",
"import requests\n",
"import subprocess\n",
"from threading import Timer\n",
"\n",
"\n",
"def _run_ngrok(port):\n",
" ngrok = subprocess.Popen([\"ngrok\", 'http', str(port)])\n",
" atexit.register(ngrok.terminate)\n",
" localhost_url = \"http://localhost:4040/api/tunnels\" # Url with tunnel details\n",
" time.sleep(1)\n",
" tunnel_url = requests.get(localhost_url).text # Get the tunnel information\n",
" j = json.loads(tunnel_url)\n",
"\n",
" tunnel_url = j['tunnels'][0]['public_url'] # Do the parsing of the get\n",
" return tunnel_url\n",
"\n",
"\n",
"def start_ngrok(port):\n",
" ngrok_address = _run_ngrok(port)\n",
" print(f\" * Running on {ngrok_address}\")\n",
" print(f\" * Traffic stats available on http://127.0.0.1:4040\")\n",
"\n",
"def run_with_ngrok(app):\n",
" \"\"\"\n",
" The provided Flask app will be securely exposed to the public internet\n",
" via ngrok when run, and the its ngrok address will be printed to stdout\n",
" :param app: a Flask application object\n",
" :return: None\n",
" \"\"\"\n",
" old_run = app.run\n",
"\n",
" def new_run(*args, **kwargs):\n",
" port = kwargs.get('port', 5000)\n",
" thread = Timer(1, start_ngrok, args=(port,))\n",
" thread.setDaemon(True)\n",
" thread.start()\n",
" old_run(*args, **kwargs)\n",
" app.run = new_run"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"execution": {}
},
"outputs": [],
"source": [
"# Imports\n",
"import io\n",
"import platform\n",
"from PIL import Image\n",
"from urllib.request import urlopen\n",
"\n",
"import flasgger\n",
"from flask_restful import Api\n",
"from flask_restful import Resource, fields, marshal\n",
"from flask import Flask, render_template_string, request, redirect\n",
"\n",
"import torch\n",
"from torchvision import models\n",
"import torchvision.transforms as transforms"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"## Setup ngrok\n",
"\n",
"In order to be able to access the web app running in the notebook, we need to use a service called `ngrok`. Since recently, `ngrok` requires the user to register for a free account and setup an authentication token. Follow the steps below to set it up.\n",
"\n",
"1. Go to [ngrok.com](https://ngrok.com/) and create a free account. Do not forget to verify your e-mail address right after!\n",
"2. Go to [dashboard.ngrok.com/get-started/your-authtoken](https://dashboard.ngrok.com/get-started/your-authtoken) and copy your authtoken.\n",
"3. Paste it in the cell below by replacing `YOUR_NGROK_AUTHTOKEN`, uncomment the last line and run it.\n",
"\n",
"You should see this output:\n",
"\n",
"```python\n",
"Authtoken saved to configuration file: /root/.ngrok2/ngrok.yml\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"execution": {}
},
"outputs": [],
"source": [
"# Paste your ngrok authtoken below and run the cell\n",
"\n",
"## Uncomment the line below\n",
"# !ngrok authtoken YOUR_NGROK_AUTHTOKEN"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"**Note:** If you want to delete at some point your account, visit this page [here](https://dashboard.ngrok.com/user/settings) and at the bottom of the page click on `Delete User`."
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"---\n",
"# Section 1: Introduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Video 1: Deploying Neural Networks on the Web\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"# @title Video 1: Deploying Neural Networks on the Web\n",
"from ipywidgets import widgets\n",
"from IPython.display import YouTubeVideo\n",
"from IPython.display import IFrame\n",
"from IPython.display import display\n",
"\n",
"\n",
"class PlayVideo(IFrame):\n",
" def __init__(self, id, source, page=1, width=400, height=300, **kwargs):\n",
" self.id = id\n",
" if source == 'Bilibili':\n",
" src = f'https://player.bilibili.com/player.html?bvid={id}&page={page}'\n",
" elif source == 'Osf':\n",
" src = f'https://mfr.ca-1.osf.io/render?url=https://osf.io/download/{id}/?direct%26mode=render'\n",
" super(PlayVideo, self).__init__(src, width, height, **kwargs)\n",
"\n",
"\n",
"def display_videos(video_ids, W=400, H=300, fs=1):\n",
" tab_contents = []\n",
" for i, video_id in enumerate(video_ids):\n",
" out = widgets.Output()\n",
" with out:\n",
" if video_ids[i][0] == 'Youtube':\n",
" video = YouTubeVideo(id=video_ids[i][1], width=W,\n",
" height=H, fs=fs, rel=0)\n",
" print(f'Video available at https://youtube.com/watch?v={video.id}')\n",
" else:\n",
" video = PlayVideo(id=video_ids[i][1], source=video_ids[i][0], width=W,\n",
" height=H, fs=fs, autoplay=False)\n",
" if video_ids[i][0] == 'Bilibili':\n",
" print(f'Video available at https://www.bilibili.com/video/{video.id}')\n",
" elif video_ids[i][0] == 'Osf':\n",
" print(f'Video available at https://osf.io/{video.id}')\n",
" display(video)\n",
" tab_contents.append(out)\n",
" return tab_contents\n",
"\n",
"\n",
"video_ids = [('Youtube', 'yQtPGtz4jDI'), ('Bilibili', 'BV1754y1E7Qf')]\n",
"tab_contents = display_videos(video_ids, W=730, H=410)\n",
"tabs = widgets.Tab()\n",
"tabs.children = tab_contents\n",
"for i in range(len(tab_contents)):\n",
" tabs.set_title(i, video_ids[i][0])\n",
"display(tabs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Submit your feedback\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"# @title Submit your feedback\n",
"content_review(f\"{feedback_prefix}_Deploying_Neural_Networks_on_the_Web_Video\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"We will start by building a simple web application in Flask, which we'll keep extending throughout the tutorial. In the end, you will have a web app where you can upload an image and have it classified automatically by a neural network model."
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"---\n",
"# Section 2: Flask"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"Flask is a web application micro-framework built with Python. Flask is popular because it's lightweight, easy to use, scalable, and has tons of great extensions. Nowadays, Flask is used for many different applications like web applications, REST APIs, socket-based services, and by companies like LinkedIn or Pinterest.\n",
"\n",
"In this section, you will learn to create simple Flask websites."
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"## Section 2.1: Your First Flask App\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Video 2: Flask\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"# @title Video 2: Flask\n",
"from ipywidgets import widgets\n",
"from IPython.display import YouTubeVideo\n",
"from IPython.display import IFrame\n",
"from IPython.display import display\n",
"\n",
"\n",
"class PlayVideo(IFrame):\n",
" def __init__(self, id, source, page=1, width=400, height=300, **kwargs):\n",
" self.id = id\n",
" if source == 'Bilibili':\n",
" src = f'https://player.bilibili.com/player.html?bvid={id}&page={page}'\n",
" elif source == 'Osf':\n",
" src = f'https://mfr.ca-1.osf.io/render?url=https://osf.io/download/{id}/?direct%26mode=render'\n",
" super(PlayVideo, self).__init__(src, width, height, **kwargs)\n",
"\n",
"\n",
"def display_videos(video_ids, W=400, H=300, fs=1):\n",
" tab_contents = []\n",
" for i, video_id in enumerate(video_ids):\n",
" out = widgets.Output()\n",
" with out:\n",
" if video_ids[i][0] == 'Youtube':\n",
" video = YouTubeVideo(id=video_ids[i][1], width=W,\n",
" height=H, fs=fs, rel=0)\n",
" print(f'Video available at https://youtube.com/watch?v={video.id}')\n",
" else:\n",
" video = PlayVideo(id=video_ids[i][1], source=video_ids[i][0], width=W,\n",
" height=H, fs=fs, autoplay=False)\n",
" if video_ids[i][0] == 'Bilibili':\n",
" print(f'Video available at https://www.bilibili.com/video/{video.id}')\n",
" elif video_ids[i][0] == 'Osf':\n",
" print(f'Video available at https://osf.io/{video.id}')\n",
" display(video)\n",
" tab_contents.append(out)\n",
" return tab_contents\n",
"\n",
"\n",
"video_ids = [('Youtube', 'uVqu-9IBIRg'), ('Bilibili', 'BV1sA411P7Rq')]\n",
"tab_contents = display_videos(video_ids, W=730, H=410)\n",
"tabs = widgets.Tab()\n",
"tabs.children = tab_contents\n",
"for i in range(len(tab_contents)):\n",
" tabs.set_title(i, video_ids[i][0])\n",
"display(tabs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Submit your feedback\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"execution": {},
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"# @title Submit your feedback\n",
"content_review(f\"{feedback_prefix}_Flask_Video\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"execution": {}
},
"source": [
"Creating a minimal Flask app is very simple. You need to create a `Flask` object and define the handler for the root URL returning the HTML response. You need to provide the applications module or package, but we can use `__name__` as a convenient shortcut.\n",
"\n",
"We need one small trick because the app will be running in a notebook. If you just run the app, it will be accessible at `http://127.0.0.1:5000`. The problem is that this is a local address to the server where the notebook is running, so you can't access it. This is where `ngrok` helps - it creates a tunnel from the notebook server to the outside world. Make sure you use the ngrok URL when testing your app.\n",
"\n",
"Uncommenting the `app.run()` command below, you will see this output:\n",
"\n",
"```\n",
" * Serving Flask app \"__main__\" (lazy loading)\n",
" * Environment: production\n",
" WARNING: This is a development server. Do not use it in a production deployment.\n",
" Use a production WSGI server instead.\n",
" * Debug mode: off\n",
" * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)\n",
" * Running on https://XXX-XX-XX-XXX-XXX.ngrok.io\n",
" * Traffic stats available on http://127.0.0.1:4040\n",
"```\n",
"\n",
"The URL you have to visit is: `https://XXX-XX-XX-XXX-XXX.ngrok.io`\n",
"\n",
"
\n",
"\n",
"> **Note:** the call to `app.run()` will not return on its own. Make sure to stop the running cell when you want to move to the next one. You can do this by either clicking on the cell run/stop button or through _settings>Interrupt runtime_ or Ctrl+M+I.\n",
"\n",
"> **Warning:** if the \"visit site\" option is not working, check the URL. `ngrok` redirects some links from `https` to `http`, so you have to manually make the change!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"execution": {}
},
"outputs": [],
"source": [
"# Create a Flask app object\n",
"app = Flask(__name__)\n",
"\n",
"# Define a function to be called when the user accesses the root URL (/)\n",
"# Handler\n",
"@app.route(\"/\")\n",
"def home():\n",
" # Return a very simple HTML response\n",
" return \"
Property | \n", "Value | \n", "
---|---|
{{ key }} | \n", "{{ value }} | \n", "