Setting up AWS EC2 for Running Jupyter Notebook on GPU in the Cloud

5 min readMar 26, 2018

I set up an Amazon Elastic Compute Cloud (EC2) instance on AWS to run Jupyter Notebook on GPU as part of my study with Udacity Deep Learning Nano Degree. I’m sharing with you my notes to help with your setup in case you decide to choose AWS for running Jupyter Notebook on GPU in the cloud. Take a look at my post about Floydhub here and next up I will write about setup on Google Cloud.

High level summary

It’s helpful to see a high level summary of the setup to get an idea of what is going on before diving into the details, especially since the setup takes many steps. So here is the setup summary:

Change region to US East (N. Virginia)
Increase service limits for p2.xlarge instance type (by default it’s 0)

2. Create an instance and configure instance image and type

3. Set up security group to configure port 8888 in order to access the Jupyter notebook

4. Create key pair for authentication

5. Launch instance from EC2console, and then connect to instance via command line, with the .pem file and instance ip address

7. Configure Jupyter Notebook settings

8. Download source code and install any necessary tools such as Keras, Tensorflow etc

9. Start Jupyter Notebook, do your work, then remember to stop the instance when done.

Detailed step by step

Here is the detailed steps I took:

Log in to AWS

Change region to US East (N. Virginia)
View EC2 Service Limits — click on EC2/EC2 Dashboard/Limits or directly go to this link https://console.aws.amazon.com/ec2/
Find p2.xlarge instance type and click on “Request limit increase” and set it to 1 or a number higher than 1.

Wait for approval and you will get an email notification on this.

Configure instance — select image

Go back to EC2 Management Console,

Click on the dropdown next to the “Launch Instance” button
Select “Launch instance”. (Note even though it says “Launch instance”, you can’t really launch it until you set the instance image and instance type etc. and finish all the configuration steps below. )

Click on AWS Marketplace
Search for Deep Learning AMI with Source Code (CUDA 8, Ubuntu) and click on select.

Configure instance — select instance type

Afterwards I see a window with pricing details with a very long list which I’m not sure what is the sorting order and I can’t seem to see the p2.xlarge in the list. So I just click on “Continue”.
Filter by “GPU compute”
Select type = p2.xlarge

Click on “Review and Launch” then you will jump to step 7. Review

Next we need to set up a security group.

Configure instance — set up Security Group

Click on “Edit Security groups” on “step 7. Review” tab, or click on the 6. Configure Security Group tab. Note in order to access the Jupyter notebook, must configure to allow access to port 8888.

So add a new rule as follows:

Type = Custom TCP
Protocol=TCP
Port Range=8888
Source=Anywhere

Then click on Review and Launch which will take you to “step 7. Review” again

Launch instance-create key pair for authentication

Click on Launch
You will be prompted to select or create a new key pair
Select “create new key pair” if you don’t have one already, and give it a name

Click on Download Key Pair button then a .pem file will be downloaded
Click on “Launch Instance” and now your instance is actually finally launching!
(Optional) click on billing alerts to set up email alerts

Login to the Instance

When the status Checks changed from “initializing” to “2/2 checks”, the instance is ready for log in. Navigate to where the .pem file is located and in terminal, type the command to connect to your AWS instance:

ssh -i [key-name].pem ubuntu@[xx.xx.xxx.xx]

If you get error like this:

Permissions 0644 for ‘key-name.pem’ are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key “[key-name].pem”: bad permissions
ubuntu@52.23.237.74: Permission denied (publickey)

Then update the permission of the .pem as follows:

chmod 400 key-name.pem

Try connect again:

ssh -i [key-name].pem ubuntu@[xx.xx.xxx.xx]

And hopefully now you are connected to your AWS EC2 instance.

Configure Jupyter notebook settings

While connected to your ASW instance, at terminal type

jupyter notebook --generate-config

Then you will see -

Writing default config to: /home/ubuntu/.jupyter/jupyter_notebook_config.py

Use the instance

Clone source
Install and or upgrade any required tools: Numpy, Keras & tensorflow etc.
Start Jupyter Notebook from command line

jupyter notebook --ip=0.0.0.0 --no-browser

Copy/pate the url to browser but replace the 0.0.0.0 before :8888 with your instance ip address (Ipv4 Public IP)

Stop the instance

Make sure to stop the instance to avoid charges!

Click on Actions/Instance State/Stop to stop the instance.

Note: if you don’t need the instance at all in the future, remember to terminate it in order to avoid being charged for the storage (EBS volume).

Start the instance again

Now that I had all the configuration done, here is how to start the instance again after it’s stopped. These steps are repeat of some of the previous steps, without those one-time initial setup/configuration etc.

Click on Actions/Instance State/Start
Wait for the state to go from initialize to “2/2 checks”, then note the instance ip address
At terminal use command to login ssh -i [key-name].pem ubuntu@[xx.xx.xxx.xx]
Navigate to your source code
Launch jupyter notebook-

jupyter notebook --ip=0.0.0.0 --no-browser

Copy/pate the url to browser but replace the 0.0.0.0 before :8888 with your instance ip address (Ipv4 Public IP)

References:

Udacity Deep Learning Nano degree, Part 3 CNN, Lesson 2 CNN, Section 1 Cloud Computing
Running Jupyter notebooks on GPU on AWS: a starter guide