(formerly titled: “Full-Stack Computer Projects”)
If you are reading this on mitchellmclaughlin.com, you are experiencing a technology stack (hardware, software, and design) I built using the tutorials listed below.
Table Of Contents: (more tutorials coming soon…)
- “Setting Up A Personal Web Server With The Raspberry Pi 3” (Appeared as “How To Set Up A Personal Web Server With A Raspberry Pi” on Opensource.com)
- “Building A Personal Website Using A Domain Name, GitHub Pages, Jekyll, Dynamic DNS (Domain Name Server), and Apache”
- “Securing A Website With HTTPS (Let’s Encrypt)”
- “Protecting A Website With CloudFlare”
- “Setting Up Web Analytics & Website Monitoring”
- “Creating A Custom 404 Page For Your Github Page Site”
- “Redirecting www Traffic To non-www With Apache”
- “Creating A Secure VPN (Virtual Private Network)”
- “Setting Up Cloud-based Storage”
- “Creating A Secure, Private E-Mail Server”
- “Basic Linux Commands”
- “Screenshots on Linux”
Setting Up A Personal Web Server With The Raspberry Pi 3
What is the value of building a personal web server?
A personal web server is “the cloud” essentially, except, in this case, you will own and control it. As opposed to a large corporation. And owning a little cloud has a lot of benefits. Customization. Free storage. Free internet services. A path into open-source software. High-quality security. Full control over your content. The ability to make quick changes. A place to experiment with code. And many more. Most of these benefits are immeasurable. But, financially these benefits can save you $100+/month.
AWS (Amazon Web Services) can give you some of these benefits for a small price. I could of used AWS. But, I prefer complete freedom, full control over security, and learning how things are built.
- Self Web-Hosting – No BlueHost or DreamHost
- HTTPS – Let’s Encrypt
- Analytics – Google
- OpenVPN – Do Not Need Private Internet Access $7/month
- Cloud Storage – No Dropbox, Box, Google Drive, Microsoft Azure, iCloud, or AWS
- On-Premise Security
Things I Used:
- Raspberry Pi 3 Model B
- MicroSD Card (32GB recommended, Raspberry Pi Compatible SD Cards)
- USB MicroSD Card Reader
- Ethernet Cable
- Router Connected To WiFi
- Raspberry Pi Case
- Amazon Basics MicroUSB Cable
- Apple Wall Charger
- USB Mouse
- USB Keyboard
- Amazon Basics HDMI Cable
- Monitor (With HDMI input)
- MacBook Pro Running OS X El Capitan
STEP 1: SETTING UP THE RASPBERRY PI
Download the most recent release of Raspbian (the Raspberry Pi operating system). Raspbian Jessie ZIP version is ideal. Unzip/extract the downloaded file. Now, you need to copy it onto the SD card. Pi Filler makes this process easy. Download Pi Filer 1.3 or the most recent version. Unzip/extract the downloaded file again and open it. You should be greeted with this prompt.
Make sure the USB card reader has NOT been inserted yet. If it has, eject it. Proceed by clicking continue. A file explorer should appear. Locate the uncompressed Raspberry Pi OS file from your Mac or PC and select it. You should see another prompt like the one pictured below.
Insert the MicroSD card (32GB recommended, 16GB minimum) into the USB MicroSD Card Reader. Then, insert the USB reader into the Mac or PC. You can rename the SD card to “Raspberry” to distinguish it from others. Click continue. Make sure the SD card is empty. Pi Filler will ERASE all previous storage at runtime. If you need to backup the card do it now. When you are ready to continue, the Raspbian OS will be written to the SD card. It should take between 1-3 minutes. Once the write is completed, eject the USB reader, remove the SD card and insert it into the Raspberry Pi SD card slot. Give the Raspberry Pi power by plugging the power cord into the wall. It should start booting up. The Raspberry Pi default login is:
When the Raspberry Pi has completed booting for the first time, a configuration screen titled “Setup Options” should appear like the image below.
Select the “Expand Filesystem” option and hit the enter key.
Also, I recommend selecting the 2nd option “change user password”. It is important for security. It also personalizes your Raspberry Pi.
Select the 3rd option in the setup options list, “Enable Boot To Desktop/Scratch” and hit the enter key. It will take you to another window titled “choose boot option” like the image below.
In the “choose boot option” window, select the 2nd option, “Desktop Log in as user ‘pi’ at the graphical desktop” and hit the enter button. Once this is done you will be taken back to the “Setup Options” page. If not, select the “OK” button at the bottom of this window and you will be taken back to the previous window.
Once both these steps are done, select the “finish” button at the bottom of the page and it should reboot automatically. If it does not, then use the following command in the terminal to reboot.
After the reboot from the previous step, if everything went right, you will end up on the desktop similar to the image below.
Once you are on the desktop, open a terminal and enter the following commands to update the firmware of the Raspberry Pi.
This may take a few minutes. Great, now the Raspberry Pi is up-to-date and running.
STEP 2: CONFIGURING THE RASPBERRY PI
SSH, which stands for Secure Shell, is a cryptographic network protocol that lets you securely transfer data between your computer and your Raspberry Pi. You can control your Raspberry Pi from your Mac’s command line without a monitor or keyboard.
To use SSH, first you need your Pi’s IP address. Open the terminal and type:
If you are using ethernet, look at the “eth0” section. If you are using wifi, look at the “wlan0” section.
Find “inet addr” followed by an IP address - something like 192.168.1.115, a common default IP I will use for the duration of this article.
With this address, open terminal and type:
SSH on PC, see footnote.
Enter the default password “raspberry” when prompted, unless you changed it.
You are now logged in via SSH.
Using a GUI (graphical-user interface) is easier than a command line. On the Raspberry Pi’s command line (using ssh) type:
Xrdp supports the Microsoft Remote Desktop Client for Mac and PC.
On Mac, navigate to the app store and search for “Microsoft Remote Desktop”. Download it. On PC, see footnote.
After installation, search your Mac for a program called “Microsoft Remote Desktop”. Open it. You should see this.
Click “New” to setup a remote connection. Fill in the blanks as shown below.
Save it by exiting out of the “New” window.
You should now see the remote connection listed under “My Desktops”. Double-click it.
After briefly loading, you should see your Raspberry Pi desktop in a window on your screen. Like this.
Perfect. Now, you don’t need a separate mouse, keyboard, or monitor to control the Pi. This is a much more elegant setup.
STATIC LOCAL IP ADDRESS
Sometimes the local IP address 192.168.1.115 will change. We need to make it static. Type:
Write down from the “eth0” section or the “wlan0” section, the “inet addr” (Pi’s current IP), the “bcast” (the broadcast IP range), and the “mask” (subnet mask address). Then, type:
Write down the “destination” and the “gateway/network”.
The culumlative records should look something like this:
With this information, it is easy to set a static internal IP. Type:
Do not use /etc/network/interfaces.
Then all you need to do is append this to the bottom of the file, substituting the correct IP address you want.
Once you have set the static internal IP address, reboot the Raspberry Pi with:
After rebooting, from terminal type:
Your new static settings should appear for your Raspberry Pi.
STATIC GLOBAL IP ADDRESS
If your ISP (internet service provider) already gives you a static external IP address you can skip ahead to the port forwarding section. If not, continue.
You set up SSH, remote desktop, and a static internal IP address. So, computers inside the local network will know where to find the Pi. But, you still can not access your Raspberry Pi from outside the local wifi network. You need your Raspberry Pi to be publicly accessible from anywhere on the internet. This requires a static external IP address.
It can be a fragile process initially. Call your ISP and request a static external (sometimes referred to as static global) IP address. The ISP holds the decision-making power, so I would be extremely careful dealing with them. They may refuse your static external IP address request. You can not fault the ISP. There is legal and operational risk with this type of request. They particularly do not want customers running medium-to-large scale internet services. They might explicitly ask why you need a static external IP address. It is probably best to be honest and tell them you plan on hosting a low-traffic personal website or a similar small not-for-profit service. If all goes well, they should open a ticket and call you in a week or 2 with an address.
Great, now you obtained a static global IP address. However, this IP address your ISP assigned is for accessing the router. The Raspberry Pi is still unreachable. You need to set up port forwarding to access the Raspberry Pi specifically.
Ports are virtual pathways where information travels on the internet. You sometimes need to forward a port in order to make a computer, like the Raspberry Pi, accessible to the internet since it is behind a network router. A YouTube video helped me visually understand ports.
Port forwarding can be used for projects like a Raspberry Pi web server, or applications like VoIP, or peer-to-peer downloading. There are 65,000+ ports to choose from, so you can assign a different port for every internet applcation you build.
The way to set up port forwarding can depend on your router. If you have a Linksys like I do, this video explains how to set it up. If you don’t have a Linksys, read the documentation that comes with your router in order to customize and define ports to forward.
You will need to port forward for SSH as well as the remote desktop.
Once you believe you have port forwarding configured, check to see if it is working via ssh by typing:
It should prompt you for the password.
Check to see if port forwarding is working for remote desktop as well. Open Microsoft Remote Desktop. Your previous remote connection settings should be saved. But, you need to update the “PC name” field with the static external IP address (for example, 22.214.171.124) instead of the static internal address (for example, 192.168.1.115).
Now, try connecting via remote desktop. It should briefly load and arrive at the Pi’s desktop.
Good job. The Raspberry Pi is now accessible from the internet and ready for advanced projects.
As a bonus option, you can maintain 2 remote connections to your Pi. One via the internet and the other via the LAN (local area network). It’s easy to setup. In Microsoft Remote Desktop, keep one remote connection called “Pi Internet” and another called “Pi Local”. Configure Pi Internet’s “PC name” to the static external IP address. For example, 126.96.36.199. Configure Pi Local’s, “PC name” to the static internal IP address. For exmaple, 192.168.1.115. Now, you have the option to connect golbally or locally.
Watch this awesome video as a transition into project 2. It will show you the techincal architecture behind your project. In this case, you are using a Raspberry Pi instead of a Ubuntu server. The dynamic DNS sits between the domain company and your router, which he leaves out. Beside this subtlety, the video is spot on explaining visually how the system works. You might notice project 1 covered the Raspberry Pi setup and port forwarding. This is the server-side or back end. Project 2 covers the the domain name, dynamic DNS, Jekyll (static HTML generator) and Apache (web hosting). This is the client-side or front end.
Building A Personal Website Using A Domain Name, GitHub Pages, Jekyll, Dynamic DNS (Domain Name Server), and Apache
GitHub Pages are websites built on Jekyll and versioned using a git hosted repository on GitHub. Jekyll is a simple, blog aware, static site generator. It takes Markdown, a lightweight markup language, and spits out a complete, static HTML website suitable for serving with Apache or similar web server.
Let’s get started by installing Ruby. Ruby is a programming language that uses a package manager called gems. Gems can install Jekyll. From your Raspberry Pi, in terminal type (using ssh or remote desktop):
If ‘gem install jekyll’ fails try:
You can find Jekyll documentation here. Once you have Jekyll installed, you will want to create a directory for your blog.
If ‘jekyll new myblog’ fails, again try:
Navigate to the myblog directory by typing:
If you plan on writing to your blog, I recommend installing git. The blog will evolve over time as content is added. Git creates a snapshot of your blog at some point in time. So, you have the option to rollback a mistake or revert to some point in the past. Git is an amazing tool. It has become widely-accepted as a standard for version control of software. Version control is a far-reaching idea itself. It’s a solution to capture the evolution of an information system over time. For exmaple, writing a book, producing a movie, or writing software. It has endless applications in the arts. I witnessed organizations using it as a communication tool. I could forsee it being applied in genetic engineering. If Microsoft Word or PowerPoint crashes, you might recall the ability to recover a document from an earlier time. This is the same underlying version control technology. I could go on. Anyways, to begin install git, type:
Add the current folder to git by typing:
Once you have git installed and made the initial commit, create a blog home page by typing:
In index.html, type “hello world”. Save the file by hitting control+o (for Mac), then enter. Followed by control+x to exit. Changes have been made so don’t forget to reflect those changes to git. Type:
To test if hello world is working, type:
This command tells Jekyll to serve myblog to the local network. You can test your web server by opening a web browser on your server and heading to the URL http://127.0.0.1/.
Check that you can see the website from http://127.0.0.1/. If you see the site, continue. If your server does not run, you may need to reconfigure.
Great, everything is installed and you have generated your site locally. This is a good checkpoint. Moving on, you will install Apache to generate your site to a domain name. Type:
Apache HTTP Server is the most popular web server technology in the world. As of November 2015, it serves 50% of all websites. It also happens to be the easiest to implement. When I was setting up my web server at first, I mistakenly thought I needed a LAMP architecture (pictured below). The solution is actually much simpler with Apache.
Create a new user account with this free, open-source dynamic DNS service here. Write down your user name and password. Once logged in, click the “domains” tab in the navigation panel on the left. You should be taken to this screen (pictured below).
Add your domain name. Once your domain is saved, click “edit secondaries”. You should see a list of name servers (pictured below).
Write down the first 2 name servers. For example, ns1.afraid.org and ns2.afraid.org.
Click “Dynamic DNS” in the navigation panel on the left. You should see a table appear at the bottom of the page (pictured below).
Again, if your domain name is not listed, add it. Then, click “edit record”. Fill in the fields.
Repeat this process for the “edit record” of www.yourdomain.com.
DNS settings are now configured. These changes also need to be updated with your domain name provider (Google Domains, GoDaddy, or other).
I used GoDaddy. Upon logging in, I reached this screen (pictured below).
In the domain row, click “Manage”. This is the screen I arrived at.
Click the gear icon and select “Manage DNS”. Then, update the name servers (pictured below).
Allow up to 24 hours for the changes to be processed. After the waiting period, you can test the new configuration by typing:
If all goes well, the pings should return data packets from your global IP address (pictured below).
To stop pinging, hit control+c.
Your domain name is now configured with a dynamic DNS.
The last hurdle is connecting all these pieces together, the domain name on top of the dynamic DNS (server-side), the GitHub Page on top of Jekyll and Apache (client-side).
Apache uses a VirtualHost to serve HTML files to the web. Some minor configurations are needed to tell it where to find the HTML files. From the Raspberry Pi, open terminal and type:
Copy the configurations (pictured below).
Repeat this process for the encrypted VirtualHost.
Copy the configurations (pictured below) except for the Let’s Encrypt section.
For security reasons, the Apache web server does not listen to port 80 (http) or port 443 (https) by default. You need to open these for incoming web traffic requests. Type:
Your shell should return something similar to this (pictured below).
If you want to understand iptables, read the documentation.
To test ports at anytime, go to a browser and type your_domain_name:80 or your_domain_name:443.
Port 443 probably will not work yet since HTTPS is not set up. Port 80 should work. We will get to HTTPS in a later tutorial.
You need build your website with Jekyll in the root directory of Apache. Navigate to the myblog directory. Type:
This is Jekyll’s way of spiting out the HTML documents to Apache’s root directory to be served to the web. The command line should return something like this (pictured below).
Enter your domain name into a browser as a comprehensive final test. If you did it right, it should reveal your website. This was mine after completion (pictured below).
If you got this far, you are doing a hell of a job. Keep it up.
If you ever want to change the design of your website, check out this list of examples.
Watching this video again might crystallize your understanding of how the system works.
Securing A Website With HTTPS (Let’s Encrypt)
Let’s Encrypt is a certificate authority that provides free certificates for transport layer security (TLS) encryption via an automated process.
Have you ever landed on a red-padlocked webpage? It’s because HTTPS is not enabled.
In order to install Let’s Encrypt on your server, you need to have the git package installed. If you installed git during the <a href=”#project2”Jekyll tutorial</a>, skip this step. Otherwise, open terminal and type:
You need to navigate to the location you want to install Let’s Encrypt. I recommend /myblog. Type:
Start the process of obtaining a SSL Certificate for Apache by typing:
This should launch you into the blue setup screen.
Agree to the Terms of Service. Enter your email address.
“No names were found in your configuration files.” Continue by selecting “Yes”.
Enter your domain names separated by commas. For example, I entered “mitchellmclaughlin.com, www.mitchellmclaughlin.com”.
Select and modify option #3 “000-default-le-ssl.conf” file as shown below.
Choose whether HTTPS access is required or optional. I recommend using “Secure”.
If the process worked properly, you will receive a follow-up message.
You should also receive a congratulatory message and a URL to test your HTTPS.
Test your domain.
At this point, you have one final job. The certificates issued by Let’s Encrypt expire every 90 days for security reasons. When they expire, you can execute a simple command to renew certificates. Type:
Your certificates should be up to date. Success.
Protecting A Website With CloudFlare
CloudFlare is an internet security service. It partly functions like a content delivery network (CDN). A content delivery network is a fancy term to describe a software networking technology which stores assets of a website like text, images, and other metadata locally for a geographic area. It speeds up user-side access to your website. CloudFlare also protects your website from unwanted threats like a distributed denial-of-service (DDoS) attack. For DDoS attacks, CloudFlare functions like a reverse proxy load-balancer against incoming web requests.
Let’s get CloudFlare setup now. Go to cloudflare.com. Create a new user account. It should then prompt you for your domain name. Continue by verifying your DNS records. Select a CloudFlare plan. I chose the free/basic plan. If all goes well, it should return CloudFlare name servers. Similar to my name servers pictured below.
Change your old DNS name servers to the new CloudFlare name servers. Once you make the change, allow up to 24 hours for processing.
Setting Up Web Analytics & Website Monitoring
I like data. Web analytics is an easy way to learn more about your audience. If you try something new on your website, it’s nice to receive feedback. Over time, analytics will help you feed the audience’s desires.
I considered Google Analytics, Piwik, and Hotjar. Piwik was clunky and hard to implement. Hotjar was missing basic website metrics. So, I chose Google Analytics. It’s easy to implement and lightweight.
Go to Google Analytics. Create an account. If you already have a gmail account you might be able to sign in with it. After signing up, you will be given a “tracking code”. This is code to be embedded into the HTML code of your website. Embed it. It should take a few hours to process. Once the waiting period is over, you can start creating dashboards with a variety of metrics.
Website monitoring can be used as a notification tool to alert you when your website crashes. I use UptimeRobot. Create an account and link your domain and you will be alerted if something crashes.
Redirecting www Traffic To non-www With Apache
Shortly after I got my website working, I tested a variety of URL entry points. For example, mitchellmclaughlin.com, http://mitchellmclaughlin.com, https://mitchellmclaughlin.com, and mitchellmclaughlin.com:80 all correctly redirected to https://mitchellmclaughlin. But, www.mitchellmclaughlin.com and http://www.mitchellmclaughlin.com both landed at www.mitchellmclaughlin.com. Although viewable, there is a subtle problem. In the latter instances, HTTPS is not enabled. Ideally, all www traffic should be redirected to mitchellmclaughlin.com which further redirects to https://mitchellmclaughlin.com (a HTTPS enabled URL). The www and non-www discrepancy orignated from the invention of the world wide web. And “Why Do Some Websites Include www In The URL While Others Don’t?”.
Fortunately, the solution is quick and easy. I will warn you, I strongly advise against using the RewriteEngine solution. In the Pi terminal, type:
Find “</VirtualHost>”, above that line type and insert:
The results should look like my configuration settings below.
Save the changes to the file. There is nothing magical about doing it in the other direction. Simply swap www.youdomain and yourdomain.
Please remember, if you have configured Apache differently than I have in the preceeding projects this probably will not work.
After the changes have been saved, restart Apache.
Test your new configuration settings by typing the www URLs into a browser. It if worked, you are successful.
I purposefully did not mention https://www.mitchellmclaughlin.com because as far as I know, it is not a valid http request. Same goes for mitchellmclaughlin.com:443.
Setting Up Cloud-based Storage
Things I Used:
- Seagate 1TB External Hard Drive
- USB 2.0 Powered Hub
First, decide what you will be using for external storage. I picked a 1TB USB hard drive from Seagate. The Raspberry Pi does not have enough power to run most external hard drives or SSDs, so unless you have a drive with its own power supply, you will probably need a powered USB hub. Once you have the supplies, plug in the USB powered hub into a USB port on the Raspberry Pi. Then, plug in the external hard drive into a USB port on the USB powered hub. A folder should appear on the desktop of the Raspberry Pi. That’s it. You can now download and upload files to this folder on-demand. To increase the amount of storage, simply add more external hard drives.
Basic Linux Commands
How To Take Screenshots On Raspberry Pi
To take screenshots, open terminal and execute:
Here is how you would run scrot with a 10 second delay:
If you want to contribute to this project, liked it, got stuck, or found errors, drop me a message at mitch(dot)mclaughlin1(at)gmail(dot)com.
 I do not recommend starting with the NOOBS operating system. I prefer starting with the fully-functional Raspbian Jessie operating system.
 If “Setup Options” did not pop-up, you can always find it by opening terminal and executing this command:
 We do this to make use of all the space present on the SD card as a full-partition. All this does is, expand the operating system to fit the whole space on the SD card which can then be used as storage memory for the Raspberry Pi.
 We do this because we want to boot into the desktop environment which we are familiar with. If we do not do this step, the Raspberry Pi boots into a terminal each time with no graphical-user interface.
Download and run PuTTY or another SSH client for Windows. Enter your IP address in the field, as shown in the above screenshot. Keep the default port at 22. Hit enter, and PuTTY will open a terminal window which will prompt you for your username and password. Fill those in, and begin working remotely on your Pi.
 If it is not already installed, download Microsoft Remote Desktop. Search your computer for Microsoft Remote Desktop. Run it. Input the IP address when prompted. Next, an xrdp window will pop up, prompting you for your username and password.
 The router has a dynamically assigned external IP address. So in theory, it can be reached from the internet momentarily. But, we need the help of your ISP to make it permanently accessible. If this was not the case, the remote connection would need to be reconfigured on each use.
Orsini, Lauren. Raspberry Pi Projects SSH Remote Desktop Static IP Tutorial
Pai, Akshay. How To Install Raspbian On Raspberry Pi