About

What is this?

A FREE, no-bullshit, step-by-step guide how web apps are built, explained in a human language.

Who is this for?

Anyone that wants to build something, but doesn't know where to start, or is frustrated by all the needless complexity of programming.

Why was this made?

There are countless tutorials on how to learn programming, but almost none on what to actually do with it.

What will I learn?

You will learn the core technologies, and how to use them together to build something you've always wanted.

What will I build?

A full-featured "TODO list" web app. With an actual back-end, authentication, database and deployed on a server.

Are there any requirements?

None. I will explain everything essential. I'll leave the in-depth research to you.

What's the next step?

Follow the whole thing in order, or select a specific topic that interests you.

TLDR

The truth

Most of programming is really just taking data from place A, transforming it, and putting it in place B.

90% of real-world programs are database frontends, held by duct tape.

The internet is just a collection of people making todo lists i.e. CRUD apps. Almost everything on the internet is some database getting inserts, updates, reads and deletes.

How web apps work

These are all the concepts and moving parts for a web app.

All of the steps are explained in detail below.

Internet

Server

Any computer that is connected to the internet with the purpose of doing something asked by a client.

Client

Any device that asks a server to do something. Can be a laptop, phone, tablet, car, fridge...

HTTP (Hypertext Transfer Protocol)

The language used by clients and servers to talk to each other, by sending requests and getting responses, via special commands called methods.

HTTP Methods

Commands the client sends to the server.

  • GET (read data)
  • POST (create data)
  • PATCH (update data)
  • DELETE (delete data)

Status Codes

With every command you send, you get information back telling you what happened, in the form of Response Stauts Codes.

Here are some of the most common ones:

  • 200 - Everything worked as expected.
  • 400 - You have a mistake in your HTTP request.
  • 401 - You don't have access.
  • 404 - Not found.
  • 500 - Server error.

Front-end

HTML (Hypertext Markup Language)

Structure of websites.

Here's the HTML code for a todo list:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>TODO list</title>
    </head>

    <body>
        <div class="container">
            <div class="form">
                <input type="text" />
                <button>Add</button>
            </div>

            <div class="todos">
                <div class="todo"><input type="checkbox" />Buy milk</div>
                <div class="todo"><input type="checkbox" />Walk dog</div>
            </div>

            <div class="completed">
                <div class="todo"><input type="checkbox" checked />Hate React</div>
            </div>
        </div>
    </body>
</html>

Which looks like this:

Buy milk
Walk dog
Hate React

CSS (Cascading Style Sheets)

Looks of websites.

This CSS code, in combination with the HTML above...

.container {
    display: flex;
    flex-direction: column;
    gap: 30px;
}

.form {
    display: flex;
    gap: 10px;
}

.todos,
.completed {
    display: flex;
    flex-direction: column;
    gap: 10px;
    overflow: auto;
    max-height: 200px;
    padding: 0px 10px 10px 0px;
}

.todo {
    display: flex;
    background: white;
    padding: 10px;
    border-radius: 5px;
    box-shadow: 0px 5px 5px -5px rgba(0, 0, 0, 0.2);
    cursor: pointer;
    align-items: center;
    gap: 10px;
}

.todo:hover {
    box-shadow: 0px 5px 5px -5px rgba(0, 0, 0, 0.5);
}

.completed {
    opacity: 0.5;
}

input[type="text"] {
    font-size: 16px;
    padding: 7px;
    border-radius: 3px;
    border: 1px solid #ddd;
    width: 100%;
}

input[type="checkbox"] {
    height: 20px;
    width: 20px;
    cursor: pointer;
}

button {
    font-size: 16px;
    border-radius: 3px;
    border: 0px;
    cursor: pointer;
    background: #555;
    color: white;
}

button:hover {
    background: #222;
}

Gives us this:

Buy milk
Walk dog
Hate React

Responsive

This is simply using different CSS for different screen sizes.

This makes the text smaller on phones:

@media only screen and (max-device-width: 600px) {
    body {
        font-size: 10px;
    }
}

Javascript

There are many programming languages, but ONLY Javascript works in the browser i.e. front-end, so you must learn it.

Objects

These represent a thing.

{ id: 1, title: "Buy milk", completed: false }

Arrays

This is a collection of things.

[
    { id: 1, title: "Buy milk", completed: false },
    { id: 2, title: "Walk dog", completed: false },
    { id: 3, title: "Hate React", completed: true },
];

Variables

We can store things as a value for further use.

let todos = [
    { id: 1, title: "Buy milk", completed: false },
    { id: 2, title: "Walk dog", completed: false },
    { id: 3, title: "Hate React", completed: true },
];

Loops

This is used to go throught every item, and possibly do something with it. There are 2 ways to do this.

for loop

for (let i = 0; i < todos.length; i++) {
    console.log(todo);
}

forEach loop

todos.forEach((todo) => {
    console.log(todo);
});

Conditions

We can define rules when something can happen.

for (let i = 0; i < todos.length; i++) {
    if (todo.completed) {
        console.log(todo);
    }
}

Functions

Give a name to a group of code, and make it executable more than once.

// We define it
function showCompletedTodos() {
    for (let i = 0; i < todos.length; i++) {
        if (todo.completed) {
            console.log(todo);
        }
    }
}

// And we use it
showCompletedTodos();

DOM (Document Object Model)

It represents the HTML structure of a page as a collection of objects.

This allows changing the UI via code, instead of HTML and CSS.

For example, we can add new todo item like this:

let todos = document.querySelector(".todos");

let todo = document.createElement("div");
todo.className = "todo";

let checkbox = document.createElement("input");
checkbox.setAttribute("type", "checkbox");

todo.append(checkbox);
todo.append("Learn DOM events next");

todos.append(todo);

Or, even simpler, we can do this:

let todos = document.querySelector(".todos");

todos.innerHTML += `
    <div class="todo">
        <input type="checkbox" />
        Learn DOM events next
    </div>
`;

And we can create a function for it:

function addTodo() {
    let input = document.querySelector("input");

    let todos = document.querySelector(".todos");

    todos.innerHTML += `
        <div class="todo">
            <input type="checkbox" />
            ${input.value}
        </div>
    `;

    todos.scrollTop = todos.scrollHeight;

    input.value = "";
    input.focus();
}

We will use this function below.

Events

Every time we interact with HTML, events are sent. They can be moving the mouse, mouse clicks, key presses, hovering over an element...

And we can exexute some code when they happen.

Like this:

<button onclick="addTodo()">Add TODO</button>

Or like this:

let button = document.querySelector("button");

btn.onclick = function () {
    addTodo();
};

addEventListener

A more flexible and modern approach for handling events.

When a button is clicked...

document.querySelector("button").addEventListener("click", addTodo);

Or when the "Enter" key is pressed.

document.querySelector("input").addEventListener("keyup", (e) => {
    if (e.key == "Enter") {
        addTodo();
    }
});

Try it here:

Buy milk
Walk dog
Hate React

The new todos are not permanent because they are not stored in a database. When you refresh the page, they will be gone.

I order to save them, or retrieve them from a database, we need to use AJAX.

AJAX (Asynchronous JavaScript And XML)

Used to send HTTP requests from client to server, without reloading the web page.

Instead of getting the whole page as HTML from the server, we can only get just the data and we can show it in the client via the DOM.

The data is received in the JSON format.

JSON (JavaScript Object Notation)

The data recieved from servers looks like this:

[
    {
        "userId": 1,
        "id": 1,
        "title": "delectus aut autem",
        "completed": false
    },
    {
        "userId": 1,
        "id": 2,
        "title": "quis ut nam facilis et officia qui",
        "completed": false
    },
    {
        "userId": 1,
        "id": 3,
        "title": "fugiat veniam minus",
        "completed": false
    }
]

The same format is used for sending data.

XHR (XMLHttpRequest)

The original way to do AJAX looked like this:

let xhr = new XMLHttpRequest();

xhr.open("GET", "https://jsonplaceholder.typicode.com/todos", true);

xhr.onload = function () {
    console.log(JSON.parse(xhr.responseText));
};

xhr.send();

fetch

A much nicer way to do AJAX:

fetch("https://jsonplaceholder.typicode.com/todos")
    .then((res) => res.json())
    .then((data) => console.log(data));

async/await

An cleaner way to do fetch.

let response = await fetch("https://jsonplaceholder.typicode.com/todos");
let data = await response.json();
console.log(data);

API (Application Programming Interface)

API = server that sends and receives data

This can also be called a web service or back-end.

When the button receives a click EVENT, we will use AJAX to make an HTTP GET request to the typicode API, which will send us JSON, which we then convert it into HTML and add it to the DOM.

JSON
HTML

This is the code:

document.querySelector("button").addEventListener("click", async () => {
    let response = await fetch("https://jsonplaceholder.typicode.com/todos");
    let todos = await response.json();

    let json = document.querySelector(".json");

    json.innerHTML = JSON.stringify(todos);

    let html = document.querySelector(".html");

    todos.forEach((todo) => {
        html.innerHTML += `
            <div class="todo">
                <input type="checkbox" ${todo.completed ? "checked" : ""}/>
                ${todo.title}
            </div>
        `;
    });
});

This was someone else's API/back-end. Now it's time to build your own, but first, some Linux!

Linux

Linux

Operating system like Windows, but free. 90% of all servers run on Linux. Your's will too.

There are many versions of it called distros, the most popular one is Ubuntu.

You can either install Ubuntu directly as an OS on your computer instead of Windows...

Or, even better, use it from inside Windows via terminals with something called WSL (Windows Subsystem for Linux).

WSL (Windows Subsystem for Linux)

Allows you to run Linux inside Windows via terminals.

To install it, first you have to enable it:

Start Menu / Turn Windows features on or off / Enabling "Windows Subsystem for Linux"

Then you can download it via the Microsoft Store...

Open it...

Wait to finish installing...

Enter a username...

And that's it! You no have Linux installed inside Windows.

Linux (Ubuntu) is installed in Windows here:

C:\Users\User\AppData\Local\lxss

You can access the Windows C: drive inside the Linux terminal like this:

cd /mnt/c

Terminal

It allows you to control a computer with text commands, instead of a mouse and a User Interface.

It's also called CLI (Command Line Interface) or shell.

At first, it looks stupid and ugly to work like this, BUT TRUST ME, you will LOVE it later. It's insanely powerful.

Normally, you open folders and files by double-clicking them...

But, you can to the same with a terminal:

Here are the most useful commands for...

Navigation

ls -a               # List ALL folders and files

cd folder           # Go inside folder
cd ..               # Go back outside folder

.                   # Shorthand for current directory
cd ~                # Go to your Linux "desktop"
cd /                # Go to the Linux "C:" directory

cat file          # Print file content
less file         # Display file content

Creating folders and files

mkdir folder              # Create folder
touch file.txt            # Create file

cp file.txt file2.txt     # Create a copy
rm file2.txt              # Delete file
mv file file3             # Rename file

Editing files

nano file.txt     # Edit file with nano

# Save with CTRL + x then y then Enter
# Exit with CTR + x then n

vim file.txt      # Edit file with vim

# Save with ESC then :wq then Enter
# Exit with ESC then :q then Enter

Stop a command

CTRL + c

Users

You can create a new user like this:

adduser USERNAME

And swith between users with:

su - USERNAME

There are 2 user types. One is a regular one, like the one you created during installation. The other one is called root, which is basically god in the OS.

root has full control over Linux, and it is NOT advisable to run commands with it. You should use your own username.

Why am I telling you this? Because sometimes commands need root access to execute commands, and you do it with sudo which stands for super user do.

Like changing a username...

sudo usermod -l NEW_USERNAME OLD_USERNAME

Or changing a password.

sudo passwd

binaries

All of these commands are actually mini programs, called binaries or executables, which are just files.

So, instead of doing this command to list all files:

ls

You can manually execute it by running the actual binary:

/usr/bin/ls

This is where it's located. You can check where other commands are located with this:

which COMMAND_NAME

The connections between commands and binaries is defined in an environment variable called $PATH.

PATH

$PATH is an enviroment variable which includes a list of important DIRECTORIES that include program executables i.e. allows you to run commands.

Each time you run a command, the shell checks all the DIRECTORIES in the $PATH variable if they have the executable.

The value of $PATH looks like this:

/home/name/.nvm/versions/node/v20.13.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

It's just a list of folders separated by : i.e. this:

/home/name/.nvm/versions/node/v20.13.1/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin

Why, am I telling you this?

Because if you get a "command not found" error when executing a command, it means it's either no installed i.e not in the directories listed in $PATH, or the directory itself is not listed in the $PATH.

You can add something to it with:

export PATH=$PATH:/usr/hitech/picc/9.82/bin

Services

Services are programs/binaries/executables that run continuously in the background.

You start them with:

sudo service SERVICE_NAME start

And end them with:

sudo service SERVICE_NAME stop

You can check if they are running with:

sudo service SERVICE_NAME status

apt (Advanced Packaging Tool)

Terminal command used to install programs/binaries/executables, also called packages.

Just like Windows, Linux has its own programs, and they are all located in one central place you download them from, unlike Windows wher you have to look for the programs.

First, update the list of available programs:

sudo apt update

Then install a specific program with:

sudo apt install PACKAGE_NAME

You can update already installed programs with this:

sudo apt upgrade

Update

You can use this single command to both update and upgrade the packages:

sudo apt update && sudo apt upgrade

Back-end

Node

Allows you to run Javascript outside of browsers i.e. on your computer directly, with access to your OS.

Don't install it directly, use nvm.

nvm (Node Version Manager)

Allows you to install multiple node versions and switch between them.

In your terminal, use this command to install it:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

This command uses the curl program to download the install.sh file from the link, and then execute it by piping it to the bash program.

MUST RESTART TERMINAL AFTER INSTALL FOR nvm TO WORK!

Check if it works with:

nvm -v

Now install node:

nvm install node

Check if it works with:

node -v

Script

You can execute this hello.js script...

console.log("Hello World!");

Like this in the terminal:

node hello.js

npm

Installs node packages/libraries/modules...

Which are then used inside of your code.

Basically, you download someone else's code, and use it, instead of writing it.

You do it like this:

npm install PACKAGE_NAME

This creates a node_modules folder where all the packages go.

Now, instead of manually installing mulitple packages, you can just do...

npm install

Which will look for all packages listed inside a file called package.json

package.json

This file includes all the packages used in your project. You can create one with:

npm init

It looks like this:

{
    "name": "app",
    "version": "1.0.0",
    "main": "index.js",
    "scripts": {},
    "author": "",
    "license": "ISC",
    "description": ""
}

To add a package, you can either write it manually, or add --save to your install command:

npm install mysql --save

The package.json file will look like this now:

{
    "name": "app",
    "version": "1.0.0",
    "main": "index.js",
    "scripts": {},
    "author": "",
    "license": "ISC",
    "description": "",
    "dependencies": {
        "mysql": "^2.18.1"
    }
}

You can install all the packages listed in dependencies at once with:

npm install

express

This is an npm package used for creating an API i.e. back-end, like the todos one shown before.

npm install express --save

Here's a very simple server/API, similar to the one we used before...

Put this code inside a server.js file:

let express = require("express");
let app = express();
let port = 3000;

app.use(express.json());

app.get("/api/todos", (req, res) => {
    let todos = [
        { id: 1, title: "Buy milk", completed: false },
        { id: 2, title: "Walk dog", completed: false },
        { id: 3, title: "Hate React", completed: true },
    ];

    res.status(200).send(todos);
});

app.listen(port, () => {
    console.log(`Server is running on http://localhost:${port}`);
});

And run it with:

node server.js

Now, in your browser, go to this URL:

http://localhost:3000/api/todos

And you should see the todos JSON.

Note that the todos are hard-coded in the API. In a real API, these would come form the database.

nodemon

An npm package for continuously running a node script, especially when it fails.

You install it like this:

npm install nodemon

And then you run the server with:

nodemon server.js

You can do it like this:

REST

A mutually-agreed convention how to write API endpoints/routes/URLs/URIs.

Method Resource Result
GET /todos Get a list of todos
GET /todos/12 Get a specific todo
POST /todos Create a new todo
PATCH /todos/12 Update todo with ID 12
DELETE /todos/12 Delete todo with ID 12

Here's what the full server.js file would look like.

let express = require("express");
let app = express();
let port = 3000;

app.use(express.json());

let todos = [
    { id: 1, title: "Buy milk", completed: false },
    { id: 2, title: "Walk dog", completed: false },
    { id: 3, title: "Hate React", completed: true },
];

app.get("/api/todos", (req, res) => {
    res.status(200).send(todos);
});

app.get("/api/todos/:todoId", (req, res) => {
    let { todoId } = req.params;
    let todo = todos.find((item) => item.id == todoId);

    res.status(200).send(todo);
});

app.post(`/api/todos`, async (req, res) => {
    let todo = req.body;

    let newTodo = {
        id: todos.length + 1,
        title: todo.title,
        completed: todo.completed,
    };

    todos.push(newTodo);

    res.status(200).send(newTodo);
});

router.patch(`/api/todos/:todoId`, async (req, res) => {
    let { todoId } = req.params;
    let todo = req.body;

    let updatedTodo = todos.find((item) => item.id == todoId);

    updatedTodo.title = todo.title;
    updatedTodo.completed = todo.completed;

    res.status(200).send(updatedTodo);
});

router.delete(`/api/todos/:todoId`, async (req, res) => {
    let { todoId } = req.params;

    todos = todos.filter((item) => item.id != todoId);

    res.status(200);
});

app.listen(port, () => {
    console.log(`Server is running on http://localhost:${port}`);
});

Database

DBMS (Database Management System)

Software for creating databases and handling the data in them.

The most popular ones are:

  • MySQL
  • SQL Server
  • Postgresql
  • SQLite.

MySQL

The most widely used one. Here are some popular users:

  • Facebook
  • YouTube
  • Twitter
  • Netflix
  • Google
  • WordPress

You install it like this:

sudo apt install mysql-server -y

Start it with:

sudo service mysql start

Check if it runs:

sudo service mysql status

Log in with:

sudo mysql

Once logged in, you need to update the authentication method for the root user to use a password.

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'root';
FLUSH PRIVILEGES;

Log out:

exit

Now try logging in with the new password:

sudo mysql -u root -p

If you can't login, this is probably the problem:

  • MySQL is not started. Start it with sudo service mysql start.
  • You are not using sudo.
  • There is no password. Login with just sudo mysql.

SQL (Structured Query Language)

The language used to interact with the database via commands called queries.

Once you are logged in the DBMS i.e. MySQL, you use SQL.

Workbench

You can execute SQL queries in the terminal or in an app with an actual user interface.

You can download it here:

https://www.mysql.com/products/workbench

And it looks like this:

We will use the terminal for the practice.

Schema

The database is all the data. Think of it as an Excel file.

The schema is the blueprint for a database i.e. everything that is not the actual data.

Ex. table definitions, colum definitions, data types, relationships, indexes...

Think of it as the settings in the Excel file.

You can see the databases with this SQL query:

show databases;

Or you can create your own:

create database todo;

Note that you must select the database in order to do things in it.

use todo;

Table

Data is organized in tables. Think of them like sheets in an Excel file.

Every table has columns and rows. Exactly like in Excel.

You can see the all the tables in a database with:

show tables;

Or you can create one:

create table todo (
    id int primary key auto_increment,
    title varchar(255),
    completed int
);

Columns

When you created the table, you also defined the columns it will have and their names, along with the type of data they will store.

This of them like the titles of Excel columns.

You can also add a column after a table is created:

alter table todo add created datetime;

Rows

The actual data is store in rows, the same as Excel.

Every row is a collection of values that correspond to the columns.

You add rows like this:

insert into todo (title, completed, created) values ('learn SQL', 0, now());

You can add multiple at once:

insert into todo
    (title, completed, created)
values
    ('Create you dream idea', 0, now()),
    ('???', 0, now()),
    ('Profit!', 0, now())
;

You can see the result of this with:

select * from todo;

If you want to update a value, you do this:

update todo set completed = 1 where id = 1;

IT IS SUPER IMPORTANT TO USE A where CLAUSE, BECAUSE THIS WILL UPDATE EVERY ROW, NOT JUST THE ONE YOU WANT!!!

THE SAME APPLIES WHERE DELETING DATA!!!

delete from todo where id = 1;

A GOOD PRACTICE IS TO FIRST SELECT THE DATA, THEN UPDATE OR DELETE IT.

node + mysql

This is a database driver, which is a fancy way to say an npm package for connecting to your database.

There are many, but the best one is the default mysql one.

npm install mysql --save

Connection

You connect to your database like this:

let mysql = require("mysql");

let connection = mysql.createConnection({
    host: "localhost",
    database: "todo",
    user: "root",
    password: "root",
});

connection.connect();

And then you run your queries:

connection.query("select * from toro");

And you end the connection.

connection.end();

Pool

Opening and closing conections is costly, so a better way is to connect once, create a bunch of connections, and then just borrow them for executing queries, and finally returning them.

The best way to do this is to create the following files:

db.js

module.exports.db = {
    host: "localhost",
    database: "todo",
    user: "root",
    password: "root",
    connectionLimit: 100,
};

pool.js

const mysql = require("mysql");
const db = require("./db.js");

const pool = mysql.createPool(db);

module.exports = pool;

Optionally add this "promisify" code in the pool.js file to make the query function asynchronous i.e. async/await.

const util = require("util");
pool.query = util.promisify(pool.query);

Finally, you can now use actual SQL in your server.js.

It's a good idea to handle errors when executing queries.

let express = require("express");
let pool = require("./pool.js");

let app = express();
let port = 3000;

app.use(express.json());

app.get("/api/todos", (req, res) => {
    try {
        let query = `
            select *
            from todo
            ;
        `

        let todos = await pool.query(query);

        res.status(200).send(todos);
    } catch (error) {
        console.log(error)
    }
});


app.get("/api/todos/:todoId", (req, res) => {
    try {
        let { todoId } = req.params;

        let query = `
            select *
            from todo
            where id = ?
            ;
        `

        let todo = await pool.query(query, [todoId]);

        res.status(200).send(todo);
    } catch (error) {
        console.log(error)
    }
});

app.post(`/api/todos`, async (req, res) => {
    try {
        let todo = req.body;

        let query = `
            insert into todo
                (
                    title,
                    completed,
                    created
                )
            values
                (
                    ?,
                    ?,
                    now()
                )
            ;
        `;

        let todo = await pool.query(query, [userId,todo.completed, todo.title]);

        res.status(200).send(todo);
    } catch (error) {
        console.log(error)
    }
});

app.patch(`/api/todos/:todoId`, async (req, res) => {
    try {
        let { todoId } = req.params;
        let todo = req.body;

        let query = `
            update todo
            set
                title = ?,
                completed = ?
            where id = ?
            ;
        `;

        let todo = await pool.query(query, [todo.title, todo.completed, todoId]);

        res.status(200).send(todo);
    } catch (error) {
        console.log(error)
    }
});

router.delete(`/api/todos/:todoId`, async (req, res) => {
    try {
        let { todoId } = req.params;

        let query = `
            delete from todo
            where id = ?
            ;
        `;

        let todo = await pool.query(query, [todoId]);

        res.status(200)
    } catch (error) {
        console.log(error)
    }
});

app.listen(port, () => {
    console.log(`Server is running on http://localhost:${port}`);
});

Webserver

Webserver

A program inside the server that listens for HTTP commands on port 80, and sends back content, either directly or by redirecting to another program via ports.

Without a webserver, your website cannot be accessed.

Ports

Used to locate programs inside the server, similar to IP addresses locating devices.

NGINX

The most popular webserver.

sudo apt install nginx -y;

Start it with:

sudo service nginx start

Check if it runs:

sudo service nginx status

Servers are configured in the /etx/nginx/nginx.conf file.

Here's what the simplest setup, which serves a static website, looks like:

events {}                              # Needed to be a valid conf file.

http {
    include mime.types;                # Recognize filetypes.

    server {
        listen 8080;                   # Port number

        root /home/website;             # Match URIs to files here.
        index index.html;              # index is inside folder
    }
}

You can see your website at:

localhost:8080

You can if nginx.conf is configured properly with:

sudo nginx -t

If it doesn't work, it's most probably because of permissions. You can add them like this:

# Make user the owner
sudo chown -R user:user /home/user/website

# Make user part of www-data group
sudo usermod -aG www-data user

# Read, write and execute permissions
sudo chmod -R 777 /home/user/website

CORS (Cross-Origin Resource Sharing)

This is WAY simpler than it looks.

It's just a rule saying your front-end (client) and back-end (server) must be on the same computer.

If they are not, you need to tell the back-end which front-ends can access it.

This is a security measure so that www.hacker.ru can't ask your www.bank.com do to stuff.

Proxy-pass

Instead of accessing your back-end from your client with localhost:3000/api, you use a proxy_pass so you can use it just with /api.

Here's what the configuration looks like for the todo app, that uses an API:

events {}

http {
    include mime.types;

    server {
        listen 8081;

        root /home/todo; # Path to directory
        index index.html;

        location / {
            # Fixes refresh resulting in 404 error for single page apps
            try_files $uri $uri/ /index.html;
        }

        location /auth {
            proxy_pass 'http://127.0.0.1:3001';
        }

        location /api {
            proxy_pass 'http://127.0.0.1:3001';
        }
    }
}

include

You can also split an nginx.conf file into mulitple files and link them in the main one.

website/nginx.conf (static)

server {
    listen 80;

    root /home/user/website;
    index index.html;
}

docs/nginx.conf (SSR - Server Side Rendering)

server {
    listen 80;

    location / {
        proxy_pass http://127.0.0.1:5000;
    }
}

todo/nginx.conf (SPA - Single Page App i.e. front-end + back-end)

server {
    listen 80;

    root /home/user/todo/client;
    index index.html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /auth {
        proxy_pass http://127.0.0.1:3000;
    }

    location /api {
        proxy_pass http://127.0.0.1:3000;
    }
}

nginx.conf

events {}

http {
    include mime.types;

    include /home/user/website/nginx.conf;
    include /home/user/docs/nginx.conf;
    include /home/user/todo/nginx.conf;
}

Troubleshoot

Here are the most common reason the webserver doesn't work:

  • Forgot to restart it after changing nginx.conf.
  • You have an error. Check with sudo nginx -t.
  • You might have more than 1 process running. Stop them with sudo killall nginx.
  • You forgot to add:
    • events {}
    • port
    • server_name
    • index

Security

.env

It's not a good idea to store your database credentials in code.

A better way is to store them in a .env file.

This file is nothing magical, just a agreed upon way.

It looks like this:

DB_NAME=todo
DB_USER=root
DB_PASSWORD=root

This is a package that reads .env files, and makes the content available in server.js.

Install it like this:

npm install dotenv --save

And use this .env file...

DB_NAME=todo
DB_USER=root
DB_PASSWORD=root

In your db.js file like this:

const path = require("path");

require("dotenv").config({ path: path.resolve(__dirname, ".env") });

module.exports.db = {
    server: "localhost",
    database: process.env.DB_NAME,
    user: process.env.DB_USER,
    password: process.env.DB_PASSWORD,
    connectionLimit: 100,
};

Authentication

In order to implement authentication, the app needs to be modified in several places:

  • Create the user database table.
  • Add a userId column in the todo table, so we which todo belongs to which user.
  • Create a signup page so users can be created.
  • Create a login page for existing users.
  • Create the API endpoints for signup and login.
  • Install the JWT (tokens) issuing library.
  • Install the SHA256 hashing algorithm for safely storing passwords.
  • Add a router so that we can redirect anyone without access to the login page.
  • Modify the SQL queries to have a userId where clause, so users get only their todos.
  • Add middleware for token verification.

You can see the whole code in the TODO topic.

Before we do that, let's see how JWT and SHA256 work.

JWT (JSON Web Token)

A token looks like this:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOjEsImlhdCI6MTcyNDI1NTczNywiZXhwIjoxNzU1NzkxNzM3fQ.WW218MHiBX2VaU-GmTXmXvzTBe-4ZwSeg2E_HNlbTr0

This is actually a base64 encoded string separated by a . in 3 parts.

base64 is NOT encryption, it just saves characters and makes it URL friendly. Anyone can decode it.

If you decode it, it looks like this:

The first part eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9 (header) decoded is:

{
    "alg": "HS256",
    "typ": "JWT"
}

The second part eyJ1c2VySWQiOjEsImlhdCI6MTcyNDI1NTczNywiZXhwIjoxNzU1NzkxNzM3fQ (payload) decoded is:

{
    "userId": 1,
    "iat": 1724255737,
    "exp": 1755791737
}

The third part WW218MHiBX2VaU-GmTXmXvzTBe-4ZwSeg2E_HNlbTr0 (verification) decoded is a SHA256 hash of 3 parts:

  • The base64 of the first part (header).
  • The base64 of the second part (payload).
  • The token secret i.e. a password you define that only the server knows.

So, what is SHA256?

httpOnly cookies

let token = jwt.sign(
    {
        username: user.username,
        userId: user.id,
    },
    config.tokenSecret,
    {
        expiresIn: "7d",
    }
);

res.cookie("token", token, {
    httpOnly: true,
    secure: true,
});

Middleware

This is some code that is executed between:

  • Receiving the request and executing the query. ex. Authentication.
  • Executing the query and sending the response back. ex. Error logging.
let express = require("express");
let app = express();
let port = 3000;

app.use(express.json());

function authenticationMiddleware(req, res, next) {
    let token = req.cookies.token;

    jwt.verify(token, config.tokenSecret, (err, decoded) => {
        if (err) {
            res.status(401).send({
                status: 401,
                message: "No access!",
            });
        } else {
            req.userId = decoded.userId;
            next();
        }
    });
}

function errorLogger(err, req, res, next) {
    console.error(`Error occurred: ${err.message}`);
    res.status(500).send({ error: "Internal Server Error" });
}

app.use(authenticationMiddleware);

app.get("/api/todos", (req, res, next) => {
    try {
        let query = `
            select *
            from todo
            ;
        `

        let todos = await pool.query(query);

        res.status(200).send(todos);
    } catch (error) {
        next(error)
    }
});

app.use(errorLogger);

app.listen(port, () => {
    console.log(`Server is running on http://localhost:${port}`);
});

SHA256

It's an algorithm that converts ANY sized string to a fixed length random-character string, that is IMPOSSIBLE to convert back.

For example, the string a hashed is:

ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb

The password 12345678 is:

ef797c8118f02dfb649607dd5d3f8c7623048c9c063d532cc95c5ed7a898a64f

The USA Declaration of Independence is this:

c803200a5f8119e0c16b4c5da548fd9559742a86a0af2befc54416d9305554d6

Fun fact: This is how crypto works. Each block in the chain contains the hash of the previous blocks. When you are mining, you are actually looking for a specific number, that when added to the block results in hash that starts with several zeros.

SHA256 is useful for 2 things:

  • Verify that some content has not been changed.
  • Make it impossible to know the content.

In our case, we store the SHA256 hash of the user passwords, not the actual passwords. This way you don't know them, as well as anyone that steals them.

How it works in practice is, a user logs in, and his real passwords is hashed, and then compared to the hash you have in the database.

SQL Injection

This is THE MOST DANGEROUS vulnerability a back-end can have.

It is INCREDIBLY important to NOT trust ANYTHING that comes from the client.

Everything MUST be sanitized before it is executed as a query.

This is already done in the code above.

THIS IS WHAT YOU SHOULD NEVER DO!!!

app.get("/api/todos/:todoId", (req, res) => {
    try {
        let { todoId } = req.params;

        let query = `
            select *
            from todo
            where id = ${todoId}    -- <-- THIS HERE!!!
            ;
        `

        let todo = await pool.query(query);

        res.status(200).send(todo);
    } catch (error) {
        console.log(error)
    }
});

Notice how the todoId is directly used in the query, instead of being sanitized by the mysql library.

This is how you get hacked.

Instead of a doing sending a todoId in the API request...

GET https://localhost:3000/api/todos/1

Someone can send SQL code, and ruin your day, like this:

GET https://localhost:3000/api/todos/1%20or%201%3D1

Which is this:

GET https://localhost:3000/api/todos/1 or 1=1

Which turns your SQL query into:

select *
from todo
where id = 1 or 1=1
;

And since 1=1 is always true, the where id = 1 is ignored, and all the todos will be returned, instead of one.

An even more fun one is this:

GET https://localhost:3000/api/todos/1%3B%20drop%20database%20todo%3B%20--

Which is:

GET https://localhost:3000/api/todos/1; drops database todo; --

Which is:

select *
from todo
where id = 1; drop database todo; -- ;

Which is bye bye to your whole database.

Server

Cloud

This is just a term for "a bunch of servers in datacenters".

Host

This is a business that manages datacenters and offers cloud services.

One of the services is creating your own "virtual" server (VPS), instead of creating your own physical machine.

The most user-friendly host is Digital Ocean.

You can create an account here:

https://www.digitalocean.com

VPS (Virtual Private Server)

After you create an account, you create a server like this...

Click on Create and select Droplets (their fancy name for VPS):

Select the region closest to your users:

Choose your operating system i.e. Linux distro and version:

Choose your machine:

Insert your server password:

Give your server a name, and click "Create":

Wait for the creation to finish:

And that's it! You now have your own server, living on a dedicated IP address.

IP Address

This is used to identify and locate servers. Like a phone number.

In this case, the address is:

46.101.132.192

We can now connect to it by using SSH.

SSH (Secure Socket Shell)

Think of this like Teamviewer via the terminal i.e. without a UI.

You use it like this:

ssh root@46.101.132.192

It will ask you for the server password.

After that, you are connected to the server you created.

To change your root password, use:

passwd root

You now MUST create a user other than root. This is for security reasons:

adduser user

# You can delete users with:
userdel user

And give it root access.

usermod -aG sudo user

Then you login as the new user.

su - user

You can now exit by tying, well... exit.

Here's the whole thing:

Keys

Using your password for every SSH login is both highly inneficient and insecure.

To avoid that, you can use private and public keys.

You create them like this:

ssh-keygen

The keys will be located in ~/.ssh.

Now you just transfer them to the server like this:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@46.101.132.192

You need to use your own public key name.

Now you can connect without typing your password:

ssh user@46.101.132.192

Environment

You can now prepare your server for your app.

You need to replicate whatever you have on your local machine.

In this case that would be...

pm2 (Process Manager)

You can run your server.js file with:

node server.js

But, to run it forever in the background and restart it automatically when it crashes, you need a daemon, and pm2 is a popular choice.

You install it like this:

npm install pm2 -g

It is advisable to intall it globally since many apps can use it.

And then you can run your server.js like this:

pm2 start server.js --name "todo"

You can check all your running apps with:

pm2 ls

And you can monitor them with:

pm2 monit

To stop an app, use this (you need to be inside the directory):

pm2 stop server.js

In oder for all this to work, you need the actual code files, which we can transfer with rsync.

Deployment

rsync

This is a Linux package for transfering files.

We use this to transfer our code from our local computer to the remote server.

Here's the command, which you run on your local machine:

rsync -av -e 'ssh' ./client ./server --exclude 'server/node_modules' --exclude 'server/package-lock.json' --exclude 'server/.env' user@46.101.132.192:~/todo/

In order to see any changes, you need to restart your pm2 daemon:

ssh user@46.101.132.192 pm2 reload /home/user/todo/server/server.js --name "todo"

You can put both commands inside a deploy.sh file...

rsync -av -e 'ssh' ./server --exclude 'server/node_modules' --exclude 'server/package-lock.json' --exclude 'server/.env' user@46.101.132.192:~/todo/ \
&& ssh user@46.101.132.192 pm2 reload /home/user/todo/server/server.js --name "todo"

So you can run the file with:

./deploy.sh

This will use SSH to transfer the client and server folders from your local machine, inside a todo folder on the remote machine.

It will not transfer the node_modules folder, as well as the .env. file.

You will need to manually create the .env file with your production credentials.

You can connect to the server to check if everything was transferred correctly.

ssh user@46.101.132.192

Production db

You need to create the database by running this command:

mysql -u root -p < schema.sql

Production .env

Create this file in the server directory:

touch .env

Edit it with:

vim .env

And put the credentials inside like you did in your local environment, but use your production (server) ones.

npm install

After you've transferred the files, you can now install the packages.

First go to the server folder

cd todo/server

And then do

npm install

In order for everything to work, you need to configure your webserver i.e. nginx.

nginx.conf

You can edit the nginx.conf file with:

sudo vim /etc/nginx/nginx.conf

The configuration should look like this:

events {} # Needed to be a valid conf file.

    http {
    include mime.types; # Recognize filetypes.

    server {
        listen 80;

        server_name 46.101.132.192;

        root /home/user/todo/client; # Path to directory
        index index.html;

        location / {
            try_files $uri $uri/ /index.html; # Fixes refreshing resulting in 404 error for single page apps
        }

        location /auth {
            proxy_pass http://127.0.0.1:3001;
        }

        location /api {
            proxy_pass http://127.0.0.1:3001;
        }
    }
}

The app is now available at:

46.101.132.192

Domain

Domain

User-friendly name for an IP addresse, like contact name.

They are sold and managed by domain registrars.

You can buy a domain here:

https://www.namecheap.com

After you buy it, you need to go to the Networking page in your Host (cloud provider):

And add your domain:

After this, you have to configure the NS i.e. name servers.

NS (Name Servers)

Name servers define which DNS server to use i.e which server contains the actual DNS records.

You need to configure this on both the reigstrar's side, as we all the host's side.

For the registrar, you go to your domain and click on manage.

And inside, you add your host's nameservers under custom DNS.

And, for the host's side, you do the same. Usually done automatically when you add the domain.

DNS (Domain Name System)

Container domains into IP addresses, like a phonebook.

It contains various DNS records which associate server locations to letters.

It's done because computers only understand numbers (IP address), not letters (domain).

It works automatically in the background.

A Record

Connects the domain name to the IP address.

It's done like this:

  1. Type @ in the "hostname" field.
  2. Select your server's IP address in the "Will direct to" field.
  3. Click "Create record"

CNAME

This record is for defining an alias. This is used so that the www in www.domain.com can work.

Propagation

You need to wait for some time for the configurations to go live, first for the NS (name servers) and then for the domain.

It's not uncommon to take 24+ hours.

You can check how far they've propagated with this tool:

https://www.whatsmydns.net

For the NS (name server) record:

For the A (domain) record:

SSL (Secure Sockets Layer)

This is https instead of http i.e. the data sent via HTTP is encrypted.

To add this to your server, you can use certbot to generate the certificates, and then add them in the NGINX configuration.

certbot

Linux package for issuing and renewing SSL certificates for domains and subdomains.

Note that they need SEPARATE certificates.

Install certbot:

sudo snap install --classic certbot

BEFORE USING IT, YOU MUST DISABLE NGINX BECAUSE certbot USES IT'S OWN VERSION. YOU MUST KILL EVERY PROCESS.

DO NOT ADD THE CERTIFICATES IN NGINX BEFORE THEY ARE CREATED.

MAKE SURE THAT A RECORDS EXIST IN THE DNS BEFORE ISSUING.

sudo killall nginx

You can now create an SSL certificate:

sudo certbot certonly --nginx -d codepreneuring.com

You can see the certificate i.e. keys like this:

The nginx.conf file must now be changed to include them.

Here's the FULL nginx.conf for this website.

website/nginx.conf (static)

server {
    listen 80;

    server_name codepreneuring.com www.codepreneuring.com;

    return 301 https://codepreneuring.com$request_uri;
}


server {
    listen 443 ssl;

    server_name codepreneuring.com;

    ssl_certificate /etc/letsencrypt/live/codepreneuring.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/codepreneuring.com/privkey.pem;

    root /home/user/website;
    index index.html;
}

docs/nginx.conf (SSR - Server Side Rendering)

server {
    listen 80;

    server_name docs.codepreneuring.com;

    return 301 https://docs.codepreneuring.com$request_uri;
}

server {
    listen 443 ssl;

    server_name docs.codepreneuring.com;

    ssl_certificate /etc/letsencrypt/live/docs.codepreneuring.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/docs.codepreneuring.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:5000;
    }
}

todo/nginx.conf (SPA - Single Page App i.e. front-end + back-end)

server {
    listen 80;

    server_name todo.codepreneuring.com;

    return 301 https://todo.codepreneuring.com$request_uri;
}

server {
    listen 443 ssl;

    server_name todo.codepreneuring.com;

    ssl_certificate /etc/letsencrypt/live/todo.codepreneuring.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/todo.codepreneuring.com/privkey.pem;

    root /home/user/todo/client;
    index index.html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /auth {
        proxy_pass http://127.0.0.1:3000;
    }

    location /api {
        proxy_pass http://127.0.0.1:3000;
    }
}

nginx.conf

events {}

http {
    include mime.types;

    include /home/user/website/nginx.conf;
    include /home/user/docs/nginx.conf;
    include /home/user/todo/nginx.conf;
}

You can now start/restart/reload NGINX for this to work:

service nginx start

The app will now be available with SSL at:

https://codepreneuring.com

If it still doesn't work, it's probably a caching issue. Try opening the domain:

  • In an incognito window.
  • On your phone via 5G instead of WiFI.

Todo

Code

The code is located here:

https://github.com/codepreneuring/todo

It is uploaded there for 2 reasons:

  • It would take too much space here.
  • Everyone should keep their code on Github. There's an explanation why and how to do this a bit later.

Download

You can download it in 3 ways:

  1. Clicking on the Code/Download ZIP button.

  1. Using this download link:
https://github.com/codepreneuring/todo/archive/refs/heads/main.zip
  1. By running this terminal command:
git clone https://github.com/codepreneuring/todo.git

This will create a todo folder with all the code, in the directory you execute the command in.

Setup

After the download, go inside the todo server folder via the terminal.

cd todo/server
  1. Install all the packages.
npm install
  1. Start the server"
nodemon server.js
  1. Configure nginx.conf
events {}

http {
    include mime.types;

    server {
        listen 8081;

        root /mnt/c/user/codepreneuring/todo/client; # Path to directory
        index index.html;

        location / {
            try_files $uri $uri/ /index.html; # Fixes refreshing resulting in 404 error for single page apps
        }

        location /auth {
            proxy_pass 'http://127.0.0.1:3001';
        }

        location /api {
            proxy_pass 'http://127.0.0.1:3001';
        }
    }

}
  1. Go to:
localhost:8081

Editor

VS Code

Download and install it:

https://code.visualstudio.com

It looks like this:

If you are using WSL, it's a good idea to connect to it with CTRL + Shift + P and choosing WSL: Connect to WSL.

Shortcuts

The most useful productivity shortcuts for VS Code.

# Duplicates

CTRL + d                    # Select next duplicate value
CTRL + u                    # Unselect duplicate value
CTRL + SHIFT + l            # Select all duplicate values

# Multiple cursors

ALT + click (empty)         # Multiple cursors
ALT + SHIFT + click         # Multiple selections in range

# Line

ALT + up/down               # Move line up or down
ALT + SHIFT + up/down       # Duplicate line

# Jumping

CTRL + click (variable)     # Jump to definition

# Utility

CTRL + SPACE                # Show code completion suggestions

jsbin.com

Excellent tool for quick experimentations.

https://jsbin.com

Git

Git

This is a program used for code versioning.

It saves you from doing file.txt, file-updated.txt, file-revision-2.txt, file-FINAL.txt... Instead, all of this is tracked in the background.

This program usually is built-in in all modern Linux distros, or you can install it with this:

sudo apt install git

.gitignore

This is a simple file where you list which folders and files you do NOT want to save.

The file has to be named .gitignore and placed in the root of the project folder.

Putting this inside the file...

index.html

Means index.html will not be saved with git.

Git commands

To start using Git, initialize it inside your project folder:

git init

This will create a .git folder (invisible), inside your project, where all the changes are stored. This is called a repository, repo for short.

Each repo has its own configurations. You can see them with:

git config --list

Here are the essential configurations you need to add:

# Configure owner of changes just for current repo
git config user.email "codepreneuring@gmail.com"
git config user.name "codepreneuring"

Github contributions count only if the repo email is the same as the github email.

You can check which files are new (untracked) or unsaved (unstaged) with this command:

git status

To save something, first you need to mark the files as "to be saved" i.e. "tracked" or" staged":

git add .    # . means everything in current directory

The actual saving is done like this:

git commit -m "Any message you want for this save"

Here's how this looks in the terminal:

You can do as many saves as you want.

Let's change our file with our VS Code editor.

You can compare the changes between the saved file and the updated file with:

git diff

Save the changes like you did before:

You can see all the saves with this command:

git log

Notice the weird random characters (hash) in yellow. These are the names of the saves (commits).

Github

This is where you upload your code, which you can share with others. Like Google Drive is for files and pictures.

Github is NOT part of Git, it just works with it. There are other websites that offer the same services, like Gitlab and Bitbucket... Github is just the most popular one.

If you want to upload your code there, which is HIGHLY recommended, you can create an account here:

https://github.com

Your Github username and email MUST match the ones you used in your Git configuration, so that the commits can show up on Github. I'm talking about these:

git config user.email "codepreneuring@gmail.com"
git config user.name "codepreneuring"

This is what your Github account will look like:

Github password

After you've created your account, you will need to create a separate password for uploading your code, like this:

  1. Go to...
Settings / Developer Settings / Personal Access Token / Tokens (classic)
  1. Click on Generate New Token (classic)...

  1. Add a note. Select no expiration. Toggle the repo checkbox. Click on Generate token...

  1. Save the token somewhere because you won't be able to see it again.

Github repository

Now that you have an account and password (token), you can finally upload your code to github.

First you need to create a new repository (repo) via the top menu...

Fill up the repo details...

And that's it. The repo you've created is now empty, and needs some code to be uploaded.

You will be asked for the Github password, so to save it for future saves, use this command:

git config credential.helper store

It saves the password in a .git-credentials file.

Since we already have the code that we saved with Git, we can upload that.

The first time you upload code, you use these commands, shown in the empty repo at the bottom:

git remote add origin https://github.com/codepreneuring/todo-file.git
git branch -M main
git push -u origin main

After that, you only use:

git push

After code is uploaded, your repo will look like this:

You can download repos via the terminal:

git clone https://github.com/codepreneuring/todo.git

Or directly: