About
What is this?
A FREE, no-bullshit, step-by-step guide how web apps are built, explained in a human language.
Who is this for?
Anyone that wants to build something, but doesn't know where to start, or is frustrated by all the needless complexity of programming.
Why was this made?
There are countless tutorials on how to learn programming, but almost none on what to actually do with it.
What will I learn?
You will learn the core technologies, and how to use them together to build something you've always wanted.
What will I build?
A full-featured "TODO list" web app. With an actual back-end, authentication, database and deployed on a server.
Are there any requirements?
None. I will explain everything essential. I'll leave the in-depth research to you.
What's the next step?
Follow the whole thing in order, or select a specific topic that interests you.
TLDR
The truth
Most of programming is really just taking data from place A, transforming it, and putting it in place B.
90% of real-world programs are database frontends, held by duct tape.
The internet is just a collection of people making todo lists i.e. CRUD apps. Almost everything on the internet is some database getting inserts, updates, reads and deletes.
How web apps work
These are all the concepts and moving parts for a web app.
All of the steps are explained in detail below.
Internet
Server
Any computer that is connected to the internet with the purpose of doing something asked by a client.
Client
Any device that asks a server to do something. Can be a laptop, phone, tablet, car, fridge...
HTTP (Hypertext Transfer Protocol)
The language used by clients and servers to talk to each other, by sending requests and getting responses, via special commands called methods.
HTTP Methods
Commands the client sends to the server.
- GET (read data)
- POST (create data)
- PATCH (update data)
- DELETE (delete data)
Status Codes
With every command you send, you get information back telling you what happened, in the form of Response Stauts Codes
.
Here are some of the most common ones:
200
- Everything worked as expected.400
- You have a mistake in your HTTP request.401
- You don't have access.404
- Not found.500
- Server error.
Front-end
HTML (Hypertext Markup Language)
Structure of websites.
Here's the HTML code for a todo list:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>TODO list</title>
</head>
<body>
<div class="container">
<div class="form">
<input type="text" />
<button>Add</button>
</div>
<div class="todos">
<div class="todo"><input type="checkbox" />Buy milk</div>
<div class="todo"><input type="checkbox" />Walk dog</div>
</div>
<div class="completed">
<div class="todo"><input type="checkbox" checked />Hate React</div>
</div>
</div>
</body>
</html>
Which looks like this:
CSS (Cascading Style Sheets)
Looks of websites.
This CSS code, in combination with the HTML above...
.container {
display: flex;
flex-direction: column;
gap: 30px;
}
.form {
display: flex;
gap: 10px;
}
.todos,
.completed {
display: flex;
flex-direction: column;
gap: 10px;
overflow: auto;
max-height: 200px;
padding: 0px 10px 10px 0px;
}
.todo {
display: flex;
background: white;
padding: 10px;
border-radius: 5px;
box-shadow: 0px 5px 5px -5px rgba(0, 0, 0, 0.2);
cursor: pointer;
align-items: center;
gap: 10px;
}
.todo:hover {
box-shadow: 0px 5px 5px -5px rgba(0, 0, 0, 0.5);
}
.completed {
opacity: 0.5;
}
input[type="text"] {
font-size: 16px;
padding: 7px;
border-radius: 3px;
border: 1px solid #ddd;
width: 100%;
}
input[type="checkbox"] {
height: 20px;
width: 20px;
cursor: pointer;
}
button {
font-size: 16px;
border-radius: 3px;
border: 0px;
cursor: pointer;
background: #555;
color: white;
}
button:hover {
background: #222;
}
Gives us this:
Responsive
This is simply using different CSS for different screen sizes.
This makes the text smaller on phones:
@media only screen and (max-device-width: 600px) {
body {
font-size: 10px;
}
}
Javascript
There are many programming languages, but ONLY Javascript works in the browser i.e. front-end, so you must learn it.
Objects
These represent a thing.
{ id: 1, title: "Buy milk", completed: false }
Arrays
This is a collection of things.
[
{ id: 1, title: "Buy milk", completed: false },
{ id: 2, title: "Walk dog", completed: false },
{ id: 3, title: "Hate React", completed: true },
];
Variables
We can store things as a value for further use.
let todos = [
{ id: 1, title: "Buy milk", completed: false },
{ id: 2, title: "Walk dog", completed: false },
{ id: 3, title: "Hate React", completed: true },
];
Loops
This is used to go throught every item, and possibly do something with it. There are 2 ways to do this.
for loop
for (let i = 0; i < todos.length; i++) {
console.log(todo);
}
forEach loop
todos.forEach((todo) => {
console.log(todo);
});
Conditions
We can define rules when something can happen.
for (let i = 0; i < todos.length; i++) {
if (todo.completed) {
console.log(todo);
}
}
Functions
Give a name to a group of code, and make it executable more than once.
// We define it
function showCompletedTodos() {
for (let i = 0; i < todos.length; i++) {
if (todo.completed) {
console.log(todo);
}
}
}
// And we use it
showCompletedTodos();
DOM (Document Object Model)
It represents the HTML structure of a page as a collection of objects.
This allows changing the UI via code, instead of HTML and CSS.
For example, we can add new todo item like this:
let todos = document.querySelector(".todos");
let todo = document.createElement("div");
todo.className = "todo";
let checkbox = document.createElement("input");
checkbox.setAttribute("type", "checkbox");
todo.append(checkbox);
todo.append("Learn DOM events next");
todos.append(todo);
Or, even simpler, we can do this:
let todos = document.querySelector(".todos");
todos.innerHTML += `
<div class="todo">
<input type="checkbox" />
Learn DOM events next
</div>
`;
And we can create a function for it:
function addTodo() {
let input = document.querySelector("input");
let todos = document.querySelector(".todos");
todos.innerHTML += `
<div class="todo">
<input type="checkbox" />
${input.value}
</div>
`;
todos.scrollTop = todos.scrollHeight;
input.value = "";
input.focus();
}
We will use this function below.
Events
Every time we interact with HTML, events are sent. They can be moving the mouse, mouse clicks, key presses, hovering over an element...
And we can exexute some code when they happen.
Like this:
<button onclick="addTodo()">Add TODO</button>
Or like this:
let button = document.querySelector("button");
btn.onclick = function () {
addTodo();
};
addEventListener
A more flexible and modern approach for handling events.
When a button is clicked...
document.querySelector("button").addEventListener("click", addTodo);
Or when the "Enter" key is pressed.
document.querySelector("input").addEventListener("keyup", (e) => {
if (e.key == "Enter") {
addTodo();
}
});
Try it here:
The new todos are not permanent because they are not stored in a database. When you refresh the page, they will be gone.
I order to save them, or retrieve them from a database, we need to use AJAX.
AJAX (Asynchronous JavaScript And XML)
Used to send HTTP requests from client to server, without reloading the web page.
Instead of getting the whole page as HTML from the server, we can only get just the data and we can show it in the client via the DOM.
The data is received in the JSON format.
JSON (JavaScript Object Notation)
The data recieved from servers looks like this:
[
{
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
},
{
"userId": 1,
"id": 2,
"title": "quis ut nam facilis et officia qui",
"completed": false
},
{
"userId": 1,
"id": 3,
"title": "fugiat veniam minus",
"completed": false
}
]
The same format is used for sending data.
XHR (XMLHttpRequest)
The original way to do AJAX looked like this:
let xhr = new XMLHttpRequest();
xhr.open("GET", "https://jsonplaceholder.typicode.com/todos", true);
xhr.onload = function () {
console.log(JSON.parse(xhr.responseText));
};
xhr.send();
fetch
A much nicer way to do AJAX:
fetch("https://jsonplaceholder.typicode.com/todos")
.then((res) => res.json())
.then((data) => console.log(data));
async/await
An cleaner way to do fetch.
let response = await fetch("https://jsonplaceholder.typicode.com/todos");
let data = await response.json();
console.log(data);
API (Application Programming Interface)
API = server that sends and receives data
This can also be called a web service or back-end.
When the button receives a click EVENT, we will use AJAX to make an HTTP GET request to the typicode API, which will send us JSON, which we then convert it into HTML and add it to the DOM.
This is the code:
document.querySelector("button").addEventListener("click", async () => {
let response = await fetch("https://jsonplaceholder.typicode.com/todos");
let todos = await response.json();
let json = document.querySelector(".json");
json.innerHTML = JSON.stringify(todos);
let html = document.querySelector(".html");
todos.forEach((todo) => {
html.innerHTML += `
<div class="todo">
<input type="checkbox" ${todo.completed ? "checked" : ""}/>
${todo.title}
</div>
`;
});
});
This was someone else's API/back-end. Now it's time to build your own, but first, some Linux!
Linux
Linux
Operating system like Windows, but free. 90% of all servers run on Linux. Your's will too.
There are many versions of it called distros, the most popular one is Ubuntu.
You can either install Ubuntu directly as an OS on your computer instead of Windows...
Or, even better, use it from inside Windows via terminals with something called WSL (Windows Subsystem for Linux).
WSL (Windows Subsystem for Linux)
Allows you to run Linux inside Windows via terminals.
To install it, first you have to enable it:
Start Menu / Turn Windows features on or off / Enabling "Windows Subsystem for Linux"
Then you can download it via the Microsoft Store...
Open it...
Wait to finish installing...
Enter a username...
And that's it! You no have Linux installed inside Windows.
Linux (Ubuntu) is installed in Windows here:
C:\Users\User\AppData\Local\lxss
You can access the Windows C: drive inside the Linux terminal like this:
cd /mnt/c
Terminal
It allows you to control a computer with text commands, instead of a mouse and a User Interface.
It's also called CLI (Command Line Interface) or shell.
At first, it looks stupid and ugly to work like this, BUT TRUST ME, you will LOVE it later. It's insanely powerful.
Normally, you open folders and files by double-clicking them...
But, you can to the same with a terminal:
Here are the most useful commands for...
Navigation
ls -a # List ALL folders and files
cd folder # Go inside folder
cd .. # Go back outside folder
. # Shorthand for current directory
cd ~ # Go to your Linux "desktop"
cd / # Go to the Linux "C:" directory
cat file # Print file content
less file # Display file content
Creating folders and files
mkdir folder # Create folder
touch file.txt # Create file
cp file.txt file2.txt # Create a copy
rm file2.txt # Delete file
mv file file3 # Rename file
Editing files
nano file.txt # Edit file with nano
# Save with CTRL + x then y then Enter
# Exit with CTR + x then n
vim file.txt # Edit file with vim
# Save with ESC then :wq then Enter
# Exit with ESC then :q then Enter
Stop a command
CTRL + c
Users
You can create a new user like this:
adduser USERNAME
And swith between users with:
su - USERNAME
There are 2 user types. One is a regular one, like the one you created during installation. The other one is called root
, which is basically god in the OS.
root
has full control over Linux, and it is NOT advisable to run commands with it. You should use your own username.
Why am I telling you this? Because sometimes commands need root
access to execute commands, and you do it with sudo
which stands for super user do
.
Like changing a username...
sudo usermod -l NEW_USERNAME OLD_USERNAME
Or changing a password.
sudo passwd
binaries
All of these commands are actually mini programs, called binaries or executables, which are just files.
So, instead of doing this command to list all files:
ls
You can manually execute it by running the actual binary:
/usr/bin/ls
This is where it's located. You can check where other commands are located with this:
which COMMAND_NAME
The connections between commands and binaries is defined in an environment variable called $PATH
.
PATH
$PATH
is an enviroment variable which includes a list of important DIRECTORIES that include program executables i.e. allows you to run commands.
Each time you run a command, the shell checks all the DIRECTORIES in the $PATH
variable if they have the executable.
The value of $PATH
looks like this:
/home/name/.nvm/versions/node/v20.13.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
It's just a list of folders separated by :
i.e. this:
/home/name/.nvm/versions/node/v20.13.1/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
Why, am I telling you this?
Because if you get a "command not found" error when executing a command, it means it's either no installed i.e not in the directories listed in $PATH
, or the directory itself is not listed in the $PATH
.
You can add something to it with:
export PATH=$PATH:/usr/hitech/picc/9.82/bin
Services
Services are programs/binaries/executables that run continuously in the background.
You start them with:
sudo service SERVICE_NAME start
And end them with:
sudo service SERVICE_NAME stop
You can check if they are running with:
sudo service SERVICE_NAME status
apt (Advanced Packaging Tool)
Terminal command used to install programs/binaries/executables, also called packages.
Just like Windows, Linux has its own programs, and they are all located in one central place you download them from, unlike Windows wher you have to look for the programs.
First, update the list of available programs:
sudo apt update
Then install a specific program with:
sudo apt install PACKAGE_NAME
You can update already installed programs with this:
sudo apt upgrade
Update
You can use this single command to both update and upgrade the packages:
sudo apt update && sudo apt upgrade
Back-end
Node
Allows you to run Javascript outside of browsers i.e. on your computer directly, with access to your OS.
Don't install it directly, use nvm
.
nvm (Node Version Manager)
Allows you to install multiple node versions and switch between them.
In your terminal, use this command to install it:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
This command uses the curl
program to download the install.sh
file from the link, and then execute it by piping
it to the bash
program.
MUST RESTART TERMINAL AFTER INSTALL FOR nvm
TO WORK!
Check if it works with:
nvm -v
Now install node:
nvm install node
Check if it works with:
node -v
Script
You can execute this hello.js
script...
console.log("Hello World!");
Like this in the terminal:
node hello.js
npm
Installs node
packages/libraries/modules...
Which are then used inside of your code.
Basically, you download someone else's code, and use it, instead of writing it.
You do it like this:
npm install PACKAGE_NAME
This creates a node_modules
folder where all the packages go.
Now, instead of manually installing mulitple packages, you can just do...
npm install
Which will look for all packages listed inside a file called package.json
package.json
This file includes all the packages used in your project. You can create one with:
npm init
It looks like this:
{
"name": "app",
"version": "1.0.0",
"main": "index.js",
"scripts": {},
"author": "",
"license": "ISC",
"description": ""
}
To add a package, you can either write it manually, or add --save
to your install command:
npm install mysql --save
The package.json
file will look like this now:
{
"name": "app",
"version": "1.0.0",
"main": "index.js",
"scripts": {},
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"mysql": "^2.18.1"
}
}
You can install all the packages listed in dependencies at once with:
npm install
express
This is an npm
package used for creating an API
i.e. back-end, like the todos one shown before.
npm install express --save
Here's a very simple server/API, similar to the one we used before...
Put this code inside a server.js
file:
let express = require("express");
let app = express();
let port = 3000;
app.use(express.json());
app.get("/api/todos", (req, res) => {
let todos = [
{ id: 1, title: "Buy milk", completed: false },
{ id: 2, title: "Walk dog", completed: false },
{ id: 3, title: "Hate React", completed: true },
];
res.status(200).send(todos);
});
app.listen(port, () => {
console.log(`Server is running on http://localhost:${port}`);
});
And run it with:
node server.js
Now, in your browser, go to this URL:
http://localhost:3000/api/todos
And you should see the todos JSON
.
Note that the todos are hard-coded in the API. In a real API, these would come form the database.
nodemon
An npm
package for continuously running a node script, especially when it fails.
You install it like this:
npm install nodemon
And then you run the server with:
nodemon server.js
You can do it like this:
REST
A mutually-agreed convention how to write API endpoints/routes/URLs/URIs.
Method | Resource | Result |
---|---|---|
GET | /todos | Get a list of todos |
GET | /todos/12 | Get a specific todo |
POST | /todos | Create a new todo |
PATCH | /todos/12 | Update todo with ID 12 |
DELETE | /todos/12 | Delete todo with ID 12 |
Here's what the full server.js
file would look like.
let express = require("express");
let app = express();
let port = 3000;
app.use(express.json());
let todos = [
{ id: 1, title: "Buy milk", completed: false },
{ id: 2, title: "Walk dog", completed: false },
{ id: 3, title: "Hate React", completed: true },
];
app.get("/api/todos", (req, res) => {
res.status(200).send(todos);
});
app.get("/api/todos/:todoId", (req, res) => {
let { todoId } = req.params;
let todo = todos.find((item) => item.id == todoId);
res.status(200).send(todo);
});
app.post(`/api/todos`, async (req, res) => {
let todo = req.body;
let newTodo = {
id: todos.length + 1,
title: todo.title,
completed: todo.completed,
};
todos.push(newTodo);
res.status(200).send(newTodo);
});
router.patch(`/api/todos/:todoId`, async (req, res) => {
let { todoId } = req.params;
let todo = req.body;
let updatedTodo = todos.find((item) => item.id == todoId);
updatedTodo.title = todo.title;
updatedTodo.completed = todo.completed;
res.status(200).send(updatedTodo);
});
router.delete(`/api/todos/:todoId`, async (req, res) => {
let { todoId } = req.params;
todos = todos.filter((item) => item.id != todoId);
res.status(200);
});
app.listen(port, () => {
console.log(`Server is running on http://localhost:${port}`);
});
Database
DBMS (Database Management System)
Software for creating databases and handling the data in them.
The most popular ones are:
- MySQL
- SQL Server
- Postgresql
- SQLite.
MySQL
The most widely used one. Here are some popular users:
- YouTube
- Netflix
- WordPress
You install it like this:
sudo apt install mysql-server -y
Start it with:
sudo service mysql start
Check if it runs:
sudo service mysql status
Log in with:
sudo mysql
Once logged in, you need to update the authentication method for the root
user to use a password.
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'root';
FLUSH PRIVILEGES;
Log out:
exit
Now try logging in with the new password:
sudo mysql -u root -p
If you can't login, this is probably the problem:
- MySQL is not started. Start it with
sudo service mysql start
. - You are not using
sudo
. - There is no password. Login with just
sudo mysql
.
SQL (Structured Query Language)
The language used to interact with the database via commands called queries.
Once you are logged in the DBMS
i.e. MySQL
, you use SQL
.
Workbench
You can execute SQL
queries in the terminal or in an app with an actual user interface.
You can download it here:
https://www.mysql.com/products/workbench
And it looks like this:
We will use the terminal for the practice.
Schema
The database is all the data. Think of it as an Excel file.
The schema is the blueprint for a database i.e. everything that is not the actual data.
Ex. table definitions, colum definitions, data types, relationships, indexes...
Think of it as the settings in the Excel file.
You can see the databases with this SQL query:
show databases;
Or you can create your own:
create database todo;
Note that you must select the database in order to do things in it.
use todo;
Table
Data is organized in tables. Think of them like sheets in an Excel file.
Every table has columns and rows. Exactly like in Excel.
You can see the all the tables in a database with:
show tables;
Or you can create one:
create table todo (
id int primary key auto_increment,
title varchar(255),
completed int
);
Columns
When you created the table, you also defined the columns it will have and their names, along with the type of data they will store.
This of them like the titles of Excel columns.
You can also add a column after a table is created:
alter table todo add created datetime;
Rows
The actual data is store in rows, the same as Excel.
Every row is a collection of values that correspond to the columns.
You add rows like this:
insert into todo (title, completed, created) values ('learn SQL', 0, now());
You can add multiple at once:
insert into todo
(title, completed, created)
values
('Create you dream idea', 0, now()),
('???', 0, now()),
('Profit!', 0, now())
;
You can see the result of this with:
select * from todo;
If you want to update a value, you do this:
update todo set completed = 1 where id = 1;
IT IS SUPER IMPORTANT TO USE A where
CLAUSE, BECAUSE THIS WILL UPDATE EVERY ROW, NOT JUST THE ONE YOU WANT!!!
THE SAME APPLIES WHERE DELETING DATA!!!
delete from todo where id = 1;
A GOOD PRACTICE IS TO FIRST SELECT THE DATA, THEN UPDATE OR DELETE IT.
node + mysql
This is a database driver, which is a fancy way to say an npm package for connecting to your database.
There are many, but the best one is the default mysql
one.
npm install mysql --save
Connection
You connect to your database like this:
let mysql = require("mysql");
let connection = mysql.createConnection({
host: "localhost",
database: "todo",
user: "root",
password: "root",
});
connection.connect();
And then you run your queries:
connection.query("select * from toro");
And you end the connection.
connection.end();
Pool
Opening and closing conections is costly, so a better way is to connect once, create a bunch of connections, and then just borrow them for executing queries, and finally returning them.
The best way to do this is to create the following files:
db.js
module.exports.db = {
host: "localhost",
database: "todo",
user: "root",
password: "root",
connectionLimit: 100,
};
pool.js
const mysql = require("mysql");
const db = require("./db.js");
const pool = mysql.createPool(db);
module.exports = pool;
Optionally add this "promisify" code in the pool.js
file to make the query
function asynchronous i.e. async/await
.
const util = require("util");
pool.query = util.promisify(pool.query);
Finally, you can now use actual SQL
in your server.js
.
It's a good idea to handle errors when executing queries.
let express = require("express");
let pool = require("./pool.js");
let app = express();
let port = 3000;
app.use(express.json());
app.get("/api/todos", (req, res) => {
try {
let query = `
select *
from todo
;
`
let todos = await pool.query(query);
res.status(200).send(todos);
} catch (error) {
console.log(error)
}
});
app.get("/api/todos/:todoId", (req, res) => {
try {
let { todoId } = req.params;
let query = `
select *
from todo
where id = ?
;
`
let todo = await pool.query(query, [todoId]);
res.status(200).send(todo);
} catch (error) {
console.log(error)
}
});
app.post(`/api/todos`, async (req, res) => {
try {
let todo = req.body;
let query = `
insert into todo
(
title,
completed,
created
)
values
(
?,
?,
now()
)
;
`;
let todo = await pool.query(query, [userId,todo.completed, todo.title]);
res.status(200).send(todo);
} catch (error) {
console.log(error)
}
});
app.patch(`/api/todos/:todoId`, async (req, res) => {
try {
let { todoId } = req.params;
let todo = req.body;
let query = `
update todo
set
title = ?,
completed = ?
where id = ?
;
`;
let todo = await pool.query(query, [todo.title, todo.completed, todoId]);
res.status(200).send(todo);
} catch (error) {
console.log(error)
}
});
router.delete(`/api/todos/:todoId`, async (req, res) => {
try {
let { todoId } = req.params;
let query = `
delete from todo
where id = ?
;
`;
let todo = await pool.query(query, [todoId]);
res.status(200)
} catch (error) {
console.log(error)
}
});
app.listen(port, () => {
console.log(`Server is running on http://localhost:${port}`);
});
Webserver
Webserver
A program inside the server that listens for HTTP commands on port 80, and sends back content, either directly or by redirecting to another program via ports.
Without a webserver, your website cannot be accessed.
Ports
Used to locate programs inside the server, similar to IP addresses locating devices.
NGINX
The most popular webserver.
sudo apt install nginx -y;
Start it with:
sudo service nginx start
Check if it runs:
sudo service nginx status
Servers are configured in the /etx/nginx/nginx.conf
file.
Here's what the simplest setup, which serves a static website, looks like:
events {} # Needed to be a valid conf file.
http {
include mime.types; # Recognize filetypes.
server {
listen 8080; # Port number
root /home/website; # Match URIs to files here.
index index.html; # index is inside folder
}
}
You can see your website at:
localhost:8080
You can if nginx.conf
is configured properly with:
sudo nginx -t
If it doesn't work, it's most probably because of permissions. You can add them like this:
# Make user the owner
sudo chown -R user:user /home/user/website
# Make user part of www-data group
sudo usermod -aG www-data user
# Read, write and execute permissions
sudo chmod -R 777 /home/user/website
CORS (Cross-Origin Resource Sharing)
This is WAY simpler than it looks.
It's just a rule saying your front-end (client) and back-end (server) must be on the same computer.
If they are not, you need to tell the back-end which front-ends can access it.
This is a security measure so that www.hacker.ru
can't ask your www.bank.com
do to stuff.
Proxy-pass
Instead of accessing your back-end from your client with localhost:3000/api
, you use a proxy_pass
so you can use it just with /api
.
Here's what the configuration looks like for the todo app, that uses an API:
events {}
http {
include mime.types;
server {
listen 8081;
root /home/todo; # Path to directory
index index.html;
location / {
# Fixes refresh resulting in 404 error for single page apps
try_files $uri $uri/ /index.html;
}
location /auth {
proxy_pass 'http://127.0.0.1:3001';
}
location /api {
proxy_pass 'http://127.0.0.1:3001';
}
}
}
include
You can also split an nginx.conf
file into mulitple files and link them in the main one.
website/nginx.conf (static)
server {
listen 80;
root /home/user/website;
index index.html;
}
docs/nginx.conf (SSR - Server Side Rendering)
server {
listen 80;
location / {
proxy_pass http://127.0.0.1:5000;
}
}
todo/nginx.conf (SPA - Single Page App i.e. front-end + back-end)
server {
listen 80;
root /home/user/todo/client;
index index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /auth {
proxy_pass http://127.0.0.1:3000;
}
location /api {
proxy_pass http://127.0.0.1:3000;
}
}
nginx.conf
events {}
http {
include mime.types;
include /home/user/website/nginx.conf;
include /home/user/docs/nginx.conf;
include /home/user/todo/nginx.conf;
}
Troubleshoot
Here are the most common reason the webserver doesn't work:
- Forgot to restart it after changing
nginx.conf
. - You have an error. Check with
sudo nginx -t
. - You might have more than 1 process running. Stop them with
sudo killall nginx
. - You forgot to add:
events {}
port
server_name
index
Security
.env
It's not a good idea to store your database credentials in code.
A better way is to store them in a .env
file.
This file is nothing magical, just a agreed upon way.
It looks like this:
DB_NAME=todo
DB_USER=root
DB_PASSWORD=root
This is a package that reads .env
files, and makes the content available in server.js
.
Install it like this:
npm install dotenv --save
And use this .env
file...
DB_NAME=todo
DB_USER=root
DB_PASSWORD=root
In your db.js
file like this:
const path = require("path");
require("dotenv").config({ path: path.resolve(__dirname, ".env") });
module.exports.db = {
server: "localhost",
database: process.env.DB_NAME,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
connectionLimit: 100,
};
Authentication
In order to implement authentication, the app needs to be modified in several places:
- Create the
user
database table. - Add a
userId
column in thetodo
table, so we which todo belongs to which user. - Create a
signup
page so users can be created. - Create a
login
page for existing users. - Create the
API
endpoints for signup and login. - Install the
JWT
(tokens) issuing library. - Install the
SHA256
hashing algorithm for safely storing passwords. - Add a
router
so that we can redirect anyone without access to thelogin
page. - Modify the
SQL
queries to have auserId
where
clause, so users get only their todos. - Add
middleware
for token verification.
You can see the whole code in the TODO topic.
Before we do that, let's see how JWT
and SHA256
work.
JWT (JSON Web Token)
A token looks like this:
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOjEsImlhdCI6MTcyNDI1NTczNywiZXhwIjoxNzU1NzkxNzM3fQ.WW218MHiBX2VaU-GmTXmXvzTBe-4ZwSeg2E_HNlbTr0
This is actually a base64
encoded string separated by a .
in 3
parts.
base64
is NOT encryption, it just saves characters and makes it URL
friendly. Anyone can decode it.
If you decode it, it looks like this:
The first part eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
(header) decoded is:
{
"alg": "HS256",
"typ": "JWT"
}
The second part eyJ1c2VySWQiOjEsImlhdCI6MTcyNDI1NTczNywiZXhwIjoxNzU1NzkxNzM3fQ
(payload) decoded is:
{
"userId": 1,
"iat": 1724255737,
"exp": 1755791737
}
The third part WW218MHiBX2VaU-GmTXmXvzTBe-4ZwSeg2E_HNlbTr0
(verification) decoded is a SHA256
hash of 3 parts:
- The
base64
of the first part (header). - The
base64
of the second part (payload). - The token secret i.e. a password you define that only the server knows.
So, what is SHA256
?
Middleware
This is some code that is executed between:
- Receiving the request and executing the query. ex. Authentication.
- Executing the query and sending the response back. ex. Error logging.
let express = require("express");
let app = express();
let port = 3000;
app.use(express.json());
function authenticationMiddleware(req, res, next) {
let token = req.cookies.token;
jwt.verify(token, config.tokenSecret, (err, decoded) => {
if (err) {
res.status(401).send({
status: 401,
message: "No access!",
});
} else {
req.userId = decoded.userId;
next();
}
});
}
function errorLogger(err, req, res, next) {
console.error(`Error occurred: ${err.message}`);
res.status(500).send({ error: "Internal Server Error" });
}
app.use(authenticationMiddleware);
app.get("/api/todos", (req, res, next) => {
try {
let query = `
select *
from todo
;
`
let todos = await pool.query(query);
res.status(200).send(todos);
} catch (error) {
next(error)
}
});
app.use(errorLogger);
app.listen(port, () => {
console.log(`Server is running on http://localhost:${port}`);
});
SHA256
It's an algorithm that converts ANY sized string to a fixed length random-character string, that is IMPOSSIBLE to convert back.
For example, the string a
hashed is:
ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb
The password 12345678
is:
ef797c8118f02dfb649607dd5d3f8c7623048c9c063d532cc95c5ed7a898a64f
The USA Declaration of Independence is this:
c803200a5f8119e0c16b4c5da548fd9559742a86a0af2befc54416d9305554d6
Fun fact: This is how crypto
works. Each block
in the chain
contains the hash
of the previous blocks
. When you are mining
, you are actually looking for a specific number, that when added to the block
results in hash
that starts with several zeros.
SHA256
is useful for 2 things:
- Verify that some content has not been changed.
- Make it impossible to know the content.
In our case, we store the SHA256
hash of the user passwords, not the actual passwords. This way you don't know them, as well as anyone that steals them.
How it works in practice is, a user logs in, and his real passwords is hashed, and then compared to the hash you have in the database.
SQL Injection
This is THE MOST DANGEROUS vulnerability a back-end can have.
It is INCREDIBLY important to NOT trust ANYTHING that comes from the client.
Everything MUST be sanitized before it is executed as a query.
This is already done in the code above.
THIS IS WHAT YOU SHOULD NEVER DO!!!
app.get("/api/todos/:todoId", (req, res) => {
try {
let { todoId } = req.params;
let query = `
select *
from todo
where id = ${todoId} -- <-- THIS HERE!!!
;
`
let todo = await pool.query(query);
res.status(200).send(todo);
} catch (error) {
console.log(error)
}
});
Notice how the todoId
is directly used in the query, instead of being sanitized by the mysql
library.
This is how you get hacked.
Instead of a doing sending a todoId
in the API request...
GET https://localhost:3000/api/todos/1
Someone can send SQL
code, and ruin your day, like this:
GET https://localhost:3000/api/todos/1%20or%201%3D1
Which is this:
GET https://localhost:3000/api/todos/1 or 1=1
Which turns your SQL
query into:
select *
from todo
where id = 1 or 1=1
;
And since 1=1
is always true, the where id = 1
is ignored, and all the todos will be returned, instead of one.
An even more fun one is this:
GET https://localhost:3000/api/todos/1%3B%20drop%20database%20todo%3B%20--
Which is:
GET https://localhost:3000/api/todos/1; drops database todo; --
Which is:
select *
from todo
where id = 1; drop database todo; -- ;
Which is bye bye to your whole database.
Server
Cloud
This is just a term for "a bunch of servers in datacenters".
Host
This is a business that manages datacenters and offers cloud services.
One of the services is creating your own "virtual" server (VPS), instead of creating your own physical machine.
The most user-friendly host is Digital Ocean.
You can create an account here:
VPS (Virtual Private Server)
After you create an account, you create a server like this...
Click on Create and select Droplets (their fancy name for VPS):
Select the region closest to your users:
Choose your operating system i.e. Linux distro and version:
Choose your machine:
Insert your server password:
Give your server a name, and click "Create":
Wait for the creation to finish:
And that's it! You now have your own server, living on a dedicated IP address.
IP Address
This is used to identify and locate servers. Like a phone number.
In this case, the address is:
46.101.132.192
We can now connect to it by using SSH
.
SSH (Secure Socket Shell)
Think of this like Teamviewer via the terminal i.e. without a UI.
You use it like this:
ssh root@46.101.132.192
It will ask you for the server password.
After that, you are connected to the server you created.
To change your root
password, use:
passwd root
You now MUST create a user
other than root
. This is for security reasons:
adduser user
# You can delete users with:
userdel user
And give it root access.
usermod -aG sudo user
Then you login as the new user.
su - user
You can now exit by tying, well... exit
.
Here's the whole thing:
Keys
Using your password for every SSH
login is both highly inneficient and insecure.
To avoid that, you can use private
and public
keys.
You create them like this:
ssh-keygen
The keys will be located in ~/.ssh
.
Now you just transfer them to the server like this:
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@46.101.132.192
You need to use your own public
key name.
Now you can connect without typing your password:
ssh user@46.101.132.192
Environment
You can now prepare your server for your app.
You need to replicate whatever you have on your local machine.
In this case that would be...
- Update your Linux packages. INSTRUCTIONS
- Install
nvm
andnodejs
. INSTRUCTIONS - Install
mysql
. INSTRUCTIONS - Install
nginx
(webserver). INSTRUCTIONS
pm2 (Process Manager)
You can run your server.js
file with:
node server.js
But, to run it forever in the background and restart it automatically when it crashes, you need a daemon
, and pm2
is a popular choice.
You install it like this:
npm install pm2 -g
It is advisable to intall it globally since many apps can use it.
And then you can run your server.js
like this:
pm2 start server.js --name "todo"
You can check all your running apps with:
pm2 ls
And you can monitor them with:
pm2 monit
To stop an app, use this (you need to be inside the directory):
pm2 stop server.js
In oder for all this to work, you need the actual code files, which we can transfer with rsync
.
Deployment
rsync
This is a Linux package for transfering files.
We use this to transfer our code from our local computer to the remote server.
Here's the command, which you run on your local machine:
rsync -av -e 'ssh' ./client ./server --exclude 'server/node_modules' --exclude 'server/package-lock.json' --exclude 'server/.env' user@46.101.132.192:~/todo/
In order to see any changes, you need to restart your pm2
daemon:
ssh user@46.101.132.192 pm2 reload /home/user/todo/server/server.js --name "todo"
You can put both commands inside a deploy.sh
file...
rsync -av -e 'ssh' ./server --exclude 'server/node_modules' --exclude 'server/package-lock.json' --exclude 'server/.env' user@46.101.132.192:~/todo/ \
&& ssh user@46.101.132.192 pm2 reload /home/user/todo/server/server.js --name "todo"
So you can run the file with:
./deploy.sh
This will use SSH
to transfer the client
and server
folders from your local machine, inside a todo
folder on the remote machine.
It will not transfer the node_modules
folder, as well as the .env.
file.
You will need to manually create the .env
file with your production credentials.
You can connect to the server to check if everything was transferred correctly.
ssh user@46.101.132.192
Production db
You need to create the database by running this command:
mysql -u root -p < schema.sql
Production .env
Create this file in the server directory:
touch .env
Edit it with:
vim .env
And put the credentials inside like you did in your local environment, but use your production (server) ones.
npm install
After you've transferred the files, you can now install the packages.
First go to the server folder
cd todo/server
And then do
npm install
In order for everything to work, you need to configure your webserver
i.e. nginx
.
nginx.conf
You can edit the nginx.conf
file with:
sudo vim /etc/nginx/nginx.conf
The configuration should look like this:
events {} # Needed to be a valid conf file.
http {
include mime.types; # Recognize filetypes.
server {
listen 80;
server_name 46.101.132.192;
root /home/user/todo/client; # Path to directory
index index.html;
location / {
try_files $uri $uri/ /index.html; # Fixes refreshing resulting in 404 error for single page apps
}
location /auth {
proxy_pass http://127.0.0.1:3001;
}
location /api {
proxy_pass http://127.0.0.1:3001;
}
}
}
The app is now available at:
46.101.132.192
Domain
Domain
User-friendly name for an IP addresse, like contact name.
They are sold and managed by domain registrars.
You can buy a domain here:
After you buy it, you need to go to the Networking page in your Host (cloud provider):
And add your domain:
After this, you have to configure the NS
i.e. name servers.
NS (Name Servers)
Name servers define which DNS
server to use i.e which server contains the actual DNS records.
You need to configure this on both the reigstrar's side, as we all the host's side.
For the registrar, you go to your domain and click on manage.
And inside, you add your host's nameservers under custom DNS
.
And, for the host's side, you do the same. Usually done automatically when you add the domain.
DNS (Domain Name System)
Container domains
into IP addresses
, like a phonebook.
It contains various DNS records which associate server locations to letters.
It's done because computers only understand numbers (IP address), not letters (domain).
It works automatically in the background.
A Record
Connects the domain
name to the IP address
.
It's done like this:
- Type
@
in the "hostname" field. - Select your server's
IP address
in the "Will direct to" field. - Click "Create record"
CNAME
This record is for defining an alias. This is used so that the www
in www.domain.com
can work.
Propagation
You need to wait for some time for the configurations to go live, first for the NS
(name servers) and then for the domain
.
It's not uncommon to take 24+ hours.
You can check how far they've propagated with this tool:
For the NS
(name server) record:
For the A
(domain) record:
SSL (Secure Sockets Layer)
This is https
instead of http
i.e. the data sent via HTTP is encrypted.
To add this to your server, you can use certbot
to generate the certificates, and then add them in the NGINX
configuration.
certbot
Linux package for issuing and renewing SSL
certificates for domains
and subdomains
.
Note that they need SEPARATE certificates.
Install certbot
:
sudo snap install --classic certbot
BEFORE USING IT, YOU MUST DISABLE NGINX
BECAUSE certbot
USES IT'S OWN VERSION. YOU MUST KILL EVERY PROCESS.
DO NOT ADD THE CERTIFICATES IN NGINX
BEFORE THEY ARE CREATED.
MAKE SURE THAT A
RECORDS EXIST IN THE DNS
BEFORE ISSUING.
sudo killall nginx
You can now create an SSL
certificate:
sudo certbot certonly --nginx -d codepreneuring.com
You can see the certificate
i.e. keys
like this:
The nginx.conf
file must now be changed to include them.
Here's the FULL nginx.conf
for this website.
website/nginx.conf (static)
server {
listen 80;
server_name codepreneuring.com www.codepreneuring.com;
return 301 https://codepreneuring.com$request_uri;
}
server {
listen 443 ssl;
server_name codepreneuring.com;
ssl_certificate /etc/letsencrypt/live/codepreneuring.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/codepreneuring.com/privkey.pem;
root /home/user/website;
index index.html;
}
docs/nginx.conf (SSR - Server Side Rendering)
server {
listen 80;
server_name docs.codepreneuring.com;
return 301 https://docs.codepreneuring.com$request_uri;
}
server {
listen 443 ssl;
server_name docs.codepreneuring.com;
ssl_certificate /etc/letsencrypt/live/docs.codepreneuring.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/docs.codepreneuring.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:5000;
}
}
todo/nginx.conf (SPA - Single Page App i.e. front-end + back-end)
server {
listen 80;
server_name todo.codepreneuring.com;
return 301 https://todo.codepreneuring.com$request_uri;
}
server {
listen 443 ssl;
server_name todo.codepreneuring.com;
ssl_certificate /etc/letsencrypt/live/todo.codepreneuring.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/todo.codepreneuring.com/privkey.pem;
root /home/user/todo/client;
index index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /auth {
proxy_pass http://127.0.0.1:3000;
}
location /api {
proxy_pass http://127.0.0.1:3000;
}
}
nginx.conf
events {}
http {
include mime.types;
include /home/user/website/nginx.conf;
include /home/user/docs/nginx.conf;
include /home/user/todo/nginx.conf;
}
You can now start/restart/reload NGINX
for this to work:
service nginx start
The app will now be available with SSL
at:
https://codepreneuring.com
If it still doesn't work, it's probably a caching issue. Try opening the domain:
- In an
incognito
window. - On your phone via
5G
instead ofWiFI
.
Todo
Code
The code is located here:
https://github.com/codepreneuring/todo
It is uploaded there for 2 reasons:
- It would take too much space here.
- Everyone should keep their code on
Github
. There's an explanation why and how to do this a bit later.
Download
You can download it in 3 ways:
- Clicking on the
Code/Download ZIP
button.
- Using this download link:
https://github.com/codepreneuring/todo/archive/refs/heads/main.zip
- By running this terminal command:
git clone https://github.com/codepreneuring/todo.git
This will create a todo
folder with all the code, in the directory you execute the command in.
Setup
After the download, go inside the todo server folder via the terminal.
cd todo/server
- Install all the packages.
npm install
- Start the server"
nodemon server.js
- Configure
nginx.conf
events {}
http {
include mime.types;
server {
listen 8081;
root /mnt/c/user/codepreneuring/todo/client; # Path to directory
index index.html;
location / {
try_files $uri $uri/ /index.html; # Fixes refreshing resulting in 404 error for single page apps
}
location /auth {
proxy_pass 'http://127.0.0.1:3001';
}
location /api {
proxy_pass 'http://127.0.0.1:3001';
}
}
}
- Go to:
localhost:8081
Editor
VS Code
Download and install it:
It looks like this:
If you are using WSL, it's a good idea to connect to it with CTRL
+ Shift
+ P
and choosing WSL: Connect to WSL
.
Shortcuts
The most useful productivity shortcuts for VS Code
.
# Duplicates
CTRL + d # Select next duplicate value
CTRL + u # Unselect duplicate value
CTRL + SHIFT + l # Select all duplicate values
# Multiple cursors
ALT + click (empty) # Multiple cursors
ALT + SHIFT + click # Multiple selections in range
# Line
ALT + up/down # Move line up or down
ALT + SHIFT + up/down # Duplicate line
# Jumping
CTRL + click (variable) # Jump to definition
# Utility
CTRL + SPACE # Show code completion suggestions
jsbin.com
Excellent tool for quick experimentations.
Git
Git
This is a program used for code versioning.
It saves you from doing file.txt
, file-updated.txt
, file-revision-2.txt
, file-FINAL.txt
... Instead, all of this is tracked in the background.
This program usually is built-in in all modern Linux distros, or you can install it with this:
sudo apt install git
.gitignore
This is a simple file where you list which folders and files you do NOT want to save.
The file has to be named .gitignore
and placed in the root of the project folder.
Putting this inside the file...
index.html
Means index.html will not be saved with git.
Git commands
To start using Git, initialize it inside your project folder:
git init
This will create a .git folder (invisible), inside your project, where all the changes are stored. This is called a repository, repo for short.
Each repo has its own configurations. You can see them with:
git config --list
Here are the essential configurations you need to add:
# Configure owner of changes just for current repo
git config user.email "codepreneuring@gmail.com"
git config user.name "codepreneuring"
Github contributions count only if the repo email is the same as the github email.
You can check which files are new (untracked) or unsaved (unstaged) with this command:
git status
To save something, first you need to mark the files as "to be saved" i.e. "tracked" or" staged":
git add . # . means everything in current directory
The actual saving is done like this:
git commit -m "Any message you want for this save"
Here's how this looks in the terminal:
You can do as many saves as you want.
Let's change our file with our VS Code editor.
You can compare the changes between the saved file and the updated file with:
git diff
Save the changes like you did before:
You can see all the saves with this command:
git log
Notice the weird random characters (hash) in yellow. These are the names of the saves (commits).
Github
This is where you upload your code, which you can share with others. Like Google Drive is for files and pictures.
Github is NOT part of Git, it just works with it. There are other websites that offer the same services, like Gitlab and Bitbucket... Github is just the most popular one.
If you want to upload your code there, which is HIGHLY recommended, you can create an account here:
Your Github username and email MUST match the ones you used in your Git configuration, so that the commits can show up on Github. I'm talking about these:
git config user.email "codepreneuring@gmail.com"
git config user.name "codepreneuring"
This is what your Github account will look like:
Github password
After you've created your account, you will need to create a separate password for uploading your code, like this:
- Go to...
Settings / Developer Settings / Personal Access Token / Tokens (classic)
- Click on Generate New Token (classic)...
- Add a note. Select no expiration. Toggle the repo checkbox. Click on Generate token...
- Save the token somewhere because you won't be able to see it again.
Github repository
Now that you have an account and password (token), you can finally upload your code to github.
First you need to create a new repository (repo) via the top menu...
Fill up the repo details...
And that's it. The repo you've created is now empty, and needs some code to be uploaded.
You will be asked for the Github password, so to save it for future saves, use this command:
git config credential.helper store
It saves the password in a .git-credentials
file.
Since we already have the code that we saved with Git, we can upload that.
The first time you upload code, you use these commands, shown in the empty repo at the bottom:
git remote add origin https://github.com/codepreneuring/todo-file.git
git branch -M main
git push -u origin main
After that, you only use:
git push
After code is uploaded, your repo will look like this:
You can download repos via the terminal:
git clone https://github.com/codepreneuring/todo.git
Or directly: