From zero to a hand that copies you

The first program: variables and types

What really happens when a program "runs", how you set a value aside, and why TypeScript scolds you before running the code instead of after.

• Where you work: TypeScript Playground, nothing to install
• New concepts: statement, console.log, let and const, the types string number boolean, annotation and inference, strings with `${}`
• By the end the student can: write and run a ten-line program, read a compiler error message and understand which line it refers to

Theory

A program is a list of statements that the computer runs one after the other, from top to bottom. The most useful statement at the start is the one that writes a message:

console.log("ciao");
console.log("mondo");

Two lines, two messages, in that order. There is no magic: if you swap the lines, the result is swapped too.

A variable is a name given to a value, so you can reuse it. In TypeScript you declare two kinds:

let punteggio = 0;        // can change later
const nome = "Giulia";    // will never change

punteggio = punteggio + 10;   // ok
nome = "Marco";               // ERROR: const cannot be reassigned

Every value has a type. The three you need today: string (text, in quotes), number (numbers, whole or with a decimal point) and boolean (only true or false).

const nome: string = "Giulia";
const eta: number = 17;
const maggiorenne: boolean = false;

That : string is the type annotation: it is the part that does not exist in JavaScript and the reason this language is called TypeScript. You can often leave it out, because the compiler works out the type by itself by looking at the value (we say it infers it):

const nome = "Giulia";   // TypeScript already knows it is a string
nome.toUpperCase();      // in fact here it suggests the text methods

And what is all this for? To make errors appear while you type, not while the program runs in front of users:

let eta = 17;
eta = "diciassette";
// Error: Type 'string' is not assignable to type 'number'.
// In plain words: "you said a number goes here, and this is text".

Last piece: joining text and values. With backticks ` you write strings into which you can insert values with ${…}:

const dita = 5;
console.log(`la mano ha ${dita} dita`);
console.log(`e ${dita * 4} falangi... circa`);

Teacher demo (15')

1. Open the finished hand and move your hand in front of the webcam. Say: "in ten lessons you will write this yourselves".
2. Open the Playground and write three lines with console.log.
3. Declare const punta = 8; and explain that this number, in the final project, will be the tip of the index finger.
4. Get a type wrong on purpose and read the error together with the class.

Lab (60')

console.log("Giulia");
console.log(17);

const nome = "Giulia";
const eta = 17;
const amaLaPizza = true;

console.log(`${nome} ha ${eta} anni e la pizza le piace: ${amaLaPizza}`);

const larghezza: number = 640;
const altezza: number = 480;
const fotogrammiAlSecondo: number = 30;

console.log(`un fotogramma ha ${larghezza * altezza} pixel`);
console.log(`in un minuto arrivano ${fotogrammiAlSecondo * 60} fotogrammi`);
console.log(`il disegno gira ${fotogrammiAlSecondo} volte al secondo`);

const x = 0.5;
const y = 0.25;
const larghezza = 640;
const altezza = 480;

console.log(`pixel x: ${x * larghezza}`);   // 320
console.log(`pixel y: ${y * altezza}`);     // 120

Typical errors in class

• Wrong quotes: the operating system's autocorrect turns " into “ ”. It does not happen in the Playground, but it does if someone writes the code in a text document and pastes it.
• Backtick nowhere to be found: on an Italian keyboard it is Alt Gr + '. Worth showing on the board: without it, exercise 1.2 gets stuck.
• Comma instead of a point in decimals: 0,5 is not a number, 0.5 is.
• "I can't see anything": the Playground's message panel is on the right, Logs tab. Show it once and for all at the start.

Quick check (5 questions)

Homework (20-30 minutes)

Write a program that describes your hand: how many fingers, the name of the longest finger, whether you are left-handed. It must use at least one const, one let, the three types seen and a sentence with backticks. Then share the Playground link with the teacher (Share button: the code travels inside the address, no account needed).

Deciding and repeating: if, loops, arrays

So far the program went straight from top to bottom. Today it learns to choose one path instead of another and to repeat the same work on many values. It matters because the hand is not a number: it is twenty-one points, and all of them must be checked.

• Where you work: TypeScript Playground, still without an account
• New concepts: comparison operators, && || !, if/else if/else, for, for…of, while, typed arrays, length, union types
• By the end the student can: go through a list of numbers, count how many times a condition holds, and write a type that allows only certain values

Theory

What it is for. A program that always does the same things in the same order is not very useful. The hand in front of the webcam is sometimes there and sometimes not, the fingers are sometimes raised and sometimes closed: the program must decide. And since the hand has twenty-one points, writing the same line twenty-one times is not a workable option: the program must repeat.

We start with comparison. Comparing two values always produces a boolean, that is true or false: the same type seen last time, but this time we do not write it by hand, the computer calculates it.

const dita = 5;

console.log(dita === 5);    // true   "is equal to"
console.log(dita !== 5);    // false  "is different from"
console.log(dita > 3);      // true   "greater than"
console.log(dita < 3);      // false  "less than"
console.log(dita >= 5);     // true   "greater than or equal"
console.log(dita <= 4);     // false  "less than or equal"

Comparisons are combined with logical operators. There are three, and you read them aloud like plain English:

const x = 0.4;

console.log(x >= 0 && x <= 1);   // true   "and" → both must be true
console.log(x < 0 || x > 1);     // false  "or"  → one being true is enough
console.log(!(x < 0));           // true   "not" → flips the result

With a boolean value in hand you can finally choose. The if statement runs a block of code only if the condition is true:

const dita = 5;

if (dita === 5) {
  console.log("mano aperta");
}

The curly braces enclose the block, that is the lines that belong to that if. With else you say what to do when the condition is false, and with else if you line up several questions. The computer tries them from top to bottom and stops at the first true one:

const dita = 2;

if (dita === 0) {
  console.log("pugno");
} else if (dita === 5) {
  console.log("mano aperta");
} else if (dita === 2) {
  console.log("vittoria");
} else {
  console.log("gesto non riconosciuto");
}

// prints: vittoria

Now on to repeating. The most classic way is the for loop. Written all at once it looks scary, so read it one piece at a time:

for (let i = 0; i < 5; i++) {
  console.log(i);
}

// prints: 0 1 2 3 4

Inside the round brackets there are three things separated by semicolons, and each has a job:

1. let i = 0 — the starting point. You create a variable, usually called i (for index), set to zero.
2. i < 5 — the condition to keep going. Before every round the computer checks it: if true it does the round, if false it leaves the loop.
3. i++ — what to do at the end of every round. i++ is the short form of i = i + 1.

So the computer runs: i is 0, 0 is less than 5, print, i becomes 1… until i is 5, which is not less than 5, and the loop ends. The body ran five times, with i equal to 0, 1, 2, 3, 4.

Now you need something to repeat over. An array is an ordered list of values, all of the same type. It is written in square brackets, and its type is the type of the elements followed by []:

const numeri: number[] = [10, 20, 30, 40];
const nomi: string[] = ["pollice", "indice", "medio"];

console.log(numeri.length);   // 4   → how many elements there are
console.log(numeri[0]);       // 10  → the FIRST element
console.log(numeri[3]);       // 40  → the last one
console.log(numeri[4]);       // undefined → does not exist

Two things must be learned right away, because they cause half of beginners' errors. First: indices start from zero. An array of four elements has indices 0, 1, 2, 3. Second: as a consequence the last index is length - 1, never length.

Putting loop and array together gives the most used line of the whole course:

const numeri: number[] = [10, 20, 30, 40];

for (let i = 0; i < numeri.length; i++) {
  console.log(`l'elemento numero ${i} vale ${numeri[i]}`);
}

You write i < numeri.length and not i < 4: that way the code keeps working if tomorrow the array changes length. And in the final project it certainly does, because an array of points can hold 21 (a hand) or 33 (a whole body).

When you do not need the index, and only care about the value, there is a shorter and more readable form, for…of:

const numeri: number[] = [10, 20, 30, 40];

for (const valore of numeri) {
  console.log(valore);
}

It reads literally: "for each valore of numeri". You use const because a new variable is born at every round. When to use which: if the value is enough, for…of; if you also need to know which position you are at — and to find the highest finger you do — then the classic for with the index.

There is a third way, while, which repeats as long as a condition stays true, with no automatic counter. You use it when you do not know in advance how many rounds you will need:

let rimaste = 3;

while (rimaste > 0) {
  console.log(`mancano ${rimaste} prove`);
  rimaste = rimaste - 1;
}

It never appears in the hand project: when you work on lists of points you always know how many there are. It is useful to know it, not useful to use it today.

Last concept, and it is what makes TypeScript more than JavaScript: union types. What they are for: often a variable cannot hold any text, only two or three specific words. Telling the compiler avoids a whole family of errors.

type Stato = "spenta" | "accesa";

let camera: Stato = "spenta";
camera = "accesa";    // ok
camera = "acesa";     // ERROR: Type '"acesa"' is not assignable to type 'Stato'.

The vertical bar | reads "or". Without the union type you would have written let camera: string, and that "acesa" with a single c would have slipped through: the program would have checked camera === "accesa", found false and turned nothing on, without a single error message. An afternoon wasted, for one letter.

We close with a trap worth its thirty seconds. const on an array does not make the array immutable: it only prevents replacing it with another array.

const numeri: number[] = [1, 2, 3];

numeri.push(4);           // ALLOWED: the array is now [1, 2, 3, 4]
console.log(numeri);

numeri = [9, 9, 9];       // ERROR: Cannot assign to 'numeri' because it is a constant.

The reason: const protects the name, not the contents. The variable numeri will keep pointing at that list; what is inside the list is another matter. push adds an element at the end and is the normal way to build an array bit by bit.

Teacher demo (15')

1. Write an array of five numbers and print it with a classic for, reading aloud the three pieces in the round brackets.
2. Rewrite the same loop with for…of and ask the class what was gained and what was lost (the position was lost).
3. Remove i++ and show the tab freezing. Then put it back. It takes away the fear: it happened, you close, you reopen.
4. Declare an array of 21 numbers between 0 and 1 — "these are the horizontal positions of the twenty-one points of the hand" — and with an if inside the loop count how many are below 0.5.
5. Write type Gesto = "pugno" | "mano aperta", get a letter wrong and show the error. That is the moment union types click.

Lab (60')

const numeri: number[] = [12, 7, 30, 4, 22];

let somma = 0;
// write here the loop that adds each number to `somma`

console.log(`somma: ${somma}`);

const numeri: number[] = [12, 7, 30, 4, 22];

let somma = 0;
for (const valore of numeri) {
  somma = somma + valore;
}

const media = somma / numeri.length;

console.log(`somma: ${somma}`);    // 75
console.log(`media: ${media}`);    // 15

const x = 0.42;

if (x >= 0 && x <= 1) {
  console.log("punto dentro l'immagine");
} else {
  console.log("punto fuori dall'immagine");
}

const alzate: boolean[] = [false, true, true, false, false];

const alzate: boolean[] = [false, true, true, false, false];

let quante = 0;
for (const dito of alzate) {
  if (dito === true) {
    quante = quante + 1;
  }
}

console.log(`dita alzate: ${quante}`);   // 2

if (quante === 0) {
  console.log("pugno");
} else if (quante === 5) {
  console.log("mano aperta");
} else {
  console.log("altro");
}

const numeri: number[] = [12, 7, 30, 4, 22];

const numeri: number[] = [12, 7, 30, 4, 22];

let massimo = numeri[0];
let indiceDelMassimo = 0;

for (let i = 1; i < numeri.length; i++) {
  if (numeri[i] > massimo) {
    massimo = numeri[i];
    indiceDelMassimo = i;
  }
}

console.log(`il massimo è ${massimo}, alla posizione ${indiceDelMassimo}`);
// il massimo è 30, alla posizione 2

const x: number[] = [
  0.50, 0.42, 0.36, 0.33, 0.30,
  0.44, 0.42, 0.41, 0.40,
  0.51, 0.51, 0.51, 0.51,
  0.57, 0.58, 0.59, 0.60,
  0.63, 0.66, 0.68, 0.70,
];

const x: number[] = [
  0.50, 0.42, 0.36, 0.33, 0.30,
  0.44, 0.42, 0.41, 0.40,
  0.51, 0.51, 0.51, 0.51,
  0.57, 0.58, 0.59, 0.60,
  0.63, 0.66, 0.68, 0.70,
];

let sinistra = 0;
let destra = 0;

for (const valore of x) {
  if (valore < 0.5) {
    sinistra = sinistra + 1;
  } else {
    destra = destra + 1;
  }
}

console.log(`punti totali: ${x.length}`);              // 21
console.log(`a sinistra: ${sinistra}`);                // 8
console.log(`a destra: ${destra}`);                    // 13
console.log(`controllo: ${sinistra + destra}`);        // 21

if (sinistra > destra) {
  console.log("la mano sta più a sinistra");
} else if (destra > sinistra) {
  console.log("la mano sta più a destra");
} else {
  console.log("la mano sta al centro");
}

Typical errors in class

• Mixing up = and ===: if (dita = 5) does not compare, it assigns. The compiler helps here and flags the error, but the message is not intuitive: worth showing it before they meet it on their own.
• Index starting from 1: for (let i = 1; i <= numeri.length; i++) skips the first element and reads one that does not exist. Double damage in a single line.
• The variable declared inside the loop: let somma = 0 written inside the braces is reset at every round, and at the end only the last value is printed.
• Forgotten braces: if (x > 0) console.log("a"); console.log("b"); always prints "b". Course rule: always use braces, even for a single line.
• NaN printed on screen: almost always a numeri[i] read past the end of the array, or a division by a length equal to zero.
• Infinite loop: the tab freezes. Close it, reopen the Playground, the code is in the address. No drama.

Quick check (5 questions)

Homework (30 minutes)

Write a program that, starting from the x array of exercise 2.5, prints the rightmost point (value and position) and the leftmost point, and says whether the hand is wide or narrow: call a hand "larga" (wide) when the difference between the maximum and the minimum exceeds 0.35, "stretta" (narrow) otherwise. Declare the answer in a variable with the type type Larghezza = "larga" | "stretta". Share the Playground link with the teacher.

Functions: giving a name to a piece of work

In the last lesson you wrote almost the same loop three times. Today you learn to write it only once, give it a name and call it when needed. It is the step that turns a list of statements into a program you can read.

• Where you work: TypeScript Playground
• New concepts: function, typed parameters, return type, return, void, arrow functions, default values, optional parameters ?, map and filter
• By the end the student can: write a function with parameters and a return value, reuse it inside another function, and understand when a function returns something and when it just "does"

Theory

What it is for. Two reasons, in order of importance. The first is that giving a name to a piece of work makes it understandable: distanza(a, b) is clear at a glance, four lines of square roots are not. The second is that something written in one place is fixed in one place. If the same formula is copied in five places, sooner or later four are updated and one is not.

A function is a block of code with a name, which receives some input values and returns one as output. You declare it like this:

function raddoppia(numero: number): number {
  return numero * 2;
}

console.log(raddoppia(5));    // 10
console.log(raddoppia(21));   // 42

Piece by piece, because each part has a name the teacher will use later:

• function — the word announcing "I am about to declare a function".
• raddoppia — the name. Choose a verb, because a function does something.
• numero: number — the parameter, with its type. It is a variable that exists only inside the function.
• : number after the round brackets — the return type: what comes out.
• return — the statement that hands over the result and immediately ends the function.

raddoppia(5) is the call: the 5 goes in, takes the place of numero, and the function returns 10. The returned value can be printed, put in a variable, or fed to another function.

There can be more than one parameter, separated by commas, each with its own type:

function area(larghezza: number, altezza: number): number {
  return larghezza * altezza;
}

console.log(area(640, 480));   // 307200

Order matters. area(640, 480) and area(480, 640) give the same result here by chance, but not in general: the first value goes into the first parameter, full stop.

Some functions return nothing: they print, draw, switch on. Their return type is written void, which means "empty":

function saluta(nome: string): void {
  console.log(`ciao ${nome}`);
}

saluta("Giulia");   // prints: ciao Giulia

const risultato = saluta("Marco");
console.log(risultato);   // undefined: there was nothing to return

There is a second, shorter way to write it, called an arrow function. It does exactly the same thing:

const raddoppia = (numero: number): number => {
  return numero * 2;
};

// and when the body is a single expression, you can also drop return and the braces:
const triplica = (numero: number): number => numero * 3;

console.log(raddoppia(5));    // 10
console.log(triplica(5));     // 15

The function is put in a const as if it were any other value — because it is. This form is useful above all when a function must be passed to someone else, which is the next topic.

Parameters can have a default value, used when the caller does not pass one:

function aPixel(valore: number, dimensione: number = 640): number {
  return valore * dimensione;
}

console.log(aPixel(0.5));        // 320  → uses 640
console.log(aPixel(0.5, 480));   // 240  → uses what you pass

Or they can be optional, with a question mark. In that case, if they are not passed, they are undefined, and the function must allow for it:

function etichetta(nome: string, gesto?: string): string {
  if (gesto === undefined) {
    return nome;
  }
  return `${nome}: ${gesto}`;
}

console.log(etichetta("Destra"));               // Destra
console.log(etichetta("Destra", "pugno"));      // Destra: pugno

Functions can call other functions, and this is the normal way to build programs: you start from small, safe pieces and combine them.

function distanza(x1: number, y1: number, x2: number, y2: number): number {
  const dx = x2 - x1;
  const dy = y2 - y1;
  return Math.sqrt(dx * dx + dy * dy);
}

function dimensioneMano(xPolso: number, yPolso: number, xNocca: number, yNocca: number): number {
  return distanza(xPolso, yPolso, xNocca, yNocca);
}

console.log(dimensioneMano(0.50, 0.88, 0.51, 0.58));   // about 0.3002

Math is a set of ready-made maths tools that the language provides: Math.sqrt is the square root, Math.abs the absolute value, Math.round rounding, Math.min and Math.max the smaller and the larger of two numbers.

Last piece for today: map and filter. They are two array tools that take a function as an argument. It looks odd the first time, so read them as plain sentences:

const numeri: number[] = [1, 2, 3, 4];

// map: "make me a NEW array, applying this function to every element"
const doppi = numeri.map((n) => n * 2);          // [2, 4, 6, 8]

// filter: "make me a NEW array with only the elements for which this function says true"
const grandi = numeri.filter((n) => n > 2);      // [3, 4]

console.log(doppi);
console.log(grandi);
console.log(numeri);   // [1, 2, 3, 4] — the original was not touched

Neither of them changes the starting array: they return a new one. They are the short form of a for loop that builds an array with push, and in real code they are almost always used instead of that loop because they state the intention in one line.

We close with an idea, not a syntax. A function is called pure when, given the same inputs, it always returns the same output and touches nothing outside itself. distanza is pure: with those four numbers it will always answer the same, today and in a month. saluta is not, because it prints — it changes something in the outside world. Pure functions are easier to test and hold no surprises: when you can write one, do.

Teacher demo (15')

1. Take the code of exercise 2.1 (sum and average) and turn it into a function mediaDi. Call it twice on two different arrays: that is when you see the gain.
2. Write distanza live, with Math.sqrt, explaining that it is Pythagoras' theorem and nothing else.
3. Show Math.hypot(dx, dy), which does the same in one line, and say that the real project uses that one.
4. Compute the distance between the wrist (0.50, 0.88) and the middle-finger knuckle (0.51, 0.58): "this number, in the real application, decides how thick the fingers drawn on the screen are". Show the hand demo and move your hand closer to the camera.
5. Forget the return on purpose and show that the function prints undefined.

Lab (60')

function distanza(x1: number, y1: number, x2: number, y2: number): number {
  const dx = x2 - x1;
  const dy = y2 - y1;
  return Math.sqrt(dx * dx + dy * dy);
}

function distanzaBreve(x1: number, y1: number, x2: number, y2: number): number {
  return Math.hypot(x2 - x1, y2 - y1);
}

console.log(distanza(0, 0, 3, 4));        // 5
console.log(distanzaBreve(0, 0, 3, 4));   // 5

function aPixel(valore: number, dimensione: number = 640): number {
  return valore * dimensione;
}

console.log(aPixel(0.5, 640));   // 320
console.log(aPixel(0, 640));     // 0
console.log(aPixel(1, 480));     // 480
console.log(aPixel(0.25));       // 160 — uses the default

function mediaDi(numeri: number[]): number {
  if (numeri.length === 0) {
    return 0;
  }

  let somma = 0;
  for (const valore of numeri) {
    somma = somma + valore;
  }
  return somma / numeri.length;
}

console.log(mediaDi([12, 7, 30, 4, 22]));   // 15
console.log(mediaDi([0.4, 0.6]));           // 0.5
console.log(mediaDi([]));                   // 0

const x: number[] = [0.50, 0.42, 0.36, 0.44, 0.63, 0.70];

function aPixel(valore: number, dimensione: number): number {
  return valore * dimensione;
}

const x: number[] = [0.50, 0.42, 0.36, 0.44, 0.63, 0.70];

const inPixel = x.map((valore) => aPixel(valore, 640));
console.log(inPixel);   // [320, 268.8, 230.4, 281.6, 403.2, 448] — roughly

const aSinistra = inPixel.filter((p) => p < 320);
console.log(`a sinistra ce ne sono ${aSinistra.length}`);   // 3

type Gesto = "pugno" | "mano aperta" | "altro";

function ditaAlzate(stati: boolean[]): number {
  let quante = 0;
  for (const dito of stati) {
    if (dito) {
      quante = quante + 1;
    }
  }
  return quante;
}

function nomeGesto(stati: boolean[]): Gesto {
  const quante = ditaAlzate(stati);
  if (quante === 0) {
    return "pugno";
  }
  if (quante === 5) {
    return "mano aperta";
  }
  return "altro";
}

console.log(nomeGesto([false, false, false, false, false]));   // pugno
console.log(nomeGesto([true, true, true, true, true]));        // mano aperta
console.log(nomeGesto([false, true, true, false, false]));     // altro

Typical errors in class

• Forgotten return: the function prints but does not return, and the caller gets undefined. If TypeScript knows the return type it flags it right away — a great reason to always write it.
• Declaring and not calling: writing function distanza(…) and expecting something to happen. Declaring is like writing a recipe; calling is cooking.
• Calling without the brackets: console.log(mediaDi) prints the text of the function, not the result. You need mediaDi(numeri).
• Swapped parameters: aPixel(640, 0.5) gives 320 just like aPixel(0.5, 640) — multiplication forgives — but with distanza the wrong order produces nonsense. Better to get used to reading the signature right away.
• Using a variable declared inside another function: parameters and consts declared inside a function exist only in there. The message is Cannot find name.
• The semicolon after the arrow: const f = () => { … }; wants the final semicolon because it is a variable declaration; function f() { … } does not. Nobody remembers it, and that is perfectly fine: the formatter fixes it.

Quick check (5 questions)

Homework (30 minutes)

Write three functions and test them: massimoDi(numeri: number[]): number, minimoDi(numeri: number[]): number and larghezzaDi(numeri: number[]): number, which returns the difference between maximum and minimum reusing the first two. Test them on the array of the twenty-one horizontal positions from exercise 2.5. None of the three may contain a console.log: they must return, not print — you do the printing outside.

Objects and interfaces: the Punto type

Until now a coordinate was two separate variables, x and y, that you had to remember to keep together. Today they become one thing with a name and a type. It is the data the whole final project runs on: from the next lesson we will write almost nothing else.

• Where you work: TypeScript Playground, last lesson there
• New concepts: object literals, dot access, interface, arrays of objects, optional and readonly properties, undefined and null, ?. and ??, type narrowing
• By the end the student can: define a type with interface, write functions that take and return objects, and handle without fear a value that might not be there

Theory

What it is for. Try to imagine writing the final project with what you know now: twenty-one horizontal values in one array, twenty-one vertical ones in another, and the promise never to get the pairing wrong. A single sort on one array and the hand falls apart. An object keeps together the values that describe the same thing, and keeps them together by construction.

const polso = { x: 0.50, y: 0.88 };

console.log(polso.x);   // 0.5
console.log(polso.y);   // 0.88
console.log(polso);     // { x: 0.5, y: 0.88 }

Between braces you list the properties: a name, a colon, a value, separated by commas. You read them with the dot: polso.x. That is all. If the variable is let, or if the property is not protected, you can also change them:

const polso = { x: 0.50, y: 0.88 };

polso.x = 0.55;         // PERMESSO, anche con const (vale la stessa regola degli array)
console.log(polso.x);   // 0.55

polso.z = 0.1;          // ERRORE: Property 'z' does not exist on type '{ x: number; y: number; }'.

That last error is already TypeScript at work: it worked out the shape of the object by itself and does not accept made-up properties. But that shape is written in one place only, and if tomorrow we needed it in ten different functions we would have to copy it ten times. That is why interface exists: it gives a name to a shape.

interface Punto {
  x: number;
  y: number;
  z?: number;     // il punto interrogativo: facoltativa
}

const polso: Punto = { x: 0.50, y: 0.88 };
const puntaIndice: Punto = { x: 0.40, y: 0.37, z: -0.02 };

console.log(polso.x, puntaIndice.z);

Inside the interface, properties are separated by semicolons (or new lines) and not by commas as in objects: it is the number one confusion of the lesson. From here on Punto is a type in every respect, and it is used like number or string: as the type of a variable, of a parameter, of a return value, and inside an array.

const punti: Punto[] = [
  { x: 0.50, y: 0.88 },
  { x: 0.40, y: 0.37 },
  { x: 0.51, y: 0.58 },
];

console.log(punti.length);   // 3
console.log(punti[1].x);     // 0.4  → l'elemento 1, e di quello la x

for (const p of punti) {
  console.log(`(${p.x}, ${p.y})`);
}

punti[1].x reads from left to right: take the element at index 1, and of that object take the property x. This line, as punti[8].x, will appear dozens of times in the real project: 8 is the tip of the index finger.

interface Punto {
  x: number;
  y: number;
  z?: number;
}

const mano: Punto[] = [
  { x: 0.50, y: 0.88 },   // 0  wrist
  { x: 0.42, y: 0.84 },   // 1  base of the thumb
  { x: 0.30, y: 0.64 },   // 2  thumb tip           (in the real model it is point 4)
  { x: 0.44, y: 0.60 },   // 3  index knuckle       (in the real model it is 5)
  { x: 0.40, y: 0.37 },   // 4  index tip           (in the real model it is 8)
  { x: 0.51, y: 0.58 },   // 5  middle knuckle      (in the real model it is 9)
  { x: 0.51, y: 0.33 },   // 6  middle tip          (in the real model it is 12)
  { x: 0.63, y: 0.63 },   // 7  little finger knuckle (in the real model it is 17)
  { x: 0.70, y: 0.45 },   // 8  little finger tip   (in the real model it is 20)
];

A property can be declared readonly: it can be read, not rewritten. It says "this data, once created, must not be touched".

interface PuntoFisso {
  readonly x: number;
  readonly y: number;
}

const origine: PuntoFisso = { x: 0, y: 0 };
origine.x = 5;   // ERRORE: Cannot assign to 'x' because it is a read-only property.

Now the most important part of the lesson, the one that separates those who will write solid programs from those who will spend their evenings hunting bugs: values that might not be there. There are two, and they mean slightly different things:

• undefined — "nothing was ever put here". It is what you get from an optional property with no value, from an optional parameter that was not passed, or from an element past the end of an array.
• null — "here, on purpose, there is nothing". A programmer writes it to say "this slot is empty, and I know it".

Both appear in the final project: let stream: MediaStream | null = null means "the webcam is off right now, and it is not an oversight". The problem is that reading a property of a value that is not there blows up the program:

interface Punto { x: number; y: number }

let punta: Punto | undefined = undefined;

console.log(punta.x);   // ERRORE del compilatore: 'punta' is possibly 'undefined'.
                        // e se lo eseguissi davvero: TypeError, il programma si ferma

There are three ways to handle it, in order of elegance. The first is optional chaining ?.: it tries to read, and if there is nothing in front it returns undefined instead of blowing up.

let punta: Punto | undefined = undefined;

console.log(punta?.x);   // undefined — nessun errore

The second is the fallback value ??, which reads "otherwise": if what is on the left is undefined or null, it takes what is on the right. The two are almost always used together:

let punta: Punto | undefined = undefined;

const x = punta?.x ?? 0;
console.log(x);   // 0

punta = { x: 0.40, y: 0.37 };
console.log(punta?.x ?? 0);   // 0.4

The third way is type narrowing: an if that checks, and inside which the compiler knows the value is there. It is the clearest when you need to use the value more than once:

function stampaPunto(punto: Punto | undefined): void {
  if (punto === undefined) {
    console.log("nessun punto");
    return;
  }

  // da qui in poi TypeScript sa che `punto` esiste: niente ?. e niente errori
  console.log(`x vale ${punto.x} e y vale ${punto.y}`);
}

stampaPunto(undefined);              // nessun punto
stampaPunto({ x: 0.4, y: 0.37 });    // x vale 0.4 e y vale 0.37

The compiler follows the reasoning: after that return, the only way to still be there is that punto is not undefined. It is not magic, it is the same thing you would have thought reading the code — only it checks it for you.

Teacher demo (15')

1. Write interface Punto on the board and then in the Playground. Create the wrist and the index tip with the realistic values seen above.
2. Add the z and remove the z, showing that with z? both versions are valid.
3. Write centro(punti: Punto[]): Punto live, reusing the average idea from lesson 3. Make it return an object: it is the first function in the course that returns something composite.
4. Open the hand demo, show the white dots on the preview and say: "each of those dots is an object like the one we just wrote; there are twenty-one, and they arrive thirty times a second".
5. Write punti[99].x and show the crash at run time. Then fix it with punti[99]?.x ?? 0.

Lab (60')

interface Punto {
  x: number;
  y: number;
  z?: number;
}

const polso: Punto = { x: 0.50, y: 0.88 };
const puntaIndice: Punto = { x: 0.40, y: 0.37, z: -0.02 };

console.log(`il polso sta in (${polso.x}, ${polso.y})`);
console.log(`la punta dell'indice sta in (${puntaIndice.x}, ${puntaIndice.y})`);
console.log(`profondità della punta: ${puntaIndice.z ?? 0}`);

function centro(punti: Punto[]): Punto {
  if (punti.length === 0) {
    return { x: 0, y: 0 };
  }

  let sommaX = 0;
  let sommaY = 0;
  for (const p of punti) {
    sommaX = sommaX + p.x;
    sommaY = sommaY + p.y;
  }

  return { x: sommaX / punti.length, y: sommaY / punti.length };
}

const c = centro(mano);
console.log(`il centro della mano sta in (${c.x}, ${c.y})`);
// il centro della mano sta in (0.49, 0.5911111111111111)

function piuAlto(punti: Punto[]): Punto {
  let migliore = punti[0];

  for (const p of punti) {
    if (p.y < migliore.y) {   // SMALLER y = higher in the image
      migliore = p;
    }
  }

  return migliore;
}

const alto = piuAlto(mano);
console.log(`il punto più in alto è (${alto.x}, ${alto.y})`);
// il punto più in alto è (0.51, 0.33)  → the middle fingertip

function puntoOppureNiente(punti: Punto[], indice: number): Punto | undefined {
  if (indice < 0 || indice >= punti.length) {
    return undefined;
  }
  return punti[indice];
}

function descrivi(punto: Punto | undefined): string {
  if (punto === undefined) {
    return "punto assente";
  }
  return `x: ${punto.x}`;
}

console.log(descrivi(puntoOppureNiente(mano, 4)));    // x: 0.4
console.log(descrivi(puntoOppureNiente(mano, 99)));   // punto assente

// the short version, when you only need a number:
const x = puntoOppureNiente(mano, 99)?.x ?? 0;
console.log(x);   // 0

interface Mano {
  lato: "destra" | "sinistra";
  punti: Punto[];
  gesto?: string;
}

mano destra · 9 punti · centro (0.490, 0.591) · gesto: nessuno

interface Mano {
  lato: "destra" | "sinistra";
  punti: Punto[];
  gesto?: string;
}

function riassumi(m: Mano): string {
  const c = centro(m.punti);
  const nomeGesto = m.gesto ?? "nessuno";
  return `mano ${m.lato} · ${m.punti.length} punti · centro (${c.x.toFixed(3)}, ${c.y.toFixed(3)}) · gesto: ${nomeGesto}`;
}

const destra: Mano = { lato: "destra", punti: mano };

const sinistra: Mano = {
  lato: "sinistra",
  punti: [
    { x: 0.20, y: 0.70 },
    { x: 0.24, y: 0.62 },
  ],
  gesto: "pugno",
};

console.log(riassumi(destra));
// mano destra · 9 punti · centro (0.490, 0.591) · gesto: nessuno

console.log(riassumi(sinistra));
// mano sinistra · 2 punti · centro (0.220, 0.660) · gesto: pugno

Typical mistakes in class

• Comma inside the interface: you write x: number; with a semicolon (or just a new line). With a comma the compiler complains and the message is not very clear.
• Colon instead of the equals sign: const polso = { x: 0.5 } inside the object wants the colon, but the variable is created with the equals sign. Mixing the two up is perfectly normal on the first day.
• Braces in the return: return { x: 1, y: 2 }; returns an object. The braces here are not a code block, they are the object: people end up hunting a non-existent error because of this.
• Looking for the maximum to find the highest point: in images y grows downwards. If the drawn hand comes out upside down, the problem is almost always here.
• Using || instead of ??: it works until a legitimate zero arrives, and then it breaks in a way that is very hard to track down.
• Comparing objects with ===: { x: 1 } === { x: 1 } is false. The comparison checks whether they are the same object, not whether they have the same content. To compare content you compare the properties one by one.

Quick check (5 questions)

Homework (30-40 minutes)

Write dimensioneMano(punti: Punto[]): number, which returns the distance between the first point of the array (the wrist) and the one at index 5 (the middle knuckle, in our reduced array), reusing the distanza function from lesson 3 — but adapt it so that it takes two Punto values instead of four numbers. Then write aperta(punti: Punto[]): boolean, which answers true if the middle fingertip is higher than the middle knuckle by at least a quarter of the size of the hand. Try it on the mano array and on a modified version in which you have lowered the middle fingertip to y: 0.56.

The page and the canvas: drawing a fake hand

Until now the program only spoke through console.log. Today we move to a real page and a canvas to draw on, and by the end of the two hours a hand will appear on the screen. Fake, still, drawn by you: but it is the same hand that in four lessons will move following yours.

• Where you work: vite.new/vanilla-ts on StackBlitz, no account needed (alternatively JSFiddle for quick tests)
• New concepts: the minimal HTML page, document.getElementById, HTMLElement | null, as, getContext("2d"), beginPath, arc, moveTo/lineTo/stroke, fillStyle, lineWidth, clearRect, addEventListener
• By the end the student can: get an element from the page, obtain the drawing context, draw circles and lines at the right coordinates, and respond to a click on a button

Theory

What it is for. Nobody looks at a program that prints numbers in a console. The final project is a web page: there is a button, there is a box with the webcam and there is a canvas on which the hand appears. Today we learn the two mechanisms that hold the code and what you see together: getting an element of the page and drawing inside it.

Let us start with the page. An HTML page is made of tags, which open and close and contain other tags. This is the smallest one we need:

<!doctype html>
<html lang="it">
  <head>
    <meta charset="UTF-8" />
    <title>La mia mano</title>
  </head>
  <body>
    <canvas id="tela" width="480" height="640"></canvas>
    <script type="module" src="/src/main.ts"></script>
  </body>
</html>

<head> contains information about the page, <body> what you see. The <canvas> is the canvas: an empty rectangle, here 480 pixels wide and 640 high, inside which the code can draw. The id is the name the code will look it up by: it must be unique in the page. The last line loads our program.

Now the code has to get hold of that <canvas>. You ask the page for it by name:

const tela = document.getElementById("tela");

document is the page, as seen from the code. getElementById looks for the element with that id. But if you try to use it, TypeScript gets in the way:

const tela = document.getElementById("tela");
console.log(tela.width);
// ERRORE: 'tela' is possibly 'null'.
// ERRORE: Property 'width' does not exist on type 'HTMLElement'.

Two complaints, and both are right:

1. It might not be there. The return type is HTMLElement | null: that union type says "an element, or nothing". If you misspelled the id, or if the code starts before the page is ready, the answer is null. It is the same null as in lesson 4.
2. It does not know what kind of element it is. It knows it is an element of the page, HTMLElement, and that is all. But a heading has no drawing width, a canvas does: until the compiler knows it is a canvas, it will not let you use canvas things.

The second problem is solved with as, which reads "treat it as": it is a promise you make to the compiler.

const tela = document.getElementById("tela") as HTMLCanvasElement | null;

if (!tela) {
  throw new Error("canvas non trovato: controlla l'id nell'HTML");
}

console.log(tela.width);   // 480 — adesso va

if (!tela) reads "if tela is NOT there": it is the exclamation mark from lesson 2, the one that flips things. After that block the compiler knows that tela exists — it is the same narrowing as in lesson 4.

You do not draw directly on the canvas: you ask for the context, that is the object that holds all the drawing tools. There are several kinds; we need the two-dimensional one.

const ctx = tela.getContext("2d");

if (!ctx) {
  throw new Error("contesto 2d non disponibile");
}

This too can return null (for example if the same canvas had already been asked for a context of a different kind), so you check it as before. From here on ctx is the object you draw everything with.

Coordinates. Before drawing you need to know where things will end up. The canvas has its origin at the top left: x grows to the right, y grows downwards. It is the same convention as images seen in lesson 4 — and for once this is an advantage: the recognizer's coordinates, multiplied by the canvas size, end up in the right place without flipping anything.

// su una tela 480 × 640:
// (0, 0)     angolo in alto a sinistra
// (480, 0)   angolo in alto a destra
// (0, 640)   angolo in basso a sinistra
// (240, 320) centro esatto

function aPixel(valore: number, dimensione: number): number {
  return valore * dimensione;
}

console.log(aPixel(0.5, 480));   // 240 → metà della larghezza

Drawing a circle. You need it for the joints of the hand. It takes a few lines, always the same:

ctx.beginPath();                              // 1. inizia un disegno nuovo
ctx.arc(240, 320, 8, 0, Math.PI * 2);         // 2. descrivi un cerchio
ctx.fillStyle = "#38bdf8";                    // 3. scegli il colore di riempimento
ctx.fill();                                   // 4. riempilo

The arguments of arc, in order: centre x, centre y, radius, start angle, end angle. 0 and Math.PI * 2 are the start and the end of a full turn: it is the recipe for "whole circle", and you copy it as it is.

Drawing a line. You need it for the bones. Same pattern:

ctx.beginPath();
ctx.moveTo(100, 100);            // porta la penna qui, senza disegnare
ctx.lineTo(300, 400);            // traccia fino a qui
ctx.strokeStyle = "#f472b6";     // colore della linea
ctx.lineWidth = 4;               // spessore in pixel
ctx.stroke();                    // disegna davvero

The difference between fill and stroke is the same as between filling and outlining: fillStyle/fill for the inside, strokeStyle/stroke for the edge. Colours are written in hexadecimal ("#38bdf8") or by name ("white"). And nothing is actually drawn until you call fill or stroke: the instructions before only describe the shape.

Clearing. When the hand moves you do not draw over the previous one: you clear everything and redraw from scratch, thirty times a second.

ctx.clearRect(0, 0, tela.width, tela.height);   // svuota tutta la tela

Responding to a click. The last tool for today. You get the button the way you got the canvas, and you tell it what to do when it is pressed:

const pulsante = document.getElementById("cambia") as HTMLButtonElement | null;

if (pulsante) {
  pulsante.addEventListener("click", () => {
    console.log("premuto!");
  });
}

addEventListener wants two things: the name of the event ("click") and a function to run when it happens. That function is an arrow function with no parameters, from lesson 3. Nobody calls it explicitly: the browser calls it, when and if the user presses. In the final project this is exactly how the webcam starts, from the click on the "Turn on camera" button.

Teacher demo (15')

1. Open vite.new/vanilla-ts in front of the class: showing that in ten seconds there is a working project removes the installation anxiety.
2. Empty src/main.ts and the <body> of index.html, add the <canvas>, save: the page stays white. "The canvas is there, it is empty."
3. Write getElementById without as and show the two red errors. Add as and the check, and comment on them: they are the two problems the compiler just prevented.
4. Draw a circle. It is the "it works!" moment of the lesson: wait two seconds before going on.
5. Draw three circles forgetting beginPath and show the spider web. Then add it.
6. Wire up a button that changes the colour of the circle and redraws.

Lab (60')

<body>
  <canvas id="tela" width="480" height="640"></canvas>
  <script type="module" src="/src/main.ts"></script>
</body>

const tela = document.getElementById("tela") as HTMLCanvasElement | null;
if (!tela) {
  throw new Error("canvas non trovato");
}

const ctx = tela.getContext("2d");
if (!ctx) {
  throw new Error("contesto 2d non disponibile");
}

// the light-blue circle in the centre
ctx.beginPath();
ctx.arc(240, 320, 20, 0, Math.PI * 2);
ctx.fillStyle = "#38bdf8";
ctx.fill();

// the pink line from the corner to the centre
ctx.beginPath();
ctx.moveTo(0, 0);
ctx.lineTo(240, 320);
ctx.strokeStyle = "#f472b6";
ctx.lineWidth = 4;
ctx.stroke();

interface Punto {
  x: number;
  y: number;
}

const MANO_APERTA: Punto[] = [
  { x: 0.50, y: 0.88 },   //  0 wrist
  { x: 0.42, y: 0.84 },   //  1 thumb, base
  { x: 0.36, y: 0.77 },   //  2 thumb
  { x: 0.33, y: 0.70 },   //  3 thumb
  { x: 0.30, y: 0.64 },   //  4 thumb, TIP
  { x: 0.44, y: 0.60 },   //  5 index, knuckle
  { x: 0.42, y: 0.50 },   //  6 index
  { x: 0.41, y: 0.43 },   //  7 index
  { x: 0.40, y: 0.37 },   //  8 index, TIP
  { x: 0.51, y: 0.58 },   //  9 middle, knuckle
  { x: 0.51, y: 0.47 },   // 10 middle
  { x: 0.51, y: 0.40 },   // 11 middle
  { x: 0.51, y: 0.33 },   // 12 middle, TIP
  { x: 0.57, y: 0.60 },   // 13 ring, knuckle
  { x: 0.58, y: 0.49 },   // 14 ring
  { x: 0.59, y: 0.43 },   // 15 ring
  { x: 0.60, y: 0.37 },   // 16 ring, TIP
  { x: 0.63, y: 0.63 },   // 17 little finger, knuckle
  { x: 0.66, y: 0.55 },   // 18 little finger
  { x: 0.68, y: 0.50 },   // 19 little finger
  { x: 0.70, y: 0.45 },   // 20 little finger, TIP
];

function aPixel(valore: number, dimensione: number): number {
  return valore * dimensione;
}

ctx.clearRect(0, 0, tela.width, tela.height);

for (const p of MANO_APERTA) {
  const px = aPixel(p.x, tela.width);
  const py = aPixel(p.y, tela.height);

  ctx.beginPath();
  ctx.arc(px, py, 6, 0, Math.PI * 2);
  ctx.fillStyle = "#38bdf8";
  ctx.fill();
}

const CONNESSIONI: number[][] = [
  [0, 1], [1, 2], [2, 3], [3, 4],           // thumb
  [0, 5], [5, 6], [6, 7], [7, 8],           // index
  [5, 9], [9, 10], [10, 11], [11, 12],      // middle
  [9, 13], [13, 14], [14, 15], [15, 16],    // ring
  [13, 17], [17, 18], [18, 19], [19, 20],   // little finger
  [0, 17],                                  // the side of the palm
];

ctx.clearRect(0, 0, tela.width, tela.height);

// 1) the bones, underneath
ctx.strokeStyle = "#38bdf8";
ctx.lineWidth = 4;

for (const osso of CONNESSIONI) {
  const a = MANO_APERTA[osso[0]];
  const b = MANO_APERTA[osso[1]];

  ctx.beginPath();
  ctx.moveTo(aPixel(a.x, tela.width), aPixel(a.y, tela.height));
  ctx.lineTo(aPixel(b.x, tela.width), aPixel(b.y, tela.height));
  ctx.stroke();
}

// 2) the joints, on top
for (const p of MANO_APERTA) {
  ctx.beginPath();
  ctx.arc(aPixel(p.x, tela.width), aPixel(p.y, tela.height), 6, 0, Math.PI * 2);
  ctx.fillStyle = "#38bdf8";
  ctx.fill();
}

const PUNTE: number[] = [4, 8, 12, 16, 20];

for (let i = 0; i < MANO_APERTA.length; i++) {
  const p = MANO_APERTA[i];
  const punta = PUNTE.includes(i);

  ctx.beginPath();
  ctx.arc(aPixel(p.x, tela.width), aPixel(p.y, tela.height), punta ? 9 : 6, 0, Math.PI * 2);
  ctx.fillStyle = punta ? "#ffffff" : "#38bdf8";
  ctx.fill();
}

<body>
  <button id="cambia">Cambia posa</button>
  <canvas id="tela" width="480" height="640"></canvas>
  <script type="module" src="/src/main.ts"></script>
</body>

const PUGNO: Punto[] = [
  { x: 0.50, y: 0.88 }, { x: 0.42, y: 0.84 }, { x: 0.37, y: 0.78 },
  { x: 0.36, y: 0.72 }, { x: 0.40, y: 0.68 },
  { x: 0.44, y: 0.62 }, { x: 0.43, y: 0.54 }, { x: 0.46, y: 0.56 },
  { x: 0.48, y: 0.60 },
  { x: 0.51, y: 0.60 }, { x: 0.51, y: 0.52 }, { x: 0.53, y: 0.55 },
  { x: 0.54, y: 0.60 },
  { x: 0.57, y: 0.61 }, { x: 0.58, y: 0.53 }, { x: 0.59, y: 0.57 },
  { x: 0.59, y: 0.62 },
  { x: 0.63, y: 0.64 }, { x: 0.65, y: 0.58 }, { x: 0.65, y: 0.62 },
  { x: 0.64, y: 0.66 },
];

function disegna(punti: Punto[]): void {
  ctx.clearRect(0, 0, tela.width, tela.height);

  ctx.strokeStyle = "#38bdf8";
  ctx.lineWidth = 4;
  for (const osso of CONNESSIONI) {
    const a = punti[osso[0]];
    const b = punti[osso[1]];
    ctx.beginPath();
    ctx.moveTo(aPixel(a.x, tela.width), aPixel(a.y, tela.height));
    ctx.lineTo(aPixel(b.x, tela.width), aPixel(b.y, tela.height));
    ctx.stroke();
  }

  for (let i = 0; i < punti.length; i++) {
    const p = punti[i];
    const punta = PUNTE.includes(i);
    ctx.beginPath();
    ctx.arc(aPixel(p.x, tela.width), aPixel(p.y, tela.height), punta ? 9 : 6, 0, Math.PI * 2);
    ctx.fillStyle = punta ? "#ffffff" : "#38bdf8";
    ctx.fill();
  }
}

let aperta = true;
disegna(MANO_APERTA);
const pulsante = document.getElementById("cambia") as HTMLButtonElement | null;

if (pulsante) {
  pulsante.addEventListener("click", () => {
    aperta = !aperta;
    disegna(aperta ? MANO_APERTA : PUGNO);
  });
}

Typical mistakes in class

• White page and no error: almost always the id in the HTML and the one in the code do not match, or the line <script type="module" src="/src/main.ts"> is missing. First place to look: the browser console.
• The spider web: beginPath forgotten inside the loop. All the circles end up joined by lines.
• Nothing shows but no error: fill or stroke forgotten. The drawing was described but not executed.
• The upside-down hand: someone "fixed" y thinking 0 was at the bottom. In the canvas y grows downwards, as in the image: leave it as it is.
• The canvas sized with CSS: setting width and height outside the width and height attributes stretches the drawing instead of enlarging it. For today: attributes in the HTML only.
• Tiny hand in a corner: someone used aPixel(p.x, 1) or forgot to multiply. Coordinates 0…1 without multiplication all sit in the first pixel.
• The button does not respond: the listener was registered on a null element inside an if that is never true, or the id is wrong.

Quick check (5 questions)

Homework (40 minutes)

Start from the solution of exercise 5.5 and add a third pose of your choice — the victory sign, thumbs up, a waving hand — making up the twenty-one coordinates yourself. The button must cycle through the three poses instead of alternating two: keep a let posa = 0, move it forward on every click and bring it back to zero when it reaches three. By putting the three poses in an array of arrays — const POSE: Punto[][] = [MANO_APERTA, PUGNO, LA_TUA] — the code becomes three lines. Bring the StackBlitz link to the next lesson: it is the last time we work in the browser, because lesson six opens by installing the real tools on your own computer.

On your own computer: Node, Vite, modules

Until yesterday the code ran inside someone else's web page. From today the project is yours: it lives in a folder on your computer, starts with a command and is split into several files. It is the step that makes the rest of the course possible.

• Where you work: on your own computer, with VS Code and Node.js — plan B in the blue box further down
• New concepts: Node.js, npm, Vite, the project folder, package.json, tsconfig.json, strict, export and import, compilation
• By the end the student can: create a TypeScript project, start it, split their own code into two or three files and understand what happens between what they write and what the browser runs

Theory

In the first five lessons you wrote code inside a page that someone else kept open for you. It worked very well for learning, but now there is a precise obstacle: the webcam does not turn on inside the preview of online environments. The box that shows the result is a page framed inside another, coming from a different site, and browsers deny camera and microphone to those frames. It is not a flaw of the environment: it is a rule designed to stop any site from turning on your camera by hiding inside a frame.

From the next lesson you need the webcam. So today you need a real project.

Node.js: JavaScript outside the browser

Node.js is a program that runs JavaScript on your computer, without a browser. You do not need it to run the hand — that will run in the browser as always — but the tools need it: the TypeScript compiler, the development server, the program that puts the files together. They are all written in JavaScript and run on Node.

Together with Node you install npm (node package manager), which does two things:

• it downloads libraries written by others — in lesson 8 you will download Google's hand recognizer with one command;
• it runs the project's commands, the ones written in package.json: npm run dev, npm run build.

Vite: the ready-made project

Creating the right folder, the right configuration file and the right server by hand is boring and teaches nothing. Vite does it for you with one command:

npm create vite@latest mano-corso

It asks you two questions. For the first (Select a framework) choose Vanilla, which means "no framework, just the page and the code". For the second (Select a variant) choose TypeScript. In short people say "the vanilla-ts template", and you can also ask for it directly:

npm create vite@latest mano-corso -- --template vanilla-ts

Inside the newly created folder you find this:

mano-corso/
├── index.html          ← la pagina: il titolo, i pulsanti, il canvas
├── package.json        ← il "documento d'identità" del progetto
├── tsconfig.json       ← le regole del compilatore TypeScript
├── public/             ← file serviti così come sono (immagini, icone)
└── src/
    ├── main.ts         ← il codice che parte all'apertura della pagina
    └── style.css       ← l'aspetto

At the bottom of index.html there is a single line that links the page to the code:

<script type="module" src="/src/main.ts"></script>

Then two commands, always the same, for the rest of the course:

cd mano-corso     # entra nella cartella
npm install       # scarica gli strumenti: crea node_modules/ (pesante, non si tocca)
npm run dev       # avvia il server di sviluppo

npm run dev prints an address, usually http://localhost:5173. You open it in the browser and see your page. The server stays on: leave that terminal window open and write the code in the other window. Every time you save, the page updates by itself in a fraction of a second. To stop it: Ctrl + C.

localhost means "this computer". Nobody outside sees that page. And it is also an address the browser trusts: as you will see next lesson, the webcam turns on only on https://… or on localhost, and that is why the local project works straight away.

The two configuration files

package.json is a text file that describes the project: the name, the libraries it uses and the available commands.

{
  "name": "mano-corso",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc && vite build",
    "preview": "vite preview"
  },
  "devDependencies": {
    "typescript": "~5.9.0",
    "vite": "^7.0.0"
  }
}

What is in scripts are shortcuts: npm run dev runs what is written next to "dev". What is in devDependencies are the tools downloaded by npm install. The number next to the name is the version, and it matters: in lesson 8 you will see what happens when a library changes under a project's feet.

tsconfig.json tells the compiler how strict to be. The line we care about today is one:

{
  "compilerOptions": {
    "target": "ES2022",
    "strict": true
  },
  "include": ["src"]
}

With "strict": true TypeScript stops letting things slide. The two checks you will notice straight away:

• no accidental any: a parameter without a type becomes an error, because "I don't know" is not an acceptable answer;
• no hidden null: if something might not exist — like document.getElementById("tela"), which returns the element or null — you have to say what to do in that case.

// con strict acceso
const tela = document.getElementById("tela");
tela.textContent = "ciao";
// Errore: 'tela' is possibly 'null'.

// due modi onesti per risolverlo
const tela = document.getElementById("tela") as HTMLCanvasElement;   // "fidati, esiste"
// oppure, meglio:
const tela = document.getElementById("tela");
if (!tela) throw new Error("manca l'elemento #tela nella pagina");

Splitting the code into files: export and import

The real hand project — the one you see online — is split into five files, each with one job: turning on the camera, recognizing the hand, damping the jitter, drawing, putting it all in order. It is not fussiness: it is the only way to find things again.

A file becomes a module the moment it exports something. Whatever is not exported stays private to that file.

// src/disegna.ts
export interface Punto {
  x: number;
  y: number;
  z: number;
}

const COLORE_OSSA = "#38bdf8";   // niente export: la usa solo questo file

export function disegnaPunti(ctx: CanvasRenderingContext2D, punti: Punto[]): void {
  for (const p of punti) {
    ctx.beginPath();
    ctx.arc(p.x, p.y, 4, 0, Math.PI * 2);
    ctx.fill();
  }
}

// src/main.ts
import { disegnaPunti, type Punto } from "./disegna";

const punti: Punto[] = [{ x: 100, y: 50, z: 0 }];
// disegnaPunti(ctx, punti);

Three things to notice, and they are always the same three:

1. between braces you write the same names used in the export: if there it is disegnaPunti, here it cannot be DisegnaPunti;
2. the path starts with ./ (same folder) or ../ (folder above), and you do not write the .ts extension;
3. to import a type you write type before the name: it tells the compiler "this is only for you, it will not end up in the executed code".

What the compiler really does

The browser does not know what TypeScript is. It has never known what types are and it will not learn. What happens is simpler than it looks: the types are erased.

// quello che scrivi tu (TypeScript)
function distanza(a: Punto, b: Punto): number {
  const dx: number = a.x - b.x;
  const dy: number = a.y - b.y;
  return Math.sqrt(dx * dx + dy * dy);
}

// quello che arriva al browser (JavaScript)
function distanza(a, b) {
  const dx = a.x - b.x;
  const dy = a.y - b.y;
  return Math.sqrt(dx * dx + dy * dy);
}

Identical, minus the annotations. Two consequences follow that are worth fixing in your mind now:

• types cost nothing to whoever runs the program: by then they are already gone;
• types do not protect you while the program runs. If in lesson 8 the recognizer returns zero hands, no type stops it: you will have to check it yourself with an if.

During development Vite does this transformation, on the fly, on every save. Type checking instead is done by tsc, which runs when you launch npm run build and continuously inside VS Code, which is why the red underlines appear while you type.

Teacher demo (15')

1. Empty terminal: node -v and npm -v, to show they are there.
2. npm create vite@latest demo-aula -- --template vanilla-ts, then npm install and npm run dev: from nothing to the open page, in less than two minutes.
3. Change a line in src/main.ts, save and show the page updating by itself while nobody reloads it.
4. Create src/saluta.ts containing export function saluta(nome: string) { … } and import it in main.ts. Then make a mistake on purpose: remove the export and read the error together with the class.
5. Open tsconfig.json, set "strict": true and show the line that turns red.

Lab (60')

node -v
npm -v

v22.20.0     # an even number ≥ 20, yours may differ
10.9.3       # any number ≥ 9

npm create vite@latest mano-corso -- --template vanilla-ts
cd mano-corso
npm install
npm run dev

<!doctype html>
<html lang="it">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>La mia mano</title>
  </head>
  <body>
    <h1>La mia mano</h1>
    <button id="disegna">Disegna</button>
    <canvas id="tela" width="640" height="480"></canvas>
    <script type="module" src="/src/main.ts"></script>
  </body>
</html>

// src/main.ts
import "./style.css";

console.log("il progetto è vivo");

// src/disegna.ts
export interface Punto {
  x: number;
  y: number;
  z: number;
}

/** The 21 bones of the hand: each pair says which two points to join. */
export const OSSA: number[][] = [
  [0, 1], [1, 2], [2, 3], [3, 4],
  [0, 5], [5, 6], [6, 7], [7, 8],
  [5, 9], [9, 10], [10, 11], [11, 12],
  [9, 13], [13, 14], [14, 15], [15, 16],
  [13, 17], [17, 18], [18, 19], [19, 20],
  [0, 17],
];

export function disegnaPunti(ctx: CanvasRenderingContext2D, punti: Punto[]): void {
  ctx.fillStyle = "#ffffff";
  for (const p of punti) {
    ctx.beginPath();
    ctx.arc(p.x, p.y, 4, 0, Math.PI * 2);
    ctx.fill();
  }
}

export function disegnaOssa(ctx: CanvasRenderingContext2D, punti: Punto[]): void {
  ctx.strokeStyle = "#38bdf8";
  ctx.lineWidth = 3;
  for (const osso of OSSA) {
    const a = punti[osso[0]];
    const b = punti[osso[1]];
    ctx.beginPath();
    ctx.moveTo(a.x, a.y);
    ctx.lineTo(b.x, b.y);
    ctx.stroke();
  }
}

// src/main.ts
import "./style.css";
import { disegnaOssa, disegnaPunti, type Punto } from "./disegna";
import { MANO_FINTA } from "./punti";

const tela = document.getElementById("tela") as HTMLCanvasElement;
const ctx = tela.getContext("2d") as CanvasRenderingContext2D;

const punti: Punto[] = MANO_FINTA;
ctx.clearRect(0, 0, tela.width, tela.height);
disegnaOssa(ctx, punti);
disegnaPunti(ctx, punti);

// src/punti.ts
import type { Punto } from "./disegna";

/** A made-up open hand: coordinates between 0 and 1, like the real ones. */
export const MANO_FINTA: Punto[] = [
  { x: 0.50, y: 0.90, z: 0 },   // 0  wrist
  { x: 0.40, y: 0.84, z: 0 },   // 1  thumb
  { x: 0.33, y: 0.76, z: 0 },   // 2
  { x: 0.28, y: 0.68, z: 0 },   // 3
  { x: 0.24, y: 0.61, z: 0 },   // 4  thumb tip
  { x: 0.44, y: 0.62, z: 0 },   // 5  index
  { x: 0.43, y: 0.50, z: 0 },   // 6
  { x: 0.42, y: 0.42, z: 0 },   // 7
  { x: 0.42, y: 0.35, z: 0 },   // 8  index tip
  { x: 0.52, y: 0.60, z: 0 },   // 9  middle
  { x: 0.52, y: 0.47, z: 0 },   // 10
  { x: 0.52, y: 0.38, z: 0 },   // 11
  { x: 0.52, y: 0.31, z: 0 },   // 12 middle tip
  { x: 0.59, y: 0.62, z: 0 },   // 13 ring
  { x: 0.60, y: 0.50, z: 0 },   // 14
  { x: 0.61, y: 0.42, z: 0 },   // 15
  { x: 0.61, y: 0.35, z: 0 },   // 16 ring tip
  { x: 0.65, y: 0.67, z: 0 },   // 17 little finger
  { x: 0.68, y: 0.58, z: 0 },   // 18
  { x: 0.70, y: 0.52, z: 0 },   // 19
  { x: 0.72, y: 0.46, z: 0 },   // 20 little finger tip
];

/** From 0…1 coordinates to pixels of a canvas `larghezza` wide and `altezza` high. */
export function aPixel(p: Punto, larghezza: number, altezza: number): Punto {
  return { x: p.x * larghezza, y: p.y * altezza, z: p.z };
}

// src/main.ts — the part that changes
import { MANO_FINTA, aPixel } from "./punti";

const punti = MANO_FINTA.map((p) => aPixel(p, tela.width, tela.height));
disegnaOssa(ctx, punti);
disegnaPunti(ctx, punti);

// 1) 'const ctx' is possibly 'null'
//    getContext("2d") returns null if the canvas does not support 2D.
const ctx = tela.getContext("2d");
if (!ctx) throw new Error("questo browser non sa disegnare in 2D");

// 2) Parameter 'p' implicitly has an 'any' type
//    inside map the type is usually inferred; if it is not, you write it.
const punti = MANO_FINTA.map((p: Punto) => aPixel(p, tela.width, tela.height));

Typical mistakes in class

• Command run in the wrong folder: npm run dev answers "Could not read package.json". The cd mano-corso is missing. The terminal always shows which folder you are in: read it before typing.
• Spaces or accents in the path: a folder called Documenti scuola/progètto will sooner or later cause trouble. Use short, lowercase names, without spaces.
• The server is already on: "Port 5173 is in use". Vite picks another one and says so — or you stop the first with Ctrl + C.
• The page is white: almost always an error in main.ts. Open the browser console with F12 and read. The console is the first place to look, not the last.
• "I fixed it but the error is still there": the file was not saved. Ctrl + S. The dot next to the file name, at the top, means "not saved".
• import path without ./: import { … } from "disegna" makes it look for a library called "disegna" inside node_modules. You need "./disegna".
• node_modules "takes up too much space": yes, hundreds of megabytes, and that is normal. Do not touch it, do not copy it, do not put it on a USB stick: it is recreated in thirty seconds with npm install.

Quick check (5 questions)

Homework (30-40 minutes)

Tidy up the project and make it yours: put the fake hand in the centre of the canvas whatever the size of the canvas, add a third module src/colori.ts that exports the colours used by the drawing (so changing one file changes the whole look), and check that npx tsc --noEmit says nothing. Bring the project next time, ready to start: lesson 7 begins with npm run dev already running.

The webcam: async, permissions, frames

Today your page sees. To get there you need three new things: asking for something that takes time (async and await), reacting when it goes wrong (try/catch) and redrawing thirty times a second without jamming the browser.

• Where you work: in your own mano-corso project, with npm run dev running
• New concepts: Promise, async/await, try/catch, getUserMedia, secure context, the <video> element, drawImage, requestAnimationFrame, stopping the tracks
• By the end the student can: really turn the webcam on and off, explain to the user what went wrong, and draw every frame on the canvas while counting how many go by per second

Theory

Some things are not ready right away

Until now every line of code gave its result immediately. Turning on the webcam does not: the browser has to ask the user for permission, and the user may think about it for five seconds; then it has to turn on a physical camera. Meanwhile the program cannot stop: if it stopped, the page would freeze and not even the «Allow» button would work.

JavaScript's solution is called Promise: not the result, but the promise of a result. A function like getUserMedia gives you a promise right away, and sooner or later that promise is kept (the video stream arrives) or broken (the user said no).

To work with promises there are two words:

async function accendi() {
  const flusso = await navigator.mediaDevices.getUserMedia({ video: true });
  console.log("acceso!", flusso);
}

• await means «wait until this promise is kept, then give me the value». While it waits, the rest of the page keeps working: buttons respond, animations run.
• async in front of function is the permission to use await inside it. If you forget it, the compiler tells you right away.

async function accendiCamera(video: HTMLVideoElement): Promise<MediaStream> {
  const flusso = await navigator.mediaDevices.getUserMedia({ video: true, audio: false });
  video.srcObject = flusso;
  await video.play();
  return flusso;
}

When it goes wrong: try / catch

A promise can break, and with the webcam it breaks often: permission denied, no camera, camera already used by another program. If you do nothing, the user sees a page that does not react and does not understand why.

try {
  flusso = await accendiCamera(video);
  // you only get here if it went well
} catch (e) {
  // you only get here if it went badly
  console.log("non ha funzionato:", e);
}

The browser does not send you a sentence, it sends you an error with a name. The three you will see in class today:

Translating them is trivial and changes everything: it is the difference between an application and a homework exercise.

The secure context

The webcam only turns on if the page comes from https://… or from localhost. On a plain http://, navigator.mediaDevices does not exist at all: the code fails with an obscure error.

The reason is serious. On http:// anyone in the middle of the network — the wi-fi operator, for instance — can change the page while it travels, and so make a page that had not asked for it turn on the camera. https prevents this. localhost is allowed because it travels nowhere: it is your computer.

if (!window.isSecureContext || !navigator.mediaDevices?.getUserMedia) {
  throw new Error("La camera funziona solo su una pagina HTTPS o su localhost.");
}

It is exactly the check the real project does, before anything else. And it also explains why in lesson 10, when we talk about putting the site online, HTTPS will not be an optional detail.

Where the video ends up

The webcam stream does not show itself: it has to be attached to a <video> element.

<video id="video" autoplay muted playsinline></video>
<canvas id="tela" width="640" height="480"></canvas>

Three attributes, three reasons:

• autoplay — it starts by itself, without a second click;
• muted — without it browsers block autoplay; and we do not need audio anyway, in fact we ask for audio: false;
• playsinline — on phones it stops the video from opening full screen and swallowing the page.

The <video>, though, is only a little window. To draw the hand landmarks on top you need the canvas, and first you have to copy the image into it:

ctx.drawImage(video, 0, 0, tela.width, tela.height);

One line, and the current frame is in the canvas. From there on you draw whatever you want on top: in lesson 8 it will be 21 dots.

The frame loop

Drawing once is not enough: the image changes all the time. You need a loop that restarts at every frame, and the browser provides it:

function ciclo(): void {
  requestAnimationFrame(ciclo);          // book the next round
  ctx.drawImage(video, 0, 0, tela.width, tela.height);
}
ciclo();                                  // you call the first round yourself

requestAnimationFrame — «request an animation frame» — tells the browser: call me back just before the next screen repaint. It is usually 60 times a second.

Why not a timer, that is setInterval(ciclo, 16)? Three concrete reasons:

• the timer does not know when the screen repaints, so now and then you draw twice between one repaint and the next (wasted work) or zero times (a visible stutter);
• if the computer is slow, timers pile up and the page chokes; requestAnimationFrame instead simply skips a round;
• when you switch tab, the browser suspends requestAnimationFrame: no battery spent drawing a page nobody is looking at. The timer would keep going regardless.

The loop is stopped with cancelAnimationFrame, and to do that you must have kept the number that requestAnimationFrame returns:

let idCiclo = 0;

function ciclo(): void {
  idCiclo = requestAnimationFrame(ciclo);
  // …drawing…
}

function fermaCiclo(): void {
  cancelAnimationFrame(idCiclo);
}

Really turning the camera off

This is the part almost everyone gets wrong the first time. Pausing the <video> does not turn off the camera: the stream stays open and the little light next to the lens stays on. You have to stop the tracks, that is the individual channels of the stream, one by one:

function spegniCamera(video: HTMLVideoElement, flusso: MediaStream | null): void {
  flusso?.getTracks().forEach((t) => t.stop());   // ← this is the line that matters
  video.pause();
  video.srcObject = null;
}

The check is physical and leaves no room for argument: the webcam light must go off. If it stays on, you have turned nothing off.

Teacher demo (15')

1. A page with two buttons and a <video>. Write accendiCamera live and make the permission request appear on the projector.
2. Deny permission on purpose. Show that the page says nothing, then add the try/catch and the translated message. It is the most useful moment of the lesson.
3. Write spegniCamera first without getTracks().forEach(...): the light stays on. Then add the line: the light goes off. Show the little light, not the code.
4. Add the loop with requestAnimationFrame and the drawImage on the canvas: now there are two identical images, the video and the canvas. Ask the class why two are needed — the answer («we can draw on top of the canvas») is lesson 8.

Lab (60')

<!-- index.html -->
<button id="accendi">Accendi la camera</button>
<button id="spegni">Spegni</button>
<p id="messaggio"></p>
<video id="video" autoplay muted playsinline></video>
<canvas id="tela" width="640" height="480"></canvas>

// src/main.ts
import "./style.css";

const bottoneAccendi = document.getElementById("accendi") as HTMLButtonElement;
const video = document.getElementById("video") as HTMLVideoElement;

let flusso: MediaStream | null = null;

async function accendiCamera(): Promise<void> {
  flusso = await navigator.mediaDevices.getUserMedia({ video: true, audio: false });
  video.srcObject = flusso;
  await video.play();
}

bottoneAccendi.addEventListener("click", () => {
  void accendiCamera();
});

const messaggio = document.getElementById("messaggio") as HTMLParagraphElement;

/** The standard browser error names, translated into useful sentences. */
const MESSAGGI: Record<string, string> = {
  NotAllowedError: "Permesso negato: consenti l'uso della camera dalle impostazioni del browser e riprova.",
  NotFoundError: "Nessuna webcam trovata su questo dispositivo.",
  NotReadableError: "La webcam è occupata da un'altra applicazione.",
  OverconstrainedError: "La webcam non supporta la risoluzione richiesta.",
};

async function accendiCamera(): Promise<void> {
  messaggio.textContent = "";
  try {
    flusso = await navigator.mediaDevices.getUserMedia({ video: true, audio: false });
    video.srcObject = flusso;
    await video.play();
    messaggio.textContent = "● Camera accesa";
  } catch (e) {
    const nome = e instanceof DOMException ? e.name : "";
    messaggio.textContent = MESSAGGI[nome] ?? "Non riesco ad accendere la camera.";
  }
}

const bottoneSpegni = document.getElementById("spegni") as HTMLButtonElement;

function spegniCamera(): void {
  flusso?.getTracks().forEach((t) => t.stop());
  video.pause();
  video.srcObject = null;
  flusso = null;
  messaggio.textContent = "Camera spenta";
}

bottoneSpegni.addEventListener("click", spegniCamera);

const tela = document.getElementById("tela") as HTMLCanvasElement;
const ctx = tela.getContext("2d");
if (!ctx) throw new Error("questo browser non sa disegnare in 2D");

let idCiclo = 0;

function ciclo(): void {
  idCiclo = requestAnimationFrame(ciclo);

  ctx.save();                       // set aside the current orientation
  ctx.translate(tela.width, 0);     // move the origin to the right edge
  ctx.scale(-1, 1);                 // flip the x axis: there is the mirror
  ctx.drawImage(video, 0, 0, tela.width, tela.height);
  ctx.restore();                    // put everything back as it was
}

const etichettaFps = document.getElementById("fps") as HTMLSpanElement;

let fotogrammi = 0;
let inizioFinestra = performance.now();

function contaFps(): void {
  fotogrammi++;
  const adesso = performance.now();
  const trascorso = adesso - inizioFinestra;
  if (trascorso >= 1000) {
    etichettaFps.textContent = `${Math.round((fotogrammi * 1000) / trascorso)} fps`;
    fotogrammi = 0;
    inizioFinestra = adesso;
  }
}

document.addEventListener("visibilitychange", () => {
  if (document.hidden && flusso) spegniCamera();
});

Typical mistakes in class

• Permission denied by mistake the first time: the browser remembers and does not ask again, so the page looks broken. Fix it from the camera icon in the address bar. Show this to everyone before the lab, not after.
• await outside an async function: compiler error. The little word async goes in front of the function that contains the await.
• The video shows but the canvas is black: the loop has not started, or the canvas has zero width and height. The canvas width and height are written as HTML attributes, not as CSS.
• Everything mirrored, text included: ctx.restore() is missing.
• The light stays on after «Spegni»: getTracks().forEach((t) => t.stop()) is missing. That is why this check is done with your eyes on the webcam.
• Two loops in parallel: turn on, turn off, turn on again, and suddenly the fps doubles. Every start-up launched a loop without stopping the previous one. The fix is cancelAnimationFrame(idCiclo) before every start.
• The webcam does not start in the online environment: it is the framed preview. From here on you work locally, or in pairs with someone who has it; the preview in a separate tab is the last resort (lesson 6).

Quick check (5 questions)

Homework (30-40 minutes)

Move everything to do with the camera into a module src/camera.ts, which exports only two functions: accendiCamera(video) returning Promise<MediaStream> and spegniCamera(video, flusso). In main.ts only the buttons, the messages and the loop remain. Also add the secure-context check at the start of the start-up. It is the camera.ts file of the real project, written by you: next lesson you will use it as it is.

MediaPipe: the 21 landmarks of a real hand

This is the lesson in which the project works. At the end of these two hours you move your hand in front of the webcam and on screen a hand drawn by you moves, with your fingers and your movements. The rest of the course is finishing touches.

• Where you work: in your own project, with the working camera from lesson 7
• New concepts: artificial intelligence models, installing a library with npm install, pinned versions, FilesetResolver, GestureRecognizer, runningMode: "VIDEO", recognizeForVideo, reading a nested result with ?.
• By the end the student can: load a model, get the 21 landmarks of their own hand at every frame, convert them to pixels and draw points, bones and the gesture name

Theory

What a model is, in two paragraphs

Until now every line of your program told the computer exactly what to do: «draw a circle here», «multiply this by that». Finding a hand in an image cannot be done that way: nobody knows how to write the rules to recognize a hand — in any light, any skin, any ring, any background. People tried for twenty years.

A model is a different approach: instead of rules, examples. Someone — Google, here — collected tens of thousands of photographs of hands and marked by hand, on each one, where the 21 joints are. Then a training program adjusted millions of internal numbers, a little at a time, until the system started guessing those 21 points even on photographs it had never seen. The result of that work is a file: gesture_recognizer.task, eight megabytes of numbers. It contains no photographs, no hands: only the numbers needed to do that calculation.

Three practical things follow from this, worth having clear:

• The model is downloaded once and runs on your computer. There is no server to query: the image of your hand never leaves the browser. That is the reason this project can promise what it promises.
• The model makes mistakes. Not «may make mistakes»: it does, and you will see it today. Against the light, with the hand tilted or half out of the frame, the points jitter or the hand disappears. It is not your bug: it is the nature of the thing, and in lesson 9 you will learn to live with it.
• The model understands nothing. It gives numbers, and the numbers have a confidence. It does not know what a hand is, it does not know it is yours.

Installing a library, and pinning its version

npm install @mediapipe/tasks-vision@1.0.1

This command downloads Google's library into node_modules and adds a line to package.json. That @1.0.1 at the end is the important part of the lesson.

"dependencies": {
  "@mediapipe/tasks-vision": "1.0.1"
}

Written like this — without ^, without ~, without @latest — it means «exactly this version». Anyone who installs the project two years from now will get the very same files, and the project will work as it does today.

Loading the recognizer

You need two addresses: the runtime (the calculation engine, written in WebAssembly) and the model. Loading takes time — about twenty megabytes the first time — so it is all async, like last lesson's webcam.

// src/tracker.ts
import { FilesetResolver, GestureRecognizer } from "@mediapipe/tasks-vision";

const WASM = "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@1.0.1/wasm";
const MODELLO =
  "https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/1/gesture_recognizer.task";

export async function creaRiconoscitore(): Promise<GestureRecognizer> {
  const vision = await FilesetResolver.forVisionTasks(WASM);
  return GestureRecognizer.createFromOptions(vision, {
    baseOptions: { modelAssetPath: MODELLO },
    runningMode: "VIDEO",
    numHands: 1,
  });
}

Line by line, the three things that matter:

• FilesetResolver.forVisionTasks prepares the engine: it looks at what your processor can do and picks the right WebAssembly variant by itself.
• runningMode: "VIDEO" says «you will get frames in sequence, not unrelated photos». In this mode the recognizer follows the hand from one frame to the next instead of searching for it from scratch every time: it is much faster and much more stable. The alternative is "IMAGE", for single photographs.
• numHands: 1 — one hand is enough to start. Setting it to 2 is exercise 8.5.

Asking for the points, at every frame

const risultato = riconoscitore.recognizeForVideo(video, performance.now());

This line goes inside the loop from lesson 7. Two details, both sources of mistakes on the first day:

1. It is not await. In VIDEO mode the calculation happens immediately and returns the result immediately. It is also why, if the computer is slow, the fps collapse: drawing waits for recognition.
2. The time must always increase. The second parameter is the moment of the frame, in milliseconds. If you pass a time equal to or less than the previous one, the library stops with an error. With performance.now() it does not happen, because a clock moves forward — but you need to know the rule, because it causes an error that otherwise looks incomprehensible.

There is also a trick that saves half the work. The screen repaints 60 times a second, but the webcam sends 30 frames a second: half of the loop's rounds see the same image as before. Analysing it twice is wasted work:

let ultimoIstante = -1;

function ciclo(): void {
  idCiclo = requestAnimationFrame(ciclo);

  // readyState >= 2 means "a frame is ready"
  if (video.readyState >= 2 && video.currentTime !== ultimoIstante) {
    ultimoIstante = video.currentTime;
    const risultato = riconoscitore.recognizeForVideo(video, performance.now());
    disegnaTutto(risultato);
  }
}

What the result looks like

risultato.landmarks is an array of hands; each hand is an array of 21 points; each point has x, y, z. Three levels, and you have to keep them apart:

risultato.landmarks            // [ [21 points], [21 points] ]  ← 0, 1 or 2 hands
risultato.landmarks[0]         // the 21 points of the first hand
risultato.landmarks[0][8]      // the index fingertip
risultato.landmarks[0][8].x    // 0.42 — a number between 0 and 1

They are the same 0-to-1 coordinates as the fake hand in lesson 5, and that is no accident: the course was built so that today you have to change just one line. They are numbers relative to the image, not pixels: if tomorrow you change the size of the canvas, the points keep working.

And the point numbers are always the same, for everyone: 0 the wrist, 4 the thumb tip, 8 the index fingertip, 12 the middle finger, 16 the ring finger, 20 the little finger. They are the numbers you already used in the bone table in lesson 6.

When the hand is not there

If you take your hand out of the frame, landmarks is an empty array. Writing risultato.landmarks[0][0].x without checking blows the program up. The ?. from lesson 4 is for exactly this:

const mano = risultato.landmarks[0];
if (!mano) return;          // no hand in this frame: skip drawing

// or, in one line:
const polso = risultato.landmarks[0]?.[0];
console.log(polso?.x ?? "nessuna mano");

And it is the practical answer to what was said in lesson 6: types disappear before execution, so they do not protect you while the program runs. You have to write the check yourself.

Teacher demo (15')

1. npm install @mediapipe/tasks-vision@1.0.1 on the projector; open package.json and show the new line, without ^.
2. Write creaRiconoscitore and call it at start-up. Show the Network panel of the developer tools: you can see the model's megabytes coming down. Then reload: the second time it is instant, because the browser has them in cache.
3. In the loop, console.log(risultato.landmarks.length): move your hand in and out of the frame and comment on the 1s and 0s scrolling by.
4. Print risultato.landmarks[0]?.[8] and slowly move your index finger: have the class read the x going from 0.1 to 0.9. It is the moment the numbers become your own finger: worth letting it breathe for half a minute.
5. Make a mistake on purpose: pass 0 instead of performance.now() and read the error about increasing timestamps.

Lab (60')

// src/main.ts — the new parts
import { creaRiconoscitore } from "./tracker";
import type { GestureRecognizer } from "@mediapipe/tasks-vision";

let riconoscitore: GestureRecognizer | null = null;
let ultimoIstante = -1;

async function avvia(): Promise<void> {
  messaggio.textContent = "Carico il modello…";
  riconoscitore = await creaRiconoscitore();   // model first: it asks the user nothing
  await accendiCamera();                        // then the camera: permission is asked last
  ciclo();
}

function ciclo(): void {
  idCiclo = requestAnimationFrame(ciclo);
  if (!riconoscitore) return;
  if (video.readyState < 2 || video.currentTime === ultimoIstante) return;
  ultimoIstante = video.currentTime;

  const risultato = riconoscitore.recognizeForVideo(video, performance.now());
  console.log("mani:", risultato.landmarks.length, "punti:", risultato.landmarks[0]?.length ?? 0);
}

import { disegnaPunti, type Punto } from "./disegna";
import { aPixel } from "./punti";

function mostra(punti: Punto[]): void {
  ctx.save();
  ctx.translate(tela.width, 0);
  ctx.scale(-1, 1);
  ctx.drawImage(video, 0, 0, tela.width, tela.height);
  ctx.restore();

  const inPixel = punti.map((p) => aPixel(p, tela.width, tela.height));
  disegnaPunti(ctx, inPixel);
}

// in the loop, after getting the result:
const mano = risultato.landmarks[0];
if (mano) mostra(mano);
else ctx.clearRect(0, 0, tela.width, tela.height);

import { disegnaOssa, disegnaPunti } from "./disegna";

const inPixel = punti.map((p) => aPixel(p, tela.width, tela.height));
disegnaOssa(ctx, inPixel);    // lines first
disegnaPunti(ctx, inPixel);   // then the dots, on top

const etichettaGesto = document.getElementById("gesto") as HTMLParagraphElement;

const GESTI: Record<string, string> = {
  Closed_Fist: "✊ Pugno",
  Open_Palm: "🖐️ Mano aperta",
  Pointing_Up: "☝️ Indice in su",
  Thumb_Down: "👎 Pollice giù",
  Thumb_Up: "👍 Pollice su",
  Victory: "✌️ Vittoria",
  ILoveYou: "🤟 Ti voglio bene",
};

const gesto = risultato.gestures[0]?.[0];
const nome = gesto?.categoryName;
etichettaGesto.textContent =
  nome && nome !== "None" ? GESTI[nome] ?? nome : "nessun gesto riconosciuto";

// tracker.ts: numHands: 2

// disegna.ts: both functions take one more parameter, a colour
export function disegnaOssa(ctx: CanvasRenderingContext2D, punti: Punto[], colore: string): void {
  ctx.strokeStyle = colore;
  ctx.lineWidth = 3;
  for (const osso of OSSA) {
    const a = punti[osso[0]];
    const b = punti[osso[1]];
    ctx.beginPath();
    ctx.moveTo(a.x, a.y);
    ctx.lineTo(b.x, b.y);
    ctx.stroke();
  }
}

// main.ts
const COLORI = ["#38bdf8", "#f472b6"];

risultato.landmarks.forEach((mano, i) => {
  const inPixel = mano.map((p) => aPixel(p, tela.width, tela.height));
  disegnaOssa(ctx, inPixel, COLORI[i]);
  disegnaPunti(ctx, inPixel, "#ffffff");
});

Typical mistakes in class

• «Cannot read properties of undefined»: landmarks[0][0] was read in a frame with no hands. You need the check with if or the ?..
• Error about non-increasing timestamps: a fixed number was passed, or the same performance.now() saved in a variable and reused. The time is read at every call.
• The page freezes for ten seconds on first start: it is the twenty-megabyte download. It is not a bug — but the user must be told with a «Loading the model…» message, otherwise it looks broken.
• Points all piled up in the top-left corner: they are the 0…1 coordinates drawn as if they were pixels. aPixel is missing.
• Points mirrored relative to the image: you mirrored the video but not the coordinates (or the other way round). Decide once and be consistent.
• The trail: clearRect is missing, or the video drawImage does not cover the whole canvas.
• Two or three frames per second: recognition is running on the processor instead of the graphics card, which is common on school laptops. It is not solved today; note it down and measure it with exercise 7.5.

Quick check (5 questions)

Homework (30-40 minutes)

Make what you have presentable: a «Loading the model…» message while waiting, «No hand in view» when the hand is not there, the gesture name in large type, and the fps counter from lesson 7 left on in a corner. Then write down on a sheet of paper how many fps your computer gets with recognition on and under which conditions the hand gets lost (against the light? hand edge-on? too close?). Bring the numbers next time: lesson 9 starts from there.

Polishing: classes, state, gestures, jitter

The project already works. Today we go from «it works» to «you can hand it to someone»: the hand stops shaking, the buttons always know what state they are in, and the ghost hands stop flickering.

• Where you work: in your own project, with the working hand from lesson 8
• New concepts: class, constructor, fields, methods, private, union types as state, a single function that applies the state, moving average, confirmation over several frames
• By the end the student can: write a small class and use it, keep the interface consistent with a single setState, and damp the jitter knowing what is gained and what is lost

Theory

Classes: an object that carries its own functions

In lesson 4 you learned objects: boxes with values inside. A class is the same idea, with one thing more: inside it there are also the functions that work on those values.

Look at today's problem. To damp the jitter you need to remember the last five values of a coordinate. With what you know so far:

let ultimiValori: number[] = [];

function aggiungi(valore: number): void {
  ultimiValori.push(valore);
  if (ultimiValori.length > 5) ultimiValori.shift();
}

It works… for one coordinate. But the coordinates are 21 points × 3 axes = 63, and each one needs its own memory. With loose variables you would need 63 arrays and 63 functions. With a class, you write one and create 63.

class MediaMobile {
  private valori: number[] = [];      // field: the memory of THIS object
  private readonly quanti: number;    // how many values to keep

  constructor(quanti = 5) {           // runs when the object is born
    this.quanti = quanti;
  }

  aggiungi(valore: number): void {    // method: a function of the object
    this.valori.push(valore);
    if (this.valori.length > this.quanti) this.valori.shift();
  }

  valore(): number {
    let somma = 0;
    for (const v of this.valori) somma += v;
    return somma / this.valori.length;
  }
}

const x = new MediaMobile(5);
const y = new MediaMobile(5);   // separate memory: two objects, two histories

x.aggiungi(0.42);
x.aggiungi(0.44);
console.log(x.valore());        // 0.43

That is all the vocabulary, and it is five words:

• class — the mould: it says what an object of that type is made of and what it can do;
• new — build an object from that mould;
• constructor — what has to be done at birth;
• field — a value the object keeps (valori, quanti);
• method — a function the object can run (aggiungi, valore).

this means «this object here, not the other 62». And private means «not to be touched from outside»: whoever uses a MediaMobile calls aggiungi and valore, and does not even need to know there is an array inside. If tomorrow you change how the average is computed, no other file notices.

The interface state

Look at your lesson 8 page and count: the «Accendi» (turn on) button, the «Spegni» (turn off) one, the message, the error box, the fps counter. Five things that must be consistent with each other. If you switch them on and off one by one, scattered through the code, sooner or later you reach an absurd screen — «Camera off» written while the light is on, the «Starting…» button that stays there forever after an error.

The solution is to say first of all which states exist. It is a union type, like the ones in lesson 2:

type Stato = "spenta" | "avvio" | "accesa" | "errore";

Four words, and no others: if you write "acesa", the compiler stops you straight away. Then a single function decides how each state looks:

function setState(stato: Stato, messaggio: string): void {
  etichettaStato.textContent = messaggio;
  bottoneAccendi.disabled = stato === "avvio" || stato === "accesa";
  bottoneSpegni.disabled = stato !== "accesa";
  riquadroErrore.hidden = stato !== "errore";
  document.body.dataset.stato = stato;   // useful to the CSS and to whoever reads the page
}

From then on, in all the rest of the program, no button is touched any more: you call setState("avvio", "Carico il modello…") and that is it. The advantage is not writing fewer lines: it is that impossible states become impossible, instead of lying in wait.

Why the points jitter

Hold your hand still, really still. The points on screen keep dancing by a few pixels. It is not your fault and it is not a bug: the model analyses each frame on its own, and each frame is slightly different from the previous one — sensor noise, a little change in light. Small differences in, small differences out. Thirty times a second, and the eye reads them as a vibration.

The simplest idea to remove it is one you already know from school marks: instead of the current value, the average of the last five. The little jumps up and down cancel out and the hand calms down.

// no filter                         average over 5
// 0.412 0.418 0.409 0.421 0.414     0.415 0.415 0.416 0.415 0.416

But there is a price, and it must be well understood because it is the heart of the lesson. If you move your hand suddenly, the average still contains four old values: the hand on screen arrives late, as if it were crawling behind yours. The stronger the filter, the slower it is. With 5 values the delay is barely noticeable; with 20 the hand seems to be in treacle. Try both: it is exercise 9.2.

Confirmation over several frames

There is a second, sneakier defect. Every now and then — one frame in fifty, when the light changes or something passes through the frame — the model «sees» a hand that does not exist. It lasts a single frame, but on screen it is an annoying flash.

The remedy costs three lines: do not trust the first time. You count for how many consecutive frames the hand has been present, and draw only from three onwards.

let consecutivi = 0;
const CONFERME = 3;

if (risultato.landmarks.length > 0) consecutivi++;
else consecutivi = 0;

if (consecutivi >= CONFERME) {
  // only now is the hand drawn
}

Cost: a tenth of a second of delay when it appears, which nobody notices. Gain: the ghost hands disappear. It is the same reasoning as the filter — you pay a little delay to buy stability — and it is one of the most common choices in the whole trade.

Teacher demo (15')

1. Hold your hand still in front of the webcam and zoom in on the screen: the whole class must see the points dancing by themselves. The problem before the solution.
2. Write MediaMobile live, calmly: it is the first class they see. Point out that this.valori of two different objects are two different arrays.
3. Apply it and look at the still hand again: the points calm down. Then move your hand suddenly and show the delay. Do not hide it, show it: it is the lesson.
4. Raise the window to 20 values: the hand becomes jelly. Bring it back to 5. Ask the class which they would prefer, and why there is no answer that is right for every use.
5. Show the finished setState and try the sequence turn on → error → turn off → turn on again, to show that the buttons never get tangled.

Lab (60')

// src/filtro.ts
export class MediaMobile {
  private valori: number[] = [];
  private readonly quanti: number;

  constructor(quanti = 5) {
    this.quanti = quanti;
  }

  aggiungi(valore: number): void {
    this.valori.push(valore);
    if (this.valori.length > this.quanti) this.valori.shift();
  }

  /** The average of the values in memory. If there are none, it returns 0. */
  valore(): number {
    if (this.valori.length === 0) return 0;
    let somma = 0;
    for (const v of this.valori) somma += v;
    return somma / this.valori.length;
  }
}

// test
const m = new MediaMobile(3);
m.aggiungi(10);
console.log(m.valore());   // 10
m.aggiungi(20);
console.log(m.valore());   // 15
m.aggiungi(30);
m.aggiungi(60);            // the 10 leaves the window
console.log(m.valore());   // 36.66…

// src/filtro.ts
import type { Punto } from "./disegna";

export class FiltroMano {
  private filtri: MediaMobile[] = [];

  constructor(quanti = 5) {
    // 21 points × 3 coordinates: one filter for each, with its own memory
    for (let i = 0; i < 21 * 3; i++) this.filtri.push(new MediaMobile(quanti));
  }

  smorza(punti: Punto[]): Punto[] {
    const risultato: Punto[] = [];
    for (let i = 0; i < punti.length; i++) {
      this.filtri[i * 3].aggiungi(punti[i].x);
      this.filtri[i * 3 + 1].aggiungi(punti[i].y);
      this.filtri[i * 3 + 2].aggiungi(punti[i].z);
      risultato.push({
        x: this.filtri[i * 3].valore(),
        y: this.filtri[i * 3 + 1].valore(),
        z: this.filtri[i * 3 + 2].valore(),
      });
    }
    return risultato;
  }
}

// main.ts
import { FiltroMano } from "./filtro";

let filtro: FiltroMano | null = null;

// when the hand appears
filtro ??= new FiltroMano(5);
const addolciti = filtro.smorza(mano);
// …and from here on you draw `addolciti` instead of `mano`

// when the hand disappears: throw the filter away
filtro = null;

type Stato = "spenta" | "avvio" | "accesa" | "errore";

function setState(stato: Stato, messaggio: string): void {
  etichettaStato.textContent = messaggio;
  bottoneAccendi.disabled = stato === "avvio" || stato === "accesa";
  bottoneSpegni.disabled = stato !== "accesa";
  riquadroErrore.hidden = stato !== "errore";
  document.body.dataset.stato = stato;
}

async function avvia(): Promise<void> {
  setState("avvio", "Carico il modello…");
  try {
    riconoscitore ??= await creaRiconoscitore();
    await accendiCamera();
    setState("accesa", "● Camera accesa");
    ciclo();
  } catch (e) {
    spegni();
    riquadroErrore.textContent = messaggioErrore(e);
    setState("errore", "Camera spenta");
  }
}

function spegni(): void {
  cancelAnimationFrame(idCiclo);
  spegniCamera(video, flusso);
  flusso = null;
  filtro = null;
  setState("spenta", "Camera spenta");
}

const mano = risultato.landmarks[0];

if (mano) {
  etichettaMani.textContent = "";
  // …filter and draw…
} else {
  etichettaMani.textContent = "Nessuna mano inquadrata";
  ctx.clearRect(0, 0, tela.width, tela.height);
  filtro = null;
}

const CONFERME = 3;
let consecutivi = 0;

if (mano) {
  consecutivi++;
  filtro ??= new FiltroMano(5);
  const addolciti = filtro.smorza(mano);   // always filter…
  if (consecutivi >= CONFERME) {
    disegnaOssa(ctx, addolciti.map((p) => aPixel(p, tela.width, tela.height)), "#38bdf8");
    disegnaPunti(ctx, addolciti.map((p) => aPixel(p, tela.width, tela.height)));
  }
} else {
  consecutivi = 0;
  filtro = null;
}

let precedenti: Punto[] | null = null;
let sommaMovimento = 0;
let misure = 0;

/** Average displacement of the 21 points relative to the previous frame (in 0…1 units). */
function misuraTremolio(punti: Punto[]): void {
  if (precedenti && precedenti.length === punti.length) {
    let somma = 0;
    for (let i = 0; i < punti.length; i++) {
      const dx = punti[i].x - precedenti[i].x;
      const dy = punti[i].y - precedenti[i].y;
      somma += Math.sqrt(dx * dx + dy * dy);
    }
    sommaMovimento += somma / punti.length;
    misure++;
  }
  // copy of the values, not of the references: we need them as they were now
  precedenti = punti.map((p) => ({ x: p.x, y: p.y, z: p.z }));
}

// once a second, together with the fps:
if (misure > 0) {
  etichettaTremolio.textContent = `tremolio: ${(sommaMovimento / misure).toFixed(5)}`;
  sommaMovimento = 0;
  misure = 0;
}

Typical mistakes in class

• Forgotten this: inside a method someone writes valori.push(...) instead of this.valori.push(...). The compiler says «Cannot find name 'valori'»: the field exists only through this.
• Forgotten new: const f = FiltroMano(); builds nothing. You need new FiltroMano().
• A single filter for all the points: the hand crumples towards a single point, and it is a memorable sight. You need 63 independent filters, one per coordinate.
• Filter never thrown away: the hand coming back into the frame crawls in from where it left.
• The hand «in treacle»: window too long. Bring it back to 5 and look again.
• precedenti = punti instead of the copy: you save a reference to the array, not the values; at the next comparison the differences all come out zero and the measured jitter is 0.00000. It looks like a resounding success and it is not.
• Two setState calls in a row with contradictory messages: the last one wins, and it is often the wrong one. One path, one state.

Quick check (5 questions)

Homework (40 minutes)

Two things. One: polish the project however you like — colours, point sizes, a title, the gesture shown in large type, maybe the hand's colour changing with the recognized gesture. Two: prepare the presentation for lesson 10. Three or four minutes, in which you must say: what your project does, the part of the code you are most pleased with, the thing that cost you the most time and how you solved it. No slides: you show the project running.

The third dimension, publishing, presentations

Last lesson, and you can breathe: the hand leaves the plane and becomes a figure in space, we look at what it really means to put a site online, and then everyone presents. Twenty hours ago none of you had written a line of TypeScript.

• Where you work: in your own project, starting from a starter file provided by the teacher
• New concepts: Three.js, scene, camera, lights, mesh; from 21 2D points to 21 spheres in space; npm run build, what you need to publish, HTTPS, a quick look at CSP
• By the end the student can: connect the points of their own hand to a 3D scene, explain what happens when a site «goes online», and tell someone about their project

Theory

Three.js: scene, camera, lights, mesh

The browser can draw in three dimensions, but the interface it offers (WebGL) is ferociously awkward: matrices, programs for the graphics card, buffers of numbers. Three.js is the library that steps in between and talks the way people talk in a theatre. Four words and you have understood everything you need today:

• scene (Scene) — the stage: the list of things that exist;
• camera (PerspectiveCamera) — where you look from, and with what field of view: distant things get smaller, as in reality;
• lights — without them, everything is black. An ambient light that brightens everywhere, a directional one that gives volume;
• mesh (Mesh) — an object: a geometry (the shape: a sphere, a cylinder) plus a material (the colour, how shiny it is).

const sfera = new THREE.Mesh(
  new THREE.SphereGeometry(1, 20, 14),                    // shape
  new THREE.MeshStandardMaterial({ color: 0x38bdf8 }),    // appearance
);
sfera.position.set(0, 0, 0);
scena.add(sfera);

And then, at every frame, a single line: renderer.render(scena, camera). Three.js looks at where the objects are at that moment and draws the picture.

From 21 points to 21 spheres

Here is all the maths of the lesson, and it is three lines. The points coming from the webcam live in a world like this: x and y between 0 and 1, origin in the top-left corner, y growing downwards as in all images. The 3D world instead has its origin in the centre and the y growing upwards, as at school.

function nelloSpazio(p: Punto) {
  const larghezza = 6;    // how wide the video rectangle is in the 3D world
  const altezza = 4.5;
  return {
    x: (0.5 - p.x) * larghezza,       // centre AND mirror
    y: (0.5 - p.y) * altezza,         // centre AND flip
    z: -p.z * larghezza * 1.6,        // depth, a little exaggerated
  };
}

Look at what that 0.5 - x does, because it does two jobs in one subtraction:

• it moves the origin from the corner to the centre: the point at the centre of the image (0.5) becomes 0, that is the centre of the scene;
• it reverses the direction of the axis. On y it takes you from the «y down» of images to the «y up» of 3D graphics. On x it is the mirror effect: move your hand to the right and the hand on screen goes to the right, as in front of a mirror. Without it, it feels like watching someone else, and it is strangely unpleasant.

The z has its sign flipped (MediaPipe's depth is negative towards the camera, in 3D the observer is on the positive side) and is multiplied by 1.6: the depth estimated from a single camera is small, and without some gain bringing your hand closer would hardly be noticed. The value is chosen by eye — any higher, and the hand starts to look deformed.

Putting a site online

So far you have worked on localhost, which is just your computer. «Putting online» means three things, and none of them is magic:

1. Building: npm run build. The command checks the types, translates the TypeScript into JavaScript, joins the files, shortens them and writes everything into the dist/ folder. Inside there is a normal site — pages, scripts, images — that no longer needs either Node or Vite.
2. A computer that is always on to serve those files to whoever asks for them, and a name (a domain) pointing to that computer.
3. HTTPS, and here there is no choice: without a certificate, as you saw in lesson 7, the webcam does not turn on. Your project online on http:// would be a dead page.

The site you are looking at, for instance, runs on a small rented computer: one program serves the dist/ folder, another (Caddy) takes care of the domain and gets the HTTPS certificate by itself. Before publishing, a script checks that nothing strange has appeared in the built files and that all the pages are there; if a check fails, publishing stops and yesterday's version, which worked, stays online.

A few closing words: privacy and CSP

Your project handles images of people. It is the most sensitive data there is, and you handled it the right way almost without noticing: the images do not leave the browser, because there is nowhere to send them; the camera turns on only on click; it turns off when you switch tab. It is not by chance, it is a design choice — and the rule underneath it applies well beyond this course: the data you do not collect is the only data you cannot lose.

There is also a way not to just promise it. A site can declare to the browser what it is allowed to do — it is the Content Security Policy. The hand site declares, among other things, that the page may talk only to its own domain. From then on it is no longer the programmer who guarantees that the images are not sent to someone: it is the browser that refuses to send them, even if the code tried. The whole difference between a promise and a guarantee lies there.

In the real project this rule had an interesting side effect: it blocked a call nobody had written. It was the recognition library, which every minute sent some usage data to its servers — no images, but still a call to a stranger, and with no way to switch it off. It has been disabled. Without the CSP, nobody would ever have noticed.

Teacher demo (10')

1. Open the starter file src/scena3d.ts (you will find it below, in the teacher box) and show that everything is already there: scene, camera, lights, 21 spheres lined up in a row.
2. Connect a single point, the wrist, and move it with your hand in front of the webcam. A sphere following a wrist is already a striking effect, and it explains the mechanism better than twenty.
3. Then connect all 21 points: the hand appears. Get the class looking at the screen and bring your hand closer to the camera to show the depth.
4. Make a mistake on purpose: remove the 0.5 - from the y. The hand turns upside down. Put it back and explain the subtraction while everyone watches the effect.

Lab (80')

npm install three@0.186.0
npm install --save-dev @types/three@0.186.0

// src/scena3d.ts — ready-made 3D scene: you only need to call mostraMano() and disegna()
import * as THREE from "three";
import type { Punto } from "./disegna";

const tela3d = document.getElementById("scena") as HTMLCanvasElement;

const scena = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(35, 1, 0.1, 100);
camera.position.set(0, 0, 10);

const renderer = new THREE.WebGLRenderer({ canvas: tela3d, antialias: true, alpha: true });

function adatta(): void {
  const w = tela3d.clientWidth || window.innerWidth;
  const h = tela3d.clientHeight || window.innerHeight;
  renderer.setSize(w, h, false);
  camera.aspect = w / h;
  camera.updateProjectionMatrix();   // required after changing the proportions
}
window.addEventListener("resize", adatta);
adatta();

// Lights: an ambient one so nothing stays black, a directional one that gives volume.
scena.add(new THREE.HemisphereLight(0xdbeafe, 0x1e1b4b, 1.4));
const principale = new THREE.DirectionalLight(0xffffff, 2.2);
principale.position.set(3, 5, 8);
scena.add(principale);

// 21 spheres, created ONCE only: at every frame they are just moved.
const geometria = new THREE.SphereGeometry(1, 20, 14);
const materiale = new THREE.MeshStandardMaterial({ color: 0x38bdf8, roughness: 0.35 });
const sfere: THREE.Mesh[] = [];
for (let i = 0; i < 21; i++) {
  const s = new THREE.Mesh(geometria, materiale);
  s.position.set((i - 10) * 0.35, 0, 0);   // in a row, until a real hand arrives
  s.scale.setScalar(0.12);
  scena.add(s);
  sfere.push(s);
}

const LARGHEZZA = 6;
const ALTEZZA = 4.5;

/** Moves the 21 spheres onto the 21 points of the hand (MediaPipe 0…1 coordinates). */
export function mostraMano(punti: Punto[]): void {
  for (let i = 0; i < 21; i++) {
    const p = punti[i];
    sfere[i].position.set(
      (0.5 - p.x) * LARGHEZZA,
      (0.5 - p.y) * ALTEZZA,
      -p.z * LARGHEZZA * 1.6,
    );
  }
  mostra(true);
}

/** Shows or hides the whole hand. */
export function mostra(visibile: boolean): void {
  for (const s of sfere) s.visible = visibile;
}

/** One frame. To be called inside the loop in main.ts. */
export function disegna(): void {
  renderer.render(scena, camera);
}

<!-- index.html: the scene canvas, behind everything else -->
<canvas id="scena"></canvas>

// main.ts
import { disegna, mostra, mostraMano } from "./scena3d";

function ciclo(): void {
  idCiclo = requestAnimationFrame(ciclo);

  if (riconoscitore && video.readyState >= 2 && video.currentTime !== ultimoIstante) {
    ultimoIstante = video.currentTime;
    const risultato = riconoscitore.recognizeForVideo(video, performance.now());
    const mano = risultato.landmarks[0];

    if (mano) {
      filtro ??= new FiltroMano(5);
      mostraMano(filtro.smorza(mano));    // 0…1 points: the scene does the conversion
    } else {
      filtro = null;
      mostra(false);
    }
  }

  disegna();   // outside the if: the scene is redrawn at every frame
}

// inside mostraMano, in scena3d.ts
const PUNTE = [4, 8, 12, 16, 20];
sfere[i].scale.setScalar(PUNTE.includes(i) ? 0.16 : 0.12);

// colour: the material is shared, so one line changes the whole hand
export function coloraMano(colore: number): void {
  materiale.color.setHex(colore);
}
// in main.ts
coloraMano(nome === "Thumb_Up" ? 0x22c55e : 0x38bdf8);

// size according to distance: wrist closer = more negative z
const vicinanza = 1 - punti[0].z * 2;
sfere[i].scale.setScalar(0.12 * Math.max(0.5, Math.min(2, vicinanza)));

npm run build
npm run preview     # opens http://localhost:4173

/** Index, middle, ring, little finger: tip and knuckle of each. */
const DITA = [
  { punta: 8, nocca: 5 },
  { punta: 12, nocca: 9 },
  { punta: 16, nocca: 13 },
  { punta: 20, nocca: 17 },
];

function contaDita(punti: Punto[]): number {
  let alzate = 0;
  for (const dito of DITA) {
    // smaller y = higher up in the image
    if (punti[dito.punta].y < punti[dito.nocca].y) alzate++;
  }
  return alzate;
}

The presentations (40')

Three or four minutes each, with the project running. Four questions, always the same, so everyone knows what to prepare:

1. What does your project do? Show it, do not describe it.
2. Which piece of code are you most pleased with, and why?
3. What cost you the most time? How did you get out of it?
4. If you had two more hours, what would you add?

The third question is the most important of the four. Telling the story of a mistake and how you solved it is what, in this trade, separates those who know how to work from those who only know how to make things work: it will happen in every interview, in every meeting, for the rest of your working life. You might as well start today.

Typical mistakes in class

• Black screen: the lights are missing, or the canvas has no size in the CSS, or disegna() is never called. Check them in that order.
• The hand is upside down: the 0.5 - on the y is missing, or it has become p.y - 0.5.
• The hand moves the wrong way: the mirror on the x is missing. It is the same reasoning as the y, but for a different reason — here it is the user's comfort, there it is a conversion between two coordinate systems.
• The hand is huge or invisible: it was given points in pixels instead of between 0 and 1.
• Everything stutters: new objects are being created inside the loop. The spheres are created once, then moved.
• npm run build fails while npm run dev worked: it is tsc checking the types for real. The errors are real and must be fixed; during development they were only ignored squiggles.

Quick check (5 questions)

Homework (no deadline)

The course ends here, the project does not. Eight directions in which it can grow, from the easiest to the most ambitious:

1. Count the raised fingers properly, measuring distances from the wrist instead of heights (exercise 10.5, serious version).
2. Draw with your finger: when the index is raised and the others closed, leave a trail on the canvas. Closed fist = erase. It is thirty lines and great fun.
3. Drive a small game: a paddle that follows your index finger, a target to hit, a score. You already know everything you need.
4. Two hands together, with different colours and the right/left labels sorted out properly.
5. Your own gestures: measure the distances between the fingertips and invent your own commands — thumb and index touching, L-shaped hand, three fingers. Remember the confirmation over several frames from lesson 9.
6. The whole body: the very same loop with 33 points from head to toe. Look at the Body page of this site: only the model changes.
7. Replace the spheres with a real hand: cylinders for the bones, a semi-transparent palm. That is what the demo on this site does, and the code is commented in Italian line by line in the tutorial.
8. Publish your site and send the link to someone. It is the simplest thing on the whole list and the most satisfying: npm run build, then any platform that hosts static sites with HTTPS included.

Twenty hours ago you had never written console.log. Now you have an application that turns on a camera, asks an artificial intelligence model where your fingers are, damps the noise in the data, draws a figure in three dimensions sixty times a second and sends your data to nobody. It is not an exercise: it is a real program. Keep going.