Graphics!

I have studied that stuff before.


You should go learn about what "Vectors" are - Professor Daniel Shiffman from https://YouTube.com/TheCodingTrain will explain fine (in his "The Nature of Code!" series).


Scratch is missing *a lot* of the stuff you need. Variables are still scattered around the place. You can't make dimensional arrays (imagine a list inside a list!). It is *very* hard to pull off 3D graphics in such an environment! Also, it is completely `CPU`-based, so performance experiences a big loss.




Now, let me to try to explain a bit of this myself :)


# Some Math!

Skip ahead to the enxt 'section' if you want to know how this Math is being used game engines.

The word "vector" comes from Latin, meaning "vehicle". A vector, is "data structure", with numbers in it.


Imagine a number written on a white board - no, no! A mouse! Where can it move? Well, it can only move up, down, left... and right! ...right?! Haha! This is the idea of the 'dimensionality' of a data structure (just *something* that holds numbers!).


In a vector, we make a *straight,* up-to-down list of numbers. It is a one-dimensional data structure, since we're traveling downwards.


In Scratch, all objects have an `(x, y)` position, and guess what? THIS, is also a vector - a list of numbers! You can fit anything - even the `RGB` values for a color! ..or even put a `z`-axis and... all fo a sudden you have `(x, y, z)` coordinates.


However, our mouse can also move left and right. If we let it move in both axes (as if it is in 2D space), we'd make a "matrix". Vectors only had columns - numbers listed from the too, to the bottom. "Matrices" however, are tables of numbers, where we have both rows and columns. Also, note how a vector is basically a matrix, but it's not taking advantage of the 2D space available!


This might remind you of our good old ordinary numbers that had none of this 'dimensionality' nonsense. We call them "scalars" with love.


Now... We can do things such as adding and subtracting with both of these. If you have a `4x4` matrix, then you can indeed, add it to another `4x4` matrix. It's simple - just add or subtract the numbers that are in the same places, and... you'd have the result!


Same with vectors - as long as you know they're the same size, it works!


MULTIPLICATION AND DIVISION, however, are very complicated. Matrices only have this thing called a "dot product", whereas vectors have both a "dot" and a "cross" product. Divison in matrices only applies in some cases!

Before we discuss those, you need to understand that vectors are something you can... visualise! Remember how I said that a vector is like the `(x, y)` position? Well... yeah! In Scratch, the stage has it's "origin point" (the `(0, 0)` point) at its very center. Imagine if we have something at the position `(144, 72)`. I could, for example... draw an arrow towards it. This arrow, is supposed to 'represent' a vector. Hello, vector!


Now imagine if we had another pointing to some other `(x, y)` coordinate. If we're to write them out in the mathemtical way, and try to multiply them (AKA take the product of their `x` and `y` 'components'), we'd see these behaviours when using the `dot` and `cross` products:

The `dot product` method gives us a SCALAR number - remember? Just an ordinary number!

The `cross product`, gives us a "perpendicular vector" - that is, an arrow that is pointing *through* the middle of both of them. For two 2D vectors that are already perpendicular to each other (that is, there is a 90-degree angle between both of the arrows), we'd see a third vector, which... is actually 3D! This looks similar to "Fleming's right hand law" (or... *"left hand law"*) from Physics. **After having looked up a photograph of it on Google** - if you think of the index finger and the thumb as the two 2D arrows we drew, the cross product would give us an arrow that is represented by the thumb.


Alright, that felt like a math class. What's the good stuff?!


Well, vectors can use "trigonometry", to *basically* connect these `(x, y)` with the concept of "direction" and "length" - don't our arrows have those? They point in a direction, and are of a some size! Aren't they?


This gives us *awesome* powers such as being able to find the *direction* between two points on the screen, or making some awesome rotation animations (think of Physics Engines! Although Physics Engines use a bit more math, they HEAVILY use the concept of vectors!).


Now... we can also multiply a vector by a matrix. It sounds weird, but it is doable. And matrices can do some awesome stuff!


# And finally, Game Engines!

Please search and learn about terms such as "`GPU`s" and "`API`s" on Google, if you don't know what they are.


***NOW,*** game engines use a "Graphics API", such as... `OpenGL` ("free" software - anybody can use it, and it is present on all machines - it is maintained by the "Khronos Group"), `DirectX` (made and maintained by Microsoft as a kind of 'wrapper' on `OpenGL`, I guess. Apparently, it's a little faster on Windows.) or `Vulkan` (a newer version of `OpenGL` made by Khronos that came in 2015, promising more control over how memory is used in the `GPU`).


Scratch 3 *actually* uses `OpenGL` in the form of "`WebGL`", which is a version of `OpenGL` made for browsers so that the same code could be run on browsers on both mobile and desktop devices!


This "graphics API", allows us to contact the `GPU` on our computer, and ask it to draw things to the screen, and sometimes, we ask it to do some Math for us. It is a device that does everything very fast in both scenarios, since it is designed that way!


(At some point, you'll study about electric circuits in Physics. There, you'll be told about "series" and "parallel" circuits. The `CPU` is like a series circuit - everything is calculated one-by-one. The `GPU`, does this faster, since it takes the idea of "parallel processing", and *essnetially,* has THOUSANDS of tiny `CPU`-like units that calculate things fast by cutting out on precision and electric power.)


Recall how animations are an illusion where things move in the screen *little-by-little* after very small amounts of time? Yeah. Our computers do that when we animate things, and during each 'cycle' of animation, we send LOADS of data - images, shape color data and the `(x, y)` positions of the points of every shape we draw, to the `GPU` for it to calculate (yeah, those are vectors!).


Now, the `GPU` takes this data, and following instructions written in a "shader program", uses this data to draw things to the screen!


In this data, as stated earlier, we provide the positions for *vertices* (that is, the `(x, y)` positions for the points of a shape), and those mathematical number tables we discussed earlier - "Matrices", to the `GPU`.


Mainly, we send three of these:

- A "projection" matrix,
- A "view"/"eye"/"camera" matrix, and,
- A "model" matrix.


The "model" matrix has data about the position, rotation and size of a shape, and is used to modify the vector data (`(x, y)` positions) the `GPU` will ultimately give everything we're drawing.


The "camera" matrix, gives us the illusion of all of these objects being present in a " game world". You can *actually* give the `GPU` numbers to create zoom, rotation and "change in location in a 3D (or 2D) world"-kind of effects.


The "projection" matrix, is of two types. "Orthographic", and "Perspective". **No, nobody types out these numbers by hand anymore - *everybody* uses already available code for this!**


This "projection" matrix, is what *scales*, that is, resized everything on the screen, to make sure it fits on every screen size. Orthographic matrices create a view, where everything appears to be the same size even if they are far away somewhere on the `z`-axis. Perspective ones, give us the much wanted 3D illusion - making things that are far away, appear more near the center of the screen, and things that are closer, close to the corners. Now, as I said before - nobody types the numbers for a projection matrix themselves. We mostly use ready-made code for this. That code too, however, needs data, such as:

- `FOV` ("**F**ield **O**f **V**iew" - tells the graphics API *how much* of the world we want to able to see at once - imagine sitting in a chair. If you tilt yourself forwards, you see less. If you tilt yourself backwards, you see more at once!)
- the "aspect ratio* of the screen (`width / height`),
- the "near" and "far"... "clipping plane depths" - these tell the computer to *stop* displaying things that are too far away on the `z`-axis ("far plane's depth"), and things that are *too near* ("near plane's depth").


The problem? You'll *have* to calculate these numbers on Scratch **BY HAND**, which is... no joke, a task very hard for actual engineers, too - real engineers may understand what they're doing, but you can always get it wrong!


Tools built for actual computer graphics *business* provide you all of this, but Scratch doesn't. Making these is a hard task, given how simple Scratch is, but some people using Scratch ***were*** very hardworking, and... got these done right! Salutes to them!


The *terrifying*, terrifying part is optimization - these calculations are supposed to be done on a `GPU`, but they do it on the `CPU` on Scratch - oh God!

..it's actually not all that bad, since the actual displaying of the graphics is still done by Scratch using `WebGL`, so it may be fine. But hey - Scratch doesn't let you modify any of it - you're making another layer on whatever exists! Optimization ded!!!1!1!

...


That's how it works. Took me an hour to write this - uhm, *bye!*