-======================================================================================================-
| David Irvine - SAFE Network, Technical Overview of Vaults - Full Transcription (Clean) [beta 0.5.88] |
-======================================================================================================-

This is obviously the Vault network. So just a very brief overview of vaults, but... it's very related to that "language of the network" blog post that we've got.

So we've got certain identities on this network. We have Client Node, Client Manager, Node Manager. And Network Addressable Element manager---this guy here is where a lot of the work actually gets done.

So, we don't really need to go over again. We can understand what a Client is. It's somebody that's not in the Routing table. So for our-- when we're in the Vault library, we just consider Client as a User. A Client Node is basically an Access Node, with a Web browser, if you like. It can read information off of this network (this is the Vault network).

The basic thing that a Client is is a browser. And he can read the network as though it's just one great big huge data repository. And whatever shape we want that data to be---whether it's Web pages, bits of hard drive, bits of programs, anything---so any kind of data, your Client in general is something that can read from it.

For us, if we just consider the network-- let's do it like that. We'll just consider the network like this at the moment. So when the Clients in this browser thing, the Client Manager doesn't matter. Because, we don't manage browsers. We don't care if people are looking at the network. We don't need to do any management of them. The Client Manager is not required when you're just a browser of data on the network.

In terms of being this user, your view of the network is "Get ABC." And that thing goes to the part of the network which controls ABC, and returns the data to you. So feedback network: you ask the part that's responsible for that piece of data, and it comes back to you. So that's what you see. Just like a Web browser is not aware of the internal workings of a Web server, here a Client in browser mode is not aware or cares about the internal workings of the Maidsafe network. And, slightly different, the Maidsafe network has no ability to prevent this happening. So anybody at the moment can read any piece of data. They're not managed, and the piece of data just comes back to them.

So that's the kind of, one of the fundamental things that the Vault network does. Where it gets tricky is when we want to Mutate this whole thing, when we want to Mutate the network. That's where it gets hard. And, also, how does the network look after the Data?

So if we tried to explain the Vaults several months ago, we would have been looking at a 2-3 week presentation here. And get through things like Synchronizing, and Passing keys about, and doing all sorts of stuff. But this has been simplified a lot now, by the Routing interface.

So what the Routing interface fundamentally will do is say, "Here's a message."
And it will tell you several things about that message. So it will tell you the type of call---so Type (Get, Put, Post).
And it will tell you Your Authority (Auth).
And it will tell you From Authority (Auth).
And it will have a Data Type Identifier.
So these are the kind of fundamental things that Routing will give us, in Vaults.

And that alone gives us a lot of power. So we can tell what this thing is that's getting passed up to us. And the message from the Vault layer is Valid 100%. We don't need to do any work here. We don't need to confirm, "Am I really this Authority?" "And did it really come from there?" "Is it really that data type?" That's all 100% valid, as far as we are concerned. And it's Routing's job to make sure that that is the case.

So with that information coming into the Vaults, it gives us the ability to do certain things. So, if we are a Vault node, this thing here is going to say, "You're Authority." So you could be Client Manager, Node Manager, or Network Addressable Element manager.

Now you might be thinking, "Why do you keep calling this Network Address 'Element Manager' and not Data Manager?" Network Addressable Element manager is the group closest to the name of the Data. So this Message has got a Name, and Content. We don't know what type it is. Could be 3 or 4 things, 5 things, 2 things, 1 thing. But it's got a Name. So the group closest to that Name is your Network Addressable Element manager. And what that decides to do with that piece of data is entirely up to us, as programmers of the network.

So when we get this thing through here, then we use a sort of language of the network host, and make decisions. So let's say someone was storing a piece of data. So it's a Client, and he's going to try and store a piece of data. In other words he's going to try and Mutate the network. He's going to try and Mutate the network, so he's not doing a Read. So this is something that has to be Managed. He's trying to mutate the actual network---not Data that he owns on the network, but Mutate the actual network.

So the Call that he would use there is Put -> Mutate. And when we say Mutate here, what we mean is he's going to Put something onto the network. He is going to actually Put a piece of Data on the network. The network is going to grow an intelligence, if you like, with this Store piece of Data protocol.

So once we get this message up from Routing, our Vault is going to analyze the message. So we are going to be told (in this Store Data thing), "We are either a Client Manager." That's the first thing we're interested in, is our Authority. "Or we're a Network Addressable Element manager." "Or a Node Manager." "Or a Node." (So this Store Data comes in.)

And if we are a Client Manager, we ask ourselves the question, "Is user allowed to Store(Put)?" So is the user allowed to put this piece of Data? So we are saying here, if we are a Client Manager, is the user allowed to put the piece of Data? I.e. is From(Auth) a User? Then we ask the question, "Does User have Auth to Store?" So how do we know if the User has the Authority to Store? He just came out of the blue. We don't know him. So, we can't answer that question, "Does the User have the Auth to Store?" Because, we don't know anything about the user.

So what has to happen is, these guys obviously have to know the Users they're responsible for. So, before someone can try and Mutate Data, we are saying they have to have an Account. And that Account, from the Client Manager perspective, needs to be able to answer this question, "Is the User allowed to Store this piece of Data?"

So how do we understand if a user is allowed to store this piece of Data? So his Account has to show that he has this ability to Put more Data on the network. So eventually that will mean that he has purchased space via Safecoin. Previously what it meant was he had a Vault that had stored plenty of stuff on the network, in this quid pro quo thing. But basically we need to know, has this guy purchased space? And has he got any left? And does what he have left cover the Size of the piece of Data he's trying to Store?

So, we get this thing as a Client Manager that says, "You're a Client Manager." So we can say, "Oh, if I'm a Client Manager, it must have come from a Client." "And if it didn't come from a Client, drop it." So if it's not from a Client, drop it. We don't do anything with it. We're a Client Manager. We want to spoken to with a Put from a Client. (So we drop it.) If it's actually from a Client, we look in our Accounts List, to see, "Does that Client have space left?" If he does have space left, then what we're doing is we're going to the next step.

And the next step is basically, we are going to send the Data then, to Network Addressable Element manager. So the Network Addressable Element manager says, "I'm not going to speak to a Client!" It's going to say (our Authority is obviously Network Addressable Element manager), "From Authority should be Client Managers." And if the From Authority is not Client Managers, drop it. So, we'll put that over here, same as here: From = Client Manager. So if it's not from a Client Manager, drop it, because that's all we're speaking to just now.

So a Network Addressable Element manager, what's he asking? What questions is he asking: "Is already stored?" Or, "Does it need stored?" So, this is where he's looking at first. He's just gets the message, saying, "Is this from a Client Manager Group?" "Routing should have confirmed that it is." (And how Routing does that is kind of irrelevant to us here; we don't care. So [??] Vaults, don't even put your head in that area. It's Routing's job.)

So if it's from a Client Manager group, then we are really saying, is it already stored? If it's already stored, our job's done---we're finished. That's fine. Somebody tried to Store a piece of Data. The network already knows about that piece of Data. It's complete. That's AOK. However, if it still needs Stored, then the Network Addressable Element manager will basically return to Routing. Same as this guy did. So this guy said, "There is space left." "Pass the message on to Network Addressable Element manager." Network Addressable Element manager checks if it's Stored or if it needs Stored. If it needs Stored, it passes it back to the Routing, "Send this to Node Managers." A Node Manager group. And the Node Manager Group...

Oh, is it down there? ... Let me take this opportunity! *laughs*

So, he goes to the Node Managers. And the Node Manager accepts this piece of Data. Checks it came from the From Authority. So here the From (Network Address called Element Manager). (And if it's not, drop it.) Takes this piece of Data.

And basically, he updates the Node Account. So, the Node Account, just like the User Account, he updates that to say, "Where would the User be saying you need to have paid me to do this sort of stuff, and give me something so that I can store this piece of data?" What the network's doing down here when he is updating the Node Account is saying, "We are going to reward you." Or, "We are going to recognize that you are doing work for us, by updating your Account." "So as a network, we are aware that you are doing a little bit more than you were before we spoke to you." "We're adding to your account the fact that you are doing more work." Very important.

And then the Node Manager sends the data (I won't draw on the wall down here) to the Node that was selected by the Network Addressable Element manager. The Node will then look and say, "I'm only going to accept stuff on this network that is told to me by my Manager Group." "My manager group is who controls me." So I need to know that it's them that have said, "Store this piece of Data." Because, it's them that's recording the fact that I've Stored that I want Credit for it." "I'm only going to trust them." So the node checks, "Is it From Authority?" "His Manager Group." And if it's his Manager Group, then that's perfectly fine. He'll store the piece of Data.

So, it might look kind of complex. But when you break this down it's actually pretty straightforward. It's basically just a whole bunch of dispatching. We just dispatch as messages come in. So we're kind of saying, "If it's a Put, my Authority is a Client Manager." "The From Authority has to be the User." "And if that's all True, I'll do the work." "If there's a Put, and I'm a Network Addressable Element manager, the From Authority must have been a Client Manager Group." "And if that's True, I'll do the work." And same, "I'm a Node Manager." "I should have been told by a Network Addressable Element manager to do something." And then it'll do the work. Or, "If I'm a Node, I must have been told from a Node Manager to do the work."

And that gives us this logical network, where we can say, "We will identify this relationship here. We'll identify the relationship." "And based on the type of request that's coming in, we'll do certain things." So, the relationship first has to be something that we have defined. There is a relationship from Client to Network Address. And if there's wasn't, this message wouldn't go through. So we are defining those relationships. And those definitions of those relationships are really important. Because, any messages outside of that definition just get dropped. And Routing as I said is already guaranteeing this Authority stuff is correct.

So in the Vault network, we are really saying we have got these different types of personas.

So how can we introduce something that means that these types of personas will speak to each other? And based on the message type (Get, Put, Post), will execute a piece of code. (So if it's a Get it will do one piece of Code. Put, it will do another one. Post, it will do something else.) So, we are defining these relationships. And then we are defining actions---the actions based on what the request type has been (Get, Put, or Post).

So the relationship gets defined. And then the action gets defined. And this action here---so we'll go back to the work horse. The work horse here of the whole network is Network Addressable Element manager.

So the Client Manager is doing something pretty simple. It's saying, "I need to be told from a Client." Node Manager's managing nodes. So there's a relationship where there's a manager thing happening here, and happening here. But if you'll look from the Node perspective, you're just managed by your Close Group. From a Client perspective you're just managed by your Close Group.

So we can take those out of the picture, just now, and look at the work horse---look at the kind of Clydesdale (horse) of this whole thing---the Network Addressable Element manager. Because, that does something that the other Node types don't do. And it's all to do with this: Data Type Identifier.

So, our Network Addressable Element manager---for us at Maidsafe, because we are doing things in a certain way, and we've been thinking of a certain thing---it's "Data Manager." And that's what we think, "It's that Data Manager, because that's what sets the [??] and all the rest of it." But it's not the Data Manager. So, what extra step happens in the network here (now, we're just focused on this Network Addressable Element manager)---we get this message, and we say, "I am Network Addressable Element manager."

(So the From Authority: let's say can be from different people, different types here. And the data type-- let's ignore From Authority just now. It's going to confuse.)

So this message comes in. We've got all the Authority checks. We're doing whatever we were doing previously.

And we've got a Data Type. This Data type here is important, because that's what starts Dispatch 2. The Network Addressable Element manager based on a Data Type goes through another dispatch. So here, if our Data Type is Immutable Data, then we are doing this Data Manager's job really. So if you look at the code just now, that's your Data Manager. However, if that Data Type is Structured Data---or our SDV thing that we were (the Structured Data Versions of the version thing, of the transaction thing or whatever)---it's a Structured Data Manager at this stage. So we're getting Structured Data through. And, let's just say we're going to handle it differently from a Data Manager.

So this particular Data Type is going be handled differently from the Data Manager's data types. And that's what this 2nd Dispatch is. So it's saying, "We're coming through the network." "We're getting to a level which says this is Network Addressable Element territory." And we get into Network Addressable Element territory, the Network Addressable Element is saying, "I can be a Data Manager." "Or I can be a Version Manager." "Or I can be multiple other things (as time goes on, we'll see what those are)." "But, at the moment, I've got a couple of things I can be." "And those things I can be are determined by the Type of Data that's getting passed through to me."

So from the SDV thing, or the Transaction Manager, or whatever we want to call this thing---which seems to be lots of different names of it: Data Manager---the difference between these two is literally, "What is the Action that we do on that Data Type?" "How do we handle it on the network?" "Do we store it somewhere?" "Do we store it ourselves, on that certain account or certain container?"

Whatever it is we do, that Data Type you can imagine could be Immutable Data. It could be Structured Data. It could be a computation. With Structured Data and Immutable Data, we can handle all Data Types in terms of Data. (But we use a different mechanism.) It could be a computation that the Network Addressable Element manager is going to be handling. But the Type of Data (or Type of Message, which might be more accurate) allows is to Dispatch that Call to the correct piece of code. So we've split Transaction Manager and Data Manager into two types of code, and two different personas.

And really,