/ elang / intro

15 Minute Intro to E by Marc Stiegler

Most of E looks and acts like a conventional object-oriented programming language. A few quick examples are:

The declaration of a variable and the assignment statement look a bit like Pascal:

def a := 1 + 2
# Comments can start with a #

The if-statement looks quite familiar to anyone who has done C:

if (a == b) { a := 3 * a }

The print-statement looks much like Java:

println("Here's the answer")

The creation of an object looks a little different, but is easily understood by anyone who has written Java; in this example, we use the carMaker object (which behaves like a class) to create a car:

def car := carMaker("Mercedes")

Calling a method on an object looks somewhat like C++, Python, or Java, with a period separating the object from the method name:

car.moveTo(2,3)

For the purposes of this introduction, this ends our coverage of the ordinary object-oriented E features. Let us now launch into the features that make E unique. We start with the eventually operator, "<-":

car <- moveTo(2,3)
println("car will eventually move to 2,3. But not yet.")

The first statement can be read as, "car, eventually moveTo(2,3)". As soon as this eventual send has been made to the car, the program immediately moves on to the next sequential statement--the program does not wait for the car to move. Since the program does not wait around for the eventual send, an eventual send is very different from a traditional object-oriented method call (referred to here as an immediate call to distinguish it from an eventual send; the statement "car.moveTo(2,3)", shown earlier, is an immediate call). In general, you do not know, and cannot know, exactly when the car will move; indeed, if the car is on a remote computer, and the communication link is lost, the car may never move at all (which throws an exception,discussed later).

This brings us to the most interesting feature of an eventual send: just as you do not know when the operation will complete, you also do not know, and do not need to know, where the car is. The car could be a local object running on the same machine with the program... or it could be on a computer a thousand miles away across the Internet. Regardless, E keeps track of the car's location and delivers the message for you. Moreover, if the car is indeed remote, E sets up a secure communication link between the program and the car; and since the Universal Resource Identifier for the car includes an ungueassable random string of characters, no one can send a message to the car or extract information from the car except someone who has explicitly and intentionally received a reference to it from someone with authority. Thus a computation running on five computers scattered across five continents, all publicly accessible by the whole world of the Web,can be as secure as a computation running on a box locked in your basement.

Another very interesting feature of the eventual send: because the program continues on to the next statement immediately, without waiting for the eventual send to finish, deadlock can never occur.

Finally, note that the eventual send invoked an ordinary object method ("moveTo(x,y)")in the car. The programmer who created the carMaker defines the cars to have ordinary methods, with ordinary returns of values (or null if no value need be returned). He does not know and does not need to know whether objects that invoke those methods use calls or sends, and does not know or need to know whether those invoking objects are local or remote.

When you make an eventual send to an object (referred to hereafter simply as a send, which contrasts with a call to a local object that waits for the action to complete), even though the action may not occur for a long time, you immediately get back a promise for the result of the action:

def carPromise := carMaker <- new("Mercedes")
carPromise <- moveTo(2,3)

In this example, we have sent the carMaker object the "new(name)" message. Eventually the carMaker will create the new car on the same computer where the carMaker resides; in the meantime, we get back a promise for the car. We can make eventual sends to the promise just as if it were indeed the car, but we cannot make immediate calls to the promise even if the carMaker (and therefore the car it creates) actually live on the same computer with the program. To make immediate calls on the promised object, you must set up an action to occur when the promise resolves:

def temperaturePromise := carPromise <- getEngineTemperature()
when (temperaturePromise) -> done(temperature) {
    println(`The temperature of the car engine is: $temperature`)
} catch e {
    println(`Could not get engine temperature, error: $e`)
}
println("execution of the when-catch waits for resolution of the " +
        "promise, but the program moves on immediately to this println")

We can read the when-catch statement as, "when the promise for a temperature becomes done, and therefore the temperature is locally available, perform the main action block... but if something goes wrong, catch the error in variable e and perform the error block". Inside the when-catch statement, we say that the promise has been resolved. In this example, we have requested the engine temperature from the carPromise. Eventually the carPromise resolves to a car and receives the request for engine temperature; then eventually the temperaturePromise resolves into an actual temperature. The when-catch construct waits for the temperature to resolve into an actual (integer) temperature, but only the when-catch construct waits. The program itself does not wait: rather, it proceeds on, with methodical determination, to the next statement following the when-catch.

The temperature is an integer and is guaranteed to resolve to a local integer object upon which you can make immediate calls, even if the car is remote. Immutable objects, such as integers, floating point numbers, booleans, arrays, and the E data structures ConstLists and ConstMaps, are always passed by copy, so you always get a local copy of the object if one of these is returned by a method call or send. This local copy can of course accept immediate calls. Mutable objects like cars reside on the machine upon which they were constructed. So in general, if you created the car using an eventual send, you will probably have to interact with the car using eventual sends forever, since only such sends are guaranteed to work with remote objects.

def farCar := carMaker <- new("Mercedes")
farCar <- moveTo(2,3)
farCar <- moveTo(5,6)
farCar <- moveTo(7,3)
def fuelPromise := farCar <- fuelRemaining()

Here we have created a farCar using an eventual send to the carMaker. Earlier, we declared a variable temperaturePromise, wherein the suffix of "Promise" was a notational reminder that indeed this was only a promise, and it could not receive immediate calls except inside a when-catch construct where the promise would have resolved into an actual local copy of the temperature.

In this example, we use the prefix "far" as a notational reminder that this car may be (indeed, probably is) executing on a remote machine, and that consequently we can never make immediate calls to it. It is a reminder that we will always have to make eventual sends to it for any interactions we desire. Technically, what we get back immediately from the carMaker is still only a promise, but since the promise and the farCar both respond identically to sends, and both respond identically to calls (by throwing an error), we treat it the same as if the promise had already been resolved to a remote reference, and name the variable using the remote reference convention. In a real program, the carMaker object would probably be a remote reference as well, and would actually have the variable name farCarMaker to follow the convention.

The example moves the farCar around repeatedly and then asks for the amount of fuel remaining. This displays another important property of eventual sends: if object A sends several messages to object B, it is guaranteed that those messages will arrive and be processed in the order of sending. This is only a partial ordering, however. From B's point of view, there are no guarantees that the messages from A will not be interspersed with messages from C, D, etc. Also, from A's point of view, there are no guarantees that, if A sends messages to both B and C, the message to B will resolve before (or after) the message from C, regardless of the order in which A initiates the messages. Despite those uncertainties, however, in the example the partial ordering is sufficient to guarantee that fuelPromise will resolve to the quantity of fuel remaining after all three of the moveTo() operations have occurred, and that those moveTos will have been performed in the specified sequence.

There is one reason to use a when-catch construct to resolve the promise for a far object. This reason is to do a test for equality, i.e., to see whether the promised object is the same object you received from another activity:

when (raceTrack <- getPolePositionCar()) -> done(farPolePositionCar)          {
    if (farPolePositionCar == myCar) {
        println("My car is in pole position")
    }
} catch e {}

Note that the catch clause is empty. Because remote references across a network are much less reliable than immediate calls to objects collocated with the program, you will usually want to handle the error case. However, if you have many pending actions with a single remote object, all the catch clauses for all those actions will start up if the connection fails, so you may want to disregard most of the catch clauses and focus on handling the larger problem (loss of connection) in one place. In this example, there is no compelling special action to take if you can't find out whether your car is in pole position, so no action is taken.

If you write a function or method that must get values from a far object before returning an answer, your function should return promise for the answer, and fulfill that answer later. You can create your own promises using the Ref.promise() primitive in E:

def promiseMilesBeforeEmpty(farCar) :any{
    def [milesBeforeEmptyPromise, milesBeforeEmptyResolver] := Ref.promise()
    def fuelRemaining := farCar <- fuelRemaining()
    def milesPerGallonPromise := farCar <- efficiency()
    when (milesPerGallonPromise) -> done(milesPerGallon) {
        milesBeforeEmptyResolver.resolve(milesPerGallon * fuelRemaining)
    } catch e {
        milesBeforeEmptyResolver.smash(`Car Lost: $e`)
    }
    return milesBeforeEmptyPromise
}

# Somewhere else in the program
def milesPromise := promiseMilesBeforeEmpty(myCar)
when (milesPromise) -> done(milesBeforeEmpty) {
    println(`miles before empty = $milesBeforeEmpty`)
} catch e {
    println(`Error: $e`)
}

Once again our example highlights several different features of E. The basic idea of the example is that the function promiseMilesBeforeEmpty(farCar) will eventually compute the number of miles the car can still travel before it is out of fuel; later in the program, when this computation is resolved, the program will print the value. However, before we can do the computation, we must first get both the fuelRemaining and the milesPerGallon values from the far car.

The promise and the resolver for that promise are created as a pair using the Ref.promise() built-in E primitive. They are "normal" variables in the sense that they can be passed as arguments, returned from methods as answers, and sent messages: the resolver can even accept immediate calls, though the promise of course cannot. In this example, the function returns the milesBeforeEmptyPromise to the caller just as it would return any other kind of value.

To cause the promise to resolve, call the resolver (or send to the resolver if the resolve may be remote to your program) with the "resolve(value)" method. To break the promise (which produces an exception that will cause the catch clause of a when-catch to execute), call the resolver with the "smash(value)" method.

The naming conventions used here are worth noticing. The function has the suffix "promise", informing the user of the function that the function will return a promise rather than an actual value, and the user will therefore have to resolve the promise before using the answer. Also, the parameter for the function has the prefix "far". We have seen the "far" prefix before, as a reminder to the author not to make immediate calls. Here, as a convention in a parameter list, the "far" prefix has the following deeper implication: by calling the parameter "farCar", the author of the function is making a commitment to users that nowhere in the function will the author use immediate calls: the author will only interact with the farCar using eventual sends. So the "far" convention in this case is not only a reminder, it can actually act as an important part of the function's contract.

Another interesting naming issue arises here: You may have noticed that the variable fuelRemaining is created using an eventual send. In fact, at the moment of creation, this variable contains only a promise, not a value. Our normal convention would be to give this variable a name such as fuelRemainingPromise. However, in this case, the fuelRemaining variable is never used except in the when-catch construct defined shortly after the creation of the variable. Inside this when-catch construct, the fuelRemaining promise is guaranteed to have been resolved. How do we know the promise is resolved, even though the when-catch does not explicitly resolve the value? Because, as noted earlier, a series of messages sent to a single remote object by a single program are guaranteed to be processed (and resolved) in the sequence in which they are sent. Therefore, the milesBeforeEmptyPromise cannot resolve until after the fuelRemaining promise has been fulfilled, and since the milesBeforeEmptyPromise is guaranteed to have been fulfilled inside the when-catch, fuelRemaining is also guaranteed to have been fulfilled. Upon fulfillment of the promise, the variable containing the promise is itself filled with the actual value. Therefore, fuelRemaining does indeed contain a value (not a promise) by the time it is actually used. And as a minor observation, note that inside the when-catch construct both milesBeforeEmptyPromise and milesBeforeEmpty contain the same resolved value.

As discussed earlier, though we can guarantee the ordering of messages sent and answers received coming back to a single program from a single object, when a program sends messages to several objects, the order in which answers will be received cannot be predicted. You can nest when-catch clauses to ensure several the resolutions have taken place before doing a computation:

def mercedesDistancePromise := mercedes <- distanceToFinishLine()
def chevyDistancePromise := chevy <- distanceToFinishLine()
when (mercedesDistancePromise) -> done(mercedesDistance) {
    when (chevyDistancePromise) -> done2(chevyDistance) {
        if (chevyDistance < mercedesDistance) {
            println("Chevy is in the lead")
        } else {
            println("Mercedes is in the lead")
        }
    } catch chevyE {
        println("Chevy lost")
    }
} catch mercedesE {
    println("Mercedes lost")
}

Here, to determine which car was leading the race, we had to get the distance f rom the finish line for each of the cars. We compute the leader inside nested when-catch constructs to ensure we have all the distances before trying to compare them. One very important syntactic note: we see in this example that the "done(variable)" clause of the when-catch construct is actually the declaration of a function and a parameter. The reasons for this are beyond the scope of this introduction, but the operational consequence is that you can use different names for the "done" function, and indeed you must use different names if you have multiple when-catch clauses in the same scope. Nested when-catch clauses put the "done" functions into the same scope. In the actual example here, we gave the inner when-catch "done" function the name "done2" to prevent the name collision. We also gave the error-containing variables in the catch clauses different names, chevyE and mercedesE.

We end this introduction by answering a last, critical question: How does a program acquire its very first reference to an object on a different computer? In all our examples up to this point, we have always started out with a reference to at least one remote object, and we retrieved other remote objects by asking that object for other objects: for example, we asked a remote carMaker for a new (remote) car, and were able to immediately work with the car just like any other remote object. How did we get the reference to the carMaker in the first place?

In E, the reference to an object can be encoded as a Universal Resource Identifier string, known as a uri (the familiar url of the Web is a type of uri). This uri string can be passed around in many fashions; one good secure way of passing a uri is to save it as a text file, encrypt and sign it with PGP, and send it in email. If you wish to run a seriously secure distributed E system, encrypting the uris is crucial: indeed, the passing of the uris from machine to machine is the main security issue that E cannot address for you. Other ways uris have been passed in operational E systems have been to send the uri over an ssh connection, and (less securely) by reading the uri off over a telephone! If you are using E on a local area network and have no security concerns, but are using E simply because it is simpler, safer, and more maintainable for distributed computing, the uris can be stored in files on a shared file system and read directly by the programs on different computers as they start up.

The functions makeURIFromObject(object) and getObjectFromURI(uri) detailed in the E Quick Reference Card perform the basic transformations you need to hook up objects on multiple computers. Each program that expects to work with remote objects needs to invoke the primitive "introducer onTheAir" before starting any remote connections, including the making or using of uris. An example of these functions working together can be found in the Securit-Echat system.

The promise architecture described here is the heart of what makes E different. There are no multiple threads, no synchronize statements, no critical objects, no deadlocks. Yet the promise architecture allows the construction of all the different distributed programming behaviors that more conventional architectures allow. The consequence is that distributed systems written in E are more robust and easier to work with-- once you have grasped the implications of promises. However, though this introduction may have taken only 15 minutes, most people will find it takes more time and more thought to fully appreciate the ramifications. As noted at the beginning, the E Quick Reference Card is a compact introduction to all the features of E, and may be the next useful place to go. Beyond that, the Securit-Echat system is a very small but full featured example of E at work, documented in detail for the explicit purpose of being a learning tool.

When first using a promise architecture, people often have a sense of breathlessness, a feeling that once they have started a series of remote computations, they have lost control of the action. This sense can actually become a sense of panic if the sense of lost controlled is accompanied by a sense of lost understanding. There are multiple techniques and patterns for "reining in" the remote computations. Several of these can be found in the Securit-Edesk program, including the promiseAllDone function, the vowsMonitorMaker, the Sequencer, and the summoning pattern. Feel free to join the E Language Discussion group if these matters become of serious interest to you.

Topics Beyond the 15 minute intro, for the full concurrency chapter:

Recursion for multiple when-catch resolutions, calculating lead car if we do not know how many cars there are in the race.

promiseAllDone example, both with a loop and with a recursion.

Summoning pattern

Breathless E, the need for blockAtTop and continueAtTop for a windowed application

The whenBroken construct. A promise can only be broken once, however, if the promise is fulfilled with a remote reference, that remote reference can become broken at any time through the loss of the connection.

Subtlety that it is objects outside the vat, not outside the computer, that need a remote reference.

Data lock possibility, example, explanation of why it is rare.